Report on the
External Evaluation
For
Within and Without the State; South Sudan project
‘Civil Society Strengthening Initiative’
OXFAM INTERNATIONAL
SOUTH SUDAN COUNTRY OFFICE
Management Support Organisation (MSO)
Plot 34/35 Makindye Road
P.O. Box 616, Kampala
April 2014
WWS External Evaluation Report – April 2014 ii
Executive Summary
The WWS was designed as a pilot phase targeting the two
geographical areas of Rumbek and Juba. The strategy was to
work closely with a few CBOs and CSOs in these locations
and use the experiences and lessons learnt through a
consistent and continuous documentation of success stories in
the course of implementation. The project sought to address;
the slow progress of putting in place functional government
instruments from the national government to the state and the
local governments that was affecting delivery of vital
development services from the State governments down to the
County, the Payams, the Bomas and the villages.
Efforts by emerging CSOs to offer support is limited by weak
coordination and synergy between the CBOs/CSOs, low
collaboration and information sharing, weak funding and
management systems and limited capacity and expertise to
propose alternatives.
The project was implemented within a very difficult landscape
that had; weak CSOs operating within a fragile state,
insecurity and political uncertainty, challenges as a result of
take off, capacity gaps and social cultural issues. The
evaluation considers these as potential factors that may have
influenced the outcomes of the interventions.
Despite the obstacles above, the evaluation considers that the
project interventions achieved above average performance and
has indications of high potential when obstacles are
adequately addressed.
This conclusion arises from the findings of the assessment;
Operational community dialogues in a number of locations
with full engagement of a section of MPs. This
operationalisation is also enhanced by discussions with state
and national assembly to address policy and legal
environment. Clearly there is more trust and improved
working relationship between government and the CSOs.
Within the interventions, partner capacities have been
enhanced and a CSO network formed to coordinate activities.
As such a number of target communities are continuously
aware and engaging with the leaders to address priorities and
needs. The community attitude towards women is changing
with more men indicating they are willing to share
responsibilities and resources with their wives. More women
Acknowledgement
This report was a product of a
concerted effort from very
dedicated teams, individuals
and personalities that played
important roles in
coordinating activities at all
levels.
We want to thank the Oxfam
staff and management for the
time and resources they put to
ensure that this work is
accomplished in the time it
was. Special thanks to Mr.
Rama Anthony, Mr. Alumgbi
Abure Isaac, Ms. Rebecca
Moriku and Mr. John Makur
who were at the forefront of
the coordination.
We also recognize the time
and contribution of the
implementing partners;
CEPO, SDRDA, APARD,
AIM and SSuDEMOP who
were willing to provide
information within a tight
schedule when the evaluation
was conducted.
Particular appreciation also to
the Government of Lakes
State, and more especially the
Ministry of Local
Government and the Ministry
of Social Development, the
Members of Parliament who
were part of the exercise. At
the National assembly, special
thanks to the Committee on
Human Rights and
Humanitarian Affairs and
Defense and National
Security.
And finally to all the
stakeholders and participating
communities who shared with
us the vital information that
formed the basis of the report.
WWS External Evaluation Report – April 2014 iii
are participating in community dialogues and forming clubs
for empowerment.
However, a number of areas need to be addressed in order to
improve the outcomes of the interventions. The assessment
noted areas of; sustainability of partners and approaches,
mode of delivery of the programme, formation and
strengthening of community structures, project support and
follows up and focuses on women empowerment among
others.
Management Support
Organisation (MSO)
WWS External Evaluation Report – April 2014 iv
Table of contents
EXECUTIVE SUMMARY ..........................................................................................................II
ACKNOWLEDGEMENT ............................................................................................................II
LIST OF ACRONYMS AND ABBREVIATIONS ..................................................................... V
1.0 INTRODUCTION ............................................................................................................. 1
2.0 BACKGROUND TO THE EVALUATION ...................................................................... 1
2.1 OVERVIEW OF THE PROJECT EVALUATED: ........................................................................ 1
2.2 THE OVERALL GOAL ....................................................................................................... 2
2.3 THE PROJECT SPECIFIC OBJECTIVE ................................................................................... 2
2.4 PROJECT RESULTS (SHORT-TO MEDIUM-TERM OUTCOMES) ............................................... 2
3.0 PURPOSE OF THE EVALUATION ................................................................................. 3
3.1 DESCRIPTION OF THE SCOPE OF THE EVALUATION ............................................. 3
4.0 APPROACHES AND METHODOLOGY ......................................................................... 3
4.1 METHODOLOGY .............................................................................................................. 4
4.2 DESCRIPTION OF TARGET GROUPS/RESPONDENTS ............................................................. 4
4.3 CONCEPTUAL FRAMEWORK GUIDING THE EVALUATION .................................................... 5
4.4 INFLUENCING FACTORS ................................................................................................... 6
4.5 CHALLENGES FACED DURING THE EVALUATION ............................................................... 7
5.0 EVALUATION FINDINGS .............................................................................................. 8
5.1 SUMMARY OF KEY FINDINGS ........................................................................................... 8
5.2 DETAILED ANALYSIS OF FINDINGS ................................................................................. 10
6.0 CONCLUSIONS ............................................................................................................. 16
7.0 RECOMMENDATIONS ................................................................................................. 17
7.1 DETAILED DESCRIPTION OF RECOMMENDATIONS ............................................................ 17
7.1 ANNEXES ...................................................................................................................... 21
ANNEX I: TERMS OF REFERENCE .............................................................................................. 21
ANNEX II: LIST OF RESPONDENTS ............................................................................................. 27
ANNEX III: EVALUATION PLAN AND FRAMEWORK .................................................................... 28
8.0 REFERENCES ................................................................................................................ 33
WWS External Evaluation Report – April 2014 v
List of Acronyms and Abbreviations
ACC
AIM
ALPs
APARD
CBOs
CEPO
CISON
CSOs
DFID
FGDs
IDIs
KIIS
LGs
LSLA
MoLG
MoSD
MPs
OPT/I
PWDs
RoSS
SDRDA
SSuDEMOP
UNMIS
WWS
Anti Corruption Commission
Agency for Independent Media
Adult Learning Projects
African Partnership for Aid Rehabilitation and Development
Community Based Organisations
Community Empowerment for Progress Organization
Civil Society Organisation Network
Civil Society Organisations
Department For International Development – United Kingdom
Focus Group Discussions
In-depth Interviews
Key Informant Interviews
Local Governments
Lakes State Legislative Assembly
Ministry of Local Government
Ministry of Social Development
Members of Parliament
Occupied Palestinian Territories/Israel
People With Disabilities
Republic of South Sudan
Sudanese Disable Rehabilitation And Development Agency
Sudan Domestic Election Monitoring and Observation Project
United Nations Mission in Sudan
Within and Without the State: Civil Society Strengthening Project
1.0 Introduction
This is a report on the external evaluation of the project; within and without the state: South
Sudan Civil Society Strengthening (WWS). It covers background information on the evaluation,
the approaches and methodology of the evaluation, the evaluation findings including a detailed
description of the findings, a conclusion and recommendations.
2.0 Background to the evaluation
Within and without the state: South Sudan Civil Society Strengthening (WWS) is a project that
Oxfam GB has been implementing within the Conflict, Security and Justice, and Humanitarian
Project Partnership Arrangement (PPA) with DFID for the period 2011-14. The Humanitarian
PPA includes one objective as a global initiative aimed at improving the quality and effectiveness
of programming in fragile and conflict-affected settings. This is initially targeting three pilot
countries: South Sudan, Occupied Palestinian Territories/Israel (OPT/I) and Afghanistan - with
the intention to roll out lessons from these countries to a wider group. In South Sudan the
initiative expects to model a different approach to working with civil society with particular focus
on building greater trust between stakeholders, promoting deeper political economy and thus a
more relevant programming at the end.
The project WWS was designed as a pilot phase targeting the two geographical areas of Rumbek
in Lakes state and Juba in Central Equatorial. Working closely with a few CBOs and CSOs in
these locations, the experiences and lessons learnt through a consistent and continuous
documentation of success stories and experiences in the course of implementation. The evaluation
was designed and intended to inform Oxfam on the appropriateness of the design of
implementation through CSOs and CBOs.
The project has worked with 2 CBOs based in Rumbek and 3 based in Juba. These partners were
selected through a rigorous process 1in which their governance structures, legal status, policy
profile, Human Resource structure and management, Monitoring, Evaluation and Reporting,
Financial Management and Sustainability and advocacy competencies were assessed.
2.1 Overview of the project evaluated:
According to the project document reviewed2, the issues the project sought to address are
anchored on the slow progress of putting in place functional government instruments from the
national government to the state and the local governments. As such many critical laws and policy
frameworks remain outstanding. The fact that the administration structures run from National
Government to State Governments and filter down to the County, the Payams, the Bomas and the
villages that make up the local government structures, there is evidently limited presence of CSOs
and CBOs making it very difficult for the National Government to properly deliver and respond to
the needs of the ordinary persons.
It is estimated that South Sudan has a predominantly rural population of up to 8 million people
whose livelihood is dependent on subsistence. The non functional structures and institutions
provide very little to the majority of the population already living in abject poverty. More than 4
1 Civil Society Organisation mapping and selection report by the Civil Society Strengthening project – August 2012. 2 Civil Society _ Project full proposal
WWS External Evaluation Report – April 2014 2
million of this population are internally displaced persons or became refugees as a result of the
civil war, conflicts and their related impacts.
The impact of these conflicts on people’s lives and livelihoods has been enormous, demonstrated
by the numbers of lives lost, numbers displaced persons and indicators such as the maternal
mortality rate and adult literacy rates.
Women are among the most affected by situations in South Sudan and their participation at all
levels of society is very poor and they remain almost voiceless within their communities. In most
parts of Southern Sudan, women are commoditised against cattle exposing them to enormous
structural and cultural barriers to access education, income and participate in leadership. Very few
women have access to and control of productive resources within their families and communities.
Despite efforts by emerging Civil Society Organisations to offer support, there is still limited
coordination and synergy between the CBOs/CSOs, low collaboration and information sharing
between local NGOs/CSOs and INGOs, weak funding and management systems, lack of clear
strategic vision for local NGOs/CSOs and very limited capacity and expertise to propose
alternative solutions where the government shows weaknesses.
2.2 The Overall goal
To contribute to the building of strong foundations for both women and men in South Sudan to
equally contribute to the development of their nation.
2.3 The project specific Objective
Increased accountability and responsiveness of Republic of South Sudan (RoSS), civil society and
donors to the citizens of South Sudan.
2.4 Project Results (Short-to medium-term outcomes)
a) Levels of trust and working relationships between CSOs & Oxfam and between CSOs and
State institutions are improved.
b) The capacity of selected CSOs and CBOs to be more accountable, transparent and responsive
is increased.
c) CSO and State Projects and policies are more aligned to the needs and priorities of poor
communities.
d) Poor women’s leadership of CSOs and State institutions and participation in economic
decision making is increased.
WWS External Evaluation Report – April 2014 3
3.0 Purpose of the evaluation
The main purpose of the final evaluation is to provide an independent assessment of the results of
Oxfam GB work and build a body of evidence that can be used to improve the quality of the
Projects. The evaluation is also expected to assess the key pillars of Oxfam GB interventions such
as accountability, transparency and participation of the project so as to inform future design and
implementation strategies. The evaluation is guided by parameters defined by Oxfam within the
project interventions:
a) Relevance and appropriateness: Interest of the evaluation sought to understand whether the
objectives, project design and activities were consistent with the building of strong foundation
for men and women by equitably contributing to nation building. Focus was to be placed on
complementarities and coherence with related activities undertaken by other actors.
b) Effectiveness: The evaluation sought to assess the extent to which the project results were
realised. Key focus was on the extent to which the project purpose and results were achieved.
c) Efficiency: The evaluation also sought to examine how well the various project activities have
transformed the available resources (inputs) into the intended results (outputs, outcome)
taking into consideration value for Money.
d) Sustainability: The evaluation sought to assess the extent to which the positive effects of the
project/program will still continue after external assistance has been ended.
e) Impact: Finally, the evaluation sought to assess the wholeness of the positive and negative,
primary and secondary results as produced by the program, directly or indirectly, intended or
unintended. The main focus was to measure the impact of the program on the target group.
This includes the assessment of the intended or unintended effects on the national, state and
local population, the environment, the conflict or other factors.
The evaluation results are meant to be shared with relevant donors, Oxfam GB Head Office in
Oxford, the South Sudan Country Team, and partners.
3.1 Description of the scope of the Evaluation
Based on the Terms of Reference, the evaluation of the WWS mainly focused on the Project
interventions implemented based on the project log frame and schedule of activities from 2011 –
2014. Furthermore, the evaluation was also to examine how the findings and recommendations
emanating from the review meetings and workshops undertaken since 2011 informed the
implementation of the project.
4.0 Approaches and Methodology
In order to be practically objective, the evaluation used an appreciative inquiry approach in
carrying out the evaluation. This was to ensure that the existing strengths and experiences of the
WWS External Evaluation Report – April 2014 4
Project are used to understand what works within the prevailing settings in order analyse
alternatives for future planning from the best outcomes and lessons learnt.
Since the design of the project focused more on qualitative results, the evaluation adapted
approaches to measure changes that are as a result of the direct/indirect contribution of the
interventions without necessarily relying on statistical judgments. This enabled identification of
positive learning points, challenges, gaps and experiences as presented by the Project staff,
partners, affiliates, stakeholders and intervention beneficiaries from which recommendations are
proposed for improving future Project plan, design and delivery.
4.1 Methodology
The methodology adapted was anchored on the nature and type of results and outcomes the
project considered. A methodology mix was used in order to ensure that as much information is
accessed as possible and the exercise made as participatory as possible. Emphasis was placed on
engaging Oxfam staff, partners, community leaders, Local and State governance structures and
institutions including local and national politicians as well as members of civil society
organisations that are either directly or indirectly engaged in the Project.
Primary data for the evaluation was collected through Focus Group Discussions, Key Informant
Interviews, In-depth Interviews and Field surveys and observations while secondary data was
collected through document reviews and testimonies. Appropriate tools and instruments were
developed to ensure that respondents exhaustively address the key issues and answer the
evaluation questions.
The entry point for the evaluation was review of documents such as; the project documents,
reports, work plans, minutes of meetings, correspondences between Oxfam and the partners and
stakeholders and other relevant documentation related. This was followed by a workshop
involving the project staff and the focal point persons of the partners in Lakes State. This
discussion was intended to validate the information contained in the reports and also get an
objective impression of the actors on progress made so far. This information was then triangulated
to feedbacks received from some of the direct beneficiaries and local and national government
structures involved. These feedbacks were analysed thematically based on the expected project
results and outcomes using the evaluation parameters.
4.2 Description of target groups/Respondents
The target groups for evaluation information were mainly those that were directly or indirectly
associated with the project from the onset. This was to enable the evaluation trace a change that
could be attributed to the interventions. The table below presents a summary of the respondents
and how many were reached.
WWS External Evaluation Report – April 2014 5
Respondents Target for Evaluation Method of data collection
1) National Legislative
Assembly
1 meeting with the Committee
on
1 meeting with the committee
on defence and national security
Focused discussions
Interviews
2) Ministry of Local
Government – Lakes
State
Director for Local Government
Key informant
Interviews
In-depth interviews
3) Ministry of Social
Development – Lakes
State
Director for Social
Development
Senior inspector – vulnerable
groups
Key informant
Interviews
In-depth interviews
4) Media houses within
Lakes State
Director - Radio Good News
Projects Manager – Radio Good
News
Key informant
Interviews
In-depth interviews
5) Civil Society
Organisations
(Rumbek/Juba)
5 Non Governmental
Organisations.
Key informant
Interviews
In-depth interviews
6) Community structures 5 Community volunteers
3 Community Working Groups
Focus Group
Discussions.
7) Direct beneficiaries 4 Women groups
2 Institutions promoting girl
education
3 Youth groups
Focus Group
Discussions.
Key Informant
Interview
4.3 Conceptual framework guiding the evaluation
The evaluation was conceptualised within a framework that looks at the components and sub
components within which the evaluation was undertaken. The evaluation was considered a
structural whole in exploring how the programme components such as structure of
implementation, programme beneficiaries, programme stakeholders and programme resources
have achieved their desired results based on the parameters such as relevance, appropriateness,
effectiveness, efficiency, sustainability and impact as described and how the sub components have
interrelated with each other.
The evaluation results and outputs were considered as important as the extent to which the
evidence of the parameters relate within the sub components. This was the basis to determine the
information obtained and analysed as per the objectives of the project.
WWS External Evaluation Report – April 2014 6
4.4 Influencing factors
The evaluation considers the following as influencing factors that may have impacted on the
program delivery. Although within the proposal, some of these were considered risks and
assumptions for which mitigating strategies were proposed, due to circumstances that evidently
were outside the intervention capacity of the Project, a number of the issues prevailed and may
have influenced the extent to which the results were obtained:
Influencing factor Intervention areas affected
1) Weak Civil Society
within a fragile state.
Local capacities and structures to independently
operate and engage government.
Legislative and policy framework to operate.
Trust and confidence in the Civil Society
activities and intentions.
2) Insecurity and political
uncertainty.
Community/public confidence to converge for
dialogues.
Social order and displacement of communities.
Constitution of the county counsellors
3) Delay in take off of the
project.
Planning and scheduling activities.
Engagement of communities.
4) Capacity of
implementing Partners.
Governance and implementation structure3.
Sustainability of approaches and activities
5) Social cultural issues Women’s participation in family choices and
development issues.
Access to information
3 Particularly seen with APARD and CEPO
Oxfam Project interventions (WWS) and
review
meetings
Project
Beneficiaries
Project
Stakeholders
Project
Resources
Project
Structure
Relevance and
appropriateness
Effectiveness
Efficiency
Sustainability
Impact
Attributes
expected
Provide independent
and objective assessment of work done.
Assess the key pillars of Oxfam
Inform future design and implementation strategy
Build a body of evidence to improve
quality of Projects
Evaluation exercise Evaluation Results and output
WWS External Evaluation Report – April 2014 7
4.5 Challenges faced during the evaluation
The evaluation process ran smoothly from the beginning to the end; however a few challenges
encountered did affect the running of the exercise. These issues are examined below in the context
of how they affected the process and recommendation of how they could be avoided in the future
evaluations and similar assignments.
Challenge Recommended areas to address
1) Delayed access to relevant
documentation for review.
Documents such as baseline report,
capacity building reports and field
monthly and quarterly reports were not
readily available for review. This
affected the comparison of information
on what is documented and the findings
in the findings.
Maintain a filing all relevant documents such
as progress and financial narrative, field
reports and updates, work plans and
schedules and all relevant reports for ease of
access and review.
Avail all required document for review in the
earliest time possible to enable structuring of
information for reference and analysis.
2) Insecurity in some sections of Lakes
State:
This was evident in Eastern Rumbek
were inter clan clashes were reported
during the period of data collection. As a
result of this, not all desired locations
were reached.
Security situations are unpredictable;
however alternative locations can be planned
in time so as to avoid gaps as a result of
inadequate data.
3) Failure to access some focal persons of
partner organisations.
Particular reference was AIM and
SSuDEMOP that indicated having
conducted activities in Rumbek, however
there was no source of verification.
Project management should follow up
activities and be in touch with focal persons
to ensure that the activities carried out are
verifiable.
Focal persons should be contacted in time in
order to avail themselves for such
assignments and feedback.
WWS External Evaluation Report – April 2014 8
5.0 Evaluation findings
The evaluation findings are derived from the contribution of the interventions to the desired
changes. The findings also show how much these changes can be attributed to the transformation
of the intervention outputs to outcomes in the short, medium and long term and the relationship
with the anticipated impact. These are looked at as the basis of the evaluation and are assessed
against the criterion set within the evaluation parameters.
For the purpose of the evaluation; short term is considered the period from the onset of the project
to the 6th month onwards, medium term as the period from the 6
th month of intervention to the 14
th
month and long term as the period from the 12th
month to the evaluation period and beyond. This
is graphically illustrated below:
Short Term Medium Term Long Term
0 6th 12
th 14
th 18
th +
Implementation months
5.1 Summary of Key findings
These summary findings present a snapshot of key findings of the evaluation grouped based on
their short term, medium term and long term contribution to the desired changes in the
intervention areas.
The Short term
Basis of evaluation Key findings
1) Operationalization
and effectiveness of
the consultation and
dialogue spaces.
a) Community structures set up and in place to mobilize for
consultation and dialogue (Community Working Groups and
Community Volunteers.
b) 9 Members of Parliament (MPs) fully engaged and following up
on dialogue and use the consultations to inform, mobilize and
engage communities on issues.
c) Important experiences shared during plenary to encourage other
MPs to replicate.
2) An enabling
operational
environment.
a) Willingness at state government to support & engage with CSOs
and CBOs.
b) Positive engagements with key National Assembly committees
to discuss and support legislations and policy environment.
c) Projects being run on through media with full government
support and awareness.
Improved trust and working relationship within actors
Enhanced capacities of actors
Priorities and needs of communities
Active participation of women
Operationalization
Functionality
Enabling environment
Improved capacities of actors
Priorities of communities
Level of participation of women
Mutual accountability and responsiveness on the plight of the ordinary citizens
WWS External Evaluation Report – April 2014 9
d) Restriction of meetings in some counties as a result of insecurity.
3) Enhanced capacity of
actors and partners.
a) Improved partner compliance to Oxfam and donor requirements.
b) Governance, operational and management capacity gaps
identified within partner organisation structures.
c) Community structures set up and engaged in planning,
implementation and monitoring.
4) Focus on priorities
and needs of
communities.
a) Level of openness in raising and discussing issues affecting
communities.
b) Increased awareness on negative social/cultural practices.
c) Increased awareness on the immediate and long term needs of
the communities.
5) Level of participation
of women
a) Increased number of women engaged in dialogues and
consultations.
b) Changes in the perception of the role of women and men in
development.
c) More women participating and engaged in family decisions and
choices.
The Medium Term
Basis of assessment Key findings
1) Improved level of
trust and working
relationship
between/among actors
a) Structure of engagement and collaboration formalized through a
Memorandum of Understanding with the Local Government
Board.
b) More requests for training and setting up County Council
structures.
c) Formation of Civil Society network within lakes State
d) Increased participation and engagement in planning and
advocacy within UN agencies.
2) Focus on priorities
and needs of
communities.
a) Improved community capacity to assess social and economic
needs and priorities.
b) In cooperation of identified needs and priorities in plans and
schedules of state agenda.
c) Promotion and recognition of rights of women, children and
Persons with Disabilities (PWDs).
3) Enhanced capacity of
actors and partners.
a) Increased demand from communities to engage more on topical
issues of governance and accountability.
b) Increased level of participation at state, county and community
levels.
c) Governance, operational and management capacity enhanced
within partner structures.
d) Inadequate focus on sustainability of approaches/Interventions.
WWS External Evaluation Report – April 2014 10
“We raise our issues to the MPs through the community volunteers who inform us on when the meeting with the MPs will be..” FGD in Ciubet
4) Level of participation
of women.
a) Increased number of women enrolled in Adult Learning Project
(ALPs)
b) Increased number of women groups being formed at county
level.
c) Participation of schools/institutions in structured discussions and
dialogue on issues affecting women.
d) More dialogue on practices that prohibit women
participation/restrain their development.
e) Increased women participation in decision making.
Long Term
Basis of assessment Key findings
1) Mutual accountability
and responsiveness to
the plight of citizens.
a) Interest, commitment and engagement of CSOs in addressing
social and governance issues.
b) Collaborations and partnerships with national and government,
CSO and communities to address emerging issues.
c) Improved participation and engagement between communities
and leaders in governance issues.
d) Platform provided to share knowledge, challenges and
experiences within Local Government settings.
e) Positive support from institutions/schools to provide space for
discussions and dialogue.
f) Full engagement of media in dissemination of information to
community.
5.2 Detailed analysis of findings
The above summaries of key findings were extracted from an analysis based on themes developed
during the evaluation. This section seeks to provide detail to some of the issues raised in section
5.1 above, covering the themes; community structures, engagement of local and national leaders,
willingness of government to engage with CSOs, Media activities, Capacities of implementing
partners, awareness on issues affecting communities, level of women engagements to mention.
5.2.1 Community structures set up and operational
In the short term, there is evidence that the project has enabled creation and setting up of
structures to operationalise the dialogue spaces through which the local leaders and politicians are
engaged. This was seen in the activities of community volunteers and County Working Groups in
Wullu and Ciubet counties were these structures are used as engines of mobilisation, follow up
and feedback.
Within the target communities, members talked to are aware and recognise the role these
structures play in operationalising the dialogue spaces. The volunteers and Working groups
receive petitions from the communities and fix Projects
with the MPs on when the dialogue should take place.
They follow up the promises made by MPs and report on
progress made to the project focal persons within Lakes.
WWS External Evaluation Report – April 2014 11
“Our facilitation is very poor, we need motorcycles and better allowances to be able to reach other remote areas..” IDI – Community Volunteer
“The dialogue has enabled me address specific needs of my constituency and created an open forum for discussions on different issues..” IDI – MP
“These approaches are very good for governance in Southern Sudan, but our ability to replicate them will depend on resources and if they are adapted by whole house..” IDI – MP
The assessment noted the above as a positive contribution from the interventions in the short term.
However, in the medium term, it was noted that the provision of soda and water during the
meetings may be a counterproductive motivation,
especially for the continued sustenance of the dialogue
spaces already in existence. The same is true for the
existence and willingness of the community volunteers and
the county working groups to continue with their current
engagements. This could partly explain the absence of information and failure to reach some of
the rural communities in the counties.
5.2.2 Engagements with Members of parliament
As a result of the interventions, there is evidence of a consistent interest in the members of
parliament at state and the national level to engage more with their constituents in addressing
development issues within the communities. At the time
of the evaluation, a total of nine MPs from Lakes State out
of a total of forty nine were engaged with their
communities each having participated in at least 2
dialogues. The rest of the MPs have simply avoided
exposing themselves to their constituents in order to dodge addressing issues of accountability and
transparency.
Although viewed as a challenge at the onset, some of the MPs engaged have taken on fully with
the dialogue to support them in addressing the issues raised. They consider the dialogue as an
important ingredient in dealing with community members and understanding their needs and
aspirations.
At the national level, the Project engaged positively with 2 committees of the national assembly –
The committee on Human Rights and Humanitarian Affairs and the committee on Defence and
National Security. Both committees view the activities and engagement of CSOs as vital and key
in sharing information of national importance. Despite these developments, there are still
significant cases of suspicion, mistrust and reservations in engaging with CSOs and communities
as a result of security concerns.
The ability of MPs to influence issues and that of government to take on some of the approaches
is dependent on a number of factors including; how popular the issues being presented are, who
the originator of the issue is, resources involved in
resolving the issues and the security situation in the
country among others. This has left a number of CSOs
unable to push issues they consider urgent beyond their
current efforts. Although a time consuming process, some
the initiatives undertaken such as the Media and NGO bill and other engagements with the
political leadership are of strategic nature and therefore have long term implications that the
assessment was not able to measure.
WWS External Evaluation Report – April 2014 12
“..LG board assigns a representative who participates in all the activities of CEPO from which insights are got to guide activities of Local Governments in the states..” KII – CEPO ED
“..the CSOs have been very vital in providing feedback from the communities on the current situation in the country regarding human rights and peace talks..” KII – MP, National Assembly
“..we are not fully involved in designing what is aired.. this limits our participation and yet we can make some of the content part and parcel of our programming...we can invite lawyers, police, social workers, women leaders, youth and other groups to come and participate in the panel discussions to enrich the Projects..” IDI – Projects Manager RGN
5.2.3 Willingness of government to engage with CSOs
One of the positive trends seen in the intervention areas is the will of the government to work with
and engage the CSOs. The signing of a Memorandum of Understanding between Community
Empowerment Project Organisation (CEPO) and the
Local Government Board was an expression of such will.
The MoU recognises the critical role that CSOs are
playing in supporting and strengthening the local
governments in order to address a number of critical
issues such as governance, accountability and citizen participation.
Through such partnerships and collaborations, CSOs are able to directly or indirectly influence
opinions and choices using foras such as the Commissioners Forum and capacity building
workshops. The exchanges and experience sharing during the forum has improved working
relationships and collaborations within the counties and the state government.
The nomination by government and participation of
two partners in the ongoing peace negotiations in
Addis Ababa is also viewed within the assessment as a
part of the existing government confidence and
willingness to engage CSOs in nation building.
Although these relationships developed circumstantially, they have been strengthened on a case
by case basis over time. What the assessment could not establish is if the relationships will be
sustained beyond the circumstances. However, what is evident in the current events is that
government needs the CSOs as much as the CSOs need government.
5.2.4 Media involvement as a key player
The involvement of Media as a platform for discussions, dissemination and seeking opinion of the
communities was critical considering the role of media in reaching out. There has been an
increased public interest to question leaders in issues affecting them as a result of the dialogue
spaces. Testimony from Radio Good News (RGN) indicates more call ins from communities
where dialogue is taking place and interest in MPs to use the radio more.
Although the Media faced challenges in the onset – with the closure of RGN, arresting journalists
and restraining media houses. There has been a general ease on media operations but with a kin
government eye on the content of what is aired. The security situation in the country has created
more apprehension on level of trust and confidence
from some media sections. Besides these
challenges however, media is still vital and
seemingly more critical a partner in achieving some
of the critical Project targets.
From the assessment, the engagement with media
especially in Lakes is unstructured based on
available air time. Participation of media in planning, follow up and monitoring trends is not
being fully utilised and besides, the Projects are not entrenched in the development focus of the
station. Engaging media and jointly designing these Project interventions will enable continuity of
WWS External Evaluation Report – April 2014 13
“..one of the challenges of implementation is the structural challenge of depending on Juba office to clear activities.. this takes time and most times you cannot adequately mobilise until you are sure of the approved budget..” IDI – Partner focal person
such Projects in the long term. Proposals of holding panel discussions in local languages
involving people of different technical backgrounds, social groupings and political affiliations
will further add value to the discussions and issues aired.
Although seen as one most effective avenue to reach out to the communities and pass on vital
message in greater volume, the prevailing conditions indicate that a small fraction of households
own a radio and phones due to the extreme poverty conditions therefore may not effectively
participate in the Project activities.
5.2.5 Capacities of partners to deliver
The project carried out assessments4 of partner capacities giving consideration to different
parameters. The assessment informed the partners on the gaps within the partner implementation,
governance and management. Support given to partners in the form of training and logistical
facilitation greatly contributed to improved delivery in the short and medium term.
The Project anticipated placing experts to support capacity development of partners over time, the
assessment established that all partners received support in financial management and
organisational development during the implementation period. This support improved
organisational functioning and operations in the short and medium term. However with increased
activities and the processes of organisational transformation in the medium and long term, critical
areas such as Project design, implementation strategies, organisational development,
communication and reporting that needed continuous support were not addressed. Gaps in these
areas were noted in some of the partner structures and indeed affected their functions and
operations. For continuity, the evaluation considers these aspects as important as those addressed.
The records and evidence within partner organisations indicate capacity gaps in some of the issues
raised above that substantially influenced delivery of the Project.
There were efforts and plans to coordinate and
follow up through the steering committee meetings
and routine Project monitoring. In the short term,
this yielded positively, however in the medium and
long term, coordination and supervision gaps
emerged as partner activities were implemented
without clear follow up and supervision. It is evident that some activities carried out in Lakes by
Juba based partners were not well coordinated with the Oxfam field office. This was partly
attributed to the lack of presence in Rumbek and logistical challenges arising from distance to
Juba.
The assessment noted diversity in technical competencies and strengths among the partners in
Juba and Rumbek. This diversity was not fully utilised considering each partner towed their own
line and as such efforts were not concerted to achieve desired outcomes. Although nearly all the
partners were found to be carrying out activities in similar locations and targeting the same
structures, the inter agency synergies were not tapped and therefore operational resources were
not efficiently and effectively engaged.
4 Civil Society Organisation mapping and selection report – Juba/Rumbek. August 2013
WWS External Evaluation Report – April 2014 14
“..I can now openly discuss and share roles with my husband without fearing like it was before. This has enabled me to join the rest of the women groups to learn from them different things like business and raising children..” FGD – beneficiary
“..it was through these discussions and engagements that recommendations to set up a women’s desk and one for PWDs was reached and implemented. Now women and PWDs address their issues through their respective desks at the MoSD..” IDI – Director MoSD
“..Because of what we are taught, I can now go and fetch water, bath the children and cook to support my wife while she is doing something else...I also discuss with her family issues freely unlike before..” FGD – Beneficiary, Cuiebet
In the view of the evaluation, the community and Project delivery structures that are required as a
foundation to sustain the approaches and activities have not been given adequate attention. The
current modalities can work in the short run but will not stand in the medium and long run as
interests may be diverted. The CSOs in Rumbek have formed an umbrella organisation – the
Civil Society Organisation Network (CISON) that is still at its infancy, this network will be a
critical platform for planning, follow up, advocacy and lobbying but needs more support to
overcome challenges.
5.2.6 Awareness on issues affecting communities
The evaluation established that communities that participated in the Project interventions adapted
a more open approach to raising and discussing issues affecting them. This was possible as a
result of the dialogue spaces and confidence built in communities on their rights and obligations.
From such discussions emerged knowledge on the
negative social/cultural practices affecting
communities, awareness on the immediate and long
term needs of the communities and improved
capacity to assess social and economic needs and
priorities.
Such awareness has drawn in different ministries at state level to in cooperate the identified needs
and priorities in plans and schedules of state
agenda. Through this, the rights of women,
children and Persons with Disabilities (PWDs) are
promoted and protected. It also emerged that MPs
are more in touch with their constituents and are
able to present factual arguments during debates in
the plenary.
5.2.7 Level of women’s engagement
More women according to the assessment have
participated in the community dialogues and
consultations. In the process, the social perception
of the role of men and women in decision making,
ownership of assets and community development
is gradually changing with more men appreciating the importance of engaging women in
development. There is evidence of women participating in family decisions and choices and
getting engaged in women’s groups and community activities.
As a result of this openness, more women’s groups have emerged within the communities with up
to 20 or 30 members in each payam. These women have come together to learn from each other
and share experiences on development concerns. One positive aspect of this gathering is the
number of women within these groupings that are enrolled in the Adult Learning Project (ALPs)
under a UNDP financed Project.
WWS External Evaluation Report – April 2014 15
“..the discussions were very educative and we would like to have more debates and sessions on issues such as bride price, education of women, forced marriage and violence against women..” FGD – Beneficiary student
“..many of my colleagues are not bothered about public accountability because they consider this an opportunity to amass wealth.. however, government is also constrained by resources, that is why we have not received the Community Development Grants for a long time..” MP Lakes State
Participation of schools/institutions in structured discussions and dialogue on issues affecting
women was evident during the assessment.
Although only one girl’s school and a university
participated, the result of the discussions and
dialogue were evident. Many of the girls
expressed interest in having more such discussions
and diversifying the topics to include subjects and practices seen as prohibitive to women
participation and development. Because of the relevance of the discussion points for the girls, the
school is planning to introduce regular discussions and debates adapting the approach introduced
and the girls are considering starting a “girl’s empowerment clubs” that will focus on carrying the
message back home during holidays.
5.2.8 Mutual accountability and responsiveness to the plight of citizens.
To achieve mutual interests and focus would be a farfetched reality in the span of the project.
However, the commitment, interest, and engagement of CSOs in addressing social, economic and
governance issues particularly in Lakes state is a positive development. Within the time span,
evidence of efforts to build collaborations and partnerships with state and national governments is
visible, more especially in responding to the plight
of citizens. The project has provided government
and CSOs a platform to share knowledge,
challenges and experiences within the Local
Government settings.
However, the assessment has taken note that as
much as there is a will to support the citizenry, there are still gaps in capacity to effectively be
responsive on the side of the government and a section of CSOs. Commitment issues also come at
the fore front at different levels of governance in as far as the ability to commit resources to
continuously engage with both CSOs and communities for mutual development. This arises from
the fact that the target communities and a section of leaders are still held up and recovering from a
history of dependence and handouts and so tend to focus on addressing immediate needs rather
than long term perspectives such as accountability, transparency and good governance
WWS External Evaluation Report – April 2014 16
6.0 Conclusions
The conclusions are drawn based on the 5 parameters that guided the evaluation exercise. They
are measured using a rating scale “to no extent” as the least achieved, “to some extent” as an
average achievement and “to a greater extent” as a high achievement.
Measure Conclusion Remarks
1. Extent to which project
objectives, design and
activities were consistent
with the project overall goal.
To a greater extent
As a pilot, the design was
appropriate in the context of the
implementation.
2. Extent to which the project
results were realised and or
achieved.
To some extent
Partners, institutions and
beneficiaries appreciate and have a
positive perception on the project.
Some activities scheduled late, not
implemented and or followed up.
Key aspects of the project such as
structures, synergies and
implementation capacities not
adequately addressed.
3. The extent to which the
project activities transformed
resources into results.
To some extent
Considering the time, some
partners achieved a lot within
resources provided.
Operational and structural
inefficiencies sighted in some
partner activities.
4. Extent to which the positive
effects of the project will
continue without external
assistance.
To some extent
Adaptation of some intervention
results within government and
educational institutions.
Evidence of dependence on project
for resources to implement.
5. Extent to which the program
has produced or influenced
the wholeness of positive or
negative primary and
secondary results directly or
indirectly.
To some extent
Participation of women in
domestic, community and state
development.
Participation of local leaders in
dialogue spaces.
Participation of CSOs in engaging
with government of legislations,
peace talks and other development
concerns.
WWS External Evaluation Report – April 2014 17
7.0 Recommendations
Basing on the findings and analysis of the intervention areas, the evaluation has come up with the
following recommendations. These are addressed to both Oxfam and the Implementing partners
as a basis of improving future programming and interventions. The table below presents issues the
evaluation has identified and the corresponding recommendations. The recommendations are
described in greater detail in section 7.1.
Issue identified Recommendation
1. Mode of delivery of the
Project.
a) Engage media (radio) in a more structured way.
b) Introduce more innovations in the dialogue and
engagements.
c) Build model within focused intervention areas to be able
to replicate and scale up.
d) Utilize and develop synergies among partners to be more
effective.
2. Capacity concerns within
implementing partners
a) Invest more in concrete governance structures and
institutional development for partners.
b) Deliberate follow up on issues raised and discussed
during partner meetings/trainings etc.
c) Strengthen the community structures to reach out the
entire community.
3. Project support, follow up
and monitoring of activities
a) Define specific targets for partners to achieve. This will
be a basis of follow up and monitoring.
b) Adhere to agreed Project work plan, implementation and
reporting schedule.
4. Sustainability of approaches
and activities of
partners/stakeholders.
a) Develop implementation strategies for eventual ownership
and sustainability.
b) Engage all key stakeholders along the implementation path.
c) Ensure that goal of the project is understood by all involved
to avoid diversion.
5. Emphasis on women
involvement
a) Address the gender power gaps that exist in the
organizations and governance structures as a matter of
policy.
b) Engage women in advocacy and mobilization drives to
reach out to more.
7.1 Detailed description of recommendations
The narrations below describe how the recommendations captured above can be achieved within
the framework of the project. These are based on findings of data collected from the field that
informed the opinion and recommendation of the evaluation.
R1. a) Engage media in a more structured way
Radio Good News, a member of Catholic Radio Network (CRN) has aired a number of
Programmes related to the project interventions areas and has been engaged in a number of the
activities including reporting on project activities. As a key player in Civil Society initiatives such
WWS External Evaluation Report – April 2014 18
as advocacy, mobilisation and information dissemination, it would be prudent to involve such a
partner in planning, implementation monitoring the activities of the Project.
From the views of radio management and review of the vision and focus of the station, engaging
them as part of the delivery process will enhance to programme implementation and
sustainability. This can be through joint planning, information gathering, monitoring and
dissemination. In the process, some of the project approaches and best practices can be adapted as
part and parcel of radio programming and they can become a vital source of feedback, assessment
and monitoring results.
R1. b) Introduce more innovations in dialogue and engagements
The dialogue, peace initiatives and engagements have been structured around meetings and formal
discussions with key players and stakeholders. Over time, the communities will develop fatigue
and disinterest especially when the issues become repetitive and progress not forthcoming.
Introducing more innovative approaches such as cultural dances, poetry, games and sports, posters
and messaging charts to address some of the thematic areas of the Project will attract more players
and keep youth and women more engaged in passing vital messages across.
R1. c) Build a model within a focused intervention for replication
The Project has very practical approaches that can be replicated and adapted across all local
governments when deliberately addressed. Rather than stretch out implementation capacity to
reach a farfetched targets, a model of approach and delivery should be build within selected states
and local government structures to study scenarios and deviations.
The successes of these models can then be used to mould other local governments to function in a
desired pattern. Key in achieving this is to ensure that all stakeholders are involved from the
onset.
R1. d) Utilise and develop synergies among partners to be more effective.
The partners engaged by the project have a diversity of strengths and competencies that
collectively can support achievement of the results of the project. Rather than converge their
efforts to carry out bits and pieces within particular locations and targeting isolated communities,
a deliberate approach to reach the same locations and communities with diversity of information
should be devised.
This will not only minimise strain in operational costs but also ease implementation through
mobilisation, monitoring and tracing desired results. Besides, the strength of CISON is derived
from the ability of partners to develop and adapt synergies.
R2. a) Invest more in concrete governance structures and institutional development for
partners.
Until partners understand some of the embedded values in the project such as accountability,
transparency, good governance and gender equity, it will be painstakingly difficult to pursue these
values within their areas of operation. The assessment noted some of the partners clearly have
challenges with governance and structural set ups that need to be improved.
Oxfam should prevail more concretely in proper governance and management structures and
capacities of partners in order to build a long lasting delivery vehicle for the project interventions.
WWS External Evaluation Report – April 2014 19
Such an investment through technical support, oversight and logistical support should be
conditioned as part of the partnership development process and milestones to adhere to over time.
R2. b) Deliberate follow up on issues raised and discussed during partner meetings/trainings
Trainings follow up meetings and capacity building sessions have yielded very impressive
recommendations and action points for the partners. Some of this directly addressing project
delivery. The challenge that may be attributed to workload and conflicting schedules is that little
or no follow up is done.
Project management should define a clear schedule and checklist of critical areas of
recommendations, follow up and improvements that partners are advised on. Such a schedule
should provide for feedback sessions and challenges to be addressed in order to ensure
consistency and continuous improvement.
R2. c) Strengthen the community structures to reach out the entire community.
The effectiveness of implementation is dependent on the strength and capacity of the
implementation structure from bottom to top. The structures in use such as County Working
Groups, Community Volunteers and project focal point persons should be strengthened to
penetrate the target communities.
This may involve identifying committed players, clearly defining their roles and responsibilities,
following up their activities and facilitating them to be more effective and efficient in the
discharge of their assignments.
R3. a) Define specific targets for partners to achieve. This will be a basis of follow up and
monitoring.
For the project partners to deliver effectively, defined and specific targets should be set clearly
indicating milestones and timelines in which to achieve such targets. This will help both the
implementing agency and the project management in scheduling activities, monitoring process
and reporting on activities implemented.
R3. b) Adhere to agreed Project work plan, implementation and reporting schedule.
Project management should ensure that implementing partners adhere to and follow project work
plan and other schedules that relate to the project cycle. This will ensure that management and
implementation partners are operating on the same schedule and plan.
R4. a) Develop implementation strategies for eventual ownership and sustainability.
For as long as the dialogues, community groups and activities will be attended as a result of
external motivation and enticements, there will be very little effort in ownership and sustainability
of the very good initiatives the project is proposing.
The project should adapt deliberate strategies of phasing out any direct or indirect support that
may be viewed by partners and stakeholders as a motivation. Some of the activities of the project
should be undertaken as initiatives either by government or the community in order to own them
with time.
R4. b) Engage all key stakeholders along the implementation path.
WWS External Evaluation Report – April 2014 20
All important stakeholders should be engaged in the implementation right from planning, carrying
out activities and monitoring and supervising. This helps the stakeholders understand what
information is related to the intervention and what is not.
R4. c) Ensure that goal of the project is understood by all involved to avoid diversion.
The project goal and objectives should be clearly explained to and understood by all parties
involved to ensure non diversion from focus. Such clarification will avoid duplication of
information, distortion of facts and wrong referencing.
Enough time should be allocated for this purpose and engaging key partners and stakeholders and
allow them explore and interpret the goal and objectives of the project within their context. This
will enable them share the same knowledge and information down the path.
WWS External Evaluation Report – April 2014 21
7.1 Annexes
Annex I Terms of Reference
Evaluation of Within and Without the State: South Sudan Civil Society Strengthening Project
Oxfam GB seeks qualified individual(s), group(s) or company to carry out an evaluation of the
above stated projects in its areas of operation in central equatorial and lakes states of South Sudan.
Duration. 21 days
Reporting to: Project Manager:
Background:
Oxfam GB in South Sudan:
Oxfam GB has been working in South Sudan since 1983, with initial activities in emergency
water supply and health services for refugees. It has continued to lead on humanitarian
interventions in South Sudan since then and currently has operational programming in two states
of Lakes and Upper Nile with a country office in Juba. Oxfam GB concentrates on four primary
sectors: Public Health (water, sanitation & hygiene promotion), Livelihoods and Emergency
Preparedness & Response (EP&R).
The Projects/responses are carried out by three teams: EP&R, Upper Nile and Lakes. Within these
Projects, issues such as gender, conflict sensitive programming, peace building, policy and
advocacy are furthered with country Project leadership in Juba and field offices in Rumbek,
Malakal and Maban.
Recently, Oxfam GB entered into a Conflict, Security and Justice, and Humanitarian Project
Partnership Arrangement (PPA) with DFID for 2011-14. The Humanitarian PPA includes one
objective as a global initiative aimed at improving the quality and effectiveness of programming
in fragile and conflict-affected settings5 with attention initially on three pilot countries: South
Sudan, Occupied Palestinian Territories/Israel (OPT/I) and Afghanistan - and then with the
intention to roll out lessons from these countries to a wider group. In South Sudan the initiative
expects to model a different approach to working with civil society in the interests of building
greater trust between stakeholders, promoting deeper political economy analysis, and more
relevant programming as a result.
With several decades of war and recent independence on the 9th July 2011, civil society
organisations in South Sudan are still evolving and will require meaningful support and
facilitation for them to be empowered and strengthened in order to play a proactive role to
catalyse accountability and development. The independence of South Sudan brought to the new
nation so many opportunities and challenges that the South Sudan government needs a strong
Civil Society to support in putting in place legitimate institutions and systems that can facilitate
equitable service delivery and relevant programming. As the work to define a permanent
constitution continues up to 20156, building on the interim constitution approved in July 2011,
WWS External Evaluation Report – April 2014 22
many instruments of state are being gradually developed with complications related to the
separation from the former instruments of Sudan and to appropriate definition and development of
the diverse South Sudan context.
In the context of South Sudan the meaning of Civil Society varies from national to state and
county levels. While at National level there is some civil society in the real sense of the term, at
State and county level the term civil society is used to mainly mean associations or groups and/or
local organisations without or with very limited organisational and governance structures. The
nature of civil society is very diverse, including religious and traditional leadership which
continues to have a big role and influence in changing the lives of people but also in shaping the
politics of South Sudan. Meanwhile the growth of civil society is notable but with concerns that
the invited space that CSOs occupy today may reduce over time if RoSS feels threatened by
challenges being raised by CSOs. The critical question to CSOs is about how to effectively use
that space, occupy other spaces and even initiate more spaces to work and walk with the State to
respond to the most urgent needs and priorities of citizens through deliberate accompaniment of
RoSS to become more legitimate, responsive and accountable. This project aimed at strengthening
the civil society in South Sudan details actions and approaches, over a period of three years (
2011-2013), using the secured £ 275,000 funds from DFID.
1. Overview of the project to be evaluated:
The Overall Aim
The overall goal of the project is to contribute to the building of strong foundations for both
women and men in South Sudan to equally contribute to the development of their nation.
The project specific Objective
The specific project objective is increased accountability and responsiveness of RoSS, civil
society and donors to the citizens of South Sudan
Project Results (Short-to medium-term outcomes)
a) Levels of trust and working relationships between CSOs & Oxfam and between CSOs and
State institutions are improved.
b) The capacity of selected CSOs and CBOs to be more accountable, transparent and responsive
is increased
c) CSO and State Projects and policies are more aligned to the needs and priorities of poor
communities
d) Poor women’s leadership of CSOs and State institutions and participation in economic
decision making is increased.
Purpose of evaluation and intended use
The main purpose of the final evaluation is to providing independent assessments of the results of
our work, and building a body of evidence that we can use to improve the quality of our Projects.
The evaluation is also expected to assess key pillars of Oxfam (accountability) of the project so as
to inform future design and implementation strategies.
The evaluation results will be shared with relevant donors, Oxfam GB Head Office (HO) in
Oxford, the South Sudan Country Team, and partners.
WWS External Evaluation Report – April 2014 23
Scope of the work:
With regard to South Sudan Civil Society project the evaluation will mainly focus on the Project
interventions implemented from 2011-2014. Furthermore, the evaluation will also examine how
the findings and recommendations emanating from the review meetings and workshops project
undertaken since 2011 informed the implementation of the project.
Relevance and appropriateness: Oxfam GB interest is to understand whether the objectives,
project design and activities were consistent with the building of strong foundation for men and
women by equitably contributing to the building of their nation. Focus should also be on
complementarily and coherence with related activities undertaken by other actors: Key issues of
focus for this evaluation will be as follows:
Extent to which the projects addressed the felt needs of the beneficiaries
How appropriate was the intervention logic in light of the prevailing context, beneficiary
needs and local capacities / Appropriateness of the implementation strategies and approaches
adopted
Extent to which targeted beneficiaries feel that they own the project
Examine lessons learnt with regard to relevance and appropriateness.
Effectiveness: Oxfam defines effectiveness as the extent to which the project results were
realised. Key focus is on whether the project purpose and results were achieved and to what
extent. Key Issues of focus for this evaluation will be as follows:
Extent to which project objectives/Outcomes have been achieved based on the outlined
indicators.
Beneficiary perceptions of the value, quality and quantity of the project in relation to their felt
needs
Examine extent to which project intervention logic was implemented and what factors if any
may have hindered implementation
Examine effectiveness of the linkages and synergies established with related projects.
Examine Lessons Learnt with regard to overall effectiveness of the project.
Extent to which protection issues were integrated in the program in terms of targeting,
implementation and advocacy.
Efficiency: Oxfam understands efficiency to mean how well the various project activities have
transformed the available resources (inputs) into the intended results (outputs, outcome) taking
into consideration value for Money. Key focus interest for the evaluation will be as follows:
Examination of the extent to which the project planning and implementation processes embraced
contributed towards greater efficiency in terms of time and resources inputs (both human and
material)
Assessing strategies that could have been adopted to increase the efficiency of project
implementation
Examine, document and share Lessons Learnt with regard to Efficiency.
Sustainability: At Oxfam point of view sustainability means the extent to which the positive
effects of the project/program will still continue after external assistance has been ended. The
evaluation will thus dwell on the questions below:
What are the prospects for the continuation of the changes brought by the project? Are these
likely to last or are they not likely to last beyond the project /Are the changes that have been
achieved likely to be sustained.
WWS External Evaluation Report – April 2014 24
What are the prospects for continuation of the national, state and local community capacities
developed and structures established for sustainability of the project. outcomes
Examine, document and share Lessons with regard to sustainability
Impact: Oxfam definition of impact is the wholeness of positive and negative, primary and
secondary results as produced by a development intervention, directly or indirectly, intended or
unintended. Oxfam’s main focus is to measure the impact of the program on the target group. This
should include the assessment of the intended or unintended effects on the national, state and local
population, the environment, the conflict or other factors. Key areas of focus for this evaluation
will be as follows:
Examine and document evidence/s of changes both positive and negative as a result of the
interventions
Measure of the needs of beneficiaries were specifically addressed and what has been the
outcome
Measure scenarios in which the project contributed towards the durable solutions of the civil
society in south Sudan.
Examine, document and share lessons learnt with regard to Impact
Principles of the evaluation:
The evaluation will be guided by the following ethical considerations:
Openness of information given, to the highest possible degree to all parties.
Wider and exclusive participation of interested parties
Reliability and independence.
Methodology
The evaluation will embrace a wide range of methodologies which will include though not
restricted to any or a combination of the following:
Desk study and document review: The evaluation team shall review proposals, reports, theory
of change, logocal frameworks, workplans and other documents associated with the Project.
Sample Survey: A sample survey will be undertaken mainly to establish achievements in
relation to the outcome level and some of the result level indicators. The evaluation team will
be expected to provide a detailed description of the methodology for the sample survey as well
as an outline of tools to be used.
Key stakeholder interviews: The evaluation team will conduct interviews with Oxfam staff in
Juba, Rumbek, national NGOs in Juba and in Lakes state, community based organizations,
local authorities and inter-agency networks where applicable.
Beneficiary focus groups: The evaluation team will meet with beneficiaries and community
representatives of the target population in each region. This can include focus groups, and
interviews.
Deliverables and reporting deadlines
The evaluation team will submit two reports(two printed and two electronic copies in English)
that include but not limited to Executive summary, Back ground, purpose/objectives,
introduction, methodology, findings, conclusion , recommendation with lessons for the Project
WWS External Evaluation Report – April 2014 25
going forward and appendix including evaluation terms of reference, maps, sample
framework, and bibliography.
Offer presentation to Oxfam project management Staff in Lakes state. Team members will
also provide a short presentation summarizing key findings for senior management in the Juba
office at the end of the field visit.
Inception report: Following the desk review and prior to beginning field work, the evaluation
team will produce an inception report subject to approval by the Oxfam Project manager.
This report will detail a draft work plan with a summary of the primary information needs, the
methodology to be used, and a work plan/schedule for field visits and major deadlines. With
respect to methodology, the evaluation team will provide a description of how data will be
collected, collated, analysed and a sampling framework, data sources, and drafts of suggested
data collection tools such as questionnaires and interview guides.
Once the report is finalized and accepted, the evaluation team must submit a request for any
change in strategy or approach to the Program Development Unit.
Draft report: A draft evaluation report will be submitted to the Development Project Manager,
who will review the draft and provide feedback within two weeks of receipt of the draft report.
All material collected in the undertaking of the evaluation process should be lodged with the
Program Development Unit prior to the termination of the contract.
Presentation of findings:
At the end of the field research, the evaluation team will present key findings to management
in the field for this case is lakes state (Rumbek field office).
Before the Final Evaluation Report is submitted, the evaluation team will present their
findings at Oxfam Country Office (Juba, Hai cinema, Opposite Mobil roundabout).
Time-frame and budget considerations
Proposals should present a budget in the number of expected working days over the entire
period (e.g. 21 full time days over 1 month).
Evaluation activities will be conducted during the months of May 2014, with a final
presentation of findings expected in the last week of May 2014.
A final decision on the evaluation team to be awarded the consultancy will be taken by the
Program Development Manager in consultation with the MEAL coordinator and key Project
staff. Desk review and initial conversations with the Program Development Manager and
MEAL coordinator will commence immediately the successful consultancy
firm/group/individual has been selected and approved accordingly.
The inception report will be submitted one week after the award of the consultancy.
In the event of serious problems or delays, the evaluation team leader should immediately
inform the Project Development Manager or the MEAL coordinator. Any significant changes
to review timetable shall be approved by the Development Manager in writing in advance.
Evaluation consultant team
(Required Expertise within the Team)
WWS External Evaluation Report – April 2014 26
A team consisting of individuals with multi-disciplinary skills is required for this exercise. The
team should possess demonstrable expertise/experience/skills in all of the following areas:
Democracy/ Governance Evaluations/Assessments in both development and humanitarian
settings
At least 5 years’ experience in Conducting Evaluations/Surveys/Assessments in development
and humanitarian settings particularly in the fragile states
Progressive good track record in conducting evaluations for interventions funded by key
donors such as DFIT, EU.
Fluency in written and spoken English is required of all team members.
Experience with the logical framework approach.
Prior experience in South Sudan is helpful.
Excellent skills and knowledge in training, supervision in data collection and monitoring
Excellent reporting skills.
Excellent skills in data collection, collation, analysis in qualitative and quantitative terms.
Masters degree in social sciences preferably in statistics for individual applications
Application process:
Interested and experienced consultant firms/individual can drop their applications (proposals)
clearly indicating: Consultancy Application: within and without the State: South Sudan Civil
Society Strengthening Project to [email protected] cc. [email protected]. March
28th 2014. Late submissions will not be considered.
WWS External Evaluation Report – April 2014 27
Annex II List of respondents
Agency/Stakeholder/Category No of Respondents
1. National Legislative Assembly 03Committee on Human Rights and Humanitarian
Affairs
02 Committee on defence and national security
2. Ministry of Local Government –
Lakes State
01 Director for Local Government
3. Ministry of Social Development –
Lakes State
01 Director for Social Development
01 Senior inspector – vulnerable groups
4. Media houses within Lakes State 01 Director - Radio Good News
01 Programmes Manager – Radio Good News
5. Implementing partners
(Rumbek/Juba)
5 Non Governmental Organisations.
6. Community structures 5 Community volunteers
3 County Working Groups
7. Direct beneficiaries 4 Women groups/Clubs
10 girls from a girls school
5 girls from a higher institution
3 Youth groups
5 Rural women
8. Project staff 3 staff of Oxfam
WWS External Evaluation Report – April 2014 28
Annex III Evaluation plan and Framework
EVALUATION PLAN AND FRAMEWORK
“Within and without the state; South Sudan civil society strengthening project”. Rumbek –
Wednesday 10th
April 2014 to Saturday 12th
April 2014.
Agency/
Partner
Date/Time of
meeting
Target
group/person for
data collection
Method of data
collection Evaluation focus areas
1) APARD
Date:
Wednesday 9th
April 2014
Time:
Community
mobilisers.
FGDs
Observations
a) How have you participated
in the implementation,
development and
monitoring the project?
b) Important lessons from the
project that women can
carry on without the project.
c) How differently would women address the issues
raised in the project?
d) What kind of support
reached women from the
agency/partner?
Date:
Wednesday 9th
April 2014
Time:
Director
General –
Ministry of
Social
Development
Key informant
Interview
a) Participation of MoSD in
implementation,
development and
monitoring the project.
b) Important aspects from the
project (women’s rights)
that govt would want to
carry on with.
c) How differently would govt address the issues (esp
women’s rights) raised in
the project.
d) What kind of government
support would be
anticipated and why?
Date:
Wednesday 9th
April 2014
Time:
Vice chancellor
Rumbek
University
HM Loreto
Girls S.S
Key informant
Interview
a) Participation of academic
institutions in
implementation,
development and
monitoring the project.
b) Important aspects from the
project that institutions
would want to carry on
with. c) How differently would
institutions address the
issues raised in the project?
d) What kind of institutional
support would be
anticipated and why?
Date:
Thursday 10th
April 2014
Time:
Radio Good
News (RGN)
Key informant
Interview
a) Participation of media in
implementation,
development and
monitoring the project.
b) Important aspects from the
project that the media would
want to carry on with.
c) How differently would media address the issues
raised in the project?
WWS External Evaluation Report – April 2014 29
d) What kind of media support
would be anticipated and
why?
2) SDRDA
Date:
Thursday 10th
April 2014
Time:
Chairpersons
CWGs
Community
Mobilisers
FGDs
Observations
a) Participation of community
in implementation,
development and
monitoring the project.
b) Important aspects from the
project that the community
would want to carry on with.
c) How differently would
communities address the
issues raised in the project.
d) What kind of community
support would be
anticipated and why?
Date:
Thursday 10th
April 2014
Time:
Speaker –
LSLA
Targeted
members of
parliament.
Key informant
Interview
FGDs
a) Participation of LSLA in
implementation,
development and
monitoring the project.
b) Important aspects from the
project that the LSLA would
want to carry on with. c) How differently would the
LSLA address policy and
legislative dialogue issues
raised in the project.
d) What kind of policy and
legislative support would be
anticipated and why?
Date:
Friday 11th April
2014
Time:
General
Secretary
CISON
Key informant
Interview
a) Participation of CSOs in
implementation,
development and
monitoring the project.
b) Important aspects from the
project that the CSO
network would want to
carry on with. c) How differently would the
CSO network address the
critical issues raised in the
project.
d) What kind of support would
the CBOs and partner
members receive and why?
3) CEPO
Date:
Friday 11th April
2014
Time:
Director
General,
Ministry of
Local
Government
Key informant
Interview
a) Participation of MoLG in
implementation,
development and
monitoring the project.
b) Important aspects of the
project (good governance)
that govt would want to carry on with.
c) How differently would govt
address the issues raised in
the project?
d) What kind of support from
government to CBOs/CSOs
would be anticipated and
why?
Date: Chairpersons, Focus Group a) Participation of youth and
WWS External Evaluation Report – April 2014 30
Friday 11th April
2014
Time:
youth and
women
associations
Discussions
Observations
women assns in
implementation,
development and
monitoring the project.
b) Important aspects from the
project (good
governance/accountability) that youth and women
associations would want to
carry on with.
c) How differently would these
associations address the
issues raised in the project?
d) What kind of support would
the associations anticipate
and why?
Date:
Friday 11th April
2014
Time:
Director, Anti
corruption
commission
Key informant
Interview
a) Participation of ACC in
implementation,
development and
monitoring the project.
b) Important aspects from the project (good
governance/accountability)
that ACC would want to
carry on with.
c) How differently would ACC
address the issues raised in
the project?
d) What kind of support would
ACC anticipate to
offer/receive and why?
Date:
Saturday 12th
April 2014
Time:
Community
members
Focus Group
Discussions
Observations
a) Participation of community
in implementation,
development and
monitoring the project.
b) Important aspects from the project that the community
would want to carry on
with.
c) How differently would
communities address the
issues raised in the project.
d) What kind of community
support would be
anticipated and why?
Date:
Saturday 12th
April 2014
Time:
Coordinator,
Peace
Commission
Key informant
Interview
a) Participation of ACC in
implementation,
development and
monitoring the project.
b) Important aspects from the project (good
governance/accountability)
that ACC would want to
carry on with.
c) How differently would ACC
address the issues raised in
the project?
d) What kind of support would
ACC anticipate to
offer/receive and why?
4) Oxfam staff Saturday 12th
April 2014 Project Officer Key informant
Interview
a) Project design and
compliance
WWS External Evaluation Report – April 2014 31
5) APARD Focal Person Key informant
Interview
b) Project structure and
implementation schedule
c) Reporting schedule and
requirements (activity and
financial reporting)
d) Project modifications and
adjustments. e) Sustainability of the Project
f) Project challenges and
limitations
6) CEPO
7) SDRDA
1) CEPO
Tuesday 15th
April 2014
Agency staff
Government
departments
and
institutions.
Community
beneficiaries
Focus Group
Discussions
Observations
a) Participation of in
implementation,
development and
monitoring the project.
b) Important aspects from the
project (good
governance/accountability)
that agency and partners
would want to carry on
with.
c) How differently would partner and agencies address
the issues raised in the
project?
d) What kind of support would
partners and agencies
anticipate to offer/receive
and why?
2) AIM
Wednesday
16th April 2014
Agency staff
Government
departments
and
institutions.
Community
beneficiaries
Focus Group
Discussions
Observations
a) Participation of in
implementation,
development and
monitoring the project.
b) Important aspects from the
project (good
governance/accountability)
that agency and partners would want to carry on
with.
c) How differently would
partner and agencies address
the issues raised in the
project?
d) What kind of support would
partners and agencies
anticipate to offer/receive
and why?
3) SSUDEMOP
Thursday 17th
April 2014
Agency staff
Government
departments
and institutions.
Community
beneficiaries.
Focus Group
Discussions
Observations
a) Participation of in
implementation,
development and
monitoring the project. b) Important aspects from the
project (good
governance/accountability)
that agency and partners
would want to carry on
with.
c) How differently would
partner and agencies address
the issues raised in the
project?
d) What kind of support would
partners and agencies
WWS External Evaluation Report – April 2014 32
anticipate to offer/receive
and why?
4) Presentation of findings and de
brief to the project management
at country office
Thursday 17th
April 2014
9:30pm to 10:30
am
MEL
Manager
Project
Manager
Country
Director.
WWS External Evaluation Report – April 2014 33
8.0 References
a) Civil Society Organisation mapping and selection report – Juba/Rumbek, August 2013
b) Civil Society Organisation MEAL training report – Destelia Nwenya, Dec 2012
c) Global Indicator assessment report – WWS, January 2013
d) Media relations and advocacy training report – MSO, July 2013
e) Minutes of Steering Committee Meetings – WWS, 2012, 2013
f) Partner capacity and assessment report – Oxfam GB, August 2013
g) PPA Annual report – WWS, December 2013
h) Situation analysis – Lakes State (Oxfam), 2012
i) South Sudan Civil Society project proposal.