Report on the external evaluation for "within or without the state" south Sudan project; civil...

Report on the

External Evaluation

For

Within and Without the State; South Sudan project

‘Civil Society Strengthening Initiative’

OXFAM INTERNATIONAL

SOUTH SUDAN COUNTRY OFFICE

Management Support Organisation (MSO)

Plot 34/35 Makindye Road

P.O. Box 616, Kampala

April 2014

WWS External Evaluation Report – April 2014 ii

Executive Summary

The WWS was designed as a pilot phase targeting the two

geographical areas of Rumbek and Juba. The strategy was to

work closely with a few CBOs and CSOs in these locations

and use the experiences and lessons learnt through a

consistent and continuous documentation of success stories in

the course of implementation. The project sought to address;

the slow progress of putting in place functional government

instruments from the national government to the state and the

local governments that was affecting delivery of vital

development services from the State governments down to the

County, the Payams, the Bomas and the villages.

Efforts by emerging CSOs to offer support is limited by weak

coordination and synergy between the CBOs/CSOs, low

collaboration and information sharing, weak funding and

management systems and limited capacity and expertise to

propose alternatives.

The project was implemented within a very difficult landscape

that had; weak CSOs operating within a fragile state,

insecurity and political uncertainty, challenges as a result of

take off, capacity gaps and social cultural issues. The

evaluation considers these as potential factors that may have

influenced the outcomes of the interventions.

Despite the obstacles above, the evaluation considers that the

project interventions achieved above average performance and

has indications of high potential when obstacles are

adequately addressed.

This conclusion arises from the findings of the assessment;

Operational community dialogues in a number of locations

with full engagement of a section of MPs. This

operationalisation is also enhanced by discussions with state

and national assembly to address policy and legal

environment. Clearly there is more trust and improved

working relationship between government and the CSOs.

Within the interventions, partner capacities have been

enhanced and a CSO network formed to coordinate activities.

As such a number of target communities are continuously

aware and engaging with the leaders to address priorities and

needs. The community attitude towards women is changing

with more men indicating they are willing to share

responsibilities and resources with their wives. More women

Acknowledgement

This report was a product of a

concerted effort from very

dedicated teams, individuals

and personalities that played

important roles in

coordinating activities at all

levels.

We want to thank the Oxfam

staff and management for the

time and resources they put to

ensure that this work is

accomplished in the time it

was. Special thanks to Mr.

Rama Anthony, Mr. Alumgbi

Abure Isaac, Ms. Rebecca

Moriku and Mr. John Makur

who were at the forefront of

the coordination.

We also recognize the time

and contribution of the

implementing partners;

CEPO, SDRDA, APARD,

AIM and SSuDEMOP who

were willing to provide

information within a tight

schedule when the evaluation

was conducted.

Particular appreciation also to

the Government of Lakes

State, and more especially the

Ministry of Local

Government and the Ministry

of Social Development, the

Members of Parliament who

were part of the exercise. At

the National assembly, special

thanks to the Committee on

Human Rights and

Humanitarian Affairs and

Defense and National

Security.

And finally to all the

stakeholders and participating

communities who shared with

us the vital information that

formed the basis of the report.

WWS External Evaluation Report – April 2014 iii

are participating in community dialogues and forming clubs

for empowerment.

However, a number of areas need to be addressed in order to

improve the outcomes of the interventions. The assessment

noted areas of; sustainability of partners and approaches,

mode of delivery of the programme, formation and

strengthening of community structures, project support and

follows up and focuses on women empowerment among

others.

Management Support

Organisation (MSO)

WWS External Evaluation Report – April 2014 iv

Table of contents

EXECUTIVE SUMMARY ..........................................................................................................II

ACKNOWLEDGEMENT ............................................................................................................II

LIST OF ACRONYMS AND ABBREVIATIONS ..................................................................... V

1.0 INTRODUCTION ............................................................................................................. 1

2.0 BACKGROUND TO THE EVALUATION ...................................................................... 1

2.1 OVERVIEW OF THE PROJECT EVALUATED: ........................................................................ 1

2.2 THE OVERALL GOAL ....................................................................................................... 2

2.3 THE PROJECT SPECIFIC OBJECTIVE ................................................................................... 2

2.4 PROJECT RESULTS (SHORT-TO MEDIUM-TERM OUTCOMES) ............................................... 2

3.0 PURPOSE OF THE EVALUATION ................................................................................. 3

3.1 DESCRIPTION OF THE SCOPE OF THE EVALUATION ............................................. 3

4.0 APPROACHES AND METHODOLOGY ......................................................................... 3

4.1 METHODOLOGY .............................................................................................................. 4

4.2 DESCRIPTION OF TARGET GROUPS/RESPONDENTS ............................................................. 4

4.3 CONCEPTUAL FRAMEWORK GUIDING THE EVALUATION .................................................... 5

4.4 INFLUENCING FACTORS ................................................................................................... 6

4.5 CHALLENGES FACED DURING THE EVALUATION ............................................................... 7

5.0 EVALUATION FINDINGS .............................................................................................. 8

5.1 SUMMARY OF KEY FINDINGS ........................................................................................... 8

5.2 DETAILED ANALYSIS OF FINDINGS ................................................................................. 10

6.0 CONCLUSIONS ............................................................................................................. 16

7.0 RECOMMENDATIONS ................................................................................................. 17

7.1 DETAILED DESCRIPTION OF RECOMMENDATIONS ............................................................ 17

7.1 ANNEXES ...................................................................................................................... 21

ANNEX I: TERMS OF REFERENCE .............................................................................................. 21

ANNEX II: LIST OF RESPONDENTS ............................................................................................. 27

ANNEX III: EVALUATION PLAN AND FRAMEWORK .................................................................... 28

8.0 REFERENCES ................................................................................................................ 33

WWS External Evaluation Report – April 2014 v

List of Acronyms and Abbreviations

ACC

AIM

ALPs

APARD

CBOs

CEPO

CISON

CSOs

DFID

FGDs

IDIs

KIIS

LGs

LSLA

MoLG

MoSD

MPs

OPT/I

PWDs

RoSS

SDRDA

SSuDEMOP

UNMIS

WWS

Anti Corruption Commission

Agency for Independent Media

Adult Learning Projects

African Partnership for Aid Rehabilitation and Development

Community Based Organisations

Community Empowerment for Progress Organization

Civil Society Organisation Network

Civil Society Organisations

Department For International Development – United Kingdom

Focus Group Discussions

In-depth Interviews

Key Informant Interviews

Local Governments

Lakes State Legislative Assembly

Ministry of Local Government

Ministry of Social Development

Members of Parliament

Occupied Palestinian Territories/Israel

People With Disabilities

Republic of South Sudan

Sudanese Disable Rehabilitation And Development Agency

Sudan Domestic Election Monitoring and Observation Project

United Nations Mission in Sudan

Within and Without the State: Civil Society Strengthening Project

1.0 Introduction

This is a report on the external evaluation of the project; within and without the state: South

Sudan Civil Society Strengthening (WWS). It covers background information on the evaluation,

the approaches and methodology of the evaluation, the evaluation findings including a detailed

description of the findings, a conclusion and recommendations.

2.0 Background to the evaluation

Within and without the state: South Sudan Civil Society Strengthening (WWS) is a project that

Oxfam GB has been implementing within the Conflict, Security and Justice, and Humanitarian

Project Partnership Arrangement (PPA) with DFID for the period 2011-14. The Humanitarian

PPA includes one objective as a global initiative aimed at improving the quality and effectiveness

of programming in fragile and conflict-affected settings. This is initially targeting three pilot

countries: South Sudan, Occupied Palestinian Territories/Israel (OPT/I) and Afghanistan - with

the intention to roll out lessons from these countries to a wider group. In South Sudan the

initiative expects to model a different approach to working with civil society with particular focus

on building greater trust between stakeholders, promoting deeper political economy and thus a

more relevant programming at the end.

The project WWS was designed as a pilot phase targeting the two geographical areas of Rumbek

in Lakes state and Juba in Central Equatorial. Working closely with a few CBOs and CSOs in

these locations, the experiences and lessons learnt through a consistent and continuous

documentation of success stories and experiences in the course of implementation. The evaluation

was designed and intended to inform Oxfam on the appropriateness of the design of

implementation through CSOs and CBOs.

The project has worked with 2 CBOs based in Rumbek and 3 based in Juba. These partners were

selected through a rigorous process 1in which their governance structures, legal status, policy

profile, Human Resource structure and management, Monitoring, Evaluation and Reporting,

Financial Management and Sustainability and advocacy competencies were assessed.

2.1 Overview of the project evaluated:

According to the project document reviewed2, the issues the project sought to address are

anchored on the slow progress of putting in place functional government instruments from the

national government to the state and the local governments. As such many critical laws and policy

frameworks remain outstanding. The fact that the administration structures run from National

Government to State Governments and filter down to the County, the Payams, the Bomas and the

villages that make up the local government structures, there is evidently limited presence of CSOs

and CBOs making it very difficult for the National Government to properly deliver and respond to

the needs of the ordinary persons.

It is estimated that South Sudan has a predominantly rural population of up to 8 million people

whose livelihood is dependent on subsistence. The non functional structures and institutions

provide very little to the majority of the population already living in abject poverty. More than 4

1 Civil Society Organisation mapping and selection report by the Civil Society Strengthening project – August 2012. 2 Civil Society _ Project full proposal

WWS External Evaluation Report – April 2014 2

million of this population are internally displaced persons or became refugees as a result of the

civil war, conflicts and their related impacts.

The impact of these conflicts on people’s lives and livelihoods has been enormous, demonstrated

by the numbers of lives lost, numbers displaced persons and indicators such as the maternal

mortality rate and adult literacy rates.

Women are among the most affected by situations in South Sudan and their participation at all

levels of society is very poor and they remain almost voiceless within their communities. In most

parts of Southern Sudan, women are commoditised against cattle exposing them to enormous

structural and cultural barriers to access education, income and participate in leadership. Very few

women have access to and control of productive resources within their families and communities.

Despite efforts by emerging Civil Society Organisations to offer support, there is still limited

coordination and synergy between the CBOs/CSOs, low collaboration and information sharing

between local NGOs/CSOs and INGOs, weak funding and management systems, lack of clear

strategic vision for local NGOs/CSOs and very limited capacity and expertise to propose

alternative solutions where the government shows weaknesses.

2.2 The Overall goal

To contribute to the building of strong foundations for both women and men in South Sudan to

equally contribute to the development of their nation.

2.3 The project specific Objective

Increased accountability and responsiveness of Republic of South Sudan (RoSS), civil society and

donors to the citizens of South Sudan.

2.4 Project Results (Short-to medium-term outcomes)

a) Levels of trust and working relationships between CSOs & Oxfam and between CSOs and

State institutions are improved.

b) The capacity of selected CSOs and CBOs to be more accountable, transparent and responsive

is increased.

c) CSO and State Projects and policies are more aligned to the needs and priorities of poor

communities.

d) Poor women’s leadership of CSOs and State institutions and participation in economic

decision making is increased.

http://en.wikipedia.org/wiki/Internally_displaced_person

http://en.wikipedia.org/wiki/Refugee


3.0 Purpose of the evaluation

The main purpose of the final evaluation is to provide an independent assessment of the results of

Oxfam GB work and build a body of evidence that can be used to improve the quality of the

Projects. The evaluation is also expected to assess the key pillars of Oxfam GB interventions such

as accountability, transparency and participation of the project so as to inform future design and

implementation strategies. The evaluation is guided by parameters defined by Oxfam within the

project interventions:

a) Relevance and appropriateness: Interest of the evaluation sought to understand whether the

objectives, project design and activities were consistent with the building of strong foundation

for men and women by equitably contributing to nation building. Focus was to be placed on

complementarities and coherence with related activities undertaken by other actors.

b) Effectiveness: The evaluation sought to assess the extent to which the project results were

realised. Key focus was on the extent to which the project purpose and results were achieved.

c) Efficiency: The evaluation also sought to examine how well the various project activities have

transformed the available resources (inputs) into the intended results (outputs, outcome)

taking into consideration value for Money.

d) Sustainability: The evaluation sought to assess the extent to which the positive effects of the

project/program will still continue after external assistance has been ended.

e) Impact: Finally, the evaluation sought to assess the wholeness of the positive and negative,

primary and secondary results as produced by the program, directly or indirectly, intended or

unintended. The main focus was to measure the impact of the program on the target group.

This includes the assessment of the intended or unintended effects on the national, state and

local population, the environment, the conflict or other factors.

The evaluation results are meant to be shared with relevant donors, Oxfam GB Head Office in

Oxford, the South Sudan Country Team, and partners.

3.1 Description of the scope of the Evaluation

Based on the Terms of Reference, the evaluation of the WWS mainly focused on the Project

interventions implemented based on the project log frame and schedule of activities from 2011 –

2014. Furthermore, the evaluation was also to examine how the findings and recommendations

emanating from the review meetings and workshops undertaken since 2011 informed the

implementation of the project.

4.0 Approaches and Methodology

In order to be practically objective, the evaluation used an appreciative inquiry approach in

carrying out the evaluation. This was to ensure that the existing strengths and experiences of the


Project are used to understand what works within the prevailing settings in order analyse

alternatives for future planning from the best outcomes and lessons learnt.

Since the design of the project focused more on qualitative results, the evaluation adapted

approaches to measure changes that are as a result of the direct/indirect contribution of the

interventions without necessarily relying on statistical judgments. This enabled identification of

positive learning points, challenges, gaps and experiences as presented by the Project staff,

partners, affiliates, stakeholders and intervention beneficiaries from which recommendations are

proposed for improving future Project plan, design and delivery.

4.1 Methodology

The methodology adapted was anchored on the nature and type of results and outcomes the

project considered. A methodology mix was used in order to ensure that as much information is

accessed as possible and the exercise made as participatory as possible. Emphasis was placed on

engaging Oxfam staff, partners, community leaders, Local and State governance structures and

institutions including local and national politicians as well as members of civil society

organisations that are either directly or indirectly engaged in the Project.

Primary data for the evaluation was collected through Focus Group Discussions, Key Informant

Interviews, In-depth Interviews and Field surveys and observations while secondary data was

collected through document reviews and testimonies. Appropriate tools and instruments were

developed to ensure that respondents exhaustively address the key issues and answer the

evaluation questions.

The entry point for the evaluation was review of documents such as; the project documents,

reports, work plans, minutes of meetings, correspondences between Oxfam and the partners and

stakeholders and other relevant documentation related. This was followed by a workshop

involving the project staff and the focal point persons of the partners in Lakes State. This

discussion was intended to validate the information contained in the reports and also get an

objective impression of the actors on progress made so far. This information was then triangulated

to feedbacks received from some of the direct beneficiaries and local and national government

structures involved. These feedbacks were analysed thematically based on the expected project

results and outcomes using the evaluation parameters.

4.2 Description of target groups/Respondents

The target groups for evaluation information were mainly those that were directly or indirectly

associated with the project from the onset. This was to enable the evaluation trace a change that

could be attributed to the interventions. The table below presents a summary of the respondents

and how many were reached.


Respondents Target for Evaluation Method of data collection

1) National Legislative

Assembly

1 meeting with the Committee

on

1 meeting with the committee

on defence and national security

Focused discussions

Interviews

2) Ministry of Local

Government – Lakes

State

Director for Local Government

Key informant

Interviews

In-depth interviews

3) Ministry of Social

Development – Lakes

State

Director for Social

Development

Senior inspector – vulnerable

groups

Key informant

Interviews

In-depth interviews

4) Media houses within

Lakes State

Director - Radio Good News

Projects Manager – Radio Good

News

Key informant

Interviews

In-depth interviews

5) Civil Society

Organisations

(Rumbek/Juba)

5 Non Governmental

Organisations.

Key informant

Interviews

In-depth interviews

6) Community structures 5 Community volunteers

3 Community Working Groups

Focus Group

Discussions.

7) Direct beneficiaries 4 Women groups

2 Institutions promoting girl

education

3 Youth groups

Focus Group

Discussions.

Key Informant

Interview

4.3 Conceptual framework guiding the evaluation

The evaluation was conceptualised within a framework that looks at the components and sub

components within which the evaluation was undertaken. The evaluation was considered a

structural whole in exploring how the programme components such as structure of

implementation, programme beneficiaries, programme stakeholders and programme resources

have achieved their desired results based on the parameters such as relevance, appropriateness,

effectiveness, efficiency, sustainability and impact as described and how the sub components have

interrelated with each other.

The evaluation results and outputs were considered as important as the extent to which the

evidence of the parameters relate within the sub components. This was the basis to determine the

information obtained and analysed as per the objectives of the project.


4.4 Influencing factors

The evaluation considers the following as influencing factors that may have impacted on the

program delivery. Although within the proposal, some of these were considered risks and

assumptions for which mitigating strategies were proposed, due to circumstances that evidently

were outside the intervention capacity of the Project, a number of the issues prevailed and may

have influenced the extent to which the results were obtained:

Influencing factor Intervention areas affected

1) Weak Civil Society

within a fragile state.

Local capacities and structures to independently

operate and engage government.

Legislative and policy framework to operate.

Trust and confidence in the Civil Society

activities and intentions.

2) Insecurity and political

uncertainty.

Community/public confidence to converge for

dialogues.

Social order and displacement of communities.

Constitution of the county counsellors

3) Delay in take off of the

project.

Planning and scheduling activities.

Engagement of communities.

4) Capacity of

implementing Partners.

Governance and implementation structure3.

Sustainability of approaches and activities

5) Social cultural issues Women’s participation in family choices and

development issues.

Access to information

3 Particularly seen with APARD and CEPO

Oxfam Project interventions (WWS) and

review

meetings

Project

Beneficiaries

Project

Stakeholders

Project

Resources

Project

Structure

Relevance and

appropriateness

Effectiveness

Efficiency

Sustainability

Impact

Attributes

expected

Provide independent

and objective assessment of work done.

Assess the key pillars of Oxfam

Inform future design and implementation strategy

Build a body of evidence to improve

quality of Projects

Evaluation exercise Evaluation Results and output


4.5 Challenges faced during the evaluation

The evaluation process ran smoothly from the beginning to the end; however a few challenges

encountered did affect the running of the exercise. These issues are examined below in the context

of how they affected the process and recommendation of how they could be avoided in the future

evaluations and similar assignments.

Challenge Recommended areas to address

1) Delayed access to relevant

documentation for review.

Documents such as baseline report,

capacity building reports and field

monthly and quarterly reports were not

readily available for review. This

affected the comparison of information

on what is documented and the findings

in the findings.

Maintain a filing all relevant documents such

as progress and financial narrative, field

reports and updates, work plans and

schedules and all relevant reports for ease of

access and review.

Avail all required document for review in the

earliest time possible to enable structuring of

information for reference and analysis.

2) Insecurity in some sections of Lakes

State:

This was evident in Eastern Rumbek

were inter clan clashes were reported

during the period of data collection. As a

result of this, not all desired locations

were reached.

Security situations are unpredictable;

however alternative locations can be planned

in time so as to avoid gaps as a result of

inadequate data.

3) Failure to access some focal persons of

partner organisations.

Particular reference was AIM and

SSuDEMOP that indicated having

conducted activities in Rumbek, however

there was no source of verification.

Project management should follow up

activities and be in touch with focal persons

to ensure that the activities carried out are

verifiable.

Focal persons should be contacted in time in

order to avail themselves for such

assignments and feedback.


5.0 Evaluation findings

The evaluation findings are derived from the contribution of the interventions to the desired

changes. The findings also show how much these changes can be attributed to the transformation

of the intervention outputs to outcomes in the short, medium and long term and the relationship

with the anticipated impact. These are looked at as the basis of the evaluation and are assessed

against the criterion set within the evaluation parameters.

For the purpose of the evaluation; short term is considered the period from the onset of the project

to the 6th month onwards, medium term as the period from the 6

th month of intervention to the 14

th

month and long term as the period from the 12th

month to the evaluation period and beyond. This

is graphically illustrated below:

Short Term Medium Term Long Term

0 6th 12

th 14

th 18

th +

Implementation months

5.1 Summary of Key findings

These summary findings present a snapshot of key findings of the evaluation grouped based on

their short term, medium term and long term contribution to the desired changes in the

intervention areas.

The Short term

Basis of evaluation Key findings

1) Operationalization

and effectiveness of

the consultation and

dialogue spaces.

a) Community structures set up and in place to mobilize for

consultation and dialogue (Community Working Groups and

Community Volunteers.

b) 9 Members of Parliament (MPs) fully engaged and following up

on dialogue and use the consultations to inform, mobilize and

engage communities on issues.

c) Important experiences shared during plenary to encourage other

MPs to replicate.

2) An enabling

operational

environment.

a) Willingness at state government to support & engage with CSOs

and CBOs.

b) Positive engagements with key National Assembly committees

to discuss and support legislations and policy environment.

c) Projects being run on through media with full government

support and awareness.

Improved trust and working relationship within actors

Enhanced capacities of actors

Priorities and needs of communities

Active participation of women

Operationalization

Functionality

Enabling environment

Improved capacities of actors

Priorities of communities

Level of participation of women

Mutual accountability and responsiveness on the plight of the ordinary citizens


d) Restriction of meetings in some counties as a result of insecurity.

3) Enhanced capacity of

actors and partners.

a) Improved partner compliance to Oxfam and donor requirements.

b) Governance, operational and management capacity gaps

identified within partner organisation structures.

c) Community structures set up and engaged in planning,

implementation and monitoring.

4) Focus on priorities

and needs of

communities.

a) Level of openness in raising and discussing issues affecting

communities.

b) Increased awareness on negative social/cultural practices.

c) Increased awareness on the immediate and long term needs of

the communities.

5) Level of participation

of women

a) Increased number of women engaged in dialogues and

consultations.

b) Changes in the perception of the role of women and men in

development.

c) More women participating and engaged in family decisions and

choices.

The Medium Term

Basis of assessment Key findings

1) Improved level of

trust and working

relationship

between/among actors

a) Structure of engagement and collaboration formalized through a

Memorandum of Understanding with the Local Government

Board.

b) More requests for training and setting up County Council

structures.

c) Formation of Civil Society network within lakes State

d) Increased participation and engagement in planning and

advocacy within UN agencies.

2) Focus on priorities

and needs of

communities.

a) Improved community capacity to assess social and economic

needs and priorities.

b) In cooperation of identified needs and priorities in plans and

schedules of state agenda.

c) Promotion and recognition of rights of women, children and

Persons with Disabilities (PWDs).

3) Enhanced capacity of

actors and partners.

a) Increased demand from communities to engage more on topical

issues of governance and accountability.

b) Increased level of participation at state, county and community

levels.

c) Governance, operational and management capacity enhanced

within partner structures.

d) Inadequate focus on sustainability of approaches/Interventions.


“We raise our issues to the MPs through the community volunteers who inform us on when the meeting with the MPs will be..” FGD in Ciubet

4) Level of participation

of women.

a) Increased number of women enrolled in Adult Learning Project

(ALPs)

b) Increased number of women groups being formed at county

level.

c) Participation of schools/institutions in structured discussions and

dialogue on issues affecting women.

d) More dialogue on practices that prohibit women

participation/restrain their development.

e) Increased women participation in decision making.

Long Term

Basis of assessment Key findings

1) Mutual accountability

and responsiveness to

the plight of citizens.

a) Interest, commitment and engagement of CSOs in addressing

social and governance issues.

b) Collaborations and partnerships with national and government,

CSO and communities to address emerging issues.

c) Improved participation and engagement between communities

and leaders in governance issues.

d) Platform provided to share knowledge, challenges and

experiences within Local Government settings.

e) Positive support from institutions/schools to provide space for

discussions and dialogue.

f) Full engagement of media in dissemination of information to

community.

5.2 Detailed analysis of findings

The above summaries of key findings were extracted from an analysis based on themes developed

during the evaluation. This section seeks to provide detail to some of the issues raised in section

5.1 above, covering the themes; community structures, engagement of local and national leaders,

willingness of government to engage with CSOs, Media activities, Capacities of implementing

partners, awareness on issues affecting communities, level of women engagements to mention.

5.2.1 Community structures set up and operational

In the short term, there is evidence that the project has enabled creation and setting up of

structures to operationalise the dialogue spaces through which the local leaders and politicians are

engaged. This was seen in the activities of community volunteers and County Working Groups in

Wullu and Ciubet counties were these structures are used as engines of mobilisation, follow up

and feedback.

Within the target communities, members talked to are aware and recognise the role these

structures play in operationalising the dialogue spaces. The volunteers and Working groups

receive petitions from the communities and fix Projects

with the MPs on when the dialogue should take place.

They follow up the promises made by MPs and report on

progress made to the project focal persons within Lakes.


“Our facilitation is very poor, we need motorcycles and better allowances to be able to reach other remote areas..” IDI – Community Volunteer

“The dialogue has enabled me address specific needs of my constituency and created an open forum for discussions on different issues..” IDI – MP

“These approaches are very good for governance in Southern Sudan, but our ability to replicate them will depend on resources and if they are adapted by whole house..” IDI – MP

The assessment noted the above as a positive contribution from the interventions in the short term.

However, in the medium term, it was noted that the provision of soda and water during the

meetings may be a counterproductive motivation,

especially for the continued sustenance of the dialogue

spaces already in existence. The same is true for the

existence and willingness of the community volunteers and

the county working groups to continue with their current

engagements. This could partly explain the absence of information and failure to reach some of

the rural communities in the counties.

5.2.2 Engagements with Members of parliament

As a result of the interventions, there is evidence of a consistent interest in the members of

parliament at state and the national level to engage more with their constituents in addressing

development issues within the communities. At the time

of the evaluation, a total of nine MPs from Lakes State out

of a total of forty nine were engaged with their

communities each having participated in at least 2

dialogues. The rest of the MPs have simply avoided

exposing themselves to their constituents in order to dodge addressing issues of accountability and

transparency.

Although viewed as a challenge at the onset, some of the MPs engaged have taken on fully with

the dialogue to support them in addressing the issues raised. They consider the dialogue as an

important ingredient in dealing with community members and understanding their needs and

aspirations.

At the national level, the Project engaged positively with 2 committees of the national assembly –

The committee on Human Rights and Humanitarian Affairs and the committee on Defence and

National Security. Both committees view the activities and engagement of CSOs as vital and key

in sharing information of national importance. Despite these developments, there are still

significant cases of suspicion, mistrust and reservations in engaging with CSOs and communities

as a result of security concerns.

The ability of MPs to influence issues and that of government to take on some of the approaches

is dependent on a number of factors including; how popular the issues being presented are, who

the originator of the issue is, resources involved in

resolving the issues and the security situation in the

country among others. This has left a number of CSOs

unable to push issues they consider urgent beyond their

current efforts. Although a time consuming process, some

the initiatives undertaken such as the Media and NGO bill and other engagements with the

political leadership are of strategic nature and therefore have long term implications that the

assessment was not able to measure.


“..LG board assigns a representative who participates in all the activities of CEPO from which insights are got to guide activities of Local Governments in the states..” KII – CEPO ED

“..the CSOs have been very vital in providing feedback from the communities on the current situation in the country regarding human rights and peace talks..” KII – MP, National Assembly

“..we are not fully involved in designing what is aired.. this limits our participation and yet we can make some of the content part and parcel of our programming...we can invite lawyers, police, social workers, women leaders, youth and other groups to come and participate in the panel discussions to enrich the Projects..” IDI – Projects Manager RGN

5.2.3 Willingness of government to engage with CSOs

One of the positive trends seen in the intervention areas is the will of the government to work with

and engage the CSOs. The signing of a Memorandum of Understanding between Community

Empowerment Project Organisation (CEPO) and the

Local Government Board was an expression of such will.

The MoU recognises the critical role that CSOs are

playing in supporting and strengthening the local

governments in order to address a number of critical

issues such as governance, accountability and citizen participation.

Through such partnerships and collaborations, CSOs are able to directly or indirectly influence

opinions and choices using foras such as the Commissioners Forum and capacity building

workshops. The exchanges and experience sharing during the forum has improved working

relationships and collaborations within the counties and the state government.

The nomination by government and participation of

two partners in the ongoing peace negotiations in

Addis Ababa is also viewed within the assessment as a

part of the existing government confidence and

willingness to engage CSOs in nation building.

Although these relationships developed circumstantially, they have been strengthened on a case

by case basis over time. What the assessment could not establish is if the relationships will be

sustained beyond the circumstances. However, what is evident in the current events is that

government needs the CSOs as much as the CSOs need government.

5.2.4 Media involvement as a key player

The involvement of Media as a platform for discussions, dissemination and seeking opinion of the

communities was critical considering the role of media in reaching out. There has been an

increased public interest to question leaders in issues affecting them as a result of the dialogue

spaces. Testimony from Radio Good News (RGN) indicates more call ins from communities

where dialogue is taking place and interest in MPs to use the radio more.

Although the Media faced challenges in the onset – with the closure of RGN, arresting journalists

and restraining media houses. There has been a general ease on media operations but with a kin

government eye on the content of what is aired. The security situation in the country has created

more apprehension on level of trust and confidence

from some media sections. Besides these

challenges however, media is still vital and

seemingly more critical a partner in achieving some

of the critical Project targets.

From the assessment, the engagement with media

especially in Lakes is unstructured based on

available air time. Participation of media in planning, follow up and monitoring trends is not

being fully utilised and besides, the Projects are not entrenched in the development focus of the

station. Engaging media and jointly designing these Project interventions will enable continuity of


“..one of the challenges of implementation is the structural challenge of depending on Juba office to clear activities.. this takes time and most times you cannot adequately mobilise until you are sure of the approved budget..” IDI – Partner focal person

such Projects in the long term. Proposals of holding panel discussions in local languages

involving people of different technical backgrounds, social groupings and political affiliations

will further add value to the discussions and issues aired.

Although seen as one most effective avenue to reach out to the communities and pass on vital

message in greater volume, the prevailing conditions indicate that a small fraction of households

own a radio and phones due to the extreme poverty conditions therefore may not effectively

participate in the Project activities.

5.2.5 Capacities of partners to deliver

The project carried out assessments4 of partner capacities giving consideration to different

parameters. The assessment informed the partners on the gaps within the partner implementation,

governance and management. Support given to partners in the form of training and logistical

facilitation greatly contributed to improved delivery in the short and medium term.

The Project anticipated placing experts to support capacity development of partners over time, the

assessment established that all partners received support in financial management and

organisational development during the implementation period. This support improved

organisational functioning and operations in the short and medium term. However with increased

activities and the processes of organisational transformation in the medium and long term, critical

areas such as Project design, implementation strategies, organisational development,

communication and reporting that needed continuous support were not addressed. Gaps in these

areas were noted in some of the partner structures and indeed affected their functions and

operations. For continuity, the evaluation considers these aspects as important as those addressed.

The records and evidence within partner organisations indicate capacity gaps in some of the issues

raised above that substantially influenced delivery of the Project.

There were efforts and plans to coordinate and

follow up through the steering committee meetings

and routine Project monitoring. In the short term,

this yielded positively, however in the medium and

long term, coordination and supervision gaps

emerged as partner activities were implemented

without clear follow up and supervision. It is evident that some activities carried out in Lakes by

Juba based partners were not well coordinated with the Oxfam field office. This was partly

attributed to the lack of presence in Rumbek and logistical challenges arising from distance to

Juba.

The assessment noted diversity in technical competencies and strengths among the partners in

Juba and Rumbek. This diversity was not fully utilised considering each partner towed their own

line and as such efforts were not concerted to achieve desired outcomes. Although nearly all the

partners were found to be carrying out activities in similar locations and targeting the same

structures, the inter agency synergies were not tapped and therefore operational resources were

not efficiently and effectively engaged.

4 Civil Society Organisation mapping and selection report – Juba/Rumbek. August 2013


“..I can now openly discuss and share roles with my husband without fearing like it was before. This has enabled me to join the rest of the women groups to learn from them different things like business and raising children..” FGD – beneficiary

“..it was through these discussions and engagements that recommendations to set up a women’s desk and one for PWDs was reached and implemented. Now women and PWDs address their issues through their respective desks at the MoSD..” IDI – Director MoSD

“..Because of what we are taught, I can now go and fetch water, bath the children and cook to support my wife while she is doing something else...I also discuss with her family issues freely unlike before..” FGD – Beneficiary, Cuiebet

In the view of the evaluation, the community and Project delivery structures that are required as a

foundation to sustain the approaches and activities have not been given adequate attention. The

current modalities can work in the short run but will not stand in the medium and long run as

interests may be diverted. The CSOs in Rumbek have formed an umbrella organisation – the

Civil Society Organisation Network (CISON) that is still at its infancy, this network will be a

critical platform for planning, follow up, advocacy and lobbying but needs more support to

overcome challenges.

5.2.6 Awareness on issues affecting communities

The evaluation established that communities that participated in the Project interventions adapted

a more open approach to raising and discussing issues affecting them. This was possible as a

result of the dialogue spaces and confidence built in communities on their rights and obligations.

From such discussions emerged knowledge on the

negative social/cultural practices affecting

communities, awareness on the immediate and long

term needs of the communities and improved

capacity to assess social and economic needs and

priorities.

Such awareness has drawn in different ministries at state level to in cooperate the identified needs

and priorities in plans and schedules of state

agenda. Through this, the rights of women,

children and Persons with Disabilities (PWDs) are

promoted and protected. It also emerged that MPs

are more in touch with their constituents and are

able to present factual arguments during debates in

the plenary.

5.2.7 Level of women’s engagement

More women according to the assessment have

participated in the community dialogues and

consultations. In the process, the social perception

of the role of men and women in decision making,

ownership of assets and community development

is gradually changing with more men appreciating the importance of engaging women in

development. There is evidence of women participating in family decisions and choices and

getting engaged in women’s groups and community activities.

As a result of this openness, more women’s groups have emerged within the communities with up

to 20 or 30 members in each payam. These women have come together to learn from each other

and share experiences on development concerns. One positive aspect of this gathering is the

number of women within these groupings that are enrolled in the Adult Learning Project (ALPs)

under a UNDP financed Project.


“..the discussions were very educative and we would like to have more debates and sessions on issues such as bride price, education of women, forced marriage and violence against women..” FGD – Beneficiary student

“..many of my colleagues are not bothered about public accountability because they consider this an opportunity to amass wealth.. however, government is also constrained by resources, that is why we have not received the Community Development Grants for a long time..” MP Lakes State

Participation of schools/institutions in structured discussions and dialogue on issues affecting

women was evident during the assessment.

Although only one girl’s school and a university

participated, the result of the discussions and

dialogue were evident. Many of the girls

expressed interest in having more such discussions

and diversifying the topics to include subjects and practices seen as prohibitive to women

participation and development. Because of the relevance of the discussion points for the girls, the

school is planning to introduce regular discussions and debates adapting the approach introduced

and the girls are considering starting a “girl’s empowerment clubs” that will focus on carrying the

message back home during holidays.

5.2.8 Mutual accountability and responsiveness to the plight of citizens.

To achieve mutual interests and focus would be a farfetched reality in the span of the project.

However, the commitment, interest, and engagement of CSOs in addressing social, economic and

governance issues particularly in Lakes state is a positive development. Within the time span,

evidence of efforts to build collaborations and partnerships with state and national governments is

visible, more especially in responding to the plight

of citizens. The project has provided government

and CSOs a platform to share knowledge,

challenges and experiences within the Local

Government settings.

However, the assessment has taken note that as

much as there is a will to support the citizenry, there are still gaps in capacity to effectively be

responsive on the side of the government and a section of CSOs. Commitment issues also come at

the fore front at different levels of governance in as far as the ability to commit resources to

continuously engage with both CSOs and communities for mutual development. This arises from

the fact that the target communities and a section of leaders are still held up and recovering from a

history of dependence and handouts and so tend to focus on addressing immediate needs rather

than long term perspectives such as accountability, transparency and good governance


6.0 Conclusions

The conclusions are drawn based on the 5 parameters that guided the evaluation exercise. They

are measured using a rating scale “to no extent” as the least achieved, “to some extent” as an

average achievement and “to a greater extent” as a high achievement.

Measure Conclusion Remarks

1. Extent to which project

objectives, design and

activities were consistent

with the project overall goal.

To a greater extent

As a pilot, the design was

appropriate in the context of the

implementation.

2. Extent to which the project

results were realised and or

achieved.

To some extent

Partners, institutions and

beneficiaries appreciate and have a

positive perception on the project.

Some activities scheduled late, not

implemented and or followed up.

Key aspects of the project such as

structures, synergies and

implementation capacities not

adequately addressed.

3. The extent to which the

project activities transformed

resources into results.

To some extent

Considering the time, some

partners achieved a lot within

resources provided.

Operational and structural

inefficiencies sighted in some

partner activities.

4. Extent to which the positive

effects of the project will

continue without external

assistance.

To some extent

Adaptation of some intervention

results within government and

educational institutions.

Evidence of dependence on project

for resources to implement.

5. Extent to which the program

has produced or influenced

the wholeness of positive or

negative primary and

secondary results directly or

indirectly.

To some extent

Participation of women in

domestic, community and state

development.

Participation of local leaders in

dialogue spaces.

Participation of CSOs in engaging

with government of legislations,

peace talks and other development

concerns.


7.0 Recommendations

Basing on the findings and analysis of the intervention areas, the evaluation has come up with the

following recommendations. These are addressed to both Oxfam and the Implementing partners

as a basis of improving future programming and interventions. The table below presents issues the

evaluation has identified and the corresponding recommendations. The recommendations are

described in greater detail in section 7.1.

Issue identified Recommendation

1. Mode of delivery of the

Project.

a) Engage media (radio) in a more structured way.

b) Introduce more innovations in the dialogue and

engagements.

c) Build model within focused intervention areas to be able

to replicate and scale up.

d) Utilize and develop synergies among partners to be more

effective.

2. Capacity concerns within

implementing partners

a) Invest more in concrete governance structures and

institutional development for partners.

b) Deliberate follow up on issues raised and discussed

during partner meetings/trainings etc.

c) Strengthen the community structures to reach out the

entire community.

3. Project support, follow up

and monitoring of activities

a) Define specific targets for partners to achieve. This will

be a basis of follow up and monitoring.

b) Adhere to agreed Project work plan, implementation and

reporting schedule.

4. Sustainability of approaches

and activities of

partners/stakeholders.

a) Develop implementation strategies for eventual ownership

and sustainability.

b) Engage all key stakeholders along the implementation path.

c) Ensure that goal of the project is understood by all involved

to avoid diversion.

5. Emphasis on women

involvement

a) Address the gender power gaps that exist in the

organizations and governance structures as a matter of

policy.

b) Engage women in advocacy and mobilization drives to

reach out to more.

7.1 Detailed description of recommendations

The narrations below describe how the recommendations captured above can be achieved within

the framework of the project. These are based on findings of data collected from the field that

informed the opinion and recommendation of the evaluation.

R1. a) Engage media in a more structured way

Radio Good News, a member of Catholic Radio Network (CRN) has aired a number of

Programmes related to the project interventions areas and has been engaged in a number of the

activities including reporting on project activities. As a key player in Civil Society initiatives such


as advocacy, mobilisation and information dissemination, it would be prudent to involve such a

partner in planning, implementation monitoring the activities of the Project.

From the views of radio management and review of the vision and focus of the station, engaging

them as part of the delivery process will enhance to programme implementation and

sustainability. This can be through joint planning, information gathering, monitoring and

dissemination. In the process, some of the project approaches and best practices can be adapted as

part and parcel of radio programming and they can become a vital source of feedback, assessment

and monitoring results.

R1. b) Introduce more innovations in dialogue and engagements

The dialogue, peace initiatives and engagements have been structured around meetings and formal

discussions with key players and stakeholders. Over time, the communities will develop fatigue

and disinterest especially when the issues become repetitive and progress not forthcoming.

Introducing more innovative approaches such as cultural dances, poetry, games and sports, posters

and messaging charts to address some of the thematic areas of the Project will attract more players

and keep youth and women more engaged in passing vital messages across.

R1. c) Build a model within a focused intervention for replication

The Project has very practical approaches that can be replicated and adapted across all local

governments when deliberately addressed. Rather than stretch out implementation capacity to

reach a farfetched targets, a model of approach and delivery should be build within selected states

and local government structures to study scenarios and deviations.

The successes of these models can then be used to mould other local governments to function in a

desired pattern. Key in achieving this is to ensure that all stakeholders are involved from the

onset.

R1. d) Utilise and develop synergies among partners to be more effective.

The partners engaged by the project have a diversity of strengths and competencies that

collectively can support achievement of the results of the project. Rather than converge their

efforts to carry out bits and pieces within particular locations and targeting isolated communities,

a deliberate approach to reach the same locations and communities with diversity of information

should be devised.

This will not only minimise strain in operational costs but also ease implementation through

mobilisation, monitoring and tracing desired results. Besides, the strength of CISON is derived

from the ability of partners to develop and adapt synergies.

R2. a) Invest more in concrete governance structures and institutional development for

partners.

Until partners understand some of the embedded values in the project such as accountability,

transparency, good governance and gender equity, it will be painstakingly difficult to pursue these

values within their areas of operation. The assessment noted some of the partners clearly have

challenges with governance and structural set ups that need to be improved.

Oxfam should prevail more concretely in proper governance and management structures and

capacities of partners in order to build a long lasting delivery vehicle for the project interventions.


Such an investment through technical support, oversight and logistical support should be

conditioned as part of the partnership development process and milestones to adhere to over time.

R2. b) Deliberate follow up on issues raised and discussed during partner meetings/trainings

Trainings follow up meetings and capacity building sessions have yielded very impressive

recommendations and action points for the partners. Some of this directly addressing project

delivery. The challenge that may be attributed to workload and conflicting schedules is that little

or no follow up is done.

Project management should define a clear schedule and checklist of critical areas of

recommendations, follow up and improvements that partners are advised on. Such a schedule

should provide for feedback sessions and challenges to be addressed in order to ensure

consistency and continuous improvement.

R2. c) Strengthen the community structures to reach out the entire community.

The effectiveness of implementation is dependent on the strength and capacity of the

implementation structure from bottom to top. The structures in use such as County Working

Groups, Community Volunteers and project focal point persons should be strengthened to

penetrate the target communities.

This may involve identifying committed players, clearly defining their roles and responsibilities,

following up their activities and facilitating them to be more effective and efficient in the

discharge of their assignments.

R3. a) Define specific targets for partners to achieve. This will be a basis of follow up and

monitoring.

For the project partners to deliver effectively, defined and specific targets should be set clearly

indicating milestones and timelines in which to achieve such targets. This will help both the

implementing agency and the project management in scheduling activities, monitoring process

and reporting on activities implemented.

R3. b) Adhere to agreed Project work plan, implementation and reporting schedule.

Project management should ensure that implementing partners adhere to and follow project work

plan and other schedules that relate to the project cycle. This will ensure that management and

implementation partners are operating on the same schedule and plan.

R4. a) Develop implementation strategies for eventual ownership and sustainability.

For as long as the dialogues, community groups and activities will be attended as a result of

external motivation and enticements, there will be very little effort in ownership and sustainability

of the very good initiatives the project is proposing.

The project should adapt deliberate strategies of phasing out any direct or indirect support that

may be viewed by partners and stakeholders as a motivation. Some of the activities of the project

should be undertaken as initiatives either by government or the community in order to own them

with time.

R4. b) Engage all key stakeholders along the implementation path.


All important stakeholders should be engaged in the implementation right from planning, carrying

out activities and monitoring and supervising. This helps the stakeholders understand what

information is related to the intervention and what is not.

R4. c) Ensure that goal of the project is understood by all involved to avoid diversion.

The project goal and objectives should be clearly explained to and understood by all parties

involved to ensure non diversion from focus. Such clarification will avoid duplication of

information, distortion of facts and wrong referencing.

Enough time should be allocated for this purpose and engaging key partners and stakeholders and

allow them explore and interpret the goal and objectives of the project within their context. This

will enable them share the same knowledge and information down the path.


7.1 Annexes

Annex I Terms of Reference

Evaluation of Within and Without the State: South Sudan Civil Society Strengthening Project

Oxfam GB seeks qualified individual(s), group(s) or company to carry out an evaluation of the

above stated projects in its areas of operation in central equatorial and lakes states of South Sudan.

Duration. 21 days

Reporting to: Project Manager:

Background:

Oxfam GB in South Sudan:

Oxfam GB has been working in South Sudan since 1983, with initial activities in emergency

water supply and health services for refugees. It has continued to lead on humanitarian

interventions in South Sudan since then and currently has operational programming in two states

of Lakes and Upper Nile with a country office in Juba. Oxfam GB concentrates on four primary

sectors: Public Health (water, sanitation & hygiene promotion), Livelihoods and Emergency

Preparedness & Response (EP&R).

The Projects/responses are carried out by three teams: EP&R, Upper Nile and Lakes. Within these

Projects, issues such as gender, conflict sensitive programming, peace building, policy and

advocacy are furthered with country Project leadership in Juba and field offices in Rumbek,

Malakal and Maban.

Recently, Oxfam GB entered into a Conflict, Security and Justice, and Humanitarian Project

Partnership Arrangement (PPA) with DFID for 2011-14. The Humanitarian PPA includes one

objective as a global initiative aimed at improving the quality and effectiveness of programming

in fragile and conflict-affected settings5 with attention initially on three pilot countries: South

Sudan, Occupied Palestinian Territories/Israel (OPT/I) and Afghanistan - and then with the

intention to roll out lessons from these countries to a wider group. In South Sudan the initiative

expects to model a different approach to working with civil society in the interests of building

greater trust between stakeholders, promoting deeper political economy analysis, and more

relevant programming as a result.

With several decades of war and recent independence on the 9th July 2011, civil society

organisations in South Sudan are still evolving and will require meaningful support and

facilitation for them to be empowered and strengthened in order to play a proactive role to

catalyse accountability and development. The independence of South Sudan brought to the new

nation so many opportunities and challenges that the South Sudan government needs a strong

Civil Society to support in putting in place legitimate institutions and systems that can facilitate

equitable service delivery and relevant programming. As the work to define a permanent

constitution continues up to 20156, building on the interim constitution approved in July 2011,


many instruments of state are being gradually developed with complications related to the

separation from the former instruments of Sudan and to appropriate definition and development of

the diverse South Sudan context.

In the context of South Sudan the meaning of Civil Society varies from national to state and

county levels. While at National level there is some civil society in the real sense of the term, at

State and county level the term civil society is used to mainly mean associations or groups and/or

local organisations without or with very limited organisational and governance structures. The

nature of civil society is very diverse, including religious and traditional leadership which

continues to have a big role and influence in changing the lives of people but also in shaping the

politics of South Sudan. Meanwhile the growth of civil society is notable but with concerns that

the invited space that CSOs occupy today may reduce over time if RoSS feels threatened by

challenges being raised by CSOs. The critical question to CSOs is about how to effectively use

that space, occupy other spaces and even initiate more spaces to work and walk with the State to

respond to the most urgent needs and priorities of citizens through deliberate accompaniment of

RoSS to become more legitimate, responsive and accountable. This project aimed at strengthening

the civil society in South Sudan details actions and approaches, over a period of three years (

2011-2013), using the secured £ 275,000 funds from DFID.

1. Overview of the project to be evaluated:

The Overall Aim

The overall goal of the project is to contribute to the building of strong foundations for both

women and men in South Sudan to equally contribute to the development of their nation.

The project specific Objective

The specific project objective is increased accountability and responsiveness of RoSS, civil

society and donors to the citizens of South Sudan

Project Results (Short-to medium-term outcomes)

a) Levels of trust and working relationships between CSOs & Oxfam and between CSOs and

State institutions are improved.

b) The capacity of selected CSOs and CBOs to be more accountable, transparent and responsive

is increased

c) CSO and State Projects and policies are more aligned to the needs and priorities of poor

communities

d) Poor women’s leadership of CSOs and State institutions and participation in economic

decision making is increased.

Purpose of evaluation and intended use

The main purpose of the final evaluation is to providing independent assessments of the results of

our work, and building a body of evidence that we can use to improve the quality of our Projects.

The evaluation is also expected to assess key pillars of Oxfam (accountability) of the project so as

to inform future design and implementation strategies.

The evaluation results will be shared with relevant donors, Oxfam GB Head Office (HO) in

Oxford, the South Sudan Country Team, and partners.


Scope of the work:

With regard to South Sudan Civil Society project the evaluation will mainly focus on the Project

interventions implemented from 2011-2014. Furthermore, the evaluation will also examine how

the findings and recommendations emanating from the review meetings and workshops project

undertaken since 2011 informed the implementation of the project.

Relevance and appropriateness: Oxfam GB interest is to understand whether the objectives,

project design and activities were consistent with the building of strong foundation for men and

women by equitably contributing to the building of their nation. Focus should also be on

complementarily and coherence with related activities undertaken by other actors: Key issues of

focus for this evaluation will be as follows:

Extent to which the projects addressed the felt needs of the beneficiaries

How appropriate was the intervention logic in light of the prevailing context, beneficiary

needs and local capacities / Appropriateness of the implementation strategies and approaches

adopted

Extent to which targeted beneficiaries feel that they own the project

Examine lessons learnt with regard to relevance and appropriateness.

Effectiveness: Oxfam defines effectiveness as the extent to which the project results were

realised. Key focus is on whether the project purpose and results were achieved and to what

extent. Key Issues of focus for this evaluation will be as follows:

Extent to which project objectives/Outcomes have been achieved based on the outlined

indicators.

Beneficiary perceptions of the value, quality and quantity of the project in relation to their felt

needs

Examine extent to which project intervention logic was implemented and what factors if any

may have hindered implementation

Examine effectiveness of the linkages and synergies established with related projects.

Examine Lessons Learnt with regard to overall effectiveness of the project.

Extent to which protection issues were integrated in the program in terms of targeting,

implementation and advocacy.

Efficiency: Oxfam understands efficiency to mean how well the various project activities have

transformed the available resources (inputs) into the intended results (outputs, outcome) taking

into consideration value for Money. Key focus interest for the evaluation will be as follows:

Examination of the extent to which the project planning and implementation processes embraced

contributed towards greater efficiency in terms of time and resources inputs (both human and

material)

Assessing strategies that could have been adopted to increase the efficiency of project

implementation

Examine, document and share Lessons Learnt with regard to Efficiency.

Sustainability: At Oxfam point of view sustainability means the extent to which the positive

effects of the project/program will still continue after external assistance has been ended. The

evaluation will thus dwell on the questions below:

What are the prospects for the continuation of the changes brought by the project? Are these

likely to last or are they not likely to last beyond the project /Are the changes that have been

achieved likely to be sustained.


What are the prospects for continuation of the national, state and local community capacities

developed and structures established for sustainability of the project. outcomes

Examine, document and share Lessons with regard to sustainability

Impact: Oxfam definition of impact is the wholeness of positive and negative, primary and

secondary results as produced by a development intervention, directly or indirectly, intended or

unintended. Oxfam’s main focus is to measure the impact of the program on the target group. This

should include the assessment of the intended or unintended effects on the national, state and local

population, the environment, the conflict or other factors. Key areas of focus for this evaluation

will be as follows:

Examine and document evidence/s of changes both positive and negative as a result of the

interventions

Measure of the needs of beneficiaries were specifically addressed and what has been the

outcome

Measure scenarios in which the project contributed towards the durable solutions of the civil

society in south Sudan.

Examine, document and share lessons learnt with regard to Impact

Principles of the evaluation:

The evaluation will be guided by the following ethical considerations:

Openness of information given, to the highest possible degree to all parties.

Wider and exclusive participation of interested parties

Reliability and independence.

Methodology

The evaluation will embrace a wide range of methodologies which will include though not

restricted to any or a combination of the following:

Desk study and document review: The evaluation team shall review proposals, reports, theory

of change, logocal frameworks, workplans and other documents associated with the Project.

Sample Survey: A sample survey will be undertaken mainly to establish achievements in

relation to the outcome level and some of the result level indicators. The evaluation team will

be expected to provide a detailed description of the methodology for the sample survey as well

as an outline of tools to be used.

Key stakeholder interviews: The evaluation team will conduct interviews with Oxfam staff in

Juba, Rumbek, national NGOs in Juba and in Lakes state, community based organizations,

local authorities and inter-agency networks where applicable.

Beneficiary focus groups: The evaluation team will meet with beneficiaries and community

representatives of the target population in each region. This can include focus groups, and

interviews.

Deliverables and reporting deadlines

The evaluation team will submit two reports(two printed and two electronic copies in English)

that include but not limited to Executive summary, Back ground, purpose/objectives,

introduction, methodology, findings, conclusion , recommendation with lessons for the Project


going forward and appendix including evaluation terms of reference, maps, sample

framework, and bibliography.

Offer presentation to Oxfam project management Staff in Lakes state. Team members will

also provide a short presentation summarizing key findings for senior management in the Juba

office at the end of the field visit.

Inception report: Following the desk review and prior to beginning field work, the evaluation

team will produce an inception report subject to approval by the Oxfam Project manager.

This report will detail a draft work plan with a summary of the primary information needs, the

methodology to be used, and a work plan/schedule for field visits and major deadlines. With

respect to methodology, the evaluation team will provide a description of how data will be

collected, collated, analysed and a sampling framework, data sources, and drafts of suggested

data collection tools such as questionnaires and interview guides.

Once the report is finalized and accepted, the evaluation team must submit a request for any

change in strategy or approach to the Program Development Unit.

Draft report: A draft evaluation report will be submitted to the Development Project Manager,

who will review the draft and provide feedback within two weeks of receipt of the draft report.

All material collected in the undertaking of the evaluation process should be lodged with the

Program Development Unit prior to the termination of the contract.

Presentation of findings:

At the end of the field research, the evaluation team will present key findings to management

in the field for this case is lakes state (Rumbek field office).

Before the Final Evaluation Report is submitted, the evaluation team will present their

findings at Oxfam Country Office (Juba, Hai cinema, Opposite Mobil roundabout).

Time-frame and budget considerations

Proposals should present a budget in the number of expected working days over the entire

period (e.g. 21 full time days over 1 month).

Evaluation activities will be conducted during the months of May 2014, with a final

presentation of findings expected in the last week of May 2014.

A final decision on the evaluation team to be awarded the consultancy will be taken by the

Program Development Manager in consultation with the MEAL coordinator and key Project

staff. Desk review and initial conversations with the Program Development Manager and

MEAL coordinator will commence immediately the successful consultancy

firm/group/individual has been selected and approved accordingly.

The inception report will be submitted one week after the award of the consultancy.

In the event of serious problems or delays, the evaluation team leader should immediately

inform the Project Development Manager or the MEAL coordinator. Any significant changes

to review timetable shall be approved by the Development Manager in writing in advance.

Evaluation consultant team

(Required Expertise within the Team)


A team consisting of individuals with multi-disciplinary skills is required for this exercise. The

team should possess demonstrable expertise/experience/skills in all of the following areas:

Democracy/ Governance Evaluations/Assessments in both development and humanitarian

settings

At least 5 years’ experience in Conducting Evaluations/Surveys/Assessments in development

and humanitarian settings particularly in the fragile states

Progressive good track record in conducting evaluations for interventions funded by key

donors such as DFIT, EU.

Fluency in written and spoken English is required of all team members.

Experience with the logical framework approach.

Prior experience in South Sudan is helpful.

Excellent skills and knowledge in training, supervision in data collection and monitoring

Excellent reporting skills.

Excellent skills in data collection, collation, analysis in qualitative and quantitative terms.

Masters degree in social sciences preferably in statistics for individual applications

Application process:

Interested and experienced consultant firms/individual can drop their applications (proposals)

clearly indicating: Consultancy Application: within and without the State: South Sudan Civil

Society Strengthening Project to [email protected] cc. [email protected]. March

28th 2014. Late submissions will not be considered.

mailto:[email protected]


Annex II List of respondents

Agency/Stakeholder/Category No of Respondents

1. National Legislative Assembly 03Committee on Human Rights and Humanitarian

Affairs

02 Committee on defence and national security

2. Ministry of Local Government –

Lakes State

01 Director for Local Government

3. Ministry of Social Development –

Lakes State

01 Director for Social Development

01 Senior inspector – vulnerable groups

4. Media houses within Lakes State 01 Director - Radio Good News

01 Programmes Manager – Radio Good News

5. Implementing partners

(Rumbek/Juba)

5 Non Governmental Organisations.

6. Community structures 5 Community volunteers

3 County Working Groups

7. Direct beneficiaries 4 Women groups/Clubs

10 girls from a girls school

5 girls from a higher institution

3 Youth groups

5 Rural women

8. Project staff 3 staff of Oxfam


Annex III Evaluation plan and Framework

EVALUATION PLAN AND FRAMEWORK

“Within and without the state; South Sudan civil society strengthening project”. Rumbek –

Wednesday 10th

April 2014 to Saturday 12th

April 2014.

Agency/

Partner

Date/Time of

meeting

Target

group/person for

data collection

Method of data

collection Evaluation focus areas

1) APARD

Date:

Wednesday 9th

April 2014

Time:

Community

mobilisers.

FGDs

Observations

a) How have you participated

in the implementation,

development and

monitoring the project?

b) Important lessons from the

project that women can

carry on without the project.

c) How differently would women address the issues

raised in the project?

d) What kind of support

reached women from the

agency/partner?

Date:

Wednesday 9th

April 2014

Time:

Director

General –

Ministry of

Social

Development

Key informant

Interview

a) Participation of MoSD in

implementation,

development and

monitoring the project.

b) Important aspects from the

project (women’s rights)

that govt would want to

carry on with.

c) How differently would govt address the issues (esp

women’s rights) raised in

the project.

d) What kind of government

support would be

anticipated and why?

Date:

Wednesday 9th

April 2014

Time:

Vice chancellor

Rumbek

University

HM Loreto

Girls S.S

Key informant

Interview

a) Participation of academic

institutions in

implementation,

development and



project that institutions

would want to carry on

with. c) How differently would

institutions address the

issues raised in the project?

d) What kind of institutional

support would be


Date:

Thursday 10th

April 2014

Time:

Radio Good

News (RGN)

Key informant

Interview

a) Participation of media in

implementation,

development and



project that the media would

want to carry on with.

c) How differently would media address the issues

raised in the project?


d) What kind of media support

would be anticipated and

why?

2) SDRDA

Date:

Thursday 10th

April 2014

Time:

Chairpersons

CWGs

Community

Mobilisers

FGDs

Observations

a) Participation of community

in implementation,

development and



project that the community

would want to carry on with.

c) How differently would

communities address the

issues raised in the project.

d) What kind of community

support would be


Date:

Thursday 10th

April 2014

Time:

Speaker –

LSLA

Targeted

members of

parliament.

Key informant

Interview

FGDs

a) Participation of LSLA in

implementation,

development and



project that the LSLA would

want to carry on with. c) How differently would the

LSLA address policy and

legislative dialogue issues

raised in the project.

d) What kind of policy and

legislative support would be


Date:

Friday 11th April

2014

Time:

General

Secretary

CISON

Key informant

Interview

a) Participation of CSOs in

implementation,

development and



project that the CSO

network would want to

carry on with. c) How differently would the

CSO network address the

critical issues raised in the

project.

d) What kind of support would

the CBOs and partner

members receive and why?

3) CEPO

Date:

Friday 11th April

2014

Time:

Director

General,

Ministry of

Local

Government

Key informant

Interview

a) Participation of MoLG in

implementation,

development and


b) Important aspects of the

project (good governance)

that govt would want to carry on with.

c) How differently would govt

address the issues raised in

the project?

d) What kind of support from

government to CBOs/CSOs

would be anticipated and

why?

Date: Chairpersons, Focus Group a) Participation of youth and


Friday 11th April

2014

Time:

youth and

women

associations

Discussions

Observations

women assns in

implementation,

development and



project (good

governance/accountability) that youth and women

associations would want to

carry on with.

c) How differently would these

associations address the

issues raised in the project?


the associations anticipate

and why?

Date:

Friday 11th April

2014

Time:

Director, Anti

corruption

commission

Key informant

Interview

a) Participation of ACC in

implementation,

development and


b) Important aspects from the project (good

governance/accountability)

that ACC would want to

carry on with.

c) How differently would ACC


the project?


ACC anticipate to

offer/receive and why?

Date:

Saturday 12th

April 2014

Time:

Community

members

Focus Group

Discussions

Observations

a) Participation of community

in implementation,

development and


b) Important aspects from the project that the community


with.


communities address the

issues raised in the project.

d) What kind of community

support would be


Date:

Saturday 12th

April 2014

Time:

Coordinator,

Peace

Commission

Key informant

Interview

a) Participation of ACC in

implementation,

development and


b) Important aspects from the project (good


that ACC would want to

carry on with.

c) How differently would ACC


the project?


ACC anticipate to

offer/receive and why?

4) Oxfam staff Saturday 12th

April 2014 Project Officer Key informant

Interview

a) Project design and

compliance


5) APARD Focal Person Key informant

Interview

b) Project structure and

implementation schedule

c) Reporting schedule and

requirements (activity and

financial reporting)

d) Project modifications and

adjustments. e) Sustainability of the Project

f) Project challenges and

limitations

6) CEPO

7) SDRDA

1) CEPO

Tuesday 15th

April 2014

Agency staff

Government

departments

and

institutions.

Community

beneficiaries

Focus Group

Discussions

Observations

a) Participation of in

implementation,

development and



project (good


that agency and partners


with.

c) How differently would partner and agencies address

the issues raised in the

project?


partners and agencies

anticipate to offer/receive

and why?

2) AIM

Wednesday

16th April 2014

Agency staff

Government

departments

and

institutions.

Community

beneficiaries

Focus Group

Discussions

Observations


implementation,

development and



project (good


that agency and partners would want to carry on

with.


partner and agencies address


project?




and why?

3) SSUDEMOP

Thursday 17th

April 2014

Agency staff

Government

departments

and institutions.

Community

beneficiaries.

Focus Group

Discussions

Observations


implementation,

development and

monitoring the project. b) Important aspects from the

project (good


that agency and partners


with.


partner and agencies address


project?





and why?

4) Presentation of findings and de

brief to the project management

at country office

Thursday 17th

April 2014

9:30pm to 10:30

am

MEL

Manager

Project

Manager

Country

Director.


8.0 References

a) Civil Society Organisation mapping and selection report – Juba/Rumbek, August 2013

b) Civil Society Organisation MEAL training report – Destelia Nwenya, Dec 2012

c) Global Indicator assessment report – WWS, January 2013

d) Media relations and advocacy training report – MSO, July 2013

e) Minutes of Steering Committee Meetings – WWS, 2012, 2013

f) Partner capacity and assessment report – Oxfam GB, August 2013

g) PPA Annual report – WWS, December 2013

h) Situation analysis – Lakes State (Oxfam), 2012

i) South Sudan Civil Society project proposal.

Report on the external evaluation for "within or without the state" south Sudan project; civil...

Documents

Transcript of Report on the external evaluation for "within or without the state" south Sudan project; civil...