1

Save the Children Alliance, Tsunami Response Programme in Indonesia, India, Sri Lanka, Thailand and Somalia Mid-Term Evaluation Report June 2008 Team Members: Chris Dammers (Team Leader) John Beauclerk Rod MacLeod Peter Wiles

2

Table of Contents

List of Abbreviations/Acronyms 3

Introduction 6

Summary 7

Country Chapters

Indonesia 28 India 47 Somalia 68 Sri Lanka 81 Thailand 104

Annexes (Separate document)

3

List of Abbreviations/Acronyms AERA Aceh Earthquake Relief Assistance Program ALNAP Active Learning Network for Accountability and

Performance in Humanitarian Action ANI Andaman and Nicobar Islands ANIIDCO Andaman and Nicobar Islands Integrated Development

Corporation AP Andhra Pradesh ARM Association for Rural Mass India ASSEFA Association of Sarva Seva Farms BDS Seuke Aceh Coop BPL Below the Poverty Line BRB/SCBR Bal Raksha Bharat BRR Badan Rehabilitasi dan Rekonstruksi (Aceh and Nias

Rehabilitation and Reconstruction Agency) CBO Community Based Organisation CFS Child Friendly School CFW Cash for Work CLDRR Child Led Disaster Risk Reduction CLCs Community Learning Centres CPC Child Protection Committee CRPO Child Rights Protection Committee DEC Disaster Emergency Committee DRR Disaster Risk Reduction DPC Department of Probation and Childcare ECCD Early Childhood Care and Development ECDC Early Childhood Development Centres ECE Early Childhood Education ECPAT End Child Prostitution, Child Pornography and Trafficking

of Children for Sexual Purposes ERA Economic Recovery Assistance ENABLE Enabling Aceh to Combat Exploitation through Education EPI Economic Policy Institute FCRA Foreign Currency Regulation Act GA Government Agent GBP Pound Sterling GHRE Grassroots Human Rights Education Development

Committee GoI Government of India GoSL Government of Sri Lanka HADMA Humanitarian Affairs and Disaster Management Agency ICDO International Civil Defence Organization ICDS Integrated Child Development Scheme Centres IDPs Internally Displaced Persons INGO International Non-Government Organisation INTRAC International NGO Training and Research Centre IOM International Organisation for Migration LEAD League for Education and Development LTTE Liberation Tigers of Tamil Eelam

4

LQAS Lot Quality Assurance Sampling M&E Monitoring and Evaluation MDMHR Ministry of Disaster Management and Human Rights MOS Minimum Operating Standards/Maximising Operational

Success NCA Norwegian Church Aid NCPA National Child Protection Agency NGO Non-Governmental Organisation OCHA Office for the Coordination of Humanitarian Affairs ODI Overseas Development Institute OECD-DAC Organisation of Economic Cooperation and Development –

Development Assistance Committee PD Pure Data PRI Panchayat Raj Institution (local self governance institution) PSSA Particularly Sensitive Sea Areas PTA Parent Teachers Associations ReDR Registered Engineers for Disaster Relief SACB Somalia Aid Coordination Body SAFE The Special Non-Invasive Advances in Foetal and Neo-

natal Evaluation Network SASY Social Awareness Society for Youths SC Scheduled Caste SC Save the Children SCA Save the Children Alliance SCBR Save the Children Bal Raksha (the recently established

Indian SC entity) SCUK Save the Children (UK) SCUS Save the Children (US) SCiSL Save the Children (Sri Lanka) SCS Save the Children (Sweden) SDS School Development Society SEVAI Society for Education and Village Action and Improvement SHG Self-Help Group SL Sri Lanka ST Scheduled Tribe TFG Transitional Federal Government TMSSS Thanjavore Multipurpose Social Service Society TN Tamil Nadu TNI Tentara Nasional Indonesia ( Armed Forces of Indonesia) TNTRC Tamil Nadu Tsunami Resource Centre TRO Tamil Relief Organisation TRP Tsunami Response Programme SCiSL Save the Children Sri Lanka SASY Social Awareness Society for Youth SEVAI Society for Education and Village Action and

Improvement UNDP UN Development Programme UNESCO UN Educational Scientific and Cultural Organization UNHCR UN High Commissioner for Refugees UNOCHA UN Office for the Coordination of Humanitarian Affairs

5

UNOPS UN Office for Project Services UNICEF UN Children’s Fund UNU-EHS United Nations University – Institute for Environment and Human Security USDA US Department of Agriculture VEC Village Education Committee WBVHA West Bengal Voluntary Health Association

6

Introduction The Save the Children Alliance’s response to the December 2004 tsunami concentrated in the three hardest hit countries of Indonesia, Sri Lanka and India and provided assistance to Thailand and Somalia. After the first phase of relief assistance, Save the Children Alliance created a comprehensive five-year recovery and development programme in the areas of health, protection, education, shelter/construction and livelihoods to help children and their families affected by the tsunami. Strategic programmes plans were designed and implemented at the individual country level. The Tsunami Recovery Programme (TRP) is, in essence, an umbrella programme derived from the strategic plans that guide the Alliance’s tsunami recovery programmes in each country. The mid-term evaluation of the TRP assesses the progress made in accomplishing the objectives of TRP in Aceh (Indonesia), Sri Lanka, India, Thailand and Somalia. Each of the five country programmes has specific objectives that it seeks to accomplish by 2009; each country programme has been evaluated on progress made towards the achievement of these objectives. Specifically the evaluation aimed to:

Assess whether the programme is on track to achieve its defined objectives/results, including a summary catalogue of costs and outputs to date.

Produce recommendations for improving performance for any objectives/results that are at risk of non-achievement by 2009.

Provide overall recommendations to the programme on improving its performances against the OECD-DAC criteria. As this is a mid-term and not a final programme evaluation, particular criteria for emphasis are Effectiveness, Efficiency, Sustainability, and where possible, Impact. (Appropriateness/ relevance, coherence and coverage were also addressed).

Prior to the commencement of the Evaluation process, the Steering Committee met with the Evaluation Team members to clarify the Terms of Reference, to discuss methodology and to plan the procedures to be followed. The Evaluation Team comprised five international and 3 national consultants; with the Evaluation Team Leader and Quality Assurance Adviser based in the UK. The OECD-DAC evaluation criteria were given priority to ensure that the questions of efficiency, effectiveness, impact relevance/ appropriateness, sustainability, coherence and coverage were systematically addressed. . Evaluation methodology included: qualitative and quantitative data collection and analysis using desk reviews and literature searches; meetings; semi-structured interviews and child-based focus group discussions with SC-TRP staff, key informants and primary and secondary stakeholders. Children were important participants in the evaluation process. Two-week visits were made by evaluation team members to India, Indonesia and Sri Lanka and a 5-day visit to Thailand. In Somalia, owing to limited time and resources and the difficulties faced in setting up the evaluation process, the evaluation was based on a desk exercise which comprised: documentation study; email questionnaire to key present and former field staff and a small range of external stakeholders in Puntland and interviews with two key former staff members. Each country programme received a copy

7

of the draft report which provided the opportunity to provide comments and to correct factual errors. Where circumstances necessitated modification to the standard methodology, eg: in Somalia, this was noted in the country chapters. The evaluation had to deal with the following constraints:

Limited time to cover such a large programme with so many components. Difficulties in always identifying and attributing outcomes and impact in a context

where there have been so many other actors. Feedback was given the first draft in 10 different documents, creating difficulties

in addressing all comments in the time initially available. Evaluation Team Leader was unwell for a good part of the write-up period of the

evaluation. Hard to incorporate all the findings within the page limits stipulated. Difficulty in presenting financial reports in an identical format as the information is

provided differently by the different SC lead agencies, and even within countries from one year to the next.

The draft Report was structured in line with the template proposed in the Terms of Reference and outlined in the Inception Report. The final Report has been re-structured in response to subsequent requests from the Steering Committee, notably that the Executive Summary should be combined with the Synthesis section of the draft. This combined section comprises the Summary in this final version. Each Country Chapter provides the evaluation findings against specific country objectives. Appendices are attached as a separate document. Summary Context of Tsunami Response Programme The UK Disasters Emergency Committee (DEC) Independent Evaluation (2005) estimated that it raised $586.46 for each person affected by the tsunami of 26 December, 2004, as compared to $3.37 for the Gujarat Earthquake and $1.98 for the Orissa Super Cyclone. This was from just the UK, but illustrates how the unprecedented response created opportunities and problems unique in the history of humanitarian work. The countries hit by the tsunami differed greatly, both in the scale of destruction and the circumstances of the societies affected. A common thread across all the countries was the inclusion of some of the poorest and most isolated communities amongst those overcome by the catastrophe. The different chapters of this evaluation illustrate diversity. It is nevertheless impossible to summarise our findings. According to the most recent figures available, the Save the Children Alliance has spent US$213 million in its Tsunami Response Programme (TRP. Overall, 77% of the total funding has been spent, ranging from 73% to 99% for individual countries. This mid-term evaluation of the TRP assesses the progress made in accomplishing the objectives of TRP in Indonesia (Aceh), Sri Lanka, India, Thailand and Somalia.

8

USD $ (millions)

Indonesia Sri Lanka

India Somalia Thailand Overall

Total funding 171 64 33 8 2 278Total spent 126 53 24 8 2 213Balance 45 11 9 0 0 65% spent 74% 83% 73% 99% 98% 77% Main Findings The evaluation found that the TRP is on target to meet its objectives. In other words the programme is effective. The overall efficiency of the programme is generally reasonable, but has sometimes been negatively affected by high overheads1, and by the specific case of the housing programme in Aceh, Indonesia. Impact generally appears good though is usually difficult to assess. Sustainability is variable. Overall, appropriateness, coherence and coverage are good. International standards have been adhered to.2 1. Effectiveness In overall terms, the implementation of TRP is effective in terms of meeting stated objectives, although in some cases the stated objectives were re-defined (for example in India) and programmes sometimes ran behind schedule (for example in Aceh). Most country programmes had overlapping objectives thus providing a challenge for the evaluators. Nevertheless, given the unprecedented scale of the programme, it has been a major achievement on the part of Save the Children to have achieved so much work to date. The varying levels of effectiveness across countries are examined below on a sector by sector basis taking examples from each country. The TRP in Sri Lanka effectively scaled up a wide range of protection activities that pre-dated the tsunami, for example their work with child soldiers; protection work in and around schools (including conflict resolution and peer mediation); and children’s clubs and child protection committees at community level. The decrease in the numbers of children who became institutionalised in SCiSL locations is an indicator of its effectiveness. In India, while many Child Protection Groups have been established, the assessors found that significantly more work is required if interventions, although found to be effective, are to be sustainable. In Thailand the evaluation found that the ‘Child Safe Organisations’ project was effective in its response to the direct training needs of Thai NGO staff in child protection and rights issues. The most effective DRR initiative in Sri Lanka was found to be the ‘hazard mapping’ and ‘mitigation’ with 92 children’s clubs in six tsunami-affected districts where SCiSL

1 Overhead and support costs refer to costs related to Save the Children’s own staff, management, administration and associated costs. As per the suggestion of the Steering Committee, a Recommendation on this issue has been included. 2 Specifically, the Sphere Standards and the Red Cross Code of Conduct. Responsiveness to a less rigid international yardstick, the recommendations of the Clinton Report, were found to vary between countries.

9

works. In 2006 and 2007, SCiSL helped schools in these areas to set up DRR school committees; altogether SCiSL estimates that it reaches 17,600 children of the 110 child-friendly schools it works with. Building on this success, it is recommended that SCiSL take steps to ensure that the programme maintains the momentum achieved in empowering children through DRR. In India, by the end of 2007, 126 contingency plans had been developed at community level; 56 schools were covered and three strategic partners identified and trained to increase the capacity of others; the component was regarded by the assessors as effective in overall terms. Within Aceh, work under the rubric of ‘Big Ideas’ has comprised a component of the DRR work (most DRR work is undertaken in Nias and elsewhere in Indonesia) and a recent proposal for a Construction Technical Support Unit. In connection with DRR, the links between getting youth to think through the risks facing their communities and the practical actions that can result have not yet taken shape, so as yet one cannot measure its effectiveness. Different approaches to both DRR and Child Protection work are evident in different parts of the TRP, and we recommend that both these aspects of SC’s work are the subject of a systematic review, focusing on policy issues and on how they are implemented in practice. Although SCiSL has largely met its original objectives, its effectiveness in the long-term is at risk as the Ministry of Education has concerns regarding a loss of government control to the NGOs in this sector. Recommendations suggest an alignment with education sector plans and that SCiSL work more closely with the lead Ministry; in addition recommendations include strengthening those CSOs related to schools and a reduction in coverage to provide longer term support to fewer schools. In Aceh, the basic education programmes have been largely effective in meeting their objectives, which have been realistic in relation to the resources available, and have been acceptable to government and welcomed by the schoolteachers. Education was a lesser but significant part of SC’s response in India. Initially work was done to support children in Port Blair in ANI; thereafter emphasis was more on creating child-friendly schools and an environment that was conducive to education taking place. This component was found to be effective. Similarly in Thailand a number of child-focused projects, including work to provide basic education for Burmese migrant children, the child-led DRR project and the strong emphasis on the promotion of child protection, child participatory approaches and psycho-social support, were found to be both innovative and effective. Thailand’s work on child-led DRR has been mostly effective. In Somalia, the assessment found that the involvement of communities, transparency and capacity building have strengthened programme implementation in an unstable and insecure environment and as well as increasing accountability. SCiSL avoided becoming too involved in building houses, but it was found to have made an adequate, if not effective, contribution. SCiSL has taken pains to be observant of standards, especially those of impartiality and of construction. Sometimes strict

10

observance of standards has not been entirely appropriate and a number of ’white elephants’ remain after the response, especially in the form of lavish ECD Centres that will either fall into disrepair or bankrupt the small communities. It was clear to the evaluators that the programme has not got to grips with the issues of local ownership (Clinton’s main critique), which are complex during internal conflict. The construction in India of child-friendly Integrated Child Development Scheme centres was found to be highly effective; and the total included 48 centres in ANI, which was particularly challenging. In Aceh, the assessors found that the construction programme cannot yet be judged as effective, since until 2008 it has been consistently under-spent and is currently having to move very fast to catch up. However, it appears to have a good chance of reaching nearly all of its targets by the end of 2008, or at least in early 2009. As the second largest component of the programme in India, Livelihoods accounted for 16% of the total expenditure by partners – following analysis, this component was considered to be effective by the evaluators. The main focus of the evaluation in Aceh was with regard to livelihoods was on current programmes: the Business Development Services (in partnership with a Jakarta-based NGO) and the Microfinance projects (in partnership with a local NGO, which was established by SC but is now running with a degree of autonomy). Both programmes were considered to be running effectively in relation to their stated objectives, though the impact of the first was queried. India’s specific HIV/AIDS Sector objective supported: 733 awareness-raising campaigns; it carried out mainstreaming activities with 35 partners, and identified two strategic partners. The work commenced later than planned and took place in seven of the planned nine districts. This programme was assessed as reasonably effective. In Thailand the Health Sector component was directly linked to increasing knowledge of HIV/AIDS prevention and care strategies with drama used as the media to address this. We endorse concerns cited in a number of the Thailand evaluation documents reviewed suggesting that the projects were too short in duration to maximise their effectiveness. 2. Efficiency Specifically when measuring outputs as a result of inputs, Thailand came out as a particularly efficient programme, probably due to its strategy of working through local partners. Comparatively high overhead costs, in relation to the local economy, were noted in India and in Aceh, though since he cost headings in budgets and financial reports were not used in a clear and consistent manner within and across country programmes, comparisons were difficult to make. To increase efficiency, the review also recommends, where possible, shortening lines of management, reporting and accountability and doing what is possible to minimise the high turnover of staff during the early phases of emergencies. In order to address improved efficiency, SCiSL has moved towards more holistic programming in Sri Lanka, looking at child protection from a prevention angle. In 2007 SCiSL started to provide cash grants for income-generation to households in tsunami-

11

affected areas where children were identified to be at risk of abuse and/or institutionalisation due to financial difficulties. This was regarded by the team as a step in the right direction as, until this time, thematic staff worked along parallel lines with little integrated thinking or programming. The GIM framework is not used at all in Sri Lanka. However, this may change in the coming two years now that Thematic Programme Plans have been developed for various sectors. In India the evaluators found that the ICDS centres were of much better quality than those in existence previously (in some cases nothing had existed before), although this came at a price. Analysis showed that SC was one of the more expensive agencies in terms of implementation. In DRR activities, findings showed that SCiSL has reached its targets (going beyond them in some cases); chaired by the Government Agent, 12 districts (half of the country) implement child sensitive Disaster Management Plans. However, Children’s Clubs are less efficient because the communities themselves have to maintain their own activities. In Aceh overall efficiency was found to be low in relation to the housing programme, even given that re-housing costs per beneficiary are necessarily high in comparison with most forms of rehabilitation assistance. Retrofitting more than doubled the costs of the first part of the housing programme and also affected the schools programme; however it was more cost effective than the alternative of demolishing and rebuilding houses. Findings showed that the cost of the prefabricated houses was three or four times that of most of the aid-funded re-housing projects. Additionally, inadequate control of local contractors had led to significant losses during the first phase of the programme. Altogether in Sri Lanka SCiSL invested £2.4m in construction which was 7% of the total TRP budget; of this £1.8m went to schools and was carried out by SCiSL efficiently and cost effectively – this was attributed to hiring an experienced engineer from the relevant ministry, transparent tendering to competent private sector enterprises and close supervisions with frequent site visits. Even though SCiSL went beyond its original objective of 55 pre-schools, most were built through cost-effective local arrangements. In terms of efficiency in India, the permanent houses in tsunami affected areas were found to be expensive compared to what had existed previously, but SC was constrained by guidelines which had been laid down by government and they were less expensive than those built by other agencies. In Sri Lanka there was awareness of Sphere Standards3 and, in relation to construction; partners were encouraged to adopt these. With regard to educational objectives in Sri Lanka, measuring the efficiency of SCiSL’s inputs for strengthening this component was not feasible as they are spread across too many stakeholders. In the livelihoods component in India, again the assessors found SC to be one of the more expensive agencies in terms of implementation. There was a feeling from

3 The evaluation did not find any cases where Sphere standards had been breached, though these standards were not necessarily at the forefront of Save the Children’s in-country planning.

12

some within the SC team in India that the approach adopted on livelihoods meant that a small number of lucky people benefited a lot. In Aceh the evaluation found that it is too early to judge the efficiency of the livelihood projects which are still some distance from being in a position to stand on their own feet. Similarly, with regard to other programme objectives in India, the HIV/AIDS component costs were found to be greater than other agencies, but the idea of having strategic partners who will in due course act as a resource organisation for others was found to be an interesting and potentially effective and efficient way of working. 3. Impact The evaluators faced the difficulty of assessing impact in the absence of baseline data, forcing the adoption of more intuitive or common sense methods in reaching conclusions. Nevertheless, the impact of TRP programmes was generally thought to be good. Interactions with beneficiaries (children, communities and leaders) were overwhelmingly positive about Save the Children supported work. A less common though parallel finding, particularly noticeable in Aceh, is that Save the Children invests a good deal of time in planning, though much less in corresponding evaluation of its programmes. Questions were raised about the impact of DRR programmes in India and Aceh though DRR programmes were considered excellent in Thailand and Sri Lanka. (The review team did not assess the impact of the DRR programs funded by the Indonesia TRP in Nias and regionally.) Other programmes whose rationale was queried include, the Child Protection programme in Sri Lanka and the HIV/AIDS programme in India. The impact of most of the programme in Somalia was particularly difficult to assess from a desk study. As one might expect, SC’s impact appears strongest when it sticks to its core business and specific competences. For instance, in Sri Lanka the initial achievements in education and child protection after the tsunami are seen as very positive as they helped to get all the affected children back to school within two months of the tsunami. In the longer term, the Child Friendly School system will be included in the national five-year educational plan from 2010. Equally the initial protection work by SC in Sri Lanka resulted in the prevention of institutionalisation of children affected by the tsunami. However, it was difficult to assess the impact of this since the monitoring of qualitative outcomes has not systematically taken place, with a focus on reporting on quantifying the outputs. SCiSL has invested resources in supporting the government to set up village-level Child Protection Committees to create awareness and to encourage prevention and response mechanisms from within communities. The subsequent increased awareness of child rights and child protection has resulted in attitudinal change and outcomes, including: prevention of early marriage, encouragement for drop-outs to return to school and fewer children sent to care homes. In India, the ICDS programme led to many communities now having valuable community assets which can be used for early childhood development. In child protection the evaluation findings reveal a strong initial impact in terms of children being more self-confident and aware of their rights, but the extent to which these gains can be

13

sustained is not clear. The Department of Social Welfare reported that attendance at ICDS centres covered by the programme in the Andaman and Nicobar Islands is up by 45%, which is regarded as a significant achievement. The SEVAI constructed houses in India were found to have afforded encouraging long-term impact, providing considerably better living spaces than previously. Rebuilding houses in situ meant that the relevant households could be clearly identified and involved in the process. Children played a strong role in SEVAI house construction, influencing minor design elements and monitoring the quality of construction. In Aceh, where legal and institutional frameworks are weak, the provision of valuable assets such as housing needs to be carefully controlled and monitored if the process is to avoid the risk of being prey to local vested interests. In order to assess overall impact the evaluators propose that research is undertaken into how and why beneficiaries have been allocated housing on relocation sites. With no SC programme in Puntland (NE Somalia) previously, the tsunami provided a strategic opportunity to establish a programme in an area generally neglected by local and international NGOs. Following the rehabilitation and construction of schools and the provision of technical and material educational inputs, there is evidence of increased enrolment of both boys and girls and a decrease in the school drop-out rates. Substantial information on the impact on children is weak. Most of those interviewed for the Harragin Impact Assessment (early 2006) agreed that there has been a change in attitude as far as treatment of boys and girls are concerned. In all the schools visited, the headmasters said boys and girls are treated equally, a position that was also confirmed by Community Education Committee members and 97% of the pupils interviewed. In Sri Lanka SCiSL’s work with youth was found to have positive impact in terms of developing facilitation and leadership skills. In line with the other four countries, in Aceh the impact on education was judged positive, particularly in the case of teacher training. It was difficult to assess impact because of lack of baseline information; however indications of positive impact could certainly be found. In Thailand, SCUK reports have shown that Thai children can be empowered and can interact effectively with adults in areas such as DRR. The programme has strongly demonstrated impact by showing that children have the ability to lead disaster risk reduction showing improved analytical skills, ability to take collective action and the confidence to interact with adults in their communities with a resultant increased recognition by adults. In India the children demonstrated evidence of knowing about disasters and how to respond to them. The assessors believe that the CLDRR model in the tsunami programme could contribute positively to CLDRR programming in future emergency preparedness programmes. In health programmes in Aceh it was clear that the training and involvement of volunteers and health workers has led to improved practices. From discussions in India on the HIV/AIDS-related activities with a male peer group, the assessors found that participants were well-informed about the topic, they took the

14

training seriously and were enthusiastic about spreading awareness and knowledge. The women’s group showed less awareness. There was no indication of major stigma reduction. In Aceh it was too early to assess the overall impact of the current livelihoods projects; however the microfinance programme was very promising. In Sri Lanka, income as a result of cash grants has increased – this was verified by an impact assessment carried out by SCiSL. Women reported that this has directly translated into benefits for their children – specifically children’s nutrition, health and education. Although in India participants spoke of improvements in their income as a result of SC interventions, the assessors were unable to access baseline information or systematic collection of data along these lines to analyse this. In emergency preparedness in Sri Lanka the assessors found impact was more in evidence at local level rather than national level. He found that DRR seems to have caught the imagination of children, teachers and local administrators. In India in terms of impact the livelihoods area was found to be difficult to assess with certainty as there have been many organisations involved in activities from the early relief distribution and government compensation schemes to a broad range of livelihood inputs. Participants spoke of improvements in their income as a result of SC interventions, although there was no baseline information or systematic collection of data along these lines for the assessors to analyse. 4. Sustainability Sustainability is a complex issue and the TRP presents a mixed picture in this regard, varying greatly between different types of project. The reports from India, Sri Lanka and Aceh all note that the sustainability of Save the Children interventions has often been made more difficult by the massive influx of post-tsunami funding; the evaluations from Thailand and Somalia are more positive on this issue. Another theme is that Save the Children support is likely to be needed for longer than is currently planned (and/or after 2009) for some interventions, particularly where there work has been more ‘developmental’ in nature. In Aceh, it was noted that government support and approval for many SC interventions, though very genuine, is unlikely in most cases to translate into practical long-term support. In India, the evaluation found that the sustainability of those houses built in situ to be in stark contrast to those constructed by other agencies on relocation sites, which made community participation impossible and resulted in less than 50% occupation in some instances (eg: fishing communities preferred to stay closer to their boats and the sea). Families said they were happy to have returned to where they were living before the tsunami and there were indications in the households visited of embellishments added by families. In Aceh it is clear that the great majority of housing provided by Save the Children, including in the resettlement communities, will comprise long term assets and provide long term benefits for their occupants. However in the case of the resettlement communities it is important that owners’ rights to land and housing are clearly established.

15

In Sri Lanka SCiSL avoided becoming too involved in building houses, something for which it was wholly unequipped. However, sustainability of construction in the education sector was reasonably well-assured as government was committed to operation and maintenance of the schools. To address overall sustainability, the exit strategy in Sri Lanka is to move from rehabilitation back into long term programming, with much less direct implementation and closer work with partners. In terms of the ICDS Centres constructed, their sustainability is best guaranteed by the fact that they do form part of a government programme. The ongoing conflict means SCiSL has to maintain the capacity for emergency response in education. The challenge is to maintain this humanitarian response capacity while putting programming in place for the longer term. The majority of income-earning ventures started through the cash grants in Sri Lanka have succeeded and were ongoing at the time of the review. However, certain sectors continue to lag behind others, especially due to the increased price of inputs. To ensure sustainability, the assessors recommend that more effort should be made to encourage field staff to translate the learning from impact assessments and other studies to their work in the field. In India, where new self-help groups were supported, experience gained over the past two decades indicates that it is likely that they will require several years’ further support and accompaniment before livelihoods become self-sustaining. For individual livelihood activities, it seems likeliest that these will be sustained where they have built on previous experience and competence. In Aceh, however, although the assessors found that the training in the education sector will leave a very positive legacy, it is unlikely that the government will be in a position to continue to support these programmes in the future. Even though the government is satisfied with the initiatives, they will not have the funding or the capacity required. Similarly in the health sector there is a similar positive legacy but no alternative local actors to ensure sustainability of the health-related initiatives (the NGOs which SC has supported are both international agencies). In India there are questions about the sustainability of education inputs, particularly where there have been short-term interventions. While short term improvements have undoubtedly taken place, it will require a more sustained follow-up for new structures to be consolidated and to continue to function for improved education. In general the level of coverage in relation to programming in Child Protection in Sri Lanka was found to be very high; however, to ensure sustainability and to have a more lasting impact on children’s lives, it is recommended that SCiSL needs to focus more on quality and good practice rather than quantity. The sustainability of the Child Protection work in India was regarded by the assessors as a serious question as, for knowledge to change attitudes and behaviour of adults and children, more sustained interventions than have been possible up to now will be required. Recommendations from the evaluation of the India programme include those relating to sustainability. Much of the work is developmental in nature and, while many groups have been established, significantly more work is required if interventions are to become sustainable. Sustainability concerns include SC’s work in Emergency Preparedness. Awareness and interest is high straight after a disaster; a few years later when there has

16

been no disaster, it can dwindle down quickly. Encouragingly, an Emergency Preparedness and Response Plan has been developed as a SCBR document, which is meant to be kept alive through training and updated each year. The assessment raised questions regarding the sustainability of the HIV/AIDS work in India; it is well recognised that raising awareness, changing attitudes and achieving behaviour change is a long term process when it comes to HIV/AIDS. It cannot easily be achieved in the context of an 18 month project when there are many other issues pre-occupying the community. 5. Relevance/Appropriateness Appropriate relates both to targeting of beneficiaries and to the fit with broader Save the Children policies and may be subject to slightly different interpretations. There was some variation between the degree of emphasis given to children’s rights and participation, being comparatively high in Sri Lanka and Thailand and lower in Aceh. Targeting was reasonable, though dependent on the sector – good for example in health and education in Aceh, though difficult to achieve in housing, where external factors are likely to determine the beneficiaries. In Sri Lanka the relevance and usefulness of a number of child-focussed activities conducted in welfare centres and later in resettlement area was restated by tsunami-affected children and adults. These included child-friendly spaces, play parks, distribution of play materials, setting up of community libraries and community centres. The provision of vocational training for tsunami affected youth to prevent separation from their communities, and to reduce their vulnerability to under-age recruitment and other forms of exploitation was found to be timely and relevant. In Sri Lanka, DRR was also found to be appropriate in a country affected by chronic conflict and with a high incidence of natural disaster such as cyclone and flood. Child-led DRR is particularly appropriate for SCiSL and there is substantial experience in the region to learn from. The appropriateness of housing as a priority sector in India was questioned by SC – initial plans were to carry out a major housing intervention, however, this was significantly reduced to 10% of its total expenditure. Given that many other agencies (including government) provide housing and SC has little experience in this area, this was justified. A real strength of SCiSL’s relief response was its evolving holistic and child-focused rapid response to displacement. In India, to affect a longer term impact on children, the usage of the ICDS centres was found to be a key factor. The education component in India was found to be an appropriate activity, even though much of the work was improving what existed before the tsunami rather than repairing what had been damaged. As was the HIV/AIDS component as HIV is prevalent in the target areas, albeit not at a very high level as compared to other countries. The livelihood component in India was also found to be an appropriate intervention, given that all tsunami affected communities had suffered the effects, even if they were not directly hit. Similarly the livelihood component in Aceh appears to be highly appropriate. The BDS project fits well with existing practices and activities,

17

though its economic impact was uncertain. The targeting of women is appropriate, with children (especially younger children) benefiting indirectly. In Sri Lanka cash grants were found to be very relevant to beneficiary needs. The earlier focus on replacing assets was regarded as inappropriately driven by the need to spend at the relief stage. It was also felt that too many boats had been handed out and it was considered that the purchase of boats and motors for boats was not appropriate for a child focused organisation. A point raised by the assessors is that assessments need to be realistic and not to be based on what the agencies wish to hear because they need to spend quickly. SCiSL prided itself on its level of co-ordination with other agencies and with the Fishing Ministry, however, there were far more boats on the beaches after the tsunami than before the emergency. The findings of the Sri Lanka assessment demonstrated that there can never be too much contextual knowledge (though there may be too little emergency response training) and brings into question the value of external assessment teams replacing the staff already on the ground. However, contrary to this was the experience in India, where the assessors found that SC was saved from mistakes as it was not as quick off the mark as other agencies. As a result they were able to benefit from the errors of others. The evaluators concur with the SC programme that the aftermath of a disaster is an appropriate time to raise awareness about how to respond to future disasters. People were found to be aware of the effect a disaster can have on their communities and are receptive to messages. 6. Coherence Coherence relates both to Save the Children policies and to the overall response to the tsunami by all agencies. Overall findings were positive. The Sri Lanka chapter draws attention to the reaction against international NGOs being seen to impose policy agendas on the country. In general, coordination with other agencies seems to have been best during the emergency phase of the response. The Sri Lanka findings draw attention to the reaction against INGOs being seen to impose policy agendas on the country. It was found that as government attitudes toughened, SCiSL made insufficient operational adjustments to counter the political risks they faced as a result; at district level in the education sector, however, there is generally good coordination and work with UNICEF has been positive. In India, SC ensured coherence in the education component by working within the existing government education system, which is already resourced and functioning well, particularly in Tamil Nadu. The work in India on the Shelter component was found to be coherent in that SC and its partners worked with government on the identification of sites and beneficiaries. However, it was felt that more needs to be done in India to integrate Child Protection Committees with local governance structures to enhance prospects for sustainability. The livelihood work in India sought to take account of government interventions, particularly when linking people to government social safety nets. The coherence of the livelihoods projects in Aceh with local social realities and with other SC objectives was found to be good.

18

Where possible in India, SC sought to link its disaster preparedness work to local institutions (schools and village communities). 7. Coverage Coverage relates to who the programme targeted and why, and was generally found to be satisfactory across the different countries. With regard to the child-led DRR in the community, SCiSL has implemented DRR activities through as many as 7,000 children’s clubs in all of the 255 communities in which SC works. However, although coverage in Sri Lanka has been high it is thought not always to be sustainable. SCiSL was found to have used sound assessment methods to reach the neediest people in the communities. In India, the only question over the geographical programming was the wide area of coverage – essentially everywhere affected by the tsunami in India, apart from Kerala. In terms of management, provision of technical support and building linkages between partners, it was found that it may have made more sense to focus more tightly. In terms of coverage, in Somalia and Thailand, it was noted that the TRP lacked focus on disabled children. Information in all country reviews was limited in this regard and it is recommended that all programming should ensure the inclusion of disabled children. (This recommendation applies to all countries to varying degrees.) Coverage in India, with respect to the provision of shelter, was found to be positive in that many of the beneficiaries were from Scheduled Caste communities, whose needs and livelihood opportunities tend to be more challenging. Coverage of the Early Childhood Care objective in India was found to be good, targeting communities in need, many of them in quite inaccessible places in ANI and providing assets far beyond what they had previously enjoyed. Likewise under the Childhood Protection Objective, coverage seemed good, with conscious efforts to involve marginalised ‘second line’ communities who were reached by other agencies. 8. Strategic and Management Issues With respect to ways of working, Save the Children adopted a combination of approaches, ranging from working entirely through local partners in India and Thailand, a mixture in Sri Lanka and being more directly operational in Indonesia. There were contextual reasons why there had to be variations, but in some cases, such as in Sri Lanka, it was felt that valuable previous experience was put to one side in the desire to become quickly operational. It was found, also in Sri Lanka, that the SC programme did not have a chance to benefit from the GIM system; this is likely to be the case in the other countries of the TRP. GIM operates on an outcome and impact level and planning for the TRP focused mainly on outputs, from where it is not possible to detect change in children’s lives. The prioritisation of programme objectives was mostly appropriate in terms of needs on the ground and Save the Children’s focus areas. The evaluation did not discover areas of unmet needs, which should have been addressed. The biggest questions were over Save the Children’s ability to work in some sectors (notably housing construction) where it had limitations in experience and capacity.

19

Targeting of beneficiaries was mostly good and there were some commendable efforts to reach those communities less covered by others, such as in Somalia and inland scheduled caste communities in India. There were questions raised as to whether Save the Children aimed for too wide a coverage (in India) and as to whether it had sufficiently sought to reach children with disabilities (in Somalia and Thailand), who were some of the most vulnerable. Child participation seems to have varied from one area to another. It was strong for example in Sri Lanka and Thailand, while was given less emphasis in Indonesia, in part due to the differing perspectives of the respective lead agencies. In the initial onset of a disaster, any kind of participation is time consuming when there is pressure to achieve outputs, but as the response progresses it should be possible for greater emphasis. Where it did occur, it often was more a case of involving children in implementation rather than planning. Also, adults (teachers, parents) sometimes were unconvinced about the approach. Save the Children’s approach towards advocacy (where it took place) tended to be in terms of low key influencing from a supportive position towards government, rather than the more confrontational approach that some other agencies adopted. This showed some positive results such as promoting child led disaster risk reduction in Sri Lanka, and initial indications following advocacy on orphanages in Aceh, although the evaluators felt there was scope for more advocacy work here. The scale of the response necessitated massive recruitment of new staff, particularly where Save the Children was more operational. This often meant taking staff from local organisations at a time where they were under severe stress and also presented challenges in terms of orienting new staff where there was pressure to get work done and they often had little child-oriented (and Save the Children) and/or disaster response background. There were many international staff brought in for quite short periods with limited local knowledge and at considerable cost. Sometimes the influx of expatriates was felt to be excessive, creating more problems than were solved. On the other hand, the development of local staff represents a resource for the future, as for example in India, where many staff have now been taken into Save the Children (Bal Raksha). During the review of the literature for the Sri Lanka assessment, the Wellness Initiative was alluded to in passing but it was not volunteered as relevant to the study, nor mentioned during the country visit. The evaluators understood it to be an HR initiative to ensure that staff did not become exhausted. Even though the assessment made clear that the HR systems in SCiSL were good, it is still not possible to directly link this directly to the Wellness Initiative. The training and capacity building of staff and partners was particularly strong in India, where numerous courses were carried out and seem to have had a real effect on the quality of programming. Also rated highly were the efforts made to convene different country programme staff involved in the TRP, to share experiences, develop learning and produce documents to capture best practice. While the TRP was essentially individual country programmes operating under an umbrella framework, this was a notably successful area where it went beyond this.

20

In terms of overall management, work was slow to get going in most countries reviewed due to the scale of the disaster and response, but also due to the multiple layers involved in decision making and the lack of devolved responsibility. The positive side to this was that it did enable Save the Children to learn from the mistakes of others. The Save the Children Alliance worked in various ways, with India being the most complicated with four different Save the Children actors involved in implementation. This led to some confusion, particularly with two international SC members (Finland and Canada), operating separately from the designated lead Alliance member. There remains work to be done in ensuring better coordination in this regard. Management lines were also extended in Aceh, with important aspects of the work coordinated by the Indonesia country office in Jakarta. On the other hand, there were positive examples of Alliance cooperation, as demonstrated in Sri Lanka. Recommendations The recommendations relate to the objectives of the programmes under review.4 . General Recommendations The following recommendations relate to the programme as a whole. These have been identified by bringing together the findings from the different countries, examining the common issues arising and prioritising the most important points.

1) There remain challenges to complete some work, like construction in

Aceh and new project development in the Andaman and Nicobar Islands in India, which require special and continued attention to ensure completion. The Big Ideas (Early Childhood Trust Fund, Education Trust Fund and the Construction Technical Support Unit) are still at a relatively early stage and also now need to be developed quite soon if they are to be credibly part of the TRP. Overall the evaluation found that the TRP was on track to complete its objectives by the end of 2009. There were however some specific elements in particular countries (as identified above) where emphasis needs to be placed to ensure that everything planned is implemented. In practical terms, this means ensuring that all the necessary workplans are developed with a clear schedule for implementation and this is followed through on a regular basis by management.

2) Construction was the only major sector where the evaluation raised questions

about the appropriateness of significant parts of Save the Children’s intervention. This was particularly true for domestic housing in Aceh, an important part of the TRP yet an area where SC has had comparatively little experience. Much SC construction work was successful, for example with schools in Sri Lanka and ICDS Centres in India. In Aceh too, institutional construction, of schools and health centres, was much less problematic than domestic construction. If Save the Children is to address construction

4 At the request of the Steering Committee, the principal recommendations here have been elaborated considerably so that their basis is as clear as possible. Further justification for the country-specific recommendations, if needed, can be found in the country chapters of the report.

21

work effectively for the rest of the programme and for the future, then it needs to ensure that it has the necessary technical, managerial and analytical capacity or support. Capacity to assess and analyse the communities benefitting from construction is an important component of this. The proposed Construction Technical Support Unit could be developed as a vehicle to take this forward, but will need to be adequately resourced and have capacity to cover or take on board social/analytical as well as technical aspects. Specifically the evaluation recommends that: o Save the Children should commission a review of the ways and means by

which the agency is currently involved in the provision of shelter, temporary housing, institutional buildings, domestic housing, or other kinds of construction, and make recommendations as to the extent that they should be in the future

o Link current proposals for a Construction (Technical) Support Unit to such a review. It should be recognised that support needs in the construction sector will be both technical and non-technical.

3) Many projects now being implemented under the TRP are essentially

developmental in nature (going beyond rehabilitation). This was appropriate given the funds available, but it does raise questions over sustainability. Such developmental activities need continued accompaniment and support over a period of time, often beyond 2009, if they are to be sustainable. Though context and circumstances are all, a group working on income generation, for example, may not be sustainable after 18 months of support. On HIV/AIDS awareness-raising, to change attitudes and achieve behaviour change requires a variable length of time in different contexts, but considerably longer than has been planned in India for most projects to date. Similarly, work on child rights which addresses social hierarchies which have existed for centuries cannot be sustainable after short term interventions. Save the Children does plan to continue working in many of the same areas up to the end of 2009, and in some cases beyond, notably for example in Aceh. In doing so, it will be important to assess what activities require further follow-up and the nature of the follow-up required for developmental work which has been initiated. This will vary from case to case, but is likely to involve a combination of training, facilitation, support, advice, mentoring and accompaniment, which can reduce in intensity as individuals/groups become more self-reliant. This of course has financial implications. In this regard, remaining earmarked tsunami funds should where possible be allocated to ensure sustainability, wherever such longer term work needs consolidation.

4) Whilst context is again crucial, child participation should where possible

be given more emphasis for the rest of the programme and extended to the planning of new work.. Inevitably child participation varied from one phase of the TRP to another (for example, with less being possible in the early relief phase as is common in these situations) and from one project and programme to another. Child participation is an important principle of the Save the Children and is considered invaluable in ensuring that activities meet children’s needs as well as being a children’s right, though in practice different members of the Alliance give it different emphasis in their programming. When child participation did occur under the TRP, it was

22

generally in terms of implementation, rather than in research and planning. For the remainder of the programme, there should be more scope for involving children as active members of society. This is not easy initially where children (and adults) have not been accustomed to this, but there are a range of participatory tools, which Save the Children itself has developed and used in different parts of the world, which can facilitate the process.

5) There were legitimate reasons for variations in partnership approaches,

but to improve sustainability the emphasis on partnership needs to be strengthened, particularly in Sri Lanka.. This means, where possible, working more through local partners (than implementing directly) and having more emphasis on the capacity development of partners to implement projects and for more general organisational strengthening. This applies even in environments like Aceh where possibilities are limited and such an approach is difficult. The scale of the response to the tsunami put unprecedented pressures on implementing agencies to scale up and implement work on a massive scale. Where local capacity was limited and there were time pressures, direct implementation was inevitably preferred, as working through local partners can be slow and means loosening control. However, direct implementation does not hold out such good prospects for sustainability, so now that the pace is less frenetic, increasing emphasis should be devoted to working through and supporting the capacity development of local partners.

6) For future disasters, it needs to be clear whether individual Save the

Children members will carry out their own programmes (as Canada and Finland did in India) and how a national entity (like Save the Children Bal Raksha) will be supported in the face of the need to scale up work rapidly for a major response. Save the Children is a complex organisation, going through a process of change. There are various moves towards a single Save the Children entity, whether it is a local national body, as has recently been established in India or a coming together of various national Save the Children members from other countries. During the tsunami, while the Alliance designated lead agencies, there were also individual members working outside this structure. To avoid confusion, it needs to be clear how these issues will be addressed in the future, particularly if there is another major disaster, which puts pressures on the overall system.

7) The amounts of money raised in relation to the tsunami were unprecedented

and presented genuine challenges for Save the Children (and many other organisations) as to how it could be spent effectively. Questions about whether so much money (when adding together all the government and NGO programmes) was appropriate, in relation to the scale of the needs, cannot be avoided. There are no simple answers to this, as the scale of the public response to a disaster is unpredictable, but it would be worth Save the Children reviewing the decisions it took and the messages it put out during the crucial fundraising period, in order to assess whether it acted appropriately and whether it might have done things differently. Also, in some cases, the speed with which Save the Children spent the money affected the quality of its work - in the event some of it could have been done more slowly. The evaluation was

23

confined to looking at how the funds raised were spent at the country level and did not review the fundraising operations and activities of Save the Children members. At the field level, it was clear that much emphasis was placed on spending the large amounts of money raised – quite unusual for agencies which are generally feeling short of funds. The amounts of money spent by all the agencies combined created (at least in the short term) a sense of dependency on the part of local communities and often worked to undermine longer term developmental approaches that may have existed previously. The recommendation here is that Save the Children examines whether it raised excessive funds relative to the needs and what else it might have done. The example of Médecins Sans Frontières is interesting in this context as one organisation, which made a conscious effort from an early stage to shift donors’ attention to other causes. It also looks at the speed with which it seeks to spend money in future similar disasters, and does not compromise on quality in the rush to show quick results.

8) Save the Children should review its accounting system to ensure

common headings and definitions are used and applied across all programmes when presenting financial information. Even within countries, headings varied from one year to another. There were also differences in presentation of budgets and accounts between countries. This makes it difficult to make comparisons and come to firm conclusions about whether expenditure on certain areas is excessive.

9) Save the Children should review its overhead costs in comparison with

its programming costs in terms of how it compares to similar international and local NGOs. The previous recommendation shows that it is hard to make conclusive comments here as the figures available followed varying formats which were difficult to compare. However, from information such as Save the Children spending 33% of funding on its own staff and other overhead/support costs in India, when all project implementation was done by local partners, it is felt that this area needs further examination. Similarly, transport was over-funded in Aceh. Save the Children should carefully consider whether the value it adds to the work can be justified by what it is spending to support programme work.

10) Objectives and outputs in plans and reports should as far as possible be

presented in a standardised, clear and consistent manner to allow direct comparisons to be made between different plans and see to what extent plans have actually been achieved in reports. It was hard to assess whether plans had been achieved and to make comparisons from one year to another and one country another, because different headings and indicators had been used from one document to another. There should at least be consistency within each country. Ideally, some framework should also be used across countries to allow for more meaningful presentation and understanding of data.

11) For the final evaluation at the end of the TRP, we recommend that Save

the Children should commission a range of sectoral studies/evaluations across countries to allow in-depth analysis by sector. After that the sectoral studies should be brought together, in order to examine

24

questions of coherence and other cross-cutting issues. The evaluators were acutely aware that given the scale of operations and range of activities implemented by Save the Children through the TRP, it was only possible to make limited samples of the different sectors in the course of this evaluation. Given the wealth of experience gained by Save the Children in different sectors, on an unprecedented scale, there is considerable scope for more in-depth learning than this evaluation allowed.

Recommendations for Aceh

1) Save the Children should prioritise planning for the sharp decrease in funding both at the end of the construction programme and at the end of 2009, whilst seeking to secure funding to support the post-construction components of its programme as far as possible.

2) Where programmes have to be discontinued through lack of funding (e.g. after 2009), particularly those involving training, contact should be maintained through intermittent or limited training or mentoring. This would have benefits out of all proportion to the limited costs involved. This recommendation applies to all projects with a training focus in health and education, as well as to the livelihoods programme.

3) Save the Children should undertake research in the resettlement areas where it has provided housing, and should plan to monitor the situation and advocate its resolution for as long as is needed to ensure that questions of ownership of land and housing are satisfactorily resolved.5 Such research should also survey any outstanding problems where they exist, for example in relation to water, sanitation, drainage, or liability to flooding, and make plans for facilitating the resolution of such problems.

4) Whilst sensitivity is always required, greater priority should be given to advocacy work. This should continue as now to be directed towards government in areas where there is a realistic chance of some response.

5) Careful assessment should be given to the advocacy work on children’s homes, in order to achieve the projects’ objectives of reforming these institutions.

6) As a general principle, throughout its work in Aceh, greater consideration should be given to promoting children’s participation, and a greater emphasis given, where appropriate to incorporating dialogue on children’s rights into SC programming.6

7) An external evaluation should be undertaken of the current livelihoods projects. The evaluation should give particular focus to relationship of the projects to local economic realities. External support should be maintained to the current livelihoods projects for at least twice the period currently planned for these projects - the précised details to be determined by the evaluation in consultation with project managers.

5 With reference to comments received, this does not of course imply that the resolution of these problems is entirely in SC’s hands, nor that SC is responsible for these problems in the first place. 6 We fully recognize and accept that SCUS places a lesser emphasis on these issues than many other members of the SC Alliance. However we do not feel such differences make this recommendation out of order. Although greater harmonization of objectives between different members of the Alliance seems to us to be desirable, this recommendation also derives from our independent assessment of the programme in Aceh. We recognize that this recommendation may not be accepted by SCUS, and that, even if it is, its impact on improving project implementation and securing objectives will be gradual and incremental.

25

8) Greater clarity is needed in defining and implementing the managerial responsibilities and the advisory responsibilities of sectoral managers in Aceh. In practice these are not always clear and are interpreted differently in different sectors.

9) The parallel sectoral/geographical management structure of the Aceh programme is necessarily complex. This places a high premium on systematic and transparent communication on policy issues and management decision-making.7

Please also note the general recommendations concerning work in construction, which are particularly relevant to the Aceh programme.

Recommendations for India

1) In terms of getting the rest of the programme finished, most of the funds allocated for India have already been spent. Of the £944,000 remaining with SCBR, most is already committed or earmarked to partner projects. It seems that ANI has the most to do after a gap in managers and lack of clarity about what is already decided. This needs to be pushed forward to ensure completion by March 2009.

2) Much of the work done was developmental in nature and, while many groups have been established, significantly more work is required if interventions are to be sustainable. This should be a priority for the next phase (for Child Rights Clubs, for new SHGs, to consolidate DRR plans, to follow up on ICDS usage, for HIV behaviour change etc.). If there are further funds earmarked for the tsunami which can be made available, these can be used to consolidate such work to 2011 and beyond if necessary.

3) On the other hand, given relative levels of poverty and need, it would seem hard to justify making efforts to secure new (non-tsunami related) funding for areas which have already received so much support. In particular, in ANI, given the relative levels of poverty, the numbers of inhabitants, the scale of government support and the cost and difficulty of working, it seems hard to justify making further investment, unless it is on the basis of funds which cannot be used more widely. When compared, for example, to the poorest parts of Bihar, there would seem to more justification for concentrating efforts there.

4) For the Education Trust Fund, apparently $6 million is allocated for this, but it is unclear how this will proceed. This needs to be pushed forward if it is to be credibly linked to the tsunami. The issue of the management and sustainability of a new and such a large institution needs careful consideration and planning.

5) For further projects with existing partners, greater efforts could be made to involve communities (including children) in planning, not just implementation.

6) There were various programme plans and it was hard to get a clear overall picture. The evolution of plans is inevitable, but it should be easier to follow changes and their reasons. Similarly, it should be clearer how reports relate to plans, and how individual project reports link to overall programme reports.

7 A comment on the draft of the report asked for more specific project-level recommendations for Aceh. Whilst understandable, this may be to misread what is feasible or appropriate for an overview evaluation of this kind. More detailed project recommendations would need to be developed with project managers, bearing in mind the more overarching recommendations presented here...

26

7) For future disaster management, thought needs to be given to how a major disaster would be managed by SCBR. Eg: would international SC members come in and if so, on what terms? Having independent SC operations seems questionable. Are expatriates really needed given their cost and limited experience in India? The Emergency Preparedness and Response Plan should address these issues.

8) The decision-making process in future disaster situations needs to be streamlined, with fewer layers and clearer authority given to local managers.

9) SC should go into shelter properly in such situations, or leave it alone completely. If working on permanent housing then in situ housing tends to be preferable. The capacity to address temporary shelter needs to be considered.

10) Now that SCBR is in existence, it will have the option of direct implementation as a local organisation itself in the future but, if the intention is to develop local civil society, then it would make sense to continue working through local partners.

11) Where there is limited local capacity, then supporting agencies from outside immediate affected area makes sense. However, it is also worth considering support for small local organisations to help develop capacity, even if they cannot deliver large scale projects.

12) If there is such a spread-out disaster again, it could make more sense to focus the programme area more tightly to make it logistically easier and more coherent.

13) SC should consider changing from ‘cash basis’ to ‘actual basis’ accounting (i.e. expenditure would better be shown not when a transfer is made to a partner, but when the partner has actually spent the received money in order to ensure coherence between narrative and financial reporting).

Recommendations for Sri Lanka

1) Save the Children should maintain essential operational capacity for addressing the impact on children of man-made disasters in Sri Lanka and come up with innovative measures to strengthen capacities for local ownership in managing natural disasters.

2) SCiSL needs to take steps to ensure that the programme maintains the momentum achieved in empowering children through DRR. It will specifically need to make sure that DRR’s new status as a cross-cutting issue does not mean it gets lost in other programming.

3) For broad policy impact based upon its DRR work in the tsunami affected areas, SCiSL needs to find reliable partners and make a specific contribution, such as support for the development of DRR curricular materials for schools.

4) SCiSL education sector plans should be aligned more closely with the lead Ministry. This means taking a longer-term and more strategic view on working through government in education. SC should not seek to circumvent central government by, for example, remaining operational beyond time agreed, or by replacing government at local level.

5) Save the Children should focus on strengthening civil society organisations related to schools, such as the School Development Societies, Children’s Clubs, DRR groups and so on.

6) A sharp reduction of coverage is advisable in education, as in other sectors, with longer term support to fewer schools and communities in the interests of

27

close follow-up of incremental change and closer connections with other SCiSL sectors.

7) The Big Issue Early Childhood Trust Fund has potential and should be pursued. It could be made better by conceiving it as a long term foundation rather than a time limited Trust Fund. This would ensure that it maintained its community ownership. In order to ensure that Big Issue is in the long term interest of the pre-school age child, Save the Children should commission an independent risk assessment locally before making firm commitments.

8) SCiSl and its partners UNICEF and the Ministry of Social Services should make a concerted effort over the next two years to render the 27 Social Services Centres useful. If the original purpose of joined up child protection services at divisional level cannot be met immediately, other options for using the buildings that benefit children should be explored meanwhile. At all costs SCiSL should avoid inadvertently impoverishing local authorities with high maintenance cost of buildings that serve no useful purpose.

Please note that a partners’ meeting in Sri Lanka (Ampara) also produced some interesting recommendations. Whilst not formally part of the evaluation recommendations, they have been appended to the Sri Lanka chapter. Recommendations for Thailand It would be inappropriate to make any detailed recommendations at this stage, given that nearly all the project work has been completed.

1) As the programme winds down, it might be appropriate for SCUK and SCS to jointly spend some time reflecting on lessons learned and reviewing recommendations from the various evaluations to see if any follow-up work should and can be done at this stage.

2) As indicated above, this process could also involve looking at the draft child protection situation analysis report to see if any of that work can be salvaged.

3) Follow up on the Lessons Learned exercises e.g. SCUK Bangkok May 2007 to see whether further work is needed to implement recommendations.

4) Increase SC’s responsiveness to the situation of children with disabilities.

Recommendations for Somalia Given the nature of this desk evaluation, it is not appropriate to make detailed recommendations about the remaining elements of the Tsunami-funded programme. Some more general recommendations are as follows:

1) Consider ways in which monitoring and evaluation could be strengthened in the Somali situation, perhaps by developing in-country resources through an independent NGO such as the Puntland Development Research Centre.

2) Develop a focus on the needs of children with disabilities in Puntland by assessing and responding to their particular protection and rights issues.

3) In demanding and insecure situations such as Somalia that involve long-distance management, SCUK should consider developing intensive induction procedures for new staff focusing on SC’s programming policies and priorities.

28

Aceh Programme (Indonesia) Programme description Country Context and Impact of Tsunami Aceh had been a society in conflict for nearly 30 years when the tsunami struck on 26 December 2004. The conflict between the GAM (Free Aceh Movement)-led secessionist rebels and the Government of Indonesia was usually maintained at a comparatively low level and did not have major geopolitical ramifications, so did not attract a great deal of attention from the international community. Its overall impact on Aceh was however enormous, maintaining the region in a state of poverty and insecurity and constraining the development of civic and governmental institutions, both in an absolute sense and by comparison with the rest of Indonesia. The international agencies who tried to rush to the scene after the tsunami engaged with a society damaged by decades of conflict. All found it was not an easy environment in which to work, and many underestimated or misinterpreted the implications of the legacy of conflict. The impact of the tsunami was catastrophic. Over 130,000 people died, a third of them children, and more than 645,000 were displaced. 270,000 houses were damaged or destroyed, along with 1,488 schools, two hospitals, and 26 health centres. Virtually everybody suffered greatly through loss of family, friends, land, livelihoods and opportunities. The tsunami generated an unprecedented international humanitarian response and Aceh was recognised as the worst affected area. Despite the resources available, however, security considerations hampered the initial response, with some government concern that aid might end up benefiting the rebels. Nevertheless, although more slowly than in other tsunami-affected countries, a massive relief and rehabilitation effort did get underway. There is little doubt that the political and economic impact of this intervention contributed to the ceasefire, peace settlement, and elections which comprise an enormously positive outcome to developments since the tsunami, although the legacy of instability has not disappeared. Save the Children has worked in Aceh since 1976. Before the Indonesia country programme temporarily closed in 1996,Save the Children worked mainly running small community-based initiatives on the east coast of Aceh, an area that saw a good deal of conflict. SC restarted their programme in 2000 and were one of the few agencies working in Aceh when the tsunami struck. The benefits from this, and the limitations on these benefits, are described in the section on management issues in the Aceh Chapter below.

29

Aceh: Summary of overall findings – outcomes against objectives Effectiveness Efficiency Impact Sustainability Appropriateness Coherence Coverage Overall/ general

Generally good, though uncertain in resettlement areas. Assessment hampered by high rate of change in plans and management

Low overall because of the construction programme. Overheads high by comparison with local and national agencies notably government. Adversely affected by number of management tiers.

Generally difficult to assess. SC invests time in planning but little in genuine evaluation.

Developing sustainable components post 2009 is a major challenge. Massive aid influx has made this more difficult.

Reasonable focus on children. Poverty targeting quite limited. Geographical targeting justified. Participation limited.

Coordination with other stakeholders good at first then some decline. Engagement with local society often limited. Cooperation with government has been good but impact on government limited.

Justified. Limited to coastal districts, mainly on east coast, less badly affected by tsunami but often more affected by conflict.

Construction

Consistent underspend now being rectified. Ownership of housing and land sometimes unresolved.

Very low for housing because of high costs of retrofitting and prefabricated houses. Better for institutional construction.

Good, though uncertain in resettlement areas.

Good where buildings will be continuously used. The housing programme is not a replicable programme model for the future.

Institutional building more appropriate than house building.

Low for housing. Otherwise good

Geographically appropriate. Beneficiary profiles need analysis, especially in relocation sites.

Health

Generally good and sometimes outstanding.

Reasonable, especially when working with volunteers, despite high logistics costs.

Difficult to assess but appears good

Will leave an impact of trained people and changed practices but government will not provide limited training or mentoring needed to sustain.

High Good Appears good but difficult to track

30

Effectiveness Efficiency Impact Sustainability Appropriateness Coherence Coverage Education

Variable but often very good.

Reasonable Generally appears good and sometimes outstanding, e.g. teacher training

Same situation as with health, see above.

High Good Good

Livelihoods

Early interventions problematic. Too soon to say for current programmes. Microfinance programme looks more effective than Business Development Services.

High, especially microfinance.

Potentially high Uncertain. Early “handout” environment created by many agencies made things more difficult.

High if programmes prove successful.

Reasonable, but other actors must be taken into account.

Reasonable

Child protection .

Reasonable, but parts of the programme outside standard child protection mandate.

Reasonable, with great potential in advocacy work.

Low, except for non-formal education/ vocational training programme.

Uncertain, especially with regard to non-formal education and vocational training.

Variable Low Low overall

31

Financial summary of Save the Children responses in Aceh

First line shows budget. Second line shows Actuals. Amounts in US dollars 2005 2006 2007 2008

Emergency phase 5,699,162 5,236,730

0 749

0 0

0

Shelter/construction (inc Temporary and watsan)

6,004,372 3,445,254

9,105,204 4,180,860

9,697,936 6,882,753

18,556,133

Health 4,399,767 4,225,355

2,451,200 1,926,441

2,091,072 3,140,237

1,000,000

Education

3,130,470 3,008,228

2,666,643 1,566,951

1,252,042 1,454,347

1,500,000

Child Protection 1,620,381 1,419,772

1,421,172 974,781

1,233,451 934,880

1,000,000

Livelihoods 3,398,293 3,379,760

3,523,868 1,213,810

1,567,699 1,216,824

1,000,000

2008 figures Principle Findings and Recommendations This section assesses the programme objectives against the principal programme criteria. These objectives have not been immutable, but have generally been expressed in quantitative terms which are summarized in the tables of each section. These sections develop the Summary of Findings Table shown earlier. Objective 1: Construction The extent to which Save the Children should be involved in construction, and whether it should be involved in domestic reconstruction at all, comprise major questions for the Alliance, which have been sharply illuminated by experience in Aceh.8 SC has long acknowledged that the first phase of its housing programme in Aceh was a failure. 577 houses were built in wood and to a traditional design but because of inadequate control of the construction process, they were not built to the necessary standard. Whilst only a few houses had to be demolished, the rest had to be ‘retrofitted’ at considerable cost - though this was a much better option than demolition and reconstruction. Reconstruction or retrofitting was essential, given SC’s obligations to the inhabitants, as well as to other stakeholders. SC then decided both to honour its undertaking to build 1000 houses and to reduce the risk of further setbacks by importing approximately 400

8 In relation to comments received from SCUS on the draft of this report, we fully appreciate that it is hard for any agency to decide that it is not appropriate for them to implement housing programmes when donors are offering funds for this. SC may decide it is impossible rather than hard. We do not pretend that these questions are easy, only that they should be asked.

32

prefabricated houses from Canada, despite the comparatively high costs involved. To have continued with the original programme would have been a high risk strategy, and in terms of construction quality and losses to contractors, the risks were greatly reduced. Other hazards remained, particularly since the new houses have been largely built on relocation sites where lack of certification of ownership of land and/or housing may prove to be a problem if left unresolved.9 By the end of 2008 SC should have built 42 schools (19 of which have required retrofitting), 18 health facilities and four Community Learning Centres (CLCs). We see this part of the programme as a more successful, appropriate and cost-effective intervention, given the number and nature of the beneficiaries. Summary of SC construction programme Facility constructed Budget Target

number Actual as at 30/5/08

Estimated unit cost(1)

Estimated beneficiaries(2)

Domestic housing $12,8M Traditional houses 577 577 $4810 2,500 Retrofitting of trad houses 498 498 $5900 (included in above)

Reconstruction of trad houses 79 79 $4810 (included in above)

Prefabricated houses (traditional design)

448 167 $20,500 1,900

Schools $8.7M 42 35 $208,000 (average) (3) 11,000

Retrofitting of schools 19 19 (included in

above figure) (included in above)

Health facilities $2.2M 18 9 $122,000 (average)(4) 62,000

Retrofitting of health facilities 4 0 (included in

above figure) (included in above)

Community Learning Centres $206K 4 4 $52,000

(average) 172

(1) Unit construction costs include labour but do not include other overheads. (2) Beneficiary estimates are based on programmes when completed. They are only approximate in the case of domestic housing, where no detailed research into occupancy has been undertaken. An average of 4.3 inhabitants per house has been assumed. (3) Unit costs ranged between $50K and $577K. Of 35 completed schools, 29 had been handed over. (4) Three different types of health facilities were involved. Unit costs ranged between $45K and $468K.

9 Houses rebuilt on site, as during the first part of the SC housing programme, also generally lack certification and documentary proof of ownership, but in these cases the problem is less serious because there is less likely to be any controversy or dispute about such ownership...

33

The construction programme cannot yet be judged to have been effective, since until 2008 it has been consistently under-spent and is currently having to move very fast to catch up – though it appears to have a good chance of reaching nearly all of its targets by the end of 2008 and or at least in early 2009. The programme can still only be judged effective if it can be established that occupancy and ownership of the houses on the relocation sites is satisfactory, and if outstanding problems with water, sanitation and drainage on some of these sites are resolved.10 Although overall the programme has been behind schedule, it has been catching up in recent months and there is a reasonable chance that it will be completed as planned in 2008, or very largely so. It is important to recognise that the effectiveness of a construction programme is not however primarily a question of whether buildings are constructed properly and on schedule, but depends on how effectively the buildings are used in the long term. For some of the programme it is still too soon to say. For most of the housing and virtually all the institutional buildings there is not a problem, but for the balance of the housing programme it is still too soon to say. Overall efficiency has been very low, especially in relation to the housing programme, even given that re-housing costs per beneficiary are invariably high in comparison with most forms of rehabilitation assistance. Retrofitting more than doubled the costs of the first part of the housing programme, and also affected the schools programme. The cost of the prefabricated houses was three or four times that of most of the aid-funded re-housing projects, which were themselves expensive in relation to the local economy.11 Additionally poor management and control of local contractors led to significant losses in the first phase of the programme. The impact of the construction programme is positive in that both domestic and institutional buildings are needed, though more research is needed into the second phase houses if it is to be established whether they are occupied by those in most in need. In any society the provision of very valuable assets needs to be carefully controlled and regulated if the process is to avoid being prey to local vested interests of one kind or another. This is even truer in societies like Aceh where legal and institutional frameworks are weak. In other words, it is extremely difficult to establish a level playing field for housing provision in relocation sites. Impact here could only be ascertained by research into how and why beneficiaries have been allocated. The appropriateness of the construction programme can be considered from different perspectives:

• Are the interventions appropriate as regards the beneficiaries? • Is this an appropriate type of intervention for Save the Children?

10 These possible barriers to achieving programme objectives are greatest in the second phase of the housing programme, i.e. involving prefabs and relocation sites. These issues are insufficiently recognised, partly because the progress on this second phase is viewed with understandable relief. SC has a responsibility to ensure that these problems are resolved, even where it is not the implementing agency for their resolution. 11 Whilst many features of the houses were of a higher standard than would generally be found in Aceh, this does not negate the overall point. In comments received from SC, it was pointed out that competition between local contractors also increased the cost (and so reduced the efficiency) of the construction programme, and whilst this was true it was not as significant as the other factors outlined here.

34

From the local perspective SC institutional construction is appropriate and comparatively cost-effective. The first phase of the re-housing programme, involving reconstruction on the homeowners’ original site, was also appropriate in relation to its beneficiaries, though this is less clear for the housing on relocation sites.12 The decision to build in wood and follow traditional designs was controversial, because most people prefer brick and concrete houses which are seen as more modern. Against this it can be argued that, if properly constructed, traditional housing design provides better defence against both floods and earthquakes. The design of the houses was not well adapted to modification or extension, and in May 2008 not all problems relating to water supply, sanitation, and drainage had been resolved.13 Save the Children acknowledges that its lack of experience in managing and subcontracting construction projects was a major factor behind the first phase houses failing to be built to the proper standard. Poorly managed local contractors cut corners, or worse, with serious consequences. The solution, however, is not only the upgrading of technical and managerial expertise in construction. Though this is certainly needed, it would need to be combined with much greater and more knowledgeable engagement with the local community, whose involvement is crucial if safeguards are to be provided against mismanagement of such work. In cases where homeowners were directly involved in advising on and overseeing the work, outcomes were seen as more positive.14. We would argue that there is no overriding reason why SC should have felt obliged to become involved in domestic reconstruction, even when faced with needs as great as in Aceh, and despite the ready availability of funding. Many agencies are better placed to undertake this work15 and there are strong arguments in favour of SC prioritising its areas of specific competence – a comparatively specialised niche where SC often plays a leading role. A very understandable response to the difficulties experienced in Aceh has been to propose that the solution is for the SCA worldwide to upgrade its construction capacity. Indeed this has been projected as a ‘Big Idea’ arising from the Aceh programme (see Section 6 below). To achieve this would require a considerable investment in time and personnel – greater than is envisaged in the preliminary formulations that had been tabled at the time of the evaluation. As a specialized agency with unique specific competences, SC should give thought to how much this is a priority area for its involvement. The starting point should be a survey of needs in the field of institutional construction (eg schools and clinics) as well as consideration of SC’s future involvement in domestic reconstruction. A key question is the extent to which construction can be 12 With reference to comments received from SCUS on a draft of this report, there is no suggestion that SC should have avoided involvement in relocation sites. On the contrary, families who were unable to rebuild on their original plots were amongst the neediest. 13 It was by no means possible to obtain a comprehensive overview of SC domestic reconstruction sites, but such problems clearly remained in those that were visited. 14 SCUS has commented that “this is probably true for 99 other countries in the world - but not in Aceh”. This is an important reflection of the difficulties of community involvement in a society that has been fractured by conflict and mistrust, with weak institutional capital, but does not invalidate the point. 15 This indeed proved to be the case in Aceh. Those people interviewed who were in a position to provide an overview did not support the view that these problems were equally applicable to all or most of the agencies involved in domestic reconstructions. This should not be seen as a particular criticism of SC, but as a reflection of the fact that different agencies have different specializations and experience.

35

undertaken by or outsourced to other agencies, and the extent to which capacity needs to be in-house. Even if direct implementation of construction programmes is avoided, SC managers would still benefit from good guidelines and training for involvement in this field. The sustainability of the institutional construction, and of the houses where owners’ rights are assured, is good in the sense that the buildings provide a valuable long-term asset for the beneficiaries and their communities. The sustainability of the programme as a model for intervention is of course much less certain. Problems of coherence are not so much locally based but relate to whether such work really fits with SC’s more specific priorities. As argued with other programmes, geographical coverage is reasonable, though the low overall efficiency of the programme is relevant here. Objective 2: Health Save the Children health work in Aceh focuses on neo-natal care and the promotion of good health and nutrition for young children, new-borns, and pregnant and lactating mothers living in rural areas. The programme aims to provide support to existing government health services in a manner that we found to be practical and realistic, striking a reasonable balance between innovation and the recognition of local realities. The current programme focuses mainly on upgrading the skills of community-based volunteer health workers (cadres) and other community volunteers. SC documentation categorises the work in a different ways but the table below summarises the main post-emergency phase health interventions. The construction of health facilities described above can also of course be considered a component of the health programme.

Health programmes Beneficiaries to 30/5/08*

Posyandu16 revitalisation (children, mothers, cadres)

50,905 (55,036)

Polindes (health facilities services) 15,001 School-based health and nutrition 3,190 Capacity building for health care Providers 390

Training of cadres (Siaga) 242 Immunisation programmes ( National Immunization Day) 129,882

Treated bed net distribution 17,434 Reproductive health promotion, including seminar 1,068

Breast and cervical cancer seminar 283

16 Rural health services in Indonesia are based on the posyandu, a village health system, operated by the volunteer cadres, ; the polindes, primarily a village-level maternity facility; pustu, or sub-district health centre, serving several villages; and the puskesmas, or district clinic.

36

IOM Programme (midwifery training) 423 (697)

JHPIEGO Programme - Gampong Siaga (midwifery promotion and training) 16,561

* When programmes have been completed, and even sometimes when they have not been, beneficiaries claimed in SC documentation are identical to beneficiaries planned, which illustrates weaknesses in monitoring rather than strengths in planning. Where programmes have not been completed, the planned beneficiaries are given in brackets. These figures do not differentiate between direct beneficiaries such as trainees and presumed beneficiaries such as the population covered by a health post, though this can be deduced by the context. Aggregating these figures however is not very meaningful, and makes assessment of ‘targets against actuals’ particularly problematic. Even if the exact degree of correlation between plans and outcomes is not entirely credible, the effectiveness of most of this work appears to be very good, particularly the work with volunteer village health workers – including the parallel programme run by IOM – and around posyandu revitalisation. Efficiency appears to be good given the focus on training and the involvement of volunteers, though is hampered here as elsewhere by the comparatively high costs of overheads and logistics. Impact is difficult to assess in the absence of any systematic initiatives to study changes in health status, despite numerous efforts to provide some baseline data, but there is plenty of circumstantial evidence that training of volunteers and health workers has led to improved practices and that it is clear that the SC interventions are extremely popular. Although the training will undoubtedly leave a legacy which will be partly sustainable, it is unlikely that the district or provincial government will be in a position to continue to support these programmes in the future. Although they are happy enough with the initiatives, they will not have the funding nor capacity to do so. There are presently no alternative local actors to carry this health work further (the NGOs SC has supported are both international agencies). SC health work that may continue on a reduced budget after December 2009 will no longer have to work in the tsunami-affected coastal districts and the health needs may generally be greater inland. SC’s effective training work could be maintained, however, with quite limited inputs and mentoring. This should be given serious consideration. SC’s health work is appropriate, with children in rural areas as primary beneficiaries, although more specific economic targeting is not practical. Community participation is high, and although child participation is low, this is not surprising given that we are talking here of young children. Also appropriate is the liaison with government in this field, though positive feedback here should not be mistaken for capacity to sustain the work. The work is coherent with other parts of the Aceh programme. Coverage faces similar factors to the rest of the programme, but improved practices amongst health workers can benefit large numbers of people.

37

Objective 3: Education Aceh’s educational system suffered from the conflict in a particularly disproportionate and dispiriting way. As easily targeted government buildings, hundreds of schools17 were destroyed during the conflict, either by the rebels, or (it was often claimed) by government forces wanting to discredit the rebels. This led to prolonged periods of school closures and teacher absenteeism. The damage wrought by the tsunami is only part of the problem – overall a smaller part. The case for providing as much continuity as possible for this programme beyond 2009 and/or when tsunami funds are no longer available is reinforced by such considerations. Save the Children defines its specific educational objectives in Aceh as being:

• Increased children’s access to formal and non-formal education • Enhanced quality of education • Improved stakeholders’ capacity and ownership of education • Strengthened awareness of child rights’ issues and implementation of child rights

legislation. SC’s educational work in Aceh is wide-ranging and is subject to slightly differing characterisations, but includes: • School construction (considered under Objective 1 above) • Basic education, comprising:

o Development of physical facilities in schools (latrines, hand washing facilities, play areas, desks, chairs, teaching and learning materials, etc)

o Teacher training (including Training of Trainers) in student-centred active learning methodologies

o Support to curriculum development through competency-based curriculum methods

o Training of school principals and education supervisors for more effective supervision and coaching of teachers

o Strengthening of school management through school-based management training and the establishment and/or activation of School Management Committees.

• Inclusive education (mainstreaming physically, mentally, and developmentally challenged children)

• Adolescent learning (pre-vocational training) • Early Childhood Education (pre-school care) • School Health and Nutrition (from 2008; revitalisation of school clinics; reintroduction

of health and nutrition to curriculum). Estimated beneficiaries Category Sub-category

Teachers trained to date (targeted in current plan)

Child beneficiaries (targeted in current plan)

Basic education Active learning 253 (300) 13,195 Arithmetic training 50 (4 Male, 46 Female) 3 R training 50 (8M, 50F) Active teaching learning method 748 (128F, 620F)

17 There are no précised figures and considerabledebate and controversy over precisely what took place.

38

New Teacher’s training 107 (43M, 64F) Training of Trainers 60 (22M, 48F) Inclusive Education 82 (101) (1,268) School Management Systems 300 SMCs + 60 Principals 13,195 School Health and Nutrition 19 2,588 Adolescent Learning 200 (401) 4,700 Early Childhood Development 283 2,617 Although these wide-ranging educational activities were well represented on the evaluators’ programme (with the exception of Inclusive Education) it should be pointed out that last four mentioned are new programmes whose evaluation would be premature, though some comments can be made. The construction component which takes up the greater part of the budget has already been considered. This section mainly relates to the basic education component of the programme. The basic education programmes have been largely effective in meeting their objectives, which have been realistic enough in relation to the resources available, and have been acceptable to government and welcomed by the school teachers. Children’s participation appears however to have often been less than planned.18 School Management Committees, and school management generally, seems to be highly variable. The projects have been comparatively efficient, apart from the usual caveats relating to the high overheads of salaries and transport relating to an INGO like SC: as with the health work cost, could be cut if greater attention was given to logistical planning and in the longer term to geographical concentration. The fact that many teachers work as volunteers or in return for minimal support from the community increases the efficiency of much of this work. The evaluation witnessed some startlingly high impact from the teacher training of this programme, though we must not extrapolate too much from one phenomenally impressive individual.19 Nevertheless this was an immensely encouraging encounter and did illustrate the potential of this work. The impact of filling much-needed gaps in school equipment would also appear to be high. Impact on school management would appear less certain. The Inclusive Education programme, which includes an advocacy component, would appear to have some real potential, and the Early Childhood Education to be responding to a genuine need. This is less clear in the case of the Adolescent Learning Programme. Questions of sustainability are broadly similar to those affecting SC’s health programmes. Whilst the legacy of the training will be real and long-lasting, it is unrealistic to suppose that the government will continue to give it any significant practical support, even at a mentoring level. However, even a minimal level of support or mentoring by SC could greatly prolong and maintain the value of the intervention and increase its impact. Where support has been more direct, as in the case of the Early Childhood Education, the fall away may be sharper – though it might actually be possible in this case to obtain ongoing government support, provided time and energy are invested in negotiations.

18 With reference to comments received from SCUS, it should be emphasized that this is a finding of the evaluation about what took place, not a value judgment on the degree of participation that would be appropriate. It is clear that different members of the Alliance have differing perspectives on questions of child participation, which is not surprising but which needs to be acknowledged. 19 Nor perhaps worry that her widened horizons are not shared by her head teacher.

39

Plans are in hand to maintain the education programme at a substantial level until 2010; it is to be hoped that funding can be found for this. Educational work targets children, so immediately meeting a key criterion for appropriateness, and with high school attendance rates, in primary schools at least, children from poorer families, or at least younger ones, are unlikely to be excluded. Current plans are to continue education work in the coastal provinces and whilst there are arguments for this in terms of continuity, especially if budgets have to be reduced, there is little doubt that economic conditions are worse in the interior. Other SC plans20 do indicate a geographical shift to the interior which may raise issues of coherence and coverage; otherwise these are in line with the rest of the programme, and with the realistic and pragmatic policy of fitting in with government policies whilst trying to introduce more progressive agendas. Objective 4: Child Protection The Child Protection sector of the Aceh programme includes a number of current and former initiatives, not all of which could be researched directly.21 Programmes not normally associated with Child Protection such as Early Childhood Development (formerly) and Non-Formal Education/Vocational Training (currently) have sometimes been managed as part of the Child Protection mandate, as a substitute for more standard Child Protection initiatives. The sector has been prone to policy changes associated with changes of management and has lacked consistency of approach over time. Along with some smaller initiatives the main programmes have been: • The provision of safe play areas in the aftermath of the tsunami. These have now

been handed over to communities for a variety of alternative uses.22 • Support for upgrading and managing the separated children’s database of the

Department of Social Welfare, an outgrowth of the successful work of Family Tracing and Reunification. Though difficult to assess, this appears to be a valuable initiative.

• Non-formal education and vocational skills training: the ENABLE23 programme run

through Community Learning Centres (CLCs). This seen as an initiative against child labour, and as such an important component of Child Protection. The scheme has trained 123 teachers in 28 CLCs, four of which have been built by Save the Children. This has been the largest component of the programme in Aceh, and is clearly meeting a need generated by the disruption to the education system following the long years of conflict. It comprises a major commitment which may be difficult to sustain.

20 E.g. those presented in Mike Novell’s PowerPoint of March 2008. 21 A good deal of Child Protection work is undertaken in Nias, though can access tsunami funding. 22 There is a report on this programme by Jurist Tan (August 2007) but it is difficult to know if the author’s assessment is positive or negative. 23 Enabling Aceh to Combat Exploitation through Education.

40

• Child-led research into the activities at pantis (children’s homes) and dayahs (Islamic boarding schools). After the tsunami, the institutionalisation of children who could have been settled in family homes is a major rights issue. Despite the potential sensitivity of the subject, initial response from the government to this initiative has been positive, though this programme is still at an early stage. The potential impact and significance of such advocacy work may be very high.

Child-led Disaster Risk Reduction is also considered to be part of the Child Protection mandate but as it is also a ‘Big Idea’ it is considered under Objective 6 below. Most of this work appears to have been effective in its own terms, though more recently the budget has been under-spent. Effectiveness is too soon to judge in the case of the comparative recent initiative of the children’s homes. Efficiency would seem to be reasonable – it is worth noting that advocacy work, if successful, can often be an extraordinarily efficient way of utilising funds. (The same can also be said for successfully institutionalising a self-sustaining system within a government ministry.) The impact of the non-formal education work is probably improved by the high motivation of the students; that of the safe play areas may be particularly difficult to assess. The potential impact of reforming practices in children’s homes has already been noted. There is a question mark over the sustainability of the ENABLE programme. Although the initiative is fully supported by government policy, it seems unlikely, though not impossible, that government will have the funding, capacity and motivation to maintain this programme as and when SC withdraws. However it would be worth attempting to negotiate an agreement for the continuation of this work. The sector has often struggled to establish what would be the most appropriate form of intervention and has lacked coherence, having been subjected to changes of direction from internal and external factors. The inclusion of the ENABLE programme under Child Protection is the most striking example of this. Without such rather arbitrary categorisation coverage would have been quite limited. Coverage appears to have been reasonable. Objective 5: Livelihoods Early interventions in the livelihoods field included cash for work programmes (which benefited an estimated 20,000 people, working 109,000 person days), a mangrove planting programme (18,000 mangroves planted), and an Economic Recovery Assistance Programme (essentially an asset replacement programme) with an estimated 9,000 beneficiaries of grants and loans. An internal report on the asset replacement programme concluded that although appropriately used, many of the grants and loans were too small for their purposes, were paid late, and that few of the loans were repaid. Several SC staff expressed the view that these programmes unduly prolonged the relief phase, reinforcing the ‘handout mentality’ that is a notable feature of post-tsunami Aceh and making it more difficult for development initiatives to take root, particularly in microeconomic or microfinance fields. Our focus here is on the subsequent livelihoods programmes in Business Development Services (BDS), working in Pidie and Pidie Jaya with the Pidie Pandan Women’s Empowerment Programme in collaboration with Pekerti, a Jakarta-based NGO; and with

41

the Microfinance (MF) projects working in Lhokseumawe, Aceh Utara and Bireuen with Afdhal, a locally based NGO that was effectively established by SC but is now running fairly autonomously. Both these programmes have been the outcome of extended planning initiatives. In many ways they comprise the closest SC has come in the Aceh TRP to mainstream models of developmental work, in sectors with considerable impact potential. As such they are of particular interest for the future of SC’s work in Aceh. Pandan is from fibrous leaves of the pandanus plant that grows wild and is also cultivated in Aceh. The leaves are used for weaving traditional mats, which is considered “women’s work” for extra household income. Pandan weaving is part-time for personal/family use or for sale; although the project has shown impressive increases in income and in productivity. There is no comparative economic analysis to indicate the extent to which this remains a long-term economic option, nor a serious attempt to assess how much the improved incomes are the result of the de facto subsidies in marketing, transport and other forms of support provided by Pekerti and SC. Improved incomes remain comparatively low in relation to the predominantly agricultural local economy and it has to be questioned whether the return on this investment will really be significant – though there is little doubt that there will be some benefits for those who have improved their techniques and learnt to cut their costs in what is likely to remain a part-time activity. It is far too soon to know whether the cooperatives established by this project would be able to survive without external support, whose withdrawal is being planned too soon. The microcredit schemes run by groups of women with branches in Lhokseumawe, Matang and Lhoseukon seem to be on a firmer basis, though a recent external review questions some of the admittedly outstanding achievements claimed for these initiatives. It would again seem that the timeframe for the withdrawal of external support is overly optimistic. A revealing, and encouraging, ruling of the project is that SC staff may only visit on motorbikes, never in vehicles bearing SC logos – symbolic of INGOs, wealth, and handouts! Both programmes claim to be largely meeting or sometimes exceeding their current targets, and so to be running effectively, though both of them, especially the BDS, underestimate the extent to which this is probably a result of the effective subsidies provided, especially in transport and marketing. It is too soon to judge the efficiency of these programmes, which (contrary to some the documentation) are still some distance from being in a position to stand on their own feet. It is also too soon to assess the impact of these projects, but the economic impact of the BDS programme appears to be quite low,24 whilst the potential of the microcredit scheme is very high, though currently limited by inflexibility in procedures, notably over the size and scheduling of loans. The sustainability of these programmes will depend not only on their inherent viability (which in the case of the microcredit programme appears high) but on support being maintained for a sufficient period of time, which currently seems to be seriously underestimated. 24 This is based on information from SC’s internal reports as well as from interviews.

42

The microcredit work appears to be highly appropriate, and the BDS programme fits well with existing practices and activities. Targeting of women is appropriate, with children (especially younger children) benefiting indirectly, though targeting on economic grounds is restricted since these schemes are open to all – though poverty targeting can be done to a limited extent by focusing on rural areas. The coherence of these programmes with local social realities and with other SC objectives is good, though it should be remembered that the local implementing structures have been created by external agencies rather than by local civil society. If the programmes prove to be viable and replicable their coverage could prove extensive. ‘Big Ideas’, Disaster Risk Reduction, the Construction Technical Support Unit and more This section assesses parts of the programme in Aceh not covered elsewhere, nor falling under the specific objectives already discussed. Some of these programmes were presented to the evaluators under the rubric of ‘Big Ideas’ but it later became clear that in Aceh what counted as ‘Big Ideas’ was itself a matter of debate. This is not important in relation to the success or otherwise of the work, though it does indicate that the ‘Big Ideas’ concept was interpreted differently in different parts of the Alliance and indeed within SCUS. DRR was the largest component of this work and of the work generally considered to fall under the ‘Big Ideas’ rubric. Most DRR work in Indonesia was undertaken outside Aceh, notably in Nias, and so fell outside the scope of the evaluation - even though much of it was able to utilize tsunami funding. DRR work in Aceh itself has been based in Lokhseumawe, although unfortunately the evaluators were unable to witness this work in action. However it was clear from discussions with staff and managers responsible that this programme was limited in scope and delayed in its implementation. DRR work in Aceh falls under the Child Protection unit where management problems appear to have contributed to a lack of ownership of the programme by SC staff The links between getting youth to think through the risks facing their communities and the practical actions that can result have not yet taken shape. The CTSU proposal, whose inclusion under the ‘Big Ideas’ umbrella was not in dispute, derives in large part from the recognition that SC lacks technical expertise in managing large scale construction projects, as the Aceh experience has illustrated. The solution proposed is a technical support unit with the capacity to help support and resolve construction issues wherever they arise around the world. It is important that any such unit is based on a careful assessment of the extent to which SC is and should be involved in construction projects, in what types of construction, and whether as an implementing agency. It should not be based unduly on a reactive response to events in Aceh, even though this comprises a legitimate starting point for the proposal. This does not mean there is no case for a CTSU. Even if SC were to avoid future involvement in domestic reconstruction, it will need capacity to build schools or clinics – though the skills required may often amount to managing tenders and subcontracts. SC will doubtless remain involved with temporary housing and certainly with shelter, and there may be need for support capacity here.

43

We believe, however, that construction problems should not be seen only in technical terms. In Aceh the problems were resolved by the ‘technical fix’ of importing prefabricated houses (and bringing in a very competent engineer). Problems in Aceh included understanding and managing relationships with local communities, contractors and other stakeholders. If Save the Children is to upgrade its capacity to implement construction projects successfully it should approach the question of construction holistically, and not primarily as one of improving technical capacity, important thought that is. It might for example be worth renaming the CTSU the Construction Support Unit (CSU) and ensuring that it incorporates or is properly linked to the managerial and analytical skills (social, political, economic) which are integral to the success of construction work.25 The unit would in any event need much greater resourcing and capacity than on current (admittedly only preliminary) blueprints, which centre round an individual based in Aceh.26 The ‘Wellness’ initiative, aimed at training managers how to help SC staff avoid overload and burnout also deserves mention. This is part of a global SC initiative, which appears to be well worth while, though it is not possible to provide a detailed assessment. In Aceh this was seen also seen as falling under the rubric of ‘Big Ideas’. Strategic and management issues There is no doubt that Aceh provided the most difficult working environment of all the countries covered in this evaluation.27 Thirty years of conflict had greatly hampered social, political, economic and institutional development of both government and civil society. Social capital was fragmented and the rule of law was weak. Tensions between government and society were often refracted along ethno-linguistic fault lines. In the early months of the crisis security restrictions for international agencies were considerable. Save the Children’s almost continuous work in Aceh from 1976 provided it with some genuine advantages, notably through staff links to government and opposition figures – the latter sometimes becoming government figures following the peace agreement and the elections in 2005. Some staff stayed on with the agency or returned, invaluably, from having worked with the agency in earlier years. As in Sri Lanka and India, however, the advantages of a pre-tsunami presence were largely overwhelmed by the radically new situation which followed – not only by the new realities facing the society as a whole but by the size and the nature of the new emergency programmes of the international agencies and the consequent institutional imperatives and working arrangements they adopted in addressing a situation of unprecedented scale, opportunity and challenge.

25 This is not to imply that construction projects need to be ‘community based’ - only that they need to take the wishes and realities of the local community (political, social, economic) fully into account. 26 In view of the comments made on the draft of this report, we realise that our assessment of the CTSU may not be accepted by SCUS. Although we have taken these comments fully into account and have modified our text considerably, some disagreement remains probable. We feel that the CTSU is a significant outcome of the TRP and hope that the analysis presented here will be considered very carefully. 27 Somalia perhaps provides a different kind of special case.

44

Familiar problems appeared: a big influx of new staff (though in Aceh, mainly because of the security situation, these new staff were comparatively slow to arrive) followed by a very high rate of staff turnover, in some cases accompanied by overwork and burnout. In this context it is worth noting that the comparative isolation and homogeneity of Aceh meant that staff arriving from other parts of Indonesia often shared with expatriates the experience of being outsiders. Opportunities for following Save the Children’s preferred method of working with partner agencies were limited: the NGO and civil society sector in Aceh was extremely weak; government ministries were disorganised, society was fractured, patronage was rife, and cooperation was at first confined mainly to collaboration with other international agencies. Most of these were adopting an operational approach and several, including Save the Children, were under pressure to extend their traditional mandates. In areas where it had experience and expertise, notably for example in child tracing and family reunion, SC was able to work very well, not only in the early phases and throughout the first year of the emergency (in cooperation with UNICEF and others), moreover leaving a valuable legacy for future emergencies. Other strengths included the adoption of a pragmatic and realistic approach to working with government: recognising the strengths, weaknesses, limitations, opportunities and necessities of this relationship. We argue below that the main component of such collaboration, in health and education, has been fully justified. More recently SC has worked with national and local NGOs; these relationships have generally been successful if limited in extent. More recent relationships with international NGOs such as JHPIEGO,, IOM and the Helen Keller Foundation have been justifiable and effective in the absence of a viable local NGO sector. We recognize the factors that encouraged Save the Children to believe it should implement a major domestic reconstruction programme in Aceh. We have set out the problems associated with this programme, and believe these were broader than the limitations of technical capacity which SC acknowledges - important though that was. We believe that such problems are likely to recur, especially in emergency response programmes where the capacity for engagement with local social realities may be limited. Such findings may not be welcome, but we believe they merit serious consideration. The construction programme apart, the evaluation found most programme priorities to be reasonable, though some sectors faced management difficulties, especially during the early phases of the programme. Education and health work was generally well conceived and planned, though here as elsewhere policy changes (sometimes associated with changes of senior management) were too frequent to allow for proper project cycle management. This in turn means that although a good deal of time has been devoted to planning – in which context the MOS framework has generally been found to be useful and effective – little real evaluation has been undertaken. In this all too familiar context impact can be difficult to assess. Advocacy has been on the agenda of the Aceh programme, with successful interventions at governmental level, but given that the environment is comparatively

45

open – if not always very receptive – it would seem that higher priority could be given to this aspect of SC’s work. Advocacy should continue to be focused on the government. In a cultural environment where responsibilities tend to be task-oriented and rule-bound SC has struggled to develop a participatory culture which develops motivation and creativity and promotes the flow of ideas from the bottom up as well as the top down. There may be no quick fixes to this question but SC needs to be constantly sensitive to promoting the genuine empowerment of local staff if local ownership of its programme is to be promoted. The situation is reinforced by the concentration of local staff in district offices and expatriate staff in Banda Aceh and by the high correlation between expatriate staff and senior and sectoral management posts. Different geographical configurations of staff might change dynamics and reduce stratification – the new phase of the programme will increase opportunities here. There is in any event a need to clarify the managerial versus advisory responsibilities of the sectoral managers; wherever possible, managerial responsibilities should be devolved and advisory functions emphasised. The parallel system of sectoral and regional management puts a particularly high premium on systematic communication. Overall the Aceh programme has undergone complex and changing management relationships with the Indonesia country office in Jakarta, with the Asia tsunami regional office, and with SCUS and SCUK head offices. The details of these are beyond the scope of this evaluation, but have made competing demands and pressures on management and can make it more difficult to root policies firmly in local experience. The scale of the programme in Aceh in itself presents an enormous challenge since budgets will reduce to a fraction of current levels at the end of 2009. The evaluation supports efforts to raise funds to maximise continuity in the programme but the phase down will still be drastic and all efforts should be made to avoid starting it too late and finishing it too early. Where possible, ways of investing tsunami-related funds in ways that effectively prolong their life post-2009 should be investigated, though it would appear that opportunities for this may be less in Indonesia than in other countries reviewed.28 SC does not need an exit strategy from Aceh but a continuation strategy combined with a radical downsizing strategy.29 Any evaluation of the TRP in Aceh must take account of its aftermath. There is every indication that SC intends to prolong its work in Aceh well beyond 2009. Long-term planning in Aceh will need to take the legacy of the long conflict fully into account. Even if renewed conflict can be avoided, the post-conflict environment will continue for many years, constraining or distorting normal patterns of political, economic, social and institutional development. The implications will require continuous careful assessment. Recommendations

28 The reasons for this are varied and relate to legal situations in different countries and organisations; they are nevertheless worth investigating. 29 The challenges of the unified programme and of the changing relationship between Aceh and Jakarta are very considerable, though they are beyond the scope of this evaluation.

46

1) Save the Children should prioritise planning for the sharp decrease in funding both at the end of the construction programme and at the end of 2009, whilst seeking to secure funding to support the post-construction components of its programme as far as possible.

2) Where programmes have to be discontinued through lack of funding (e.g. after 2009), particularly those involving training, contact should be maintained through intermittent or limited training or mentoring. This would have benefits out of all proportion to the limited costs involved. This recommendation applies to all projects with a training focus in health and education, as well as to the livelihoods programme.

3) Save the Children should undertake research in the resettlement areas where it has provided housing, and should plan to monitor the situation and advocate its resolution for as long as is needed to ensure that questions of ownership of land and housing are satisfactorily resolved.30 Such research should also survey any outstanding problems where they exist, for example in relation to water, sanitation, drainage, or liability to flooding, and make plans for facilitating the resolution of such problems.

4) Whilst sensitivity is always required, greater priority should be given to advocacy work. This should continue as now to be directed towards government in areas where there is a realistic chance of some response.

5) Careful assessment should be given to the advocacy work on children’s homes, in order to achieve the projects’ objectives of reforming these institutions.

6) As a general principle, throughout its work in Aceh, greater consideration should be given to promoting children’s participation, and a greater emphasis given, where appropriate to incorporating dialogue on children’s rights into SC programming.31

7) An external evaluation should be undertaken of the current livelihoods projects. The evaluation should give particular focus to relationship of the projects to local economic realities. External support should be maintained to the current livelihoods projects for at least twice the period currently planned for these projects - the précised details to be determined by the evaluation in consultation with project managers.

8) Greater clarity is needed in defining and implementing the managerial responsibilities and the advisory responsibilities of sectoral managers in Aceh. In practice these are not always clear and are interpreted differently in different sectors.

9) The parallel sectoral/geographical management structure of the Aceh programme is necessarily complex. This places a high premium on systematic and transparent communication on policy issues and management decision-making.32

30 With reference to comments received, this does not of course imply that the resolution of these problems is entirely in SC’s hands, nor that SC is responsible for these problems in the first place. 31 This is a general principle designed to harmonize the Aceh programme with other SC programmes. We recognize that many in SCUS will be sceptical of such a recommendation, and that its impact on improving project implementation and securing objectives will be gradual and incremental. 32 A comment on the draft of the report asked for more specific project-level recommendations for Aceh. Whilst understandable, this may be to misread what is feasible or appropriate for an overview evaluation of this kind. More detailed project recommendations would need to be developed with project managers,, bearing in mind the more overarching recommendations presented here..

47

Please also note the general recommendations concerning work in construction, which are particularly relevant to the Aceh programme.

48

India Programme 1. Programme description Country Context and Impact of Tsunami With a population of more than one billion and an economy that has been growing rapidly in recent years, India is increasingly seen as a major global power. It is a world leader in areas such as information technology and pharmaceuticals and its government feels sufficiently confident in its own capabilities and resources that it now only accepts bilateral aid from a few favoured donors. In spite of this, India also contains a large proportion of the global population living below the poverty line (people living on less than $1 a day) and there are many communities, such as Scheduled Castes (SCs) and Scheduled Tribes (STs) who have not significantly benefited from the economic boom and continue to be marginalised. The tsunami of 26 December 2004 affected 2,260 kilometres of India’s coastline. The official death toll in India as reported by the National Disaster Management Centre was:

State/Union Territory Dead Missing Andaman and Nicobar Islands (ANI) 1,899 5,554 Andhra Pradesh (AP) 105 11 Kerala 171 Pondicherry 591 75 Tamil Nadu (TN) 7,983 Total 10,749 5,640

In addition, it is estimated there were 430,000 people affected by the disaster, of which 163,000 were children. Damage included the loss of 110,000 homes, 87,788 boats, 31,755 livestock and 358 schools destroyed or heavily damaged (Three Years On from the Tsunami: Rebuilding Lives – Children’s Road to Recovery, Save the Children 2007). The nature of disaster varied from one location to another, but in general, the tsunami in India did not reach a great distance inland (as compared for example to Aceh in Indonesia), affecting a very long, but quite narrow strip of land. Typically there was destruction on the beaches, the first row of houses might have been destroyed, the second row badly damaged, the third somewhat damaged and so on. Meanwhile a short distance behind the sea front, there was no damage, roads were usable and communication systems and government structures continued to operate. This subsequent relief and rehabilitation operation was on an unprecedented scale, involving local groups (goods collected by local temples, clubs etc were generally the first assistance to arrive), local NGOs, INGOs and the Government of India (GoI). Vast amounts of money were raised around the world in response to a unique combination of circumstances, which gave this disaster particular resonance. The sheer volume of money raised forced the pace and scale of the response, as agencies jostled to show by visible work how they were translating donations into assistance. In Nagapattinam District in Tamil Nadu (the most prominent affected area), there were reportedly 600-700 agencies working where there were 120 directly affected communities. Inevitably there

49

was much duplication of aid, with some households receiving multiple relief distributions – five was typical, 10 was not unusual and one report even went up to 20. There were also examples of inappropriate assistance given: an over-supply of fishing boats where fishing stocks were already being depleted, distributions of bottled water, ice boxes where there was no electricity and so on. The nature of this disaster therefore raised some highly unusual issues for responding agencies to address. In particular, how could an individual organisation such as Save the Children (SC) add value and provide meaningful assistance where there was such an abundance of resources and interventions from other actors? Save the Children Response in India The Save the Children response to the tsunami in India was complicated and needs some explanation. Essentially there were four SC entities involved: Save the Children (UK) has many decades of experience of working in India on long-term development work and also on disasters when they have occurred. SCUK was designated the lead agency for the SC Alliance Tsunami Response Programme (TRP) and received financial support from SC (Netherlands, Sweden, Finland, US) and other donors. SCUK had an office in Andhra Pradesh at the time of the tsunami, had previously had one in Tamil Nadu (but had closed it some years ago due to the state being seen as better-off) and had not worked before in the Andaman and Nicobar Islands. Save the Children (Sweden) did not implement any work on the ground, but played an important supporting role on capacity building and communications through its South Asia Regional Office in Katmandhu, Nepal. Save the Children (Finland) and Save the Children (Canada) decided to run their own Tsunami response programmes, separate from the main Alliance programme. The reason reported for this was that they themselves had to use their own government’s funding. However, they did coordinate and link with the SCUK programme, particularly SC (Finland). Bal Raksha Bharat (SCBR) is the recently established Save the Children national entity in India, which was already planned prior to the tsunami (and coincidentally was legally launched the day after the tsunami). It ran some programmes by itself following the tsunami using funds donated by SCUK and from April 2008 took over responsibility for all the TRP work.33 It is intended that this will be the sole Save the Children Alliance member operational in India (which other SC members will support) and it has already taken over all of SCUK’s operations. This evaluation was commissioned and arranged by the Alliance, with SCUK as the lead agency for the tsunami response in India. This report mainly reflects that programme, which constituted 85% of total SC expenditure in response to the tsunami. However, the itinerary for the evaluation included visits to partner projects of SC (Canada), SC (Finland) and SCBR. Most findings in the report refer to SCUK’s TRP but where relevant, 33 SCBR recruited many former TRP staff into its longer term regional structures (new offices in Tamil Nadu and ANI and the existing office for Andhra Pradesh). It is called ‘Bal Raksha Bharat’ since there was already a separate ‘Save the Children India’, which holds the rights to the name in India.

50

observations are also made on other SC members’ work. Where other SC members are referred to, this is indicated as such within the report; elsewhere it can be assumed that comments refer to SCUK’s TRP. 2. Summary of Programme Objectives and Key Findings As mentioned below, the planning process for Save the Children in India went through various phases as the situation evolved. According to the team in India, while there was an earlier plan, the most significant plan which was actually used was that for 2006. The programme objectives here are therefore (mostly) quoted from the 2006 plan: Overall Objective: To protect the rights of children, their families and their communities affected by the tsunami and to support their recovery and rehabilitation, and to reduce the vulnerability of communities to future disasters. Specific Objectives: Shelter: By end 2007, children and their families affected by the tsunami and recent floods are living in improved/upgraded accommodation that meets basic international standards using locally available resources. By end 2008, affected and vulnerable families have access to child-friendly permanent housing that is appropriate to the environmental context and can withstand future shocks in TN and AP. Early Childhood Development/Integrated Child Development Scheme (ICDS): Supporting and strengthening of early childhood services, care and development for children from 0 to 6 years (No overall objective for this sector is given in the 2006 plan, so this is taken from the 2007 plan). Water and Sanitation: By end 2006, children and families affected by tsunami have access to a secure supply of potable water, better access to health services and improved health through increased nutritional and health awareness. Child Protection: To protect and promote the rights of children in all tsunami-affected areas, creating a supportive and protective environment for children in the aftermath of the tsunami by end 2009. Education: To restore and improve the continuity and quality of education; and promote children’s participation in making the education environment child-centred, inclusive, violence free and disaster prepared. Livelihoods: By 2009, targeted households and communities in the program area are able to fulfil the basic developmental rights of children with the programme support given to restore livelihoods destroyed by the tsunami, and also reduce vulnerability and risks from future disasters. Emergency Preparedness: By 2009, children’s vulnerability to the impact of disasters and community dependence on external agency support are reduced. HIV/AIDS: To promote awareness of HIV that results in behavioural change among young people, reduce stigma and discrimination towards children and families affected by HIV/AIDS and provide community based care and support and access treatment. The work completed under each objective is shown in the annex. The key findings from this evaluation are summarised in the following table:

51

Summary of Findings of the Save the Children Tsunami Response Programme in India Sector Effectiveness Efficiency Impact Appropriateness Sustainability Coherence Coverage Overall Targets were

changed as programme evolved, but final targets were mostly met.

Relatively high salaries (expatriate and local) and other costs.

Little baseline data to go on, but mainly positive feedback from communities.

The chosen sectors and activities were appropriate under the circumstances.

Mixed. Many of the softer components will require longer term follow up for sustainability.

Good attempts to link with government and coordinate with other NGOs.

Good efforts made to reach Scheduled Castes/Tribes and communities less reached by others.

1. Shelter Less done than originally planned and some still incomplete.

Cost was quite high, except for flood response housing.

Housing visited was well conceived and implemented.

Rightly deprioritised in line with what others did and SC’s own capabilities.

Once completed, families are likely to maintain their own houses.

Generally good coordination with government and others.

Communities reached included many SCs/STs.

2. Early Childhood Development

ICDS targets mostly met – a major achievement.

Centres were expensive, but ANI was problematic.

ICDS centres appreciated, but more needed on usage.

Good – this was a sensible way to use funding to create community assets.

Govt will mostly support them in the future.

Good linkages mostly with relevant govt departments.

Selected communities included many marginalised.

3. Water This component was in the original plan, but there was no significant implementation.

4. Child Protection

Targets shifted, but ultimately were achieved.

Costs of this quite high since mostly soft components.

Many positive early indications of greater awareness.

Not particularly tsunami related, but appropriate given other needs met.

Time for inputs was short, so will require longer term follow up.

More work required to link to local governance.

Coverage reached a wide range of communities.

5. Education Stated targets were exceeded.

Costs of this were quite high given this was mostly soft components.

Better attendance and improved performance were evident.

Mostly not particularly tsunami related, but appropriate given other needs met.

This is likely to require more follow up for benefits to be sustained.

Well linked with government.

Evidence of good coverage of marginalised communities with less education.

6. Livelihoods Targets shifted, but mostly achieved.

Quite high for limited beneficiaries.

Improved incomes reported.

Highly appropriate given shock caused by tsunami.

If groups new, will require further accompaniment.

Mostly Self Help Groups (SHGs) linked to NGOs.

A wide range of communities benefited.

7. Emergency Preparedness/ DRR

Revised targets were achieved.

Costs of this quite high since mostly soft components.

Hard to tell unless there is a new disaster.

Highly appropriate in context of aftermath of disaster.

Likely to be quite challenging unless further follow up.

Linked to schools and village committees.

Reached many communities, although some not at high risk.

8. HIV/AIDS Activities started late, so less done than anticipated.

Costs of this quite high since mostly soft components.

Awareness raised, but more needs doing for real impact.

Not much tsunami related, but some level of risk is present.

A lot more work will need to be done on this.

Good linkage between NGO partners.

Reached many communities with some level of risk.

52

Financial Summary of SC TRP in India (all figures given in US dollars converted at $1 = £0.50092, the 31/12/07 rate from OANDA website) 2005

Budget [1]

2005 Actuals [1]

2006 Budget [2]

2006 Actuals [2]

2007 Budget [3]

2007 Actuals [3]

Actuals up to 31/12/07

Jan-Mar 2008 Actuals [4]

Apr 2008 – Mar 2009 Budget

A. Programme Inputs 1. Relief - - - - 2. Shelter 2,231382 431,097 960,858 591,100 3. Early Childhood Devt. 2,375,074 1,269,901 2,650,505 2,270,177 4. Water - - 5. Child Protection 1,063,224 899,746 785,842 737,535 6. Education 837,291 42,978 410,954 354,542 7. Livelihoods 1,774,788 960,658 997,678 963,471 8. Emergency Preparedness 434,854 514,212 610,451 512,026 9. Health and HIV/AIDS 181,867 144,532 236,357 173,525 10. Big Ideas 1,811,283 sub-total Prog. Inputs

[65%] 3,139,837

10,709,764 4,643,125 6,652,645 5,602,376 13,385,337

B. Non-Personnel 213,126 171,890 204,444 153,691 325,581 C. Personnel 1,500,180 1,286,327 1,761,495 1,604,989 3,664,199

[16%] 772,883

D. Personnel Support 486,700 483,728 605,22 530,426 1,014,154 E. Capital 207,568 249,124 24,423 27,547 469,892

[4%] 193,221

F. Other Direct Project Costs

115,328 144,073 262,140 224,501 465,184

[2%] 96,610

G. Management Costs 218,885 26,240 654,207

[13%] 627,967

TOTAL ALLIANCE TRP Tsunami Rehab. Prog.

7,784,908 4,830,518 13,232,666 6,978,266 9,729,254 8,169,770 19,978,553 1,307,594 1,884,532

Sources: [1] Holding Steady, Annual Report 2005; [2] Save the Children Alliance Annual Report 2006; [3] Annual Report January-December 2007; [4] Figures from meeting with TRP Finance Manager.

53

Financial Summary of all SC Responses in India

2005 Actuals

2006 Actuals

2007 Actuals

Actuals up to 31/12/07 (original currency)

Exchange Rate $1 =

Actuals up to 31/12/07 (US$ estimated equivalent)

SCUK TRP: £sterling 2,419,703 3,495,553 4,092,401 10,007,657 0.50092 $19,978,533 SCBR: INR 7,889,033 19,814,469 27,703,502 39.43500 $702,510 SC (Canada): CDN$ 1,000,401 798,995 185,414 1,984,810 0.98201 $2,021,171 SC (Finland): Euro 377,652 258,036 635,688 0.67942 $935,633 TOTAL ALLIANCE TRP $23,637,867

Note: Figures from same sources as previous table. Exchange Rates taken from OANDA Currency Converter on internet, as at 31/12/07. Overall the India tsunami response was the largest programme in financial terms that Save the Children has carried out in India. In the context of the global Save the Children tsunami response, India represented slightly less than 10% of the total funds raised (in excess of $260 million), which makes sense in proportionate terms. The main part of the programme (85%) was spent through the Alliance TRP, with SC (Canada) spending 8%, SC (Finland) 4% and SCBR 3%. All SC members implemented their work through local NGOs and were not directly operational. The funding for SCBR (£358,000 or $715,000)) was allocated from the original Alliance TRP budget of £12 million ($23.9 million). For the rest, more than 90% has now been spent and of the remaining £944,000 ($1.88 million), most is already committed to, or earmarked for partner projects. 2.1 Other information Stakeholders/Mandates International child-focused organisations of particular importance were:

• UNICEF (with whom SC collaborated on some activities like ICDS centres) • Plan International (to which SCUK channelled some DEC funding) • Tamil Nadu Tsunami Resource Centre – a coordinating body established by

UNDP with strong SC involvement as one of the three key initial supporters. There were a number of government departments of particular relevance:

• General governance functions such as the Officer on Special Duty for Tsunami Relief and Rehabilitation for Tamil Nadu, District Collectors

• Specific departments like the Department of Social Welfare (for ICDS and child protection) or the Department of Education.

Save the Children worked entirely through local partners on the Tsunami Recovery Programme. Some of these were already child focused, while others had previously had other orientations (e.g. more generally poverty focused).

54

Programme Phases The programme can be summarised in terms of the following phases:

• During the first three months SC worked on relief distributions both on the mainland and in ANI. A rehabilitation plan for 2004-9 was drawn up.

• In April 2005, a completely new team was recruited who had not worked with SC before. They found the plan they had been given unrealistic and there were considerably delays in mid 2005. Eventually a new plan was developed for 2006 (although some elements went beyond that year) and implementation began.

• In 2006, implementation of rehabilitation work began in earnest, although at a slower speed than anticipated. A further annual plan was developed for 2007.

• By the first quarter of 2008, the main reports for the TRP were completed for DEC. Further activities continued however up to March 2008.

• From April 2008, all the TRP activities were absorbed into SCBR which took over SCUK’s offices and the longer term staff and recruited a number of new staff from the TRP team. Likely sectors, projects and partners were identified by SCUK for 2008/9 and the remaining TRP funding was transferred to SCBR to support them. Beyond 2009, it is expected that SCBR will raise its own funds.

• The TRP retained a small core team in its Chennai office from April 2008 to wind up the programme, finish reporting and so on, which will close by August 2008.

Magnitude of the Programme This was the biggest programme SC has ever implemented in India and was considerably larger than the longer term work that was in existence by every measure. The original budget was £12 million ($24 million) for TRP, most of which is now expended. SCUK worked through 62 partners, and signed 134 partner agreements (i.e. some partners had several projects) in seven sectors. At its height, TRP had 75 staff (including a number of expatriates), three main offices and five sub-offices. By the end of 2007, it reported its programme had covered 650 villages in 11 Districts and two Union Territories, by work which benefited 229,687 households or 1,130,712 people. SCBR had six partners, SC (Finland) had two, while SC (Canada) had one. 3. Principal Findings and Recommendations This section assesses the programme objectives against the OECD/DAC criteria. This develops the Summary of Findings Table shown earlier. Objective 1: Shelter In the original plan, there were highly ambitious plans for 10,000 families accommodated in temporary shelter and 2,000 child friendly houses, but this was significantly reduced in the 2006 plan. Measured against even the revised plan, the programme can only be described as partly effective in that it completed some of what it had planned on temporary shelter and also completed 161 permanent houses, with another 302 still to be completed by Social Awareness Society for Youths (SASY) following floods in Cuddalore District. This latter project has been a struggle since SASY had little experience in this area and is probably more suited to rights advocacy, rather than construction work. In the early stages, SC was hampered by rains, floods and the lack of a qualified engineer to support the work and monitor the construction aspects. When one was eventually recruited, this facilitated the progress and quality of the work.

55

In terms of efficiency, the permanent houses in tsunami affected areas were quite expensive compared at Rs 180,000 ($4,500) to what had existed previously (and as compared to post-earthquake housing in Gujarat which typically cost 15-30% of that), but SC was constrained by guidelines laid down by government in this respect and they were less expensive than those built by many other agencies (up to Rs 400,000 or $10,000 per house). The houses constructed by SASY were in response to floods in 2005, not the tsunami and hence could be cheaper, although communities were not happy with this, indicating how local expectations had been affected. In terms of impact, the owners of houses constructed by the Society for Education and Village Action and Improvement (SEVAI) in Nagapattinam seemed very positive; with considerably better living spaces than they had had before and their own needs being taken into account in construction. This included ‘child friendly’ aspects like shelves put in where children could keep their books. This was helped by the fact that this project involved rebuilding houses in situ, which meant that the relevant households could be clearly identified and involved in the process. This was in contrast to construction at relocation sites (which SC wisely avoided) where houses were often only allocated by government after construction was completed, thereby making community participation impossible and resulting in less than 50% occupation in some instances, as fishing communities preferred to stay closer to their boats and the sea. It is likely that the houses built will be sustainable, in that families are happy to return to where they were living before and they are likely to look after their own assets. Already there were indications in households visited of embellishments added by families. The appropriateness of housing as a priority sector was questioned by SC itself in that it started off (as did many others) planning to carry out a major housing intervention and subsequently significantly reduced it to 10% of total partner expenditure. As many other agencies (including government) provided housing and SC had little experience in the area, this was justified, although there were gaps identified in the Disasters Emergency Committee (DEC) evaluation in temporary shelter, for which the agencies collectively were criticised. It could be questioned whether it was worth SC getting involved in shelter at all if it was going to do so little, but the team justified this in terms of developing models for a ‘child-friendly’ approach, which they could then use for advocacy. If SC is to work on shelter, then it would be worth looking at how to address temporary shelter in the future. Coherence was acceptable in that SC and its partners worked with government on the identification of sites and beneficiaries. Coverage was good in that many of the beneficiaries were from Scheduled Caste and Scheduled Tribe communities, whose needs and livelihood opportunities tend to be more challenging. However, it was observed that some of the beneficiaries were not particularly poor, with children at college, household electronic equipment and so on, but this was part of the nature of the tsunami, that those most directly affected were not necessarily the poorest. Objective 2: Early Childhood Care (Integrated Child Development Scheme) This sector was the largest of SC’s response, taking 27% of total partner expenditure. In the earlier stages of the programme, 25 ICDS centres were repaired and 36 temporary

56

ICDS centres constructed. While the highly ambitious original plan for new, permanent structures was not met, the 2006 plan target of 110 new child-friendly centres was exceeded with 122 centres actually constructed, so this component can be seen as highly effective in terms of the construction side (eventually, after the early delays). The total included 48 ICDS centres in ANI, which was particularly challenging given the sensitivity of working in areas of strategic importance to India and the logistical difficulties of working on islands where everything has to be imported. Because the priority had to be put on construction, there was less done on the usage of the ICDS centres, training of anganwadi workers and so on. With respect to efficiency, there is no doubt that the ICDS centres were of much better quality than what had existed before (and in some cases nothing had existed before). This came at a price – Rs 8.5 lakhs ($22,000) was quoted by SCBR-supported Thanjavore Multipurpose Social Service Society (TMSSS) in Tamil Nadu, while on the more difficult ANI, they cost Rs 15–20 lakhs ($38-51,000) – much more than ICDS centres normally cost when constructed in India. In terms of getting the job done, working with Andaman and Nicobar Islands Integrated Development Corporation (ANIIDCO) and the Department of Social Welfare on ANI was not easy as they were coming from a completely different background and had not handled a project like this before. However, given the sensitivities of working on ANI, this may well have been the only way to access certain areas. The impact of the work can be seen in terms of the fact that many communities now have valuable community assets which can be used for early childhood development. The actual construction generally seemed to have been of a good standard and took account of possible future disasters and so on. While the Auroville designed ICDS centre visited in Villupuram District was innovative in many ways, it did seem that the upstairs room was not used as the community did not find it congenial. The Department of Social Welfare reported that attendance at ICDS centres covered by the programme in ANI is up 45%, which is a significant achievement. Furthermore the Director of Social Welfare in ANI said that she very much liked the child-friendly style of the SC construction and would seek to construct further centres in the same style if she gets the budget. As the ICDS centres are part of a government scheme, there are theoretically workers allocated towards them. In some sites visited, it could be observed that they are now fully operational, but it is not clear if this is the case in all the sites. The sustainability of the ICDS Centres constructed is best guaranteed by the fact that they do form part of a government programme. However, there seemed some doubt as to whether the TMSSS centre visited in Nagapattinam really will receive its government workers and this needs more follow up. To make a significant longer term impact on children, the usage of the centres is clearly key, and there is scope here for longer term work on improved modes for managing the centres, training the anganwadi workers and so on. This seemed to the evaluation team to be a highly appropriate way for SC to utilise the large amount of funding it had raised. It is right within SC’s niche as an organisation and was creating community assets that will be of lasting value. Coherence was also good, in that work was carried out in close collaboration with the local Department of Social Welfare, under whose auspices the centres fall. This was particularly important in ANI, where traditionally it is quite hard for NGOs to work and there was a certain amount of suspicion to overcome. There was also some linkage with

57

UNICEF, who provided support for rainwater harvesting at some centres and with whom a positive working relationship was developed. Finally the coverage was again good, targeting communities in need, many of them in quite inaccessible places in ANI and providing assets far beyond what they had previously enjoyed. Objective 3: Water This was in the original plan, but was wisely dropped subsequently given that SC had no real background in this sector and there were others who could address this. No significant work was done in this area and the evaluation visited no water projects. Objective 4: Child Protection Child protection is seen as central and integral to all SC’s work in India and was one of the largest components, consuming 15% of total partner funding. The targets for what was to be achieved varied from one plan to another, but in the final analysis there were reported to be 283 Child Protection Committees and 632 Children’s Groups covering 42,000 children so, in these terms, it can be said that the work that was carried out was effective. In addition to the committees/groups being set up, there was support in terms of educational materials being produced in local languages, which will be of lasting value in the future. In terms of efficiency, most of the work carried out comprised ‘soft’ components and did not involve capital elements like infrastructure. SC’s approach is quite expensive compared to many other agencies in that it pays relatively high salaries to local staff, involved quite a large number of expatriate staff who are much more expensive, and did not stint on items like travel and accommodation, which was appreciated by staff, but does build up the costs. It appears from the figures, that overall about 33% of the expenditure for the whole TRP within India (for all components – not just the Child Protection component) was used by SC itself, with 67% expended by the partners, who were actually implementing the projects. The evaluation had a large number of interactions with children who had been involved with child protection work and from these it was clear that the programme had had a significant initial impact on children. Children showed awareness of child rights issues and (for example with League for Education and Development (LEAD) in Cuddalore), they expressed confidence in their ability to express their views to elders and take up issues they saw as important. Adults and teachers saw these activities as positive in that it was something constructive for children to be doing, but did not necessarily see the issues raised in the same way. For example, when meeting with communities that Jeeva Jyothi was working with in Tiruvallur, parents and teachers felt that beating of children was a necessary part of maintaining control. There was also a sense that children should have responsibilities towards themselves and others and should not just be rights holders. Part of this may be understood in terms that, while a ‘child’ is internationally understood to be anyone under the age of 18, in many communities, once puberty is reached, a child is seen in quite a different way. The Child Line supported by PRAYAS in ANI was interesting, but it seems that most of the phone calls in so far are to test the line and see what it is all about, so it is not yet clear as to what extent it will enable serious

58

protection issues to be identified and addressed. The Child Rights Centre constructed by SC (Canada)’s partner Association of Sarva Seva Farms (ASSEFA) in Cuddalore seemed enormous if it really is to be used as an after-school club. If, as was hinted, it might actually be used as a school, then efforts need to be made to ensure that the poorest do really benefit and it does not end up as a private school for the better-off. The sustainability of work is a serious question in this component. While many groups have been set up, a child rights perspective takes a number of years to take hold when it is confronting centuries of a more hierarchical view of children’s place in society. In order for knowledge to change attitudes and then behaviour with adults and children, more sustained interventions (continued training, support and follow-up) will be required than have been possible up to now. Work on child protection did seem appropriate under the circumstances of a disaster for which there was a huge amount of aid available. Early fears about orphans not being taken care of and a surge in child trafficking did not seem to be realised in a major way, so the tsunami did not itself create major new child protection issues (although there were several reports of under-age marriage). This was more about using the opportunity of funds generated to initiate work which had always needed doing. If there were major unmet rehabilitation needs in other areas, then prioritising child protection may have been questionable, but since there were not, then this component was justifiable. There was some limited work done on psycho-social rehabilitation, particularly by ASSEFA, but it is questionable whether this was necessary in inland communities where the tsunami had had a limited impact. In terms of coherence, more needs to be done to integrate Child Protection Committees to local governance structures to enhance prospects for sustainability. Again coverage seemed good, with conscious efforts to involve marginalised ‘second line’ communities who were not so much covered by other agencies. Objective 5: Education Education was a lesser but still significant part of SC’s response, consuming 6% of overall partner expenditure. In the early stages, work was done to support children in ANI in Port Blair, so they could complete their exams. Thereafter emphasis was more on creating child-friendly schools and a conducive environment in which education could take place. By the end of the programme, 153 Parent Teacher Associations/Village Education Committees had been activated and strengthened, 81 bridge centres and 172 Supplementary Education Centres had been supported, so this component can be regarded as effective in reaching these milestones. Again, this mostly involved soft components, so in terms of efficiency, the same comments would apply about SC being one of the more expensive agencies in terms of its operations. In terms of impact, as a result of awareness campaigns, enrolment had improved amongst communities where many of the parents themselves had little schooling. Parents interviewed indicated that they saw education as important to give their children opportunities they had not enjoyed themselves. There was also evidence of improved understanding of what ‘child-friendly’ meant (in a context where a teacher-centred

59

approach has traditionally prevailed) and some evidence of real changes in giving children a voice to articulate their views. Children said that they could now see teachers more as their friends and that there was less beating of students (although teachers seemed somewhat uncertain as to whether this was entirely beneficial). Suggestion boxes gave an opportunity for children to suggest changes, although there were complaints that they were not acted upon. As a result of interventions, respondents reported that the quality of education had improved with a school visited in Jamilabad, Tiruvallur (supported by Jeeva Jyothi) saying that it had gone from having the lowest grades out of 40 schools in the area to 4th position. Again there are questions about the sustainability of this work, particularly where there have been quite short-term interventions. While short term improvements have undoubtedly taken place, it will require more sustained follow up for new structures to be consolidated and to continue to function for improved education. Education is certainly an appropriate activity, even though much of the work was improving what existed before the tsunami rather than repairing what had been damaged. While the states affected by the tsunami were already amongst the best in India in terms of education, there remains scope for continued improvement. One of the ‘Big Ideas’ was to establish an Education Trust Fund, to support innovative work in education and make linkages. Although there is still theoretically $6 million allocated for this, it was not clear how this is being taken forward. Certainly, India does have a experience of endowment funds (called ‘corpus funds’), so this may well be manageable based on local experience, so it is potentially workable, but there are questions as to what role the Education Trust Fund will play and how, what value it will add and whether it will be possible to create a sustainable institution from scratch. SC ensured coherence in this area by working within the existing government education system, which is already resourced and functioning well, particularly in states like Tamil Nadu. Coverage was also good, with evidence of efforts to target communities which had been less involved in education previously and ensure that schools were enrolling children from a range of different religions (where there was a mixture) and castes. Objective 6: Livelihoods Livelihoods was the second largest component of the programme, accounting for 16% of total expenditure by partners. Again the plans and targets changed over the course of the programme, so it is hard to compare how the results measured up, but with the final reported figures of 210 livelihood groups supported, with 3,353 members, 7,526 households receiving assets and 119,664 people benefiting, then this component can be regarded as effective. SC sought to work with partners who had already been working previously on livelihood issues. This made sense in terms of being able to work effectively on the issue with less training required, as was the approach in some of the projects to build on existing livelihood activities (e.g. encouraging a leather worker to make new products, or a shop to stock new goods) rather than start completely new livelihood activities in a short time. Again the previous comments about efficiency can be repeated since SC was one of the more expensive agencies in terms of implementation. There was a feeling from some

60

within SC that the approach adopted on livelihoods meant that a small number of lucky people benefited a lot. In terms of impact, this is a particularly complex area to understand as there have been so many organisation making interventions from the early relief distributions and government compensation schemes to a whole range of livelihood inputs. Participants spoke of improvements in their income as a result of SC interventions, although there was no baseline information systematic collection of data along these lines to analyse. Certainly there was no evidence of very severe unmet needs or families living in extreme poverty encountered, but most areas affected by the tsunami were never particularly poor anyway in comparison, for example, to the poorer parts of Bihar or Orissa. In addition to activities to boost income generation, there was also some work to enable communities to access government-provided social safety nets. In ANI, a Santhali community supported by West Bengal Voluntary Health Association (WBVHA) indicated that they had subsequently been able to access support for a widow, for girls and access to the Employment Guarantee Scheme introduced by the current government. For sustainability, where new Self Help Groups (SHGs) were supported, experience gained across India over the past two decades indicates that it is likely that they will require several years’ further support and accompaniment before they become self-sustaining. For individual livelihood activities, it seems likeliest that these will be sustained where they have built on previous experience and competence. Livelihood was certainly an entirely appropriate intervention, given that all tsunami affected communities had suffered from the effects, even if they were not directly hit. It was sensible that SC supported only the repair of fishing boats in the early part of the response and did not get into the provision of new boats, which ended up being over-provided. As a result, it was reported, there were not enough people to man the boats, resulting in more child labour. One of the Big Ideas in relation to livelihoods was that of cash transfers to the most marginalised. This was apparently investigated by the Overseas Development Institute (ODI) and it was considered worth pursuing, but the idea was then abandoned following an internal Save the Children discussion, because it was felt that it would be hard to manage in a context where the government has its own compensation and social welfare schemes. In terms of coherence, livelihood work sought to take account of government interventions, particularly when linking people to government social safety nets. With respect to coverage, the programme managed to include more marginalised groups like Scheduled Casts and Tribes, and was much less involved with fishing communities, who could already generate relatively high incomes. Objective 7: Emergency Preparedness/Disaster Risk Reduction Emergency Preparedness (later called Disaster Risk Reduction) was another smaller, but still important component of the programme utilising 6% of partner expenditure. In reality, the initial name was probably accurate, as the work carried out was more related to preparing to respond to a future disaster, and did not include so much about disaster mitigation or disaster prevention. Again the targets changed over time, but by the end of 2007, 126 contingency plans had been developed, 56 schools were covered and three

61

strategic partners had been identified and trained to increase the capacity of others. Hence this component was effective in overall terms. With respect to efficiency, the previous comments about efficiency can be repeated about SC being one of the more expensive agencies in terms of implementation. In terms of impact, from partners visited, such as ACTION in East Godhavri District, Andhra Pradesh, children showed evidence of knowing about disasters and how to respond to them. The knowledge gained was mostly sound, although some seemed to feel they were vulnerable to tsunamis nine kilometres inland, which may have created unnecessary fear. In reality the real test of whether emergency preparedness is effective is in the case of an actual disaster. ACTION had previously worked on emergency preparedness in an Oxfam programme in the 1990s, but seemingly it did not cover tsunami affected villages, so it was not clear how successful that had been. At least this demonstrated that SC had, once again, successfully picked a partner with a background in the issue rather than starting from scratch. There must be significant doubts about the sustainability of some of this work. Awareness and interest is high straight after a disaster; a few years later when there has been no new disaster, it can dwindle quite quickly. Certainly having weekly sessions in schools seems unlikely to be sustained in the long run and ways need to be found to integrate it into other work and continue it at a level, which is more likely to be continued. At the SC organisational level, SCUK was hampered by having no staff that had experience of previous disasters. Now SCBR has recruited 20 staff from the Tsunami Response Programme, so it is to be hoped that for a future major disaster it will be better equipped to respond in terms of personnel. Encouragingly, an Emergency Preparedness and Response Plan has been developed as an SCBR document, which is meant to be kept alive through trainings and updated each year. There is no doubt that the aftermath of a disaster is an appropriate time to raise awareness about how to respond to future disasters. People are aware of the effect a disaster can have on their communities and are receptive to messages. As well as disaster preparedness, there may have been scope for some disaster mitigation measures also, more in respect to cyclones and floods, which are higher risks in many of these communities than tsunamis, which are exceedingly rare. In terms of coherence, SC has sought to link its disaster preparedness work to local institutions (schools and village committees). The coverage of this component again seemed to focus more on second line villages and did cover marginalised communities. While not necessarily being right on the coast, there was evidence of their risk to disasters, with stories recounted of major floods and cyclones occurring every few years. Objective 8: HIV/AIDS This component originally included health, but this was soon dropped, and was just 3% of partner expenditure. The work was somewhat effective, with 733 awareness campaigns carried out and 35 partners mainstreaming and two strategic partners identified, but work started later than planned and took place in seven rather than ninre districts.

62

With respect to efficiency, the previous comments about efficiency can be repeated about SC being one of the more expensive agencies in terms of implementation. The idea of developing ‘strategic partners’ who will in due course act as resource organisation for others is an interesting and potentially efficient way to work, although it remains to be seen how effective this will be in the long run. In terms of impact, from discussions with the male peer group which Association for Rural Mass India (ARM) had been working with in Kaipannikuppam, Villupuram District, Tamil Nadu, they seemed well informed about the topic, took the trainings seriously and were enthusiastic about spreading awareness. However, the women’s group were not sure if HIV was a disease or a virus and were very shy to talk about the subject even though there were just females present. There was no indication of major stigma reduction. For the partners who were mainstreaming HIV into their other work, given the pressure on them to complete projects within the timeframe, it was not clear how much priority they had given to this where it was not a major component of their work. This underlines questions about the sustainability of this work. It is well recognised that raising awareness, changing attitudes and achieving behaviour change is a long-term process when it comes to HIV/AIDS. It cannot be easily achieved in the context of an 18 month project when there are many other issues preoccupying the community. In these terms, this component was appropriate in that HIV is prevalent in the target areas, albeit not at a very high level as compared to other countries. However, as noted above, this requires it being seen as a longer term intervention if it really is to be effective. A positive step to achieve coherence of the work was an attempt to set up a network called CHILD linking partners working on this issue. The coverage again managed to reach a range of communities, who are at some risk due to their proximity to roads down which many trucks pass. 4. Strategic and management issues Ways of working (direct, with partners, with government etc) Save the Children’s tsunami work was implemented entirely through local partners, except for the purchase and distribution of some relief materials in the early stages. The process of identifying partners was seen as quite slow in comparison to other agencies, but this had the advantage that it could more properly assess the credibility and capacity of its partners and ensure there was sufficient mutual overlap of objectives. In most instances appropriate partnerships were made, but there were exceptions such as SASY, which would probably have been more suited to rights advocacy than shelter construction, or Butterfly where there was a difference of objectives. In ANI, due to historical reasons and the lack of government encouragement, there were very few local NGOs of any size (and with the necessary Foreign Contribution Regulation Act (FCRA) approval to receive foreign funds), so SC’s partners tended to be mainland NGOs who established new programmes there. This was unavoidable in terms of implementing significant rehabilitation work, but there was possibly an opportunity missed in terms of working through an organisation with FCRA to support local groups, not so much to

63

implement large scale projects, but to lay the ground work for a more vibrant civil society in the islands in the future. With its child-focused agenda, SC clearly pushed partners towards child-oriented programming where sometimes their original mandates were different. However, there was no evidence of coercion in this direction and partners seemed happy to have worked with SC, although with most seeking further funding, maybe they did not feel able to express more critical views. The project proposal development process was slow (taking up to three months), about which there were some complaints. However, for proposals to be coherent, with clear change objectives and owned by the partner, a longer development period was necessary. While a balance needs to be struck here, it is better to spend more time to allow a partner to plan what it is going to do properly than pump in funds on the basis of a budget and a few lines which other agencies sometimes did. SCUK generally had one Programme Coordinator responsible for roughly four partners, enabling them to visit quite frequently. There were also a range of other managers and advisers who would also engage. Given that these were new partners for SC, they frequently had little disaster response experience and that they were often implementing far larger projects than they had previously, this was appropriate. However, SC (Finland) sought to place a Project Officer within each of its two partners – one accepted, while the other did not. This seems excessive in terms of seeking to exercise control and potentially undermines existing management structures. Many partners expressed their appreciation of SC’s support, its accessibility with sub-offices not too far away, and its flexibility and understanding. They generally reported considerable capacity development of their organisation as a result of working with SC. In terms of adopting and promoting international standards, staff and partners seemed aware of and had applied the Sphere Standards, notably in such areas as construction. The Red Cross Code of Conduct had substantially been adhered to and no major transgressions were noted; probably those most challenging were those in relation to involving beneficiaries in the management of relief and accountability to those assisted (as discussed further below in the section on Child Participation). The Global Impact Monitoring framework (GIM) was however not referred to during the course of the field visit. SC was also notable for its efforts to coordinate with others (it was a key founder and supporter of the Tamil Nadu Tsunami Resource Centre (TNTRC)) and to link with government. This helped ensure sustainability of work like ICDS centres and work in schools and also put SC in a good position to influence government policies. Prioritisation of programme objectives (i.e. choices re sectors) In an overcrowded context, it made complete sense for SC to focus its support on child-focused work and leave some of the more general issues to others. Some of this work (education, child protection) was not really linked to the tsunami, particularly in more inland communities and was more developmental in nature, but since these were some of the most marginalised communities, this seems entirely justifiable. SC is also to be commended for avoiding some of the mistakes made by others – partly helped by the delays in its own decision-making processes. It did not get into the

64

provision of new fishing boats, which were eventually over-provided in a context of depleting fish stocks. It did not (eventually) do much on water where it had little competence. It scaled down on large scale housing where it also lacked competence and there were issues over whether people would actually use relocated houses (some completed new settlements were said to be less than 50% occupied). It also avoided much work on psycho-social rehabilitation (apart from SC (Canada)), which was popular with others, but was of doubtful need, appropriateness and quality much of the time. It did not emerge from the evaluation that there were major unmet needs about which SC can be considered negligent in avoiding. The only possibility is in the provision of temporary shelter, which was heavily criticised by the DEC evaluation in 2005 for being inadequate. SC had little competence in this area, but could potentially have acquired this if it had been prepared to do so. Targeting of beneficiaries (by age, gender, economic status, geographical location, etc) In the early stages of the disaster, there was general concern that the coastal fishing communities were receiving the bulk of the relief and other (often lower caste and poorer) communities inland, which serviced the fishing industries/villages, were being bypassed. The DEC decision in 2005 to allow funds to be used for ‘tsunami affected districts’ rather than just tsunami affected communities, allowed more flexibility. SC took advantage of this by working to a large extent in ‘second line’ villages and reaching more marginalised communities. Repeatedly it emerged that Schedule Castes and Tribes had been involved in different SC projects and had been positively targeted, although in some instances it was too much to expect real integration between communities. In ANI, SC sought to work in some of the most difficult and isolated communities in off-lying islands. This was highly problematic in terms of staff access (government approvals are required in a militarily sensitive area), logistics where nearly everything is brought in by sea or air and resulting high costs. To manage the building of ICDS centres in the Nicobar Islands, for example, was a major achievement. The only question over the geographical programming was the very wide area SC covered – essentially everywhere affected in India, apart from Kerala. In terms of management, provision of technical support and building linkages between partners, it may have made more sense to have focused more tightly. Child Participation Child participation was an issue that staff and partners were clearly aware of and had tried to promote. Nearly all the projects visited had a child-focused orientation, although the WBVHA livelihood work in Andaman did not demonstrate this. In practice, there did not seem to have been much child participation in the planning of projects, but it was more evident at the implementation stage. In planning projects, discussions were mainly between SC staff and senior partner staff (especially directors). Probably, in the context of the need to get proposals developed (which was already seen as too slow) and where many other agencies were intervening in the same communities, much community/child participation would have been quite difficult at the initial stage. But in ‘second phase’ projects with partners, more participation should be possible. Where participation was strong was in many of the Child Rights Clubs which had been established. There was significant evidence of children able to articulate their views and

65

ask for their rights. In some instances, parents and teachers recognised the value of this and saw it positively, but there were also elements of doubt, which is to be expected after relatively short-term engagements. Children played a strong role in the SEVAI house construction, influencing minor design elements (e.g. where to put a shelf) and monitoring the quality of construction. With ICDS construction in ANI, there seems to have been less attention to child participation, probably due to the fact that the implementing partner ANIIDCO had little background in this. Advocacy SC’s approach to advocacy was generally more along the lines of softly seeking to influence government policies from a supportive position, as compared to some other agencies, who take a more confrontational approach. Positive outcomes of this approach were evident in the greater child-focused awareness of government officials such as the Special Officer on Duty for Tsunami Relief and Rehabilitation in TN and the resolution of the Director of Social Welfare in ANI to incorporate lessons learned in new ICDS construction work. SC was also a founder and strong supporter of the TNTRC, which as a civil society coordination and sharing body had the potential to take up issues with the government on behalf of the sector. Opinion seems divided how effectively this was able to fulfil its mandate and it is unclear to what extent it will be sustainable. Recruitment and orientation (of staff) The initial response was reported to have been slow mainly due to the fact there were not enough staff to respond, conduct assessments and implement programmes. The entire team from March/April 2005 onwards was entirely new to SC and it rapidly became a very large team with 75 staff – dwarfing the rest of SC’s operations in India. It proved difficult to fill many positions and it was said that for this reason it was decided to bring in expatriates, who took some of the management responsibility and a number of the advisory roles. This need is surprising in the sense that India is well known to have a large educated population, a vibrant civil society sector and experience of responding to many different disasters. An expatriate can bring in expertise, but often will not know the local context, which in the case of India with its complex social issues (caste etc) and governmental system takes a long time to understand. They are also relatively expensive and less likely to be able to speak the local languages. Nearly all had departed by the time of the evaluation, but the needs might have been met through more proactive recruitment processes within India, if for example, there had been links maintained with staff involved in the Gujarat and Orissa emergency responses. Training and capacity building (of staff and of partners) What was positive was the orientation and training of SC staff and partners, which was considerable and commendable give the time pressures facing the programme. Many staff had little experience of child-focused programming and of disaster response, so this was certainly needed. Records show 28 different trainings/workshops on general issues and 35 trainings related to each sector for partners and staff, as well as further trainings at the community level. It was hard to get a clear sense of the quality of these trainings, but both staff and partners recognised that this was a real strength of the programme approach. On the part of partners, it seems that they felt that it was obligatory for them to participate in trainings, which from a partnership point of view is somewhat questionable, but probably helped ensure that the quality of project implementation was strengthened.

66

Other management issues An issue that was highlighted strongly by the TRP staff was the delays in getting started due to multiple SCUK layers of decision making (London to Delhi to Chennai to the field offices) and a reluctance to devolve authority downwards. This resulted in the delays in getting started in 2005, particularly in the period between April and October, although this did give SC the chance to see what others were doing and learn from their mistakes. Information did not travel quickly with, for example, outcomes of a DEC meeting in London being received two weeks earlier by another agency in Chennai than by SC. It was said that the reason this culture prevails is that SCUK at least, is ‘risk averse’ in its reluctance to devolve authority. Certainly some of the examples given did seem to demonstrate excessive top-down control, probably not helped by the fact that existing staff were used to working in the more relaxed conditions of long-term development work and could not quickly adapt to an emergency context. In the end, it seems that Chennai became semi-autonomous within India, which solved the immediate problem, but probably is not healthy organisationally. These delays fed through into project implementation. Decisions were slow coming down, partner proposals were delayed and then implementation had to be rushed. Outputs may have been completed (groups formed etc.), but quality could be compromised, affecting sustainability. Most projects were of 6-18 months duration, which often made things rushed. Since this was a five year programme much of which was essentially developmental, projects could have been longer. Staff argued in response to this that they were new partners and therefore long-term commitments were inappropriate, which is a reasonable point, but this could have been covered by mid-term reviews and break clauses. The transition to SCBR complicated matters, as was shown in ANI where work stalled for a period. However, in a sense, the TRP has given SCBR a strong starting point with new staff, who now have experience of working in disaster response. It is not clear whether SC (Canada) and SC (Finland) really needed to operate independently, as it is normal for governments to allow their funds to be used by non-national actors. On a technical accounting point, transfers to partners are considered by SC expended on a ‘cash basis’ (when the transfer is made), rather than an ‘actual basis’ (when the partner has actually spent the money), which can lead to discrepancies. 5. Recommendations Key recommendations coming out of the above findings are as follows:

1) In terms of getting the rest of the programme finished, most of the funds allocated for India are already used. Of the $18,845,000 remaining with SCBR, most is already committed or earmarked to partner projects. It seems that ANI has the most to do after a gap in managers and lack of clarity about what is already decided. This needs to be pushed forward to ensure completion by March 2009.

2) Much of the work done was developmental in nature and, while many groups have been established, significantly more work is required if interventions are to be sustainable. This should be a priority for the next phase (for Child Rights Clubs, for new SHGs, to consolidate DRR plans, to follow up on ICDS usage, for HIV behaviour change etc). If there are further funds earmarked for the tsunami, these can be used to consolidate such work to 2011 and beyond if necessary.

67

3) On the other hand, given relative levels of poverty and need, it would seem hard to justify making efforts to secure new (non-tsunami related) funding for areas, which have already received so much support. In particular, in ANI, given the relative levels of poverty, the numbers of inhabitants, the scale of government support and the cost and difficulty of working, it seems hard to justify making further investment, unless it is on the basis of funds which cannot be used more widely. When compared, for example, to the poorest parts of Bihar, there would seem to more justification for concentrating efforts there. There may seem a contradiction between this recommendation and the previous one, but what is being said here is that there is a case for further work in the same tsunami affected areas to promote sustainability, but only using funds earmarked for the disaster. If it is a case of prioritising unreserved funding or finding new funding, then a strong case could be made for prioritising work into poorer areas not affected by the tsunami.

4) For the Education Trust Fund, apparently $6 million is allocated for this, but it is unclear how this will proceed. This needs to be pushed forward if it is to be credibly linked to the tsunami. The issue of the management and sustainability of a new and such a large institution needs careful consideration and planning.

5) For further projects with existing partners, greater efforts could be made to involve communities (including children) in planning, not just implementation, using the range of tools available to facilitate this process.

6) There were various programme plans and it was hard to get a clear overall picture. The evolution of plans is inevitable, but it should be easier to follow changes and their reasons. Similarly, it should be clearer how reports relate to plans, and how individual project reports link to overall programme reports.

The following recommendations relate to Future Disaster Management: 7) Thought needs to be given to how a major disaster would be managed by SCBR

in the future. Would international SC members come in and, if so, on what terms? Having independent SC operations seems questionable. Are expatriates really needed given their cost and limited experience in India? The Emergency Preparedness and Response Plan should address these issues in line with established overall SC Guidelines.

8) The decision making process in future disaster situations needs to be streamlined, with fewer layers and clearer authority given to local managers.

9) SC should go into shelter with the necessary technical expertise in such situations, or leave it alone completely. If working on permanent housing then in situ housing tends to be preferable. The capacity to address temporary shelter needs to be considered, as this was identified in the DEC Independent Evaluation as a gap, which was accepted by SC staff in India.

10) Now that SCBR is in existence, it will have the option of direct implementation as a local organisation itself in the future, but if the intention is to develop local civil society, then it would make sense to continue working through local partners so as to build local capacity for the future.

11) Where there is limited local capacity, then supporting agencies from outside immediate affected area makes sense. However, it is also worth considering support for small local organisations to help develop capacity, even if they cannot deliver large scale projects.

12) If there is such a spread-out disaster again, it could make more sense to focus the programme area more tightly to make it logistically easier and more coherent.

68

13) SC should consider changing from ‘cash basis’ to ‘actual basis’ accounting (i.e. expenditure would better be shown not when a transfer is made to a partner, but when the partner has actually spent the received money in order to ensure coherence between narrative and financial reporting).

Conclusion The Save the Children Tsunami Response Programme in India, while considerably smaller than those of Aceh, Indonesia and Sri Lanka, was still the single largest programme implemented by Save the Children in India. It posed unprecedented challenges for a largely new team, many of whom were unfamiliar with disaster response, child issues and Save the Children as an organisation. In overall terms, it must be recognised that a huge amount of work has been carried out and the outputs have been impressive, even if they did not meet the initial targets, which were aspirational and unrealistic. There is little baseline data on which to carry out comparative analysis, but the feedback from children and communities where projects had been implemented was mainly positive. There were initial delays in planning and decision making due to the hierarchical nature of the organisation and the reluctance to take risks by devolving authority to the local level. However, this did enable SC to avoid some of the mistakes that other organisations made (over-provision of fishing boats, housing which ended up only partially occupied, psycho-social rehabilitation of doubtful quality). In spite of the pressure to spend money and achieve outputs, good efforts were made to build the capacity of staff and partners throughout the programme period, and also to establish staff with specific monitoring and evaluation responsibilities. Much of the work done was essentially developmental in nature rather than rehabilitation. This was appropriate given the resources available relative to needs, but does raise questions over the sustainability of activities initiated and longer term engagement will be required to ensure that, for example, Self Help Groups are able to stand on their own feet. The TRP period saw the establishment of the Indian SC member, SCBR. Already encouraging steps have been taken to ensure that lessons leaned are incorporated and can be used for future disaster management in India.

69

Somalia Programme Methodology Because of limited time and resources, together with the difficulties of setting up an evaluation in Somalia, this report is based on a desk exercise which comprised the following elements:

• Documentation study • Thirteen email questionnaires were sent to key present and former field staff and

a small range of external stakeholders in Puntland. Nine replies were received. • Interviews with two key former staff members.34

Inevitably this methodology imposed considerable constraints on the scope of the evaluation, particularly because of very limited number of independent reviews or impact assessments available, as well as the lack of baseline data. As will be seen in later sections, this means that it has not been always possible to draw conclusions about the work. 1. Programme description Country Context and Impact of Tsunami Since the civil war that led to the collapse of the Syad Barre government in 1991, Somalia has been characterised by complex political, economic and social changes, fragmentation, the collapse of central state institutions and continuing conflict at a variety of levels. Successive attempts at the re-establishment of a central government in Mogadishu have proved elusive. Central and Southern Somalia is nominally under the administration of Transitional Federal Government (TFG) established in October 2004, but unable to sit in Mogadishu and currently supported by the intervention of the Ethiopian military. Somaliland declared its independence as a republic in 1991 and, although not recognised internationally, has developed its own political and governmental structures. North-East Somalia declared the autonomous ‘Puntland State of Somalia’ in 1998 and has also developed its administrative structures while remaining part of a federal Somalia. Apart from political instability, fragmentation and conflict, the country is also subject to natural disasters including successive droughts and occasional floods. Somalia is one of the poorest countries in the world. It is no longer ranked in the UN’s Human Development Indicator table because of the lack of reliable statistics. It has a population of between six and seven million people, including over 300,000 internally displaced persons (IDPs) and a further more than one million Somalis living as refugees or as part of a global diaspora. The estimated humanitarian indicators for Somalia are appalling. Life expectancy is 47 years. Infant and maternal mortality rates are among the world’s highest. The under five mortality rate is estimated to be 225 per 1,000 live births. Maternal mortality thought to be 1,600 per 100,000 live births and adult literacy rate of

34 One face to face; one by telephone.

70

19%. Less than 30 per cent of the country has access to safe water. Net primary school enrolment is estimated at only 13 per cent for boys – and only seven per cent for girls. The impact on Somalia of the Indian Ocean tsunami on 26 December 2004 was limited compared with Asian countries in the more immediate vicinity of the earthquake. The surge mainly affected the North-East coast of Puntland. Up to 300 people died, 2,000 houses were damaged, fishing boats were destroyed and approximately 44,000 people were affected in this remote, inaccessible and drought-affected area. The tsunami was a further turn of the screw for very poor, isolated communities in area with the minimum of infrastructure. In this context humanitarian agencies, including SCUK, responded to need in the broad sense, much of it pre-existing the tsunami.

’It was stated on more than one occasion that the tsunami has brought more benefits to Puntland than the damage that was done. That is not to deny the importance of the loss of lives and livelihoods, but to say that the scale of the humanitarian response has been greater, in proportion, than the losses suffered, and has taken place in areas that were otherwise neglected due to the (previous) ‘tarmac-bias’ of work concentrated on the main Garowe-Bosaso road access.’35

There were very few agencies working in North-East Puntland prior to this disaster. The international response came from a limited number of international NGOs and UN agencies with their back bases in Nairobi. An inter-agency assessment coordinated by the Somalia Aid Coordination Body (SACB) took some time to get organised and took place at the end of January 2005. Prior to that SCUK and Norwegian Church Aid (NCA) undertook their own assessment in mid-January. It is worth noting that Somalia (along with Thailand) has been relatively neglected in the plethora of post-Tsunami reviews, evaluations and lessons learned exercises, perhaps because of the relatively small direct humanitarian impact and probably because of being a ‘failed state’ in a different continent from those countries most badly affected. However, Save the Children’s expenditure in the Somalia response programme is not insignificant with expenditure of £4.4m/$8.3m up until March 2008. Overview of the Save the Children Alliance Response Before the tsunami, SCUK had worked for 40 years in Somalia and had a well established programme in parts of Central South Somalia and Somaliland. It is a leading international NGO in the aid coordination structures in Nairobi and the leading INGO in the education sector, working closely with other NGOs and with UNICEF and UNESCO. SCUK has been the sole Alliance member working on the tsunami response in Somalia and its programmes have been fully funded by the DEC, with some very small amounts of money from other, non-Alliance, sources. Although well-established in parts of Central South Somalia, SCUK, like most agencies, had no presence in the tsunami affected area of Puntland, or in Puntland as a whole. However, strategically SCUK had been looking to extend its geographical areas of work

35 Harragin S. ‘Impact Assessment Tsunami Response and Recovery Programme, Puntland State – Somalia’. SCUK. January 2006

71

and this event provided the opportunity for an expansion into a remote and needy region. SCUK’s underlying strategy therefore was to take this opportunity to start a long-term programme, laying the ground by developing relationships with communities, with the very few local NGOs and most importantly with the Puntland government. This strategy meant that the programme had to be established with care and with as much of a developmental approach as possible. This may have had implications in terms of speed, but can be seen as acceptable given that most of the needs being met were of a longer term nature. Security is a key constraint in Somalia programming. Thirteen aid workers have been killed in 2008 up to the end of May and seven held hostage. Puntland had been relatively peaceful in 2005, but security conditions have deteriorated. Since January 2008 international staff are advised not to travel in Puntland. A Somalia NGO Security Preparendness and Support Programme report in May 2008 found that:

’the prevailing security situation has fundamentally changed over the past six-nine months. The threat environment which faces the NGO community in Puntland, has worsened and is more unpredictable. Moreover it is assessed as not improving materially for increased program delivery over, at least, the next six months.’

2. Summary of Programme Objectives and Key Findings Overall Objectives:36 2005: Food and livelihoods security of the tsunami affected populations recovered with improved access to health and education services, especially for children and women, as well as increased overall resilience to disasters for vulnerable communities in Puntland state. 2007: By the end of 2007 more than 5,500 marginalised families are enabled to meet their children’s basic rights in the three districts of Karkaar region. Specific Objectives: Education: By the end of 2007 nearly 8,000 pupils will have access to and complete inclusive quality primary education in Karkaar Region, an increase of 60% over the 2006 enrolment. Livelihoods: By the end of 2007 more than 85037 (50%) of the affected poor households in the coastal districts of Karkaar region will have increased access to food and nutrition and income opportunities to support children and other family members.

36 Objectives have developed over the period of the response. 37 Figures based on Inter-Agency Tsunami Assessment (pgs. 21 and 24).

72

Emergency Preparedness: To promote adherence to children’s rights and needs in community disaster management and institutional emergency preparedness processes, thereby increasing the likelihood that future emergency response efforts will cater better for children (2005). Increased overall resilience to disasters for vulnerable communities in Puntland state (2006). Health: By the end of 2007 more than 4,000 (85%) children aged 0-5 years old will have improved access to quality health care services along with their mothers in Hafun and Bander Beyla districts and more than 12,000 (60 %) children in Karkaar region will have access to basic primary health care. Humanitarian Accountability: Increased accountability of Save the Children in quality and processes of programme interventions towards all the actors involved in humanitarian sector especially towards the children, participant communities, other actors and state agencies in Puntland state (2007). Construction: Objective not stated

73

Summary of Findings of the Save the Children Tsunami Response Programme in Somalia Sector Effectiveness Efficiency Impact Sustainability Appropriateness Coherence Coverage General With the limited

evidence available, the projects seem to have been generally effective.

Overall probably low because of the high costs of set up and operation in Somalia. However, offset by investments in longer term programme, so start-up costs can be written off over, say, five or more years.

Not possible to make an overall statement. Lack of reliable baseline data

SCUK has established foundation for long-term programme. Reportedly SCUK has attracted donor interest to the region. Work with government and establishing local structures likely to increase sustainability.

SCUK undertook an assessment with NCA and was a key participant in the subsequent inter-agency assessment. Appropriate sectors in which to intervene.

SCUK worked closely with and supported government departments. It also played a key role in coordination structures in Nairobi.

Disability in general not given priority Programme has begun to look at pastoralist issues.

1. Education Broadly positive outcomes.

Probably satisfactory.

Significant impact in terms of child participation and recognition.

Likely to be good

Highly appropriate

Good Probably good with exception of children with disabilities

2. Livelihoods Varied effectiveness

Not verifiable at the time of this evaluation

Positive impacts likely

Not verifiable at the time of this evaluation

Generally satisfactory

Not verifiable at the time of this evaluation

Not verifiable at the time of this evaluation

3. Emergency Preparedness

Important, but not possible to gauge outcomes.

Not verifiable at the time of this evaluation

Not verifiable at the time of this evaluation

Likely to be positive

Important given disaster and emergency prone nature of Puntland.

Good Not applicable

74

Sector Effectiveness Efficiency Impact Sustainability Appropriateness Coherence Coverage 4. Health Likely to be

significant outcomes.

Not verifiable at the time of this evaluation

Not verifiable at the time of this evaluation

Likely to be satisfactory

Highly appropriate Appears satisfactory

Not verifiable at the time of this evaluation

5. Humanitarian Accountability

Good Not verifiable at the time of this evaluation

Significant Not verifiable at the time of this evaluation

Highly relevant in a conflict situation

Not verifiable at the time of this evaluation

Not verifiable at the time of this evaluation

6. Construction Long delays. Likely to be poor.

Not verifiable at the time of this evaluation

Not verifiable at the time of this evaluation

Questionable in view of lack of expertise

Not verifiable at the time of this evaluation

Not verifiable at the time of this evaluation

75

Financial Summary of SC TRP in Somalia

SCUK 2005 budget 2005 actuals 2006 budget 2006 actuals 2007 budget 2007 actuals Actuals @

30/4/08

Emergency phase

Shelter/ construction

Health $302,132 $523,035 $379,305 $275,274 $694,062 $819,692

Water and sanitation

$74,921 $59,380 $8,590 $8,590

Education $195,821 $301,966 $276,566 $176,535 $448,304 $430,786

Child protection

Livelihoods $734,285 $658,052 $897,579 $866,373 $568,232 $597,882

Emergency prep $29,960 $43,303

Sub total Supplies/Materials

$1,337,119 $1,585,763 $1,562,040 $1,326,772 $1,710,598 $1,848,359

Non-Personnel $295,744 $452,878 $410,495 $394,285 $414,851 $364,481

Personnel $467,596 $424,997 $437,449 $369,886 $452,634 $436,939

Personnel Support

$150,353 $134,091 $143,063 $132,020 $161,747 $166,337

Capital expenditure

$84,426 $90,210 $62,655 $53,044 $26,776 $20,003

76

Other Direct Projects

$38,354 $47,120 $47,943 $32,592 $72,534 $43,785

Contingency

Management Support

$118,680 $136,757 $26,635 $4,618 $90,174 $73,397

Totals $2,492,272 $2,871,887 $2,690,280 $2,313,217 $2,929,314 $2,953,302 $8,811,057

2.1 Other information Stakeholders, mandates As stated earlier, SCUK was already collaborating with a range of international agencies in Somalia prior to the tsunami with UNICEF being an important collaborator. On the ground in Puntland, apart from the communities themselves, the government was the key stakeholder. SCUK put considerable effort in to developing transparent an supportive relationships with key ministries (particularly education and health) and in to capacity building of the Humanitarian Affairs and Disaster Management Agency (HADMA), the Puntland State coordination structure that was set up as a result of the tsunami. 3. Principle Findings and Recommendations This section provides additional information not included in the overview matrix above. Objective 1: Education

This is the one area of the Somalia tsunami response programme that has been evaluated and therefore it is possible to report on with some confidence.38 This evaluation will also provide baseline information for progress measurement. In terms of effectiveness key findings were:

• The improved learning environment has led to progressive increase in the number of pupils enrolled in primary schools within Karkaar region. If the upward trend continues, the total number of pupils enrolled will surpass the set target of 107% increase by the year 2010.

• Enrolment of girls in the region is relatively higher compared to most other

regions within Puntland.

• But enrolment of pupils with special needs is still low. Although there has been some increase in enrolment of pupils with disability since 2005, the proportion is still considered low.

Advocacy work has led to an increase in enrolment of girls with some communities volunteering to subsidise school fees for female pupils in certain circumstances. Although not specifically covered in the Ojwang evaluation, it seems likely that the programme has been reasonably efficient given its work through educational structures and its scope. In terms of impact, most of those interviewed in the evaluation cited above agreed that there has been great change in attitude as far as treatment of boys and girls are concerned. In all schools visited, the headmasters said boys and girls are treated equally, a position that was also confirmed by Community Education Committee members and 97% of the pupils interviewed. Women’s participation in the committees has increased to nearly one third.

38 Ojwang, D. ‘Puntland Education Impact Assessment’. 2008

78

In terms of changes in child participation, nearly two thirds of pupils interviewed felt that they were involved in decision making at school level. Some schools reported on child protection policies which were introduced to demonstrate child friendly schools. In the Somali context, where children are not consulted for any issue whatsoever, Ojwang notes that these are extraordinary developments that are appreciated by the community in general. Sustainability of the programme is likely to be good, given that the project has worked through and supported existing structures, including important support to the Community Education Committees. This is a highly appropriate project in the Somalia context and builds on SCUK’s considerable experience in this sector in other parts of Somalia. The project has been closely coordinated with the Puntland Ministry of Education and a good working relationship established. Coverage appears to have been generally good, except in terms of children with disabilities. It is not clear whether children of IDPs have been adequately covered. Objective 2: Livelihoods The impact assessment for this programme has been delayed because of security problems and therefore comment on this programme is limited. It seems that some of the Cash for Work (CFW) projects have been effective, including the road building and repair work. There have been concerns about the level of technical expertise available for some of this work. Some positive impacts are likely from CFW projects with some improved nutritional levels and better access to markets. It is not yet verified who has benefited from the distribution of fishing boats. In terms of sustainability, it is not certain whether the road improvements will be maintained. Generally the livelihood work has been appropriate, although choices in some areas have been questioned. For example, might it have been to focus on longer-term assets such as latrines rather than garbage collection? In terms of coverage, it is not clear whether or how effectively the projects were able directly or indirectly to assist old, the disabled or the vulnerable. Objective 3: Emergency Preparedness This has been a strategic and important capacity building project with the Puntland government’s newly formed disaster management agency (HADMA). It is not possible to gauge its effectiveness. Sustainability is likely to be good, but will depend partly on whether HADMA receives adequate political and material support from the government.

79

Objective 4: Health Lack of adequate baseline information and impact monitoring means that it is not possible in a desk study to gauge the achievements of this work.39 A study of morbidity and mortality failed to produce useful information. It seems likely that the construction of health centres, support to Gardo referral hospital, EPI and training activities will have been effective, notably by reducing distance between health facilities and referral within the region. Sustainability of the project is likely to be positive, given the relationship built with Ministry of Health and communities. Health structures appear to be maintained and used. This is a highly appropriate project given the very high levels of child and maternal mortality in Somalia. Objective 5: Humanitarian Accountability It seems highly likely that SCUK’s involvement of communities, transparency and capacity building have all strengthened programme effectiveness and accountability in very demanding operating circumstances. In terms of impact the work seems to have demonstrated that accountability to communities and to government can be carried out in Somalia and that this can have a beneficial impact on programmes and security. Objective 6: Construction There have been serious delays in this area, apparently due to lack of technical expertise and staff turnover. There is also a lack of monitoring information. It is not clear that this was an appropriate area for SCUK to intervene in given its lack of expertise. 4. Strategic and management issues Ways of working (direct, with partners, with government etc) SCUK has tried various approaches to implementing accountability to the communities in which it is working. These include the setting up of Social Audit Committees which are yet to be evaluated. Prioritisation of programme objectives (i.e. choices re sectors) Generally SCUK appears to have made sensible choices about the sectors in which it works. The areas that can be questioned are the provision of shelter and fishing boats. With shelter, lack of expertise in that sector has been a key issue. It is not yet clear what the outcomes of the distribution of high value fishing boats will be in an area of nautical insecurity, the population does not generally eat fish and most of the catch is not landed and gets trans-shipped direct to the Gulf states.

39 Report to DEC Report January 2007 – March 2008.

80

Targeting of beneficiaries (by age, gender, economic status, geographical location etc) Bearing in mind in a conflict situation that the sensitive selection of beneficiaries is a crucial issue, SCUK generally appears to have performed well. However there does appear to have been a lack of attention to reaching children with disabilities. Child participation Efforts have been made, particularly in the education programme to promote rights-based approaches and the involvement of children in project activity. However, there is a sense that progress in this area has been difficult, partly because of the operating conditions in Somalia, but also because staff new to SCUK have been recruited. It also possible that the work concentrated more service delivery and somewhat neglected advocacy approaches. Recruitment and orientation of staff Staff turnover has been cited as a constraint in reports to the DEC. As noted above, the recruitment of a staff team new to SCUK has been a challenge. Because of security restrictions, senior expatriate staff field visits can be rare. Responses to conflict A recent security assessment in Puntland noted that:

’maintaining a meritocracy, transparent processes and balance along clan and sub clan lines in all aspects of an NGO’s operation will promote trust and credibility and avert potential security implications.’

SCUK has worked hard to follow this advice, building and supporting inclusive community groups and committees and being open about its plans. As a result it appears to have had far fewer problems than some other NGOs. Coordination Generally agency coordination in Puntland appears to have weakened since the early stages of the Tsunami response. SCUK has been committed to coordination at both Puntland and Nairobi levels. Monitoring and Evaluation The 2007 report to the DEC noted that:

’there is lack of adequate operational support and capacity at the field level in order to ensure independent, systematic (rather than ad hoc) monitoring and evaluation ensuring that beneficiaries’ needs are entirely catered to.’

(See below for recommendation) Diversion In the insecure and unstable context of Somalia, theft of resources is a common threat and instances have been reported in SCUK’s programme. The instance brought to the attention of the evaluator involved a sum of about $7,000 and eight staff were dismissed as a result. Procedures and controls were also tightened.

81

5. Recommendations Given the nature of this desk evaluation, it is not appropriate to make detailed recommendations about the remaining elements of the tsunami-funded programme. Some more general recommendations are as follows:

1) Consider ways in which monitoring and evaluation could be strengthened in the Somali situation, perhaps by developing in-country resources through an independent NGO such as the Puntland Development Research Centre.

2) Develop a focus on the needs of children with disabilities in Puntland by assessing and responding to their particular protection and rights issues.

3) In demanding and insecure situations such as Somalia that involve long-distance management, SCUK should consider developing intensive induction procedures for new staff focusing on SC’s programming policies and priorities.

82

Sri Lanka Programme 1. Programme Description Country Context and Impact of Tsunami With a population of 19.3 million, Sri Lanka is a middle-income social democracy with a strong public demand for and good record in service delivery, especially in health and education. It has been plagued by civil war since 1983, when a part of the Tamil minority took up arms against what it saw as discrimination by the Sinhalese majority. The tsunami affected much of the country, but especially the north, east and south coasts that stood directly in its path. Over 230,000 families were directly affected. 39,127 people died. A large proportion of deaths were women and children, 635 children were orphaned40 and more than 3,000 children lost a parent. 550,000 people were displaced, with 250,000 people in need of temporary shelter. Fourteen of the country’s 25 administrative districts were affected to varying degrees. Mullaitivu in the North lost 3.4% of its population (4,741 people); Ampara in the more populous south east lost 1.9% (11,567 people). Batticaloa and Trincomalee lost 3,654 (0.7%) and 1,405 (0.1%) respectively. Housing suffered most damage in Ampara, Batticalao and Trincomalee.41 To bring relief to all affected areas, the catastrophe at first inspired unique cooperation between the government and its adversary, the Liberation Tigers of Tamil Eelam (LTTE). The Tamil Relief Organisation (TRO) which became a key partner of SCiSL in the north and east was not only recognised by the government in Colombo but also commended for its relief work. Unfortunately, the internationally-brokered 2002 ceasefire gradually broke down and, exacerbated by the lack of agreement on how to share donor tsunami funding, by April 2006 people were being displaced again in large numbers. The total number of new IDPs since then is almost 187,000 – a third of the number displaced by the tsunami, and all in need of shelter, relocation, schooling and so on. On January 16, 2008, the Government of Sri Lanka (GoSL) formally withdrew from the 2002 ceasefire agreement with the LTTE, leading to the departure of the international Sri Lanka Monitoring Mission. The current population of internally displaced in Sri Lanka stands at well over 400,000, consisting of more than 300,000 who remain displaced from before the 2002 ceasefire and an additional 100,000 who remain displaced from the tsunami, as well as the more recent conflict-related displacements in the Vanni.42 Many people are displaced several times by these events, which are exacerbated by regular seasonal flooding and cyclones. If the relief phase of the TRP occurred during a period of ceasefire, the recovery and development phases coincided with the much less favourable context of war, as a new government took vigorous action to bring the country as whole under control. The war is popular with the majority which, along with the government, has become increasingly impatient with the international community’s attempt at even-handedness. While humanitarian impartiality had been tolerated during the preceding decades of civil war, in the new context it was seen at least as disloyalty to the popularly elected government and at worst direct support for terrorism. Agencies,

40 According to the NCPA (National Child Protection Agency) 41 UNU-EHS, 2006 42 OCHA, 2008

83

such as UNICEF and SCiSL, with long-standing programmes in the conflict-ridden North and East where the majority is mainly Tamil, have fallen under particular censure. The reconstruction years were never going to be easy for such a prominent humanitarian agency as SCiSL but the situation was made much worse in 2007 by the Military’s capture from LTTE of two fishing boats donated by SCiSL. Fortunately SCiSL had taken the precaution of reporting the loss of the boats but there was still a public outcry. Along with most other large NGOs, SCiSL fell under scrutiny of a parliamentary committee and criticism from the media. For SCiSL, this led at one point to a demonstration of nationalist Buddhist monks outside the Colombo office. Events such as these, coinciding with slow progress at the front, fed a gathering climate of xenophobia in Sri Lanka. Restrictive measures against international NGOs soon followed. Visas became harder to obtain, some taxes were increased, and a new tax introduced on funds brought into the country by international NGOs. Ministries imposed restrictions on programmes at district level, and humanitarian access to the newly displaced became increasingly limited. A new NGO law is currently being drafted, with very severe implications for INGOs like SCiSL. The main aim seems to be to reduce the large number of expatriate staff still directing operations. Harassment has gone to the extreme of denouncing individuals on official websites, thereby exposing NGO workers to very real danger. The sheer scale of the tsunami response has not helped the reputation of international humanitarian agencies. As they struggled with unparalleled and scarcely imaginable budgets, the popular impression was of profligacy and waste. Ministries with their eye on converting international generosity into durable infrastructure also deplore the one-off missed opportunity for ’building back better’. They regret that lack of central coordination meant that resources were frittered at local level – duplication of infrastructure and unnecessary building was all too common. They regularly criticise the amounts spent on support costs, which is echoed on the streets. Local staff members who were earlier congratulated for their work with the agencies, now face criticism from their families. The operating context for long-term INGOs like SCiSL is completely different at the local level. Here, wherever offices are run by local staff in close cooperation with local NGOs and local authorities, support for SCiSL’s humanitarian work is popular and considered effective. With the support of tsunami funds, District level service provision has received a very welcome shot in the arm with valuable inputs at infrastructural and training levels. Ironically this has not helped the INGOs. Jealous ministries complain that their authority has been subverted in the districts, with local officials reporting more assiduously to their INGO donors than to the central authorities. Such complaints go beyond political point scoring and raise serious issues of accountability and democratic governance. Criticism from Sri Lanka’s NGO humanitarian umbrella, the Consortium of Humanitarian Agencies, mirrors that of the government. Its local branches commend the important humanitarian work of its SCiSL counterparts in their district offices. However the Director (of nearly 15 years) in Colombo holds trenchant views on the failure of INGOs like SCiSL to strengthen local civil society and build local capacities as an abiding legacy of the tsunami. It pertinently asks: whatever happened to the pre-tsunami talk of localising Save the Children?43 43 INGO Directors, including SCiSL’s, have as poor an opinion of CHA’s leadership as it has of INGO performance.

2. Summary of Programme Objectives and Key Findings Summary of Findings of the Save the Children Tsunami Response Programme in Sri Lanka Effectiveness Efficiency Impact Sustainability Appropriate Coherence Coverage Overall/general This was a vastly over-subscribed emergency in which visibility was an over-riding priority for SCiSL and other INGOs. Public opinion questions the INGO relief and recovery role in general and SCiSL in particular.

SCiSL set itself ambitious targets and largely met them. To meet the demands generated by these conditions, SCiSL had to transform itself. Local sensibilities and local ownership were sacrificed in the process.

There was too much waste and duplication during the initial relief phase. Some useful programming then took place. Direct operation is very costly and harder than anticipated to dismantle.

There are positive outcomes but they are obscured by the sheer quantity of activities. This was mainly an output orientated programme focused upon beneficiary numbers.

SCiSL will struggle to sustain the policy gains achieved during the TRP. SCiSL’s exit strategy is to return to where it started pre-tsunami and work towards rights-based programming.

There is a perception that SCiSL and other INGOs inappropriately used the emergency to foist unwanted change without due consultation.

As GoSL attitudes hardened, SCiSL made insufficient operational adjustments to counter the political risk. It remains very popular at district level.

SCiSL was always right to reach out to all communities despite the unpopularity this has caused it with the present GoSL.

1. Shelter/ Construction SCiSL wisely limited itself to filling gaps in transitional shelter – thereby side-stepping the

Committing to temporary shelter for 20,000 was quite unrealistic and SCiSL was very lucky not be held to it. The minor contribution it

School blocks were good value for money – but not all the ECE centres were. £1m worth of 27 Social Care Centres

Uncertain. SCiSL is not central to the main shelter issue for children and families affected by the tsunami:

Services run from the new buildings are sustainable in the case of the schools (overall this represents 40-45% of all investments in construction).

Shelter and construction are not core SC competencies. But it would have been very hard for such a prominent

SCiSL’s engagement with construction pleased GoSL as it fitted with its expressed relief and reconstruction priorities.

SCiSL rehabilitated 21 and built anew 14 of the 287 schools directly and indirectly affected by the tsunami.

85

Effectiveness Efficiency Impact Sustainability Appropriate Coherence Coverage thorny issue of space for permanent homes. It was also wise to respond positively and effectively to GoSL requests for new school blocks.

did make was well done. New school blocks were 80% on target; a major success given the challenges to construction. Built more than three times the number of pre-schools planned.

was not well conceived in design or purpose. SCiSL lost £86,000 when GoSL banned a key partner in construction of pre-schools.

lack of durable solutions in housing for the displaced. New schools and pre-schools have greatly improved the environment for affected children. Likewise rehab of schools ‘trashed’ by IDPs.

INGO to do nothing when specifically asked by government.

2. Child Protection

Largely achieved objectives. However, issues persist in relation to ownership and sustainability of community-level child protection mechanisms

Much of this work has to be directly operational, which has efficiency implications: the sector accounts for just under 20% of all SCiSL staff.

Initial protection work resulted in the prevention of institutionalisation of children affected by the tsunami. Subsequent vocational training and HES support to families

Communities and govt officials need more sustained support to bring about real change for children at community level.

As a lead INGO working on Child Protection, SCiSL work was relevant and appropriate especially during the emergency. More in-depth assessments could have made a more

Did coordinate well with UNICEF, DPCC and other stakeholders.

High – need to tighten focus to ensure quality and sustainability.

86

Effectiveness Efficiency Impact Sustainability Appropriate Coherence Coverage also prevented institutionalisation

lasting impact.

3. Education Key area of SCiSL intervention. Faces political backlash which will limit effectiveness and impact unless managed. ECE Big Idea (an annuity fund to subsidise teacher salaries needs further thought).

Has contributed to increased enrolment and quality in basic and early education. Ban on direct funding of local education authorities and ban on work with schools will affect achievement of more recent targets.

Great number of often unconnected initiatives and activities. Management systems are slow in linking services to maximum benefit of individual children and their families.

Very good – was part of getting all affected children back to school within two months of the tsunami. Child Friendly School system included in national five-year educational plan from 2010. Catch-up classes made sure many IDPs were not discriminated against in crucial exams.

History of well-established programming helps quality of interventions at each phase. Has built good capacities for training and supervision at local levels. Otherwise no exit strategy and so caught unawares by government restrictions.

Accountability to children is good via participatory Clubs, consultations and research. Reforms introduced by SCiSL and partners are popular with progressive parents and professionals.

Generally good coordination with the one critical exception of central government. Work with UNICEP in introducing the Education in Emergency Cluster has been especially important.

Starts with very wide coverage (300 schools and 17,500 children). Should focus on limited number of pilots to embed reforms.

4. Livelihoods

Specific targets were met. But the wholesale

The use of cash grants for income

Women in particular have

Uncertain because very resource

Cash grants were very relevant to

With evidence that grantees were

Used sound assessment methods to

87

Effectiveness Efficiency Impact Sustainability Appropriate Coherence Coverage replacement of

assets by SCiSL immediately after the tsunami was not properly followed up. Later, more focused work took hold.

generation was an efficient innovation. But not all activities promoted by SCiSL and its partners were viable.

responded very well to cash grants and they invest returns in children’s education and health.

intensive. A Big Idea was researched but proved too costly to implement.

needs. The earlier focus on replacing assets was inappropriate-ly driven by the need to spend at the relief stage. Too many boats were handed out

graduating to interest-bearing schemes, SCiSL’s rightly ignored Oxfam’s protests that cash grants were distorting the credit market.

reach the most- needy in the community.

5. Emergency Preparedness With SC expertise from the region, led on child-led DRR which proved a genuinely inspiring and appropriate innovation.

Specific targets surpassed but as a cross-cutting issue, EP-DRR may get now get lost in SCiSL programming. A strategic approach to institutionalising EP and DRR is required.

Opportunity was missed to link training with funding of children’s priorities for dealing with risks identified in participatory hazard mapping.

Ministry for Disaster Management includes child-led DRR in its plans and encourages Districts and schools to do so as well. DRR was an early candidate for the SCiSL Big Idea.

DRR in school syllabus is a sound exit strategy, but there is still further scope for raising much needed disaster management capacities.

Highly appropriate for SCiSL – could help it shift from doer to capacity builder in disasters. Work on EP is used as an example in other countries.

In line with both local and national priorities and approaches. DRR could be an entry point for SC to re-build local ownership for disaster management.

Very broad coverage: 7,000 children’s cubs, 114 Divisions (at sub-district level) and 100 schools.

88

Financial Summary of SC TRP in Sri Lanka 2005

budget 2005 actuals

2006 budget 2006 actuals

2007 budget 2007 actuals

2008 budget

Actuals @ 30/4/08

Support $9,087,715 $7,090,415 $5,272,080 $4,314,168 £3,095,815 $2,876,112 $1,497,975 $1,198,380Relief Assistance

$7,270,172

$5,951,954

$1,278,272 $1,677,732 £1,138,461 $1,398,110 $239,676 $239,676

Livelihoods

$4,853,439 $6,391,360 $4,254,249 $3,754,924 $1,178,407 $1,118,488 $2,456,679 $299,595

Education

$4,893,385 $4,553,844 $4,473,952 $2,776,247 $4,493,925 $3,874,762 $1,278,272 $359,514

Child protection

$3,655,059 $3,455,329 $1,697,705 $2,616,463 $2,376,787 $2,157,084 $1,977,327 $439,406

Realising Child Rights

$198,938 $139,811 $339,541 $19,973

TOTALS

$29,739,797

$27,442,902 $16,976,258 $15,139,534 $12,402,233 $11,564,367 $7,789,470 $2,556,554

DEC accounts for almost 70% of the expenditure of £29.6m. Of total expenditure to date approx 30% is through partners However, funding through partners is lower than planned and as a proportion of expenditure was lowest in 2007. The target for 2008 is 44%.

Expenditure through government partners is increasingly problematic but in line with new SCiSL policy, should be on the rise through local NGOs.

89

SCiSL has spent £28m, which equates to a vast amount of programming activity. £5 million remains to be spent by the end of the TRP. There is sufficient programming to absorb the remaining funds, though there may need to be a reallocation from government to NGOs - though this would be unpopular with central government.

Support costs (including programme staff costs as is international good practice) at just over 25% seem high even for direct implementation in emergency – especially well into the rehabilitation stage.

The projection for 2008 is however 3% above the average which may reflect high staff costs.44 Although due to reduce, SCiSL’s staff stood at around 300 at the time of the review. As of April 2008 50% of the support costs are already spent.45

Construction, which offers good value for money and which is complete, accounts for £2.4m.

Approximately £100,000 was lost when TRO was banned by the government before completing 36 pre-schools. Taking over this work involved additional expenditure. This is the only known total financial loss.

With the decision in early 2005 to shift from partnerships to direct operations, SCiSL more than tripled its staff to a total of 350 within a matter of months. It has maintained these levels of staffing since then, at an annual cost of 25-28% of budget. There are presently 16 expatriate staff, mainly concentrated in Colombo. The remainder are nationals, distributed around the coast in six offices.

44 SCiSL explains: ‘Breakdown of international staff numbers shows that operations staff (decreasing since 2007) and advocacy and strategy (increasing) reflect changes in programme focus to national level influencing’. 45 Explained by SCiSL ‘as most DEC funding ends in June and staff reductions planned post-June 2008.’

90

2.1 Other Information

- Stakeholders, relevant mandates (see Annex I for list) - Phases (See Annex 2)

3. Principle Findings and Recommendations This section assesses the programme objectives against the principal evaluation criteria. It addresses aspects of particular importance. Aspects not included here can be found in the summary table above.46 Relief Support The relief phase is not technically a part of this review; however, it has played an important and continuing role in the Sri Lanka TRP and has remained an objective throughout the TRP. The conflict has destroyed hopes that relief assistance would be a temporary need in Sri Lanka, but OCHA in Eastern Province is concerned that the departure of many NGOs has left weak capacities for emergency response. SCiSL has, however, maintained its capacity for emergency response throughout the TRP and expenditures on relief have kept in line with the other sectors at between 16%-19% of total expenditure. Only in 2006 did the relief budget fall substantially (to 9%). The Jaffna programme has dealt with displacement every year of the TRP and Batticaloa district spent its entire 2007 budget on the influx of 182,000 IDPs. However there is a risk that as TRP budgets diminish, SCiSL will gradually lose its capacity to respond to large scale natural and man-made disasters.47 This must be avoided, not least to prevent a repetition of the situation in December 2004 and January 2005 when SCiSL staff were asked to stand aside and hand over responsibility to the external Emergency Response Team. Not being allowed to do their job at the time of their people’s greatest need is etched deep in the consciousness of several staff members. Their feelings of helplessness and anger at the loss of ownership of the process endure and appear in the tsunami documentation in various forms.48 The reviewers also found such views represented quite extensively outside SCiSL both in central government and within civil society. More positively, those SCiSL members who were able to organise immediate responses developed an enduring humanitarian commitment and motivation and have gone on to manage large district programmes or departments. A real strength of SCiSL’s relief response has been its evolving holistic and child-focused rapid response to displacement. Distribution of non-food items from stock meets immediate need; tracing through the probation officers attends to separated children (and identifies any under-age recruits amongst them); child-friendly spaces offers support in the often threatening environment of welfare camps; enrolment in local schools follows rapidly, accompanied by appropriate material and technical support to the children and teachers; participatory mechanisms such as school development societies and children’s clubs are supported with small grants and mobilised around disaster risk reduction and other relevant issues; and as the situation resolves itself into re-settlement or re-location, livelihoods support is made available. 46 Please refer to the Box in Annex 3: Reconstructed Baseline for list of objectives and sub-objectives used by the review. 47 SCiSL believes that multi-year money from ECHO, OFDA and others will counter this risk. 48 With direct reference to SCiSL, see for example Forced Migration Review, Special Issue July 2005 p. 39 ‘Small fish trampled in post-tsunami stampede’. More generally the Clinton Report and TEC evaluations focus on the need to restore local ownership.

91

Objective 1: Shelter/Construction SCiSL avoided becoming too embroiled in building works – something it was wholly unequipped for; but it did make an adequate if not totally effective contribution. The scale of the original objective was completely unrealistic (’gap filling’ of transitional shelter for up to 20,000 families) – a symptom of the ’bidding fever’ that afflicted INGOs during the initial planning phase. Sense however prevailed at implementation and SCiSL restricted itself to 1,439 units in Ampara. Jaffna and Trincomalee. In Ampara District, SCiSL’s shelter design was commended by UNHCR and the local authority, but this did not deter SCiSL from withdrawing from this activity with its reputation intact. Construction was part of the sectoral objectives of education (new build schools and pre-schools) and child protection (Social Care Centres). As seen above, results were mixed : fully positive in the cases of the rehabilitation of schools damaged by IDPs, fairly positive in terms of pre-school construction and disappointing regarding Social Care Centres. Approximately 287 schools became temporary shelters for an estimated 300,000 people made homeless by the tsunami. These schools were damaged due to the over-use of their facilities. SCiSL supported the rehabilitation and renovation of 21 of these schools. New blocks were constructed in 14 of these. New blocks in three schools in Jaffna and two in Mullaitivu were not started as contractors were unavailable due to security restrictions and difficulties in getting materials up to these areas. Additionally, of the 36 schools planned for construction in the tsunami-affected areas, only 19 could be completed due to reasons of security and access. Less than half the planned number of 60 Social Care Centres were constructed, but there is no suggestion by its main sponsors SCiSL and UNICEF that the original target should be met. Essentially the project was misconceived on managerial grounds and the temptation will be to let this project quietly slip off the agenda, though this should be resisted. Altogether SCiSL invested £2.4m in construction, 7% of the total TRP budget. Of this £1.81m went to schools and was carried out by SCiSL efficiently and cost-effectively. The secret lay in hiring an experienced engineer from the relevant Ministry, transparent tendering to competent private sector enterprises and close supervision with frequent site visits. £3 million was initially invested in 36 pre-schools. 21 were built but were never considered by SCiSL local offices to give value for money. Moreover SCiSL suffered a £86,000 direct loss when SCiSL’s partner, the Tamil Rehabilitation Organisation was banned by the government before completing its share of pre-schools. Eventually SCiSL went way beyond its original objective of 55 pre-schools, but the bulk were built through more cost-effective local arrangements. SCiSL and UNICEF jointly spent £1.16m on 27 Social Care Centres. This project was fraught with inefficiencies from the start and provides a cautionary tale against trying such ambitious social engineering after an emergency. Sustainability of construction in the education sector was reasonably well assured as government was committed to operation and maintenance of the schools. Moreover there was a logical sequence for SCiSL to follow as the phases of the emergency unfolded:

• working with displaced in schools used as temporary shelters • gap filling with transitional shelter (to help vacate the schools for their primary

function) • rehabilitation of schools after the departure of the displaced • construction of new school blocks and working with child friendly schools.

92

The same connections were not so obvious in the child protection sector, which suffers from overlapping roles and responsibilities across ministries and agencies. A child protection adviser from headquarters pointed out in 2006 that the construction of Social Care Centres was no guarantee that management would be up to the task of joining up these services simply by providing a building for them. The project went ahead as the legal agreement with GoSL had already been signed. Box 1: Construction Case: Under-used Social Care Centres The idea of Social Care Centres as a one-stop-shop for social services locally had been the subject of discussions between the Ministry of Social Services and SCiSL before the tsunami. After the elections in 2005, the Ministry was split and the new Ministry of Child Development and Women’s Empowerment took on certain responsibilities. The officials responsible for the project from the respective ministries did not see eye to eye and this caused concern and delays. The idea was to build 60 centres in each division of the tsunami-affected districts. SCiSL and UNICEF eventually signed a MOU with the two ministries for half this number. This took a while, and UNOPS was requested to halt the implementation until signatures and a commitment to streamline the services were obtained from the ministries. The number was reduced to 27 due to lack of access to Northern Districts. Finally, 27 in total were constructed. Currently, some are functioning, while others are not. In addition to the coordination problems between the ministries, some of the Social Care centres are located a considerable distance from the Divisional Secretariat. The Centres were conceptualised to fall under the direct supervision of the Divisional Secretary. However, being located at a distance precluded the efficient functioning, and the transportation of officials who were expected to divide their time between the Social Care centre and the Divisional Secretariat. Inter-ministerial rivalry after the 2005 election effectively transformed the project into a damage limitation exercise which continues to embarrass SCiSL. Schools and preschools can be expected to have a positive impact on Sri Lanka’s education system in coming years. Although currently not very well served, Social Care Centres are popular with their target group and do have potential. SCiSL feels it is a good investment from the public’s point of view as long as the power struggle between the ministries can be sorted out. Objective 2: Child Protection The overall objective was to ensure that children affected by the disaster are adequately protected by their own communities from harm, abuse, exploitation and neglect. While SCiSL has largely achieved this objective, issues regarding ownership and sustainability of community-level child protection mechanisms have the potential to affect the longer-term impact of the programme. The TRP effectively scaled up a wide range of protection activities that pre-dated the tsunami, for example work with child soldiers, protection work in and around schools (including conflict resolution and peer mediation), and children’s clubs and child protection committees at community level. During the immediate aftermath of the tsunami, SCiSL focused on the protection of unaccompanied or separated children. In coordination with UNICEF and the Department of Probation and Child Care (DPCC), SCiSL worked on the registration, rapid tracing and reunification of separated children in the North and East. The most significant outcomes of this joint protection effort were that children affected by the tsunami did not enter care homes

93

in large numbers, nor did abuse of separated children take place on a significant scale. Also highly valued by the DPCC and the community was the allowance provided by SCiSL to single-parent families that were not eligible for government support. The relevance and usefulness of a number of child-focused activities conducted in welfare centres and later in resettlement areas was reiterated by tsunami-affected children and adults met during the review. These included child friendly spaces, play parks, distribution of play materials, setting up of community libraries and community centres. The provision of vocational training for tsunami-affected youth to prevent their separation from their communities and to reduce their vulnerability to under-age recruitment and other forms of exploitation was timely and relevant. Although the outcome of this work could not be verified by the reviewers, the decrease in the number of children who became institutionalised in SCiSL work locations could be used as an indicator of the effectiveness of these activities – as long as the effect of any other projects from other organisations is also taken into account. Moving towards more holistic programming, and looking at child protection from a prevention angle, SCiSL in 2007 started providing cash grants for income-generation to households in tsunami-affected areas where children were identified as at risk of abuse and/or institutionalisation due to financial difficulties. This was definitely a step in the right direction as until this time thematic staff worked along parallel lines with little integrated thinking or programming. Community members confirmed that this income had contributed towards increasing children’s access to education and better nutrition, thereby reducing their vulnerability to abuse and exploitation. National, provincial and local level officials were appreciative of the training and capacity building provided to government officers responsible for child protection, i.e. Probation Officers, Child Rights Promotion officers and Police Officers attached to the Women and Children’s Desk. In terms of coverage, SCiSL training and awareness has been provided to all Probation Officers and CRPOs, however, questions still remain as to the practical application of this knowledge and training. In a couple of villages visited during the review, the CRPOs were yet to engage with the community on child protection issues. Large numbers of children’s clubs have been formed in tsunami-affected villages in the north and east as well as southern Sri Lanka. While these have been a good forum for children to develop their skills, interact and engage in play and recreation, longer-term sustainability of these has proven to be a challenge for SCiSL, with the large majority ceasing to function once the organisation pulls out of the village. Concerted steps need to be taken to enter into partnerships with NGOs and CBOs to ensure the sustainability of these structures. SCiSL has also invested considerable resources in supporting the government to set up village-level Child Protection Committees to create awareness and to encourage prevention and response mechanisms from within communities. The capacity building and training on child rights and child protection provided to committee members has resulted in attitudinal change about the protection issues faced by children. Positive outcomes of these CPCs include prevention of early marriage, encouragement for drop-outs to return to school and fewer children sent to care homes – for which Probation Officers now serve as effective gatekeepers. Technically, the CPCs are supposed to be run by the government-recruited Child Rights Promotion Officers. However, management constraints have prevented the CRPOs from taking over responsibility for these committees. Poor coordination between the different officials responsible for child welfare is a central concern for the sustainability of these structures. For example, the CRPOs are directly managed by the centralised Department of Probation and Child Care whereas the Probation Officers report to the relevant provincial

94

authorities. Another issue of concern that affects coherence is the difficulty faced by SCiSL to ’sell’ the concept of child protection to the government authorities as the latter operates on a child development mindset and prefers to address child protection through a reactive rather than a preventive approach. In general, the level of coverage in relation to programming in Child Protection is very high, however, to ensure sustainability, and to have a more lasting impact on children’s lives, SCiSL needs to focus more on quality and good practice rather than quantity. It is also difficult to assess impact since the monitoring of qualitative outcomes or impact has not systematically taken place as the focus in reporting has been on quantifying the outputs. Objective 3: Education The objective was to improve access and quality of basic and pre-school education over the five year period of the TRP. SCiSL had substantial previous experience in the sector, with ongoing programmes that it was able to adapt to the emergency. Partly because of an increase in quality (e.g. extensive training in child-centred methods, extra curricular activities, child friendly schools, mobilisation of parents and so on) enrolment has improved across the board. The exceptionally strong motivation of all concerned in education has also played a part. It is fortunate that SCiSL has largely met its original Basic Education objectives because effectiveness is now threatened by a political backlash that has led the Ministry to adopt restrictive financial measures in an effort to regain what it sees as a loss of control over the education system. From 2007 onwards INGOs were increasingly seen as wasting tsunami resources on ’unnecessary’ training and spending too little on the official priority of infrastructure. There is divergence in views on these issues between central planners and decentralised local authorities and the reviewers always found that SCiSL’s was highly appreciated in the districts. However the restriction on direct funding of schools and of direct access to children in schools by NGOs has to be taken seriously. Apart from anything else it causes delays in disbursements to local schools and holds up work. The efficiency of inputs for strengthening the sector are hard to assess, as they are diversified across various stakeholders: for example NGOs for training and support and the local authorities for salaries (or the community in the case of pre-schools). SCiSL has calculated the annual cost of children’s clubs is £10-£12 per child, but there are no estimates for education. When the contribution of other SCiSL programmes is added, e.g. livelihoods, costs are likely to quite high. Inclusion of the popular Child Friendly School system in the national framework for education offers a good opportunity to secure real change for children during 2010-2015. In a programme of this size negative impacts are inevitable. SCiSL’s work with youth has clearly had positive impacts in terms of facilitation and leadership skills. There is anecdotal evidence however, of the LTTE specifically targeting this new youth leadership for recruitment. The risks of leadership training need to be very carefully assessed in conflict areas. SCiSL’s seeks long-term impact for children by helping the pre-school sector to become more fully a part of the education system. The Big Idea has probably correctly identified retention of trained pre-school teachers as the major barrier to the sector becoming more professional. When reliant on parents for their salaries, they tend to move on to better paying jobs once trained. SCiSL intends to overcome this by setting up an endowment fund to underwrite 50% of salary.

95

Despite inter-ministerial rivalries over the sector, the government is still keen overall to control the pre-school system. However, it is unlikely (at least in the medium term) to cover standing costs such as salaries and maybe even professional or in-service training. It would rather these costs were shared by parents, NGOs and the community. However current anti-NGO sentiment has made it reluctant to support SCiSL’s ideas for an endowment fund run jointly by representatives from government, civil society and the business sector. It wants to run the fund itself, so that it can direct investment away from Tamil areas (as SCiSL suspects). SCiSL is quite right in resisting the government’s controlling tendency. ECCD is one of the few areas in which Sri Lankan civil society has room to operate independently from the government and greater government control over the sector would even further shrink the space for civic action. There is no good reason for excluding from the Board the Sarvodaya social movement and NGO, which has established large numbers of ECD Centres over the last 50 years. Also SCiSL needs to carefully assess the risks to children of a more professional early childhood sector. Pre-schools could be used to raise standards in basic education by making an early start on literacy and numeracy – rather than focusing upon age-appropriate development through play. Given the vast expectations of education in Sri Lanka, this would doubtless be a popular approach. SCiSL is aware of this risk, which has a prominent place in its recent Education Thematic Programme Plan. Early signs of this risk and of SCiSL’s reaction are evident in the south, with parents urging pre-school teachers to start literacy and numeracy lessons as this is often considered a competitive advantage for children to gain entrance to good primary schools. To address these pressures, SCiSL has started awareness sessions for primary school teachers. In terms of sustainability, the exit strategy is to move from rehabilitation back into long-term programming, with much less direct implementation and closer work with partners. But ongoing conflict means SCiSL has to maintain the capacity for emergency response in education. The challenge is to maintain this humanitarian response capacity while putting programming in place for the longer term. In theory SCiSL is well placed to manage a hybrid programme as it has developed a strategic plan for education and can count on a well-trained staff. In practice, the task will be demanding. It was entirely relevant for SCiSL to engage fully in basic education. There is no doubting the commitment of all stakeholders, including government, parents and children, to the education system. SCiSL’s varied education response demonstrated considerable agility. Getting children back into school at the earliest opportunity is entirely appropriate in this context and makes large scale counselling/psychosocial work unnecessary. Although better than much state-funded education in middle income countries, the Sri Lankan system is dated by western European standards. The temptation to modernise it may have been a factor behind some initiatives. The case raises the issue of whether an emergency is the appropriate time to attempt systemic change within an education system from the outside. It is likely that there are similar efforts at radical change in other sectors which, collectively, filtered upwards to the capital to attract unfavourable political attention and feed hostile media. This does not argue against change per se, but suggests that there is an appropriate time and place for it. In terms of mechanisms for downward accountability, the TRP has made substantial efforts to mobilise and activate a variety of community based organisations with a focus on education. These will need continued support via local NGOs for several years.

96

Coordination between the government and SCiSL started well, on the strength of SC’s long-term presence in the country. SCiSL and other INGOs believe they are singled out for purely political reasons. While it is true that the government is very unhappy with INGO resources flowing to LTTE areas, this is not the full picture. What most upset education officials was that they could not compete with the resources available to the INGOs. To make matters worse from their perspective decentralised education zones at provincial and district level responded to the generous and flexible incentives supplied through INGOs by reporting directly to them and overlooking the Ministry. As a result the Ministry complained to the reviewers that the INGOs were better informed than they were. It is felt that INGOs like SCiSL have had the effect of upsetting arrangements under the Sector Wide Approach to education negotiated with several bilateral donors from 2004. The level of coverage is very high and cannot be sustained. The programme covers all six tsunami-affected districts, and is currently pre-positioning to address displacements from the latest offensive in the north. For example, in 2007 the programme reached 44 schools across the island, benefiting at least 17,600 children – many more if the IDPs in Batticaloa are included. If the Big Idea is successfully negotiated and attracts matching funds it will reach over 200,000 3-5 year-old children in 4,000 EC centres. This is a Very Big Idea. However size may not be so relevant at the current stage of SCiSL programming and a reduction in quantity could open up opportunities for SCiSL in promoting the rights of disabled and other very disadvantaged children, which do no feature strongly enough in the programme. Objective 4: Livelihoods The livelihoods objective was to support the income generation strategies of female-headed households and other vulnerable households. A 65% success rate for SCiSL-supported income generating activities in 2006/2007 confirms that the programme has largely met its original objective.49 This is significant, given that it was a new thematic area for the country programme. The area is relevant in view of widespread poverty and huge commitment by parents and children to education. It is also appropriate for SCiSL on the grounds of school retention and child development. The initial concentration on replacing lost assets and providing in-kind livelihood support after the tsunami was not based on sound assessments nor properly followed up; neither were issues of sustainability and direct impact on children given much consideration. The rapid proliferation of INGOs also resulted in the overlap of benefits. However in 2006 and 2007 more effort was made to maintain the focus on children. Families where children faced difficulties in continuing their education and/or were vulnerable to abuse and exploitation were provided with livelihood support. Based on a PRA exercise and lessons learnt in 2005, in-kind support was replaced in 2006 with cash grants. This has been more popular since it gave the family the opportunity to make independent decisions about livelihood options. The cash grant was coupled with professional training and sensitisation on the efficient use of income, an aspect of the programme that was highly valued by the beneficiaries. Our findings indicate that income has increased as a result of the cash grants provided in 2006 and 2007. The impact assessment conducted by SCiSL confirms these findings. Women reported that the increase in income has directly translated into benefits for children. Most significantly, the additional income was spent on children’s nutrition, health and education, especially in relation to education supplies and additional tuition classes. However, the general finding was that savings have not increased due to the sharp rise in the cost of living. 49 As evidenced by an impact assessment carried out by SCiSL.

97

In certain instances, women who received cash grants and in-kind support from SCiSL have graduated to interest bearing schemes with other local organisations. This was more evident in cases where SCiSL worked through local partners as the majority of these local NGOs had their own credit and savings schemes. This is an important consideration for future programming aimed at ensuring the sustainability of business enterprises, especially when a choice needs to be made between direct implementation and working through local partners. In addition to the cash grant, a new programme for 2008 will give an unconditional monthly cash transfer50 to 1,196 poor families in the tsunami-affected districts along the eastern coastal belt. The main reason being to provide a safety net for those living below the poverty line until the livelihoods assistance generates an income. SCiSL needs to be mindful of creating undue dependency on the part of the beneficiaries. The programme also aims to influence state-run poverty alleviation and safety net programmes to consider children and their rights. This does seem somewhat ambitious, especially given the current political context and the corruption-ridden safety net programme that currently exists and through which nearly half the country’s population receives support. Initially coverage was vast and unsustainable as in-kind support was provided through lists obtained from the local government officials. Subsequently, the focus was rightly narrowed and specific target groups were identified through community consultations, the main idea being to target smaller numbers and improve the quality of the programme to ensure sustainability. The general opinion of partners and beneficiaries was that SCiSL had reached the poorest, according to predetermined selection criteria. However, more effort needs to be made on the part of SCiSL to monitor this as evidence from one village in the east indicated that the minority ethnic group had been excluded from the selection process. The majority of income earning ventures started through the cash grants have succeeded and were ongoing at the time of the review. However, certain sectors continued to lag behind others, especially due to the increasing price of inputs. To ensure sustainability, more effort needs to be made to encourage field staff to translate the learning from impact assessments and other studies to their work in the field. There was evidence that new businesses continued to be financed (e.g. poultry farming) even after research showed that they were not very viable. The beneficiaries also need closer support and follow-up to sustain and expand including market linkages and graduation to mainstream credit schemes. Objective 5: Emergency Preparedness The objective was to strengthen the capacity of vulnerable communities to survive and recover from disasters through child sensitive Disaster Management plans at divisional level and child led DRR in 265 communities. This was not an area of previous SCiSL experience and so there was an intensive learning curve. However DRR emerged as a child-focused way of addressing emergency preparedness, which SCiSL and other agencies supported through the specialist agency RedR. SC was right to work on emergency preparedness affecting the population at large through a strategic partner, while focusing on child-led DRR itself.51 Child-led DRR was welcomed by government and became part of the post-tsunami National Disaster Risk Management Plan. Over 100 schools already have Disaster Management Committees. SCiSL has reached its targets and gone well beyond them in some cases. Chaired by the Government Agent (or their assistant) 12 districts (half of the country) implement child

50 The American Red Cross-funded Household Economic Security Programme. 51 Emergency Preparedness (simulation, pre-positioning of stocks, emergency response training). DRR (addressing the underlying impact and causes of natural hazards).

98

sensitive Disaster Management Plans. At divisional level, the number is 114, well beyond the original SCiSL target of 24. Regarding child led DRR in the community, SCiSL has implemented DRR activities through as many as 7,000 children’s clubs in all of the 265 communities in which SC works. The most effective DRR initiative was hazard mapping and mitigation with 92 children’s clubs in six tsunami-affected districts where SCiSL works. In 2006 and 2007 SCiSL helped schools in the six tsunami affected districts to set up DRR School Committees, directly involving 18,000 200 teachers and 2,700 children. Altogether SCiSL estimates that it reaches all 17,600 children of the 110 child friendly schools it works with. School is the most efficient medium for promoting DRR as it relies on existing infrastructure and human resources. There was an agreement in principle in 2005 to introduce DRR into the school curriculum for years 7 and 8 (upper basic education) on the recommendation of the Ministry of Disaster Management and Human Rights. Given that much of the expertise in DRR exists with NGOs, efficiency may suffer in the future if the Ministry of Education continues to prevent NGOs working directly with schools. Though doubtless difficult in the current climate, SCiSL may be able to contribute best with supporting the Ministry to develop appropriate learning materials for schools. Children’s clubs are less efficient as the communities themselves have to maintain their activities. However SCiSL and its NGO partners have no restrictions in working with the Clubs, suggesting that this may be where they should place their emphasis until cooperation with the Ministry of Education improves. As in other areas of SCiSL operation, impact at local levels is much more evident than at the national level, where difficult relations with ministries (or between ministries and departments) have reduced the positive impact of policy change based on local experience. Schools, District Agents and Divisional Secretaries all welcome SCiSL’s innovative DRR work – especially in the East and the South where tsunami affected districts are not currently affected by conflict. The GA of Ampara, for example, is particularly keen to work with SCiSL and UNDP on the child-focused aspects of the District Disaster Management Plan. Much depends on SCiSL’s strategic choices in coming years as to whether child-led DRR becomes embedded in Sri Lanka structures and systems. The natural tendency will be for other priorities to supplant it as the tsunami recedes in memory. This would be a shame as DRR does seem to have caught the imagination of children, teachers and local administrators and clearly demonstrates the added value of SCiSL as an international organisation. Children are enthusiastic but the work requires long term interest and commitment from local NGOs, social workers, and local government. In contrast, children leading the DRR activities in a school in the south were confident of the sustainability of their efforts, without support from an external organisation, due to the general interest prevailing at all grades within their school. It is important for SCiSL to answer to the question posed by a district manager, ’after mapping hazards, what?’. The question is pertinent. An obvious reply would be for children’s clubs to follow up on particular identified risks and mobilise the community to reduce them. This has already started happening in SCiSL programme locations in the deep south, prone to frequent floods and land slides. More broadly there is potential for SC to conceive of child-led DRR and Emergency Preparedness as an opportunity for disaster prone countries to become more systematic and self-reliant in predicting, preventing, preparing for and mitigating floods, cyclones and other regular natural occurrences. In this sense sustainability lies in institutionalising capabilities across the range of knowledge, skills and practice required for dealing effectively with disasters. In 2006 SC came close to formulating Child-led DRR as a Big Idea for the region. It certainly would have

99

fitted the visionary ambitions of the Big Idea and offered the Alliance with another role: beyond ‘doing’ emergencies itself, towards providing the know-how for local institutions to respond themselves. The work in Child-led DRR around the region gives the Alliance a base from which to credibly claim the expertise to lead in this new field. However it does need to be sustained and become more than a post-emergency response. Emergency preparedness is clearly relevant to a country affected by chronic conflict and with a high incidence of natural disaster such as cyclone and flood. Capacity building is also especially relevant in view of the determination of Sri Lanka not to be dependent a second time on international relief. Child-led DRR is particularly appropriate for SCiSL, especially as there is substantial experience in the region to learn from. 4. Strategic and management issues Ways of working – working operationally For SCiSL as an organisation, the tsunami meant that within a very short time a largely partner-based organisation became fully operational. The process was driven largely by the need to spend large amounts of money as quickly as possible: though flexible, DEC funding was strictly time-bound. SC UK’s brand image was also of great importance, and expatriate managers were very conscious that they were working to a domestic audience in the UK. In response to this stimulus, SCiSL consciously set out to become a large scale and effective ’spending machine’. Programmes and activities mushroomed, and the earlier emphasis on rights-based programming was lost. An initial preference for distribution of high cost assets and infrastructure gradually shifted with time to more developmental objectives. To a large extent SCiSL succeeded in its ambition – it rose to the occasion and in the words of an earlier review grew into a sizeable ’corporation’ with district office resources larger than medium sized SC country budgets. SCiSL continues to attract operational grants from new donors that find its programming relevant and effective. It will remain well resourced even after the end of DEC funding – at a considerably higher level than before the tsunami. Some of the drawbacks of the direct operational high-spend approach are now identifiable:

• Spent in haste, a significant amount of money was wasted (by INGOs in general – not SCiSL in particular) on duplication or ill-conceived investments. In Sri Lanka the international tsunami response is a byword for profligacy.

• At least initially, much of the highest expenditure did not focus directly on children, and so did not play to SCiSL’s natural strengths.

• The need to spend has created an operational ’culture of giving’ making it very hard for parts of SCiSL to move back into developmental ways of thinking.

• SCiSL’s learning is stuck in output mode, with little attention to outcomes. Numbers of beneficiaries reached do not provide evidence about the changes in their lives.

• SCiSL’s success may be at the cost of local NGOs – in one case even of the local branch of an international organisation.

• The focus on SC’s brand image in the UK placed the Sri Lankan audience in second place of importance, something the local press and others were quick to catch onto.

• The unfamiliar scale and profile of the new corporate INGO unsettles the government without furthering SCiSL’s positive stance on impartiality. Indeed in some of the very sensitive issues related to the conflict and the recruitment of child solders, it could be argued that a far lower-profile operation might be more effective.

Ways of working – working with partners SCiSL has already wisely determined to rebuild its partnership base. It has developed a partnership policy and committed a third of its 2007 budget to partners. It made sense to have local, decentralised offices, even though the intention was to facilitate operational

100

funding mechanisms - so the potential advantage of good contact with local partners and close supervision was not fully taken advantage of. Local partners of all sorts speak very well of SCiSL. Tellingly, the greatest compliment they pay a district office is ’that it is more like a local NGO than an international one’. Even so the SCiSL approach to partnership is fairly instrumental. It asks: ’how can we convince our partner to implement our strategy?’. Predictably, the NGO partners want more say on strategy and its implementation, more focus on capacity building and longer term funding relationships: Annex 4 summarises the assessment by 18 partners of SCiSL in Ampara during a half-day workshop. SCiSL needs to pay attention to these demands as it is dependent on these NGOs to support and continue building the capacities of the vast number of child-focused community based organisations (CBOs) that SCiSL and other INGOs created as beneficiaries. In Sri Lanka these include Child Protection Committees, Children’s Clubs, School Development Societies, Disaster Management Committees, Pre-school Parents’ groups and so on. These manifestations of civil society at a local level represent one of the more important dividends of TRP investment, which need to be nurtured. Partnership with local government is also essential as a range of local government employees are the means of embedding much of the TRP’s work. They include social workers, probation and child rights officers, education supervisors and many others. The challenge for SCiSL in the remaining years of the TRP is to develop a hybrid way of working that retains essential operational capacity for the inevitable emergencies but builds on its partnership base for sustainable benefits to children in the tsunami affected districts. This is a difficult, but essential balancing act. However the real priority for SCiSL is to ask itself what reasons other than xenophobia inform the current low opinion of the UN and INGOs, and seriously address the drawbacks. Prioritising programme objectives (i.e. choices re sectors) As a means of responding flexibly and keeping the all important expenditure flowing, SCiSL gradually reduced the number of programmes that it needed to maintain active from 7 to 4 as it moved from relief through recovery to development. Relief distribution was the first area to be scaled down, followed by shelter and eventually, construction. It remains with Education, Livelihoods/Social Protection, Child Protection and Child Rights. Suffering chronic internal conflict, relief in Sri Lanka all too regularly reappears as a priority. Child Rights, which emerged in 2006 very much as an after-thought to the tsunami response, has gained little traction within SCiSL programming. The language of rights is conspicuous by its absence, despite extensive training of staff and partners. DRR in reality amounts to a fifth programme, even if it is described as cross-cutting. The real issue taxing managers is not so much the number of programmes but their practical integration in the TRP response. Despite great efforts to harness activities for addressing children’s needs holistically, silo thinking at the level of implementation persists. This combines with a preference for ‘giving’ over more demanding development approaches – which is compounded by the tight deadlines for implementation. It is the unwanted legacy of the relief and recovery operations, in which sectors pursued their objectives independently from each other. There are however good examples of joined-up programming (see the effect of livelihoods interventions of education in the box below) and managers need to recognise that it will take more time to adjust back to development practice than it took to gear up for the relief effort, the skills and attitudes being so very different.

101

Targeting of beneficiaries (by age, gender, economic status, geographical location etc) Although SCiSL’s general principle of reaching out to all communities was sound, targeting was still based upon distinct communities. This meant that families could be left out if displacement and relocation meant they could not reside with their own communities. There was also indication during field visits that beneficiaries tended to be amongst the most vocal and articulate. The visibility of differently abled or out of school children was low in the work presented to the reviewers. This suggests that a cost of going to such scale could be the exclusion of at least some of the most vulnerable. Child participation This has been a particular strength of the Sri Lanka TRP. Children responded well to the opportunities offered and demonstrate confidence in expressing their views as well as the capacities to function in groups. Children especially appreciated having a role of their own in emergency preparedness as this empowered them. Children consulted by the team were found to have confidence and good leadership and facilitation skills – clearly the fruits of good quality training. Box 2: Children’s priorities 52

Of the activities conducted by SC, through its partner, the children prioritised the following:

1. Learning and applying leadership training/skills. They now have the confidence to speak out and adults actually listen to them.

2. Livelihoods support has improved their education – can buy supplies 3. The evening classes have been very useful 4. People from Tamil and Muslim Communities are mingling 5. Play park has been useful 6. Child-led DRR – learnt the big role children can play 7. Many school drop-outs have gone back to school

It should be noted however that school teachers do not always seem comfortable with children’s new found voice. Clearly it will take the profession longer to adjust to new methods, and the children will likely face a backlash unless teaching skills are also upgraded. Advocacy Sri Lankan ministries have been surprisingly open to changes suggested by INGOs. SCiSL examples include the Ministry of Disaster Management’s welcome of child-led DRR, the Ministry of Education’s support for child friendly schools and the same ministry’s acceptance of civics education and peer mediation in the national curriculum for 12-14 year olds. There is a question about how these innovations will actually be implemented, as well as the rafts of new standards that departments were exposed during the emergency. Within its field, SCiSL could useful help the ministries prioritise amongst these and set realistic implementation timelines. Recruitment and orientation of staff With the decision to go operational on a very large scale, recruitment of high quality personnel became a major issue. SC was amongst other INGOs that drew down on local social capital (from local NGO staff), but even so much effort was needed for induction and training of other new staff. Even for SCiSL staff, parts of the TRP response were completely new to them. Many staff members were involved in the child rights situation analysis that underpins the new strategy, as well as the thematic programme plans that came out of it. SCiSL has also invested in leadership training for district office managers. Like the Colombo 52 Results of an exercise in Central Camp, Ampara with 12 youth and 50 children.

102

office, district offices will need to retrench staff as budgets reduce and SCiSL returns to partnership ways of working. A positive initiative is to encourage skilled staff to set up their own NGOs and partner with SCiSL. Other management issues The technical advisers brought in competence to a staff much of which had no previous SC experience, but even district offices with experienced SC local employees complain they simply could not absorb the sheer quantity of advice from international advisers. At least at the outset, high turnover made it harder for short term international advisers to be attuned to Sri Lanka’s complex context. SCiSL has persisted with a large complement of international staff longer into the development phase than should be strictly necessary.53 Although there has been some conscious bringing on of permanent local staff, many advisers are hired to carry out specific tasks other than capacity building of counterparts. This could become a serious problem were the government to clamp down altogether on expatriates at short notice. Advisory support is of course necessary but the programme will sooner or later need more sustainable capacities. SCiSL has ensured that staff members are aware of SPHERE and other humanitarian standards. Construction standards have been conscientiously observed and sometimes commended – as for example when local government and UNHCR singled out SCiSl temporary accommodation as the best example of design and construction in Ampara The security of staff is taken seriously and security systems are particularly well managed, with excellent linkages between the centralised and decentralised operations. Neither management or staff are aware of the implications for the programme of the Clinton Report or the work of the Tsunami Evaluation Coalition. This represents a missed opportunity for learning within the team. Although SCiSl has participated in the wider learning opportunities presented by the TRP (e.g. lessons learned workshops in Chennai, Bangkok), there is much to be said for in-country dissemination and reflection of, for example, the executive summary of the TEC synthesis report – especially as this is to be repeated. TRP Governance SCiSL’s unified Alliance presence appears to have been a real operational success. Senior management found the steering committing unfailingly helpful when required, hands off when not. All Alliance members that were willing and able to contribute did so. If the Alliance needs an example of a working unified presence, this is it. Responses to conflict Unlike many of the INGOs that thronged the eastern coast of Sri Lanka after the tsunami, SCiSL was uniquely placed to see the disaster in the context of 30 years of war. Many affected by the tsunami were already displaced, or were to become so by further fighting or flooding. Few agencies appreciated the importance of extending support equally to different groups or of addressing the needs of those directly and indirectly affected by the tsunami. SCiSL did so. It also took pains to balance its geographical location so as to cover all three main communities: Muslim, Sinhalese and Tamil. Even so more than two thirds of the programme was implemented in Tamil areas, which did not please the hard-line government elected in 2005. SCiSL has had difficulties with both LTTE and the GoSL, which at least suggests impartiality. 53 SCiSL makes the point that donors (including UNICEF, USAID and ECHO) all consider that it is not possible to implement impartial programmes without them being overseen by international staff who also protect national staff from political and security interference.

103

In the context of a popular war, SCiSL must do what it already does – i.e. limit itself to humanitarian assistance and service provision wherever it can gain access to threatened children and their care givers. Such areas are increasingly restricted because the military limits access in areas where operations are under way, as in the Vanni; or to displaced populations that it distrusts, as in the camps of Jaffna. The composition of SCiSL’s staff enables it to field personnel safely to any part of the island. This gives it recognition as an impartial actor. There are few local civil society organisations with the same orientation, but SCiSL might benefit from closer association with Sarvodaya which, although not operational in the areas controlled by LTTE, also works with all populations of the island, but without incurring the wrath of the government. 5. Recommendations

1) Save the Children should maintain essential operational capacity for addressing the impact on children of man-made disasters in Sri Lanka and come up with innovative measures to strengthen capacities for local ownership in managing natural disasters.

2) SCiSL needs to take steps to ensure that the programme maintains the momentum achieved in empowering children through DRR. It will specifically need to make sure that DRR’s new status as a cross-cutting issue does not mean it gets lost in other programming.

3) For broad policy impact based upon its DRR work in the tsunami affected areas, SCiSL needs to find reliable partners and make a specific contribution, such as support for the development of DRR curricular materials for schools.

4) SCiSL education sector plans should be aligned more closely with the lead Ministry. This means taking a longer-term and more strategic view on working through government in education. SC should not seek to circumvent central government by, for example, remaining operational beyond time agreed, or by replacing government at local level.

5) Save the Children should focus on strengthening civil society organisations related to schools, such as the School Development Societies, Children’s Clubs, DRR groups and so on.

6) A sharp reduction of coverage is advisable in education, as in other sectors, with longer term support to fewer schools and communities in the interests of close follow-up of incremental change and closer connections with other SCiSL sectors.

7) The Big Issue Early Childhood Trust Fund has potential and should be pursued. It could be made better by conceiving it as a long term foundation rather than a time limited Trust Fund. This would ensure that it maintained its community ownership. In order to ensure that Big Issue is in the long term interest of the pre-school age child, Save the Children should commission an independent risk assessment locally before making firm commitments.

8) SCiSl and its partners UNICEF and the Ministry of Social Services should make a concerted effort over the next two years to render the 27 Social Services Centres useful. If the original purpose of joined up child protection services at divisional level cannot be met immediately, other options for using the buildings that benefit children should be explored meanwhile. At all costs SCiSL should avoid inadvertently impoverishing local authorities with high maintenance cost of buildings that serve no useful purpose.

104

Conclusions In the wider corporate interests of the Save the Children Alliance, SCiSL and its staff have loyally gone the extra mile to prove the organisation’s professionalism. Their major single achievement has been to create a gigantic spending machine and keep up with overwhelming spending requirements. It is to SCiSL’s immense credit that it has not only spent to the hilt and sometimes beyond, but has rarely done harm and has often found ways of spending usefully in such a resource saturated environment. SCiSL has achieved many of the targets it set itself (the main criterion of effectiveness in this review) and numbers its beneficiaries in tens of thousands. It was relentless in the pursuit of milestones. The energy expended was colossal and this should not be forgotten as the team coasts towards the final stretch of the TRP and hopefully finds some time for reflection as it returns to its long term task of development in the context of chronic conflict. On the down side there was often more haste and less speed. Milestones were all too often set without regard to where they led. The real outcomes are uncertain. The cost to SCiSL’s national reputation and goodwill has been heavy amongst some sections of the community and there are long serving members of national staff who will never agree that the ends justified the means. Senior international staff find it dispiriting to work in an environment where the organisation is made to feel it has outstayed its welcome. Their consolation is the overwhelming support shown SCiSL’s work at local level. With two years of the TRP to go, there is still potential for SCiSL to pull off a real victory. The Big Idea will be crucial. Will it go for ECCD, the traditional option? Or will it break new ground and start equipping South Asian society to bring contemporary skills to bear on natural disasters? The question whether SCiSL could have responded differently, less wastefully perhaps, more in keeping with local ownership, belongs to the final evaluation. Ideally this should ask if Save the Children could have called a halt to the fund raising at an earlier stage on the grounds that both its own, and the resources of other donors and INGOs were already more than required to do the job? In short, would less have eventually meant more? Had it done so it would certainly have gained credit in Sri Lanka This point raises another difficult question for all major INGOs – about whether the decision not to put an upper limit on the Tsunami appeal may in the end prove to be damaging to their overall credibility?

105

Thailand Programme 1. Programme description Methodology This evaluation included a short five-day visit to Thailand by the expatriate consultant. A national consultant was not recruited. An SCS senior team member acted as interpreter during the field visits. The evaluator did not feel that this arrangement jeopardised the integrity of the evaluation. The field visits took place during the school holidays, thus limiting meetings with children involved in the projects. SCUK’s programme finished in mid-2007 and therefore was less of a focus of the field visits, but coverage of that part of the Alliance’s work was augmented by interviews with former programme directors. Given the time elapsed and resources available, this report is inevitably dependent on secondary sources such as independent project evaluations. Country Context and Impact of Tsunami Thailand is ranked 78 in UNDP’s Human Development Index, towards the top of middle income countries, and therefore, along with the Maldives, the richest of the Tsunami affected countries.54 In spite of periods of substantial economic growth, there are substantial disparities in the status of women and children, partly related to whether they live in urban or rural areas, are members of ethnic minorities or are unregistered migrants from Burma. UNICEF identifies protection needs for more than a 1 million children, including those in orphanages, those with disabilities, living and working on the street or working in under-age employment.55 8,212 people died in Thailand, 18,000 were displaced and about 1,000 children lost one or both parents.56 Deaths amongst foreign tourists meant that Thailand received a high level of international media attention. The disaster had a substantial impact on the tourism, fishing and aquaculture industries in the affected coastal strip on the Andaman Coast. In the year after the disaster, tourism in that area fell by 80%, seriously impacting on jobs. Burmese migrant workers, mostly unregistered and unrecognised, were one of the population groups most badly hit by the disaster and their plight was to some extent ‘uncovered’ by the disaster. Officially about 800 Burmese migrants died in the disaster but, due to lack of registration, the number may have been much higher.57 The bulk of the immediate and substantial relief response and subsequent recovery work was carried out by the Thai government, armed forces, the Thai public, businesses and companies. The basic needs of most of the affected population in terms of shelter, water and sanitation, food and health care were rapidly met. By the end of 2007 all the official camps were closed and much of the reconstruction work finished, including the construction and repair of schools.

54 UNDP. Human Development Report 2007/2008 (http://hdrstats.undp.org/countries/data_sheets/cty_ds_THA.html) 55 http://www.unicef.org/infobycountry/Thailand_1022.html 56 Including 2,448 foreigners. The highest foreign death rates were of Swedes (543) and Germans (468). Telford, J and Cosgrave, J. ‘Joint Evaluation of the international response to the Indian Ocean tsunami: Synthesis Report’, Tsunami Evaluation Coalition (TEC). 57 SCUK. DRP for Tsunami Affected Countries – Thailand Submission to DEC. 29 April 2005.

106

Overview of the Save the Children Alliance Response SCUK closed its Thailand country programme in 1996, but maintained a regional office in Bangkok and continued some work in Thailand and relationships with Thai NGOs through a regional programme on the trafficking of children, managed from Bangkok. SCUK was therefore able to carry out a series of assessment very quickly after the disaster and develop a response programme implemented by newly recruited Thai staff working with local partner agencies. SCS was also involved in initial child protection assessments. Given there was an adequate, even overwhelming, response in the relief assistance sector, both SCUK and SCS mainly focused on promoting child-oriented approaches within the ongoing relief and rehabilitation effort and reaching particularly neglected population groups. SCUK closed its Tsunami response programme in mid 2007, with some residual work being carried out in relation to Burmese migrants within the regional anti-trafficking programme. SCS moved its regional office from Hanoi to Bangkok in mid 2005, a move that had been planned earlier but triggered by the Tsunami disaster and the huge impact of and response to the disaster in Sweden. Initially funding SCUK’s work, SCS then developed its own programme and continued the jointly developed child-led Disaster Risk Reduction (DRR) project after SCUK closed its programme, with part funding from SCUK. 2. Summary of Programme Objectives and Key Findings As described above, both SCUK and SCS have managed projects in Thailand, so the programme objectives are attributed accordingly.58 Sectoral objectives have changed over time and those given below are generally the most recent. Overall Objectives: SCUK: To promote the material and psychosocial recovery of children and their families in tsunami-affected areas of Thailand as well as increasing the resilience of children and their families in vulnerable communities to future disasters. SCS: The rights to development, protection and participation of children affected by the Tsunami disaster in Thailand are fulfilled. Impacts of the project are sustained in the long-term through capacity building of community based youth groups and documentation of project process and lessons learnt. SCS partners and other actors are able to integrate child rights programming and children’s participation in their programmes. Specific Objectives: Education (SCUK): To ensure that at least 80% of Burmese migrant children aged 5-11 in target communities in PhangNga Province are regularly attending quality learning centres or local schools and that the centres are officially recognised locally so that their continued operation is secured. (Overlaps with child protection and psycho-social issues below) Health (SCUK): SCUK: Knowledge of HIV/AIDS prevention and care strategies is significantly improved in secondary school children and community health workers in Ranong, PhangNga and possibly Phuket.

58 SCUK/SCS means objective shared by both agencies with the implementing agency(ies) in bold.

107

Livelihoods (SCUK/SCS): To enable 25% of families in 10 communities to meet the basic needs of their children through safe enhanced livelihood opportunities. Child protection and child participation (SCUK/SCS): Children’s Centres - 2005: To facilitate the provision of psychosocial assistance for the most vulnerable children in the most vulnerable tsunami-affected areas. Child safe organisations training project (SCUK/SCS): Children are protected from abuse and exploitation, including from the very organisations working to care for them, by the development of organisational ethics and safeguards. (Carried out in conjunction with ECPAT International and UNICEF). Case management of separated and orphaned migrant children (SCUK): To ensure that 140 children (later adjusted to 96) identified through the 2006 rapid assessment in PhanNga are in safe, durable solutions. To document a case management system that is suitable for replication in the migrant context. Child-led DRR (SCS/SCUK): Reinforce child led disaster risk reduction operations in order to reduce community vulnerability to hazards. To have shared with the Ministry of Education the value of this approach so that they can explore replication nation-wide. To strengthen children’s skills so that they understand the risk of disasters in communities and are enabled to take a lead in reducing the risks and impact of potential disasters.59. Child Friendly Spaces (SCS) Promote activities to create a child friendly physical environment in Tambon Administration Organisation (a government office at the provincial level) in Ranong. Promote and support young social volunteer, and support to children’s groups to create a child friendly physical environment with adults.

59 From CLDRR: A Practical Guide

108

Summary of Findings of Save the Children Tsunami Response Programme in Thailand

Sector Effectiveness Efficiency Impact Sustainability Appropriateness Coherence Coverage Overall

Generally good. Relatively efficient. low-cost operation.

Generally positive.

Good. Good, with possible exception of some early interventions.

Generally good.

Relatively small interventions focused on neglected or non-recognised groups.

1. Education (SCUK)

Satisfactory. Appears satisfactory.

Significant impact on school attendance.

Likely to be good.

Good. Fits well with SC emphasis on reaching neglected and vulnerable groups.

Limited by capacity of implementing partner.

2. Health (SCUK)

Project used drama.

Not verifiable. Not verifiable. Not verifiable. Good. Appears satisfactory.

Limited by capacity of implementing partners.

3. Livelihoods (SCUK/SCS)

Satisfactory. Not verifiable. Appears limited. Not known. Appropriate in the early stages of the response.

Appears satisfactory.

Small scale interventions.

4. Child protection and child participation (SCUK/SCS)

Some evidence of positive outcomes.

Not verifiable. Not verifiable.

Not known.

Good. Appears satisfactory.

Limited by capacity of implementing partners.

5. Child safe organisations training project (SCUK/SCS)

Good. Not verifiable. Satisfactory. Good in terms of training material produce and take up by other organisations.

Good. Good. Initial training covered 28 organisations.

109

6. Case management of separated and orphaned migrant children (SCUK)

Good. Not verifiable. Impact limited by short term nature of the project.

Not verifiable. Relevant in the context of Burmese migrants.

Close collaboration with and part funding by UNICEF.

Targeted at cases identified in an earlier assessment.

7. Child-led DRR (SCS/SCUK)

An effective project that has generally achieved its objectives.

Not verifiable. Good. Satisfactory. Good. Good. Piloted in 20 schools.

8. Child Friendly Spaces (SCS)

Small pilot programme that appears to be effective.

Insufficient information available.

Financial Summary of SC TRP in Thailand

SCUK 2005 budget 2005

Actual 2006 Budget

2006 Actual

2007 Budget

2007 Actual

2008 Budget

2008 Actual

Emergency phase $134,095 155729 Shelter/ construction Health / HIV/AIDS $69,420 $63,438 $125,926 $106,967 Water and sanitation Education $173,008 $169,854 Child protection / child rights $109,816 $92,247 $145,647 $172,820 $33,780 $41,656 Livelihoods $341,318 $62,246 $29,844 $23,558 Emergency prep $3,995 $1,783 $127,134 $128,155 $59,709 $52,165 Policy work $44,616 $48,570 Sub total Supplies/Materials

$658,644 $375,443 $601,559 $601,355 $138,105 $142,392

Non-Personnel $38,742 $2,778 $23,207 $22,693 $10,040 $8,233

110

Personnel $320,513 $198,480 $207,615 $212,840 $50,178 $41,875 Personnel Support $133,040 $69,220 $52,962 $65,671 $6,102 $7,919 Capital expenditure $15,581 $12,371 $2,113 $2,113 Other Direct Projects $19,661 Management Support $34,705 $32,915 $8,874 $1,810 $6,441 $5,451

$1,201,225 $710,868 $896,330 $906,483 $210,867 $205,870 TOTALS (SCUK DEC)

SCS contribution (GBP) $349,927 $360,425 TOTALS (SCUK) $1,551,152 $1,071,293 $896,330 $906,483 $210,867 $205,870 TOTAL SCUK EXPENDITURE 05-07 1,093,300

SCS Budget 2005 Actual

2005 Budget 2006

Actual 2006

Budget 2007

Actual 2007

Budget 2008

Actual 2008

Contribution to SCUK $375,120 $386,374 0 0 0 0 0 0 SCS Tsunami response $207,078 $103,158 $176,150 $174,900 $257,895 $246,954 $60,449

$32,042

DRR 0 0 0 0 $51,266 $57,049 0 0 TOTALS (USD) (1)

$582,198 $489,532 $176,150 $174,900 $309,161 $304,003 $60,449 $32,042

TOTALS (GBP) (1) £271,925 £228,636 £83,398 £82,806 £154,284 £151,710 £32,487

£17,220

TOTAL SCS EXPENDITURE 05-07 $925,053 including contribution to SCUK in 2005 $360,425 UNICEF contribution to CMMC project

$18,429 $18,429

(1) Exchange rates used for conversion at 31 December of each year

111

2.1 Other information SC worked primarily through local NGOs and in liaison with a number of Thai government departments. SC also worked closely with UNICEF and ECPAT International.60 The tsunami longer term response work effectively started from April 2005 as it took time to recruit a Thai team. The limited amount of earlier relief work was handled by staff from SCUK’s regional office in conjunction with pre-existing partner NGOs. Funding for SCUK’s work came primarily from the DEC with contributions from SCS and UNICEF and a small amount of corporate funding. SCS’s funding was mainly from its own fundraising efforts in Sweden with some contribution from SCUK after the latter’s end of programme in July 2007. 3. Principle Findings and Recommendations This section assessed the programme objectives against the principal programme criteria and develops the Summary of Findings Table shown above. Overall Programme Generally the work of both SCUK and SCS work has been effective, with positive outcomes in most areas. There is a concern in a number of evaluations that the projects were too short in duration to maximise their effectiveness. Although SC’s work consisted of post-disaster recovery projects, in fact many were developmental in nature introducing new approaches and therefore requiring developmental time spans. The projects have been low-cost and appear to be efficient. The 2006 GIM Impact Assessment suggests that SCUK programme has been good value and the same seems likely for SCS’s work. There has been an absence of heavy hardware and logistics inputs and the work has been implemented through local NGOs with national staff. The projects have shown impact by demonstrating that Thai children can be empowered and interact with adults in areas such as DRR and as evaluators. Sustainability has been worked on through capacity building of local NGOs. Production of reports and training materials has been an important part of a sustainability and dissemination strategy. There has also been work with government to mainstream child protection approaches. SCUK phased its work out during 2007 with SCS continuing programmes including DRR. Possibly SC could have done more work through the period to work with and support UNICEF in the development of national integrated child protection monitoring and response system. However this engagement was perhaps seen as inappropriate, given that there was a wish to avoid developing a long-term country programme. Effective assessments were carried out immediately after the disaster and after the short relief phase, programmes generally seem relevant and appropriate, adding value by SC’s specific focus on its core areas of child protection and participation.

60 ECPAT (End Child Prostitution, Child Pornography and Trafficking of Children for Sexual Purposes)

112

There has been good coordination with UNICEF, ECPAT and the Thai government. Apart from earlier relief and livelihoods projects, generally the projects cohere around a focus on child protection and participation. Coverage by the projects in terms of numbers has been limited because of relatively small interventions, partly due to the amount of other assistance arriving, the niches chosen by SC and the care in selection of partners. There has been a particular focus on the issues of Burmese migrants and minority ethnic groups such as the Moken (Sea Gypsies). A focus on children with disabilities has been missing. Given that SC’s projects were small, could coverage have been increased if more funds had been available? The answer is probably no, as the limiting factors were not budgets but the availability of suitable partner NGOs and the importance of working with them to increase their capacities and to deliver innovative projects. Also in terms of coverage, it is noted that the projects lacked a focus on children with disabilities, mental and physical. This appears to be a hidden issue in Thailand and one which SC might have usefully looked at in advocacy terms, in relation to issues like the physical rebuilding of schools and other public buildings. Objective 1: Education Support in the education sector has taken place through a number of child-focused projects, including work to provide basic education for Burmese migrant children and the child-led DRR project. Within this and other projects, there has been a strong emphasis on promotion of child protection and child participation approaches, as well as the provision of psycho-social support. Generally these initiatives have been innovative in the Thai context and both effective and appropriate. The education project for Burmese migrant children appears to have been effective, by providing temporary education facilities for a hitherto neglected group of migrant children. In terms of impact and sustainability, it has improved possibilities in the longer term for migrant children to attend mainstream Thai schools. Sustainability has also been helped by capacity building work with the important partner NGO, Grassroots Human Rights Education and Development Committee (GHRE) which works with Burmese migrants. This work helped to get GHRE established and registered as a Thai NGO. Objective 2: Health This was an early and relatively minor project focusing on HIV/AIDS prevention using drama. This was an appropriate intervention given that infection rates were reported as rising amongst youth and some populations in the south. Verifiable information on the effectiveness and impact of this work is not available. Objective 3: Livelihoods The SCUK’s livelihood work was an early intervention in 2005 with the provision of fishing boats, fishing equipment and credit schemes and changed focus in 2006 to skills development for small scale enterprise operation in furniture making, mushroom growing, tree nurseries, fish cages for fish breeding and growing and eco-tourism opportunities. It appears that the projects have been effective and in terms of impact have played a modest role in supporting targeted families to cope through this transition period as part of a wider

113

humanitarian/ recovery assistance programme. Nearly all affected families received support from multiple sources.61 Although the projects were appropriate in the early stages of the recovery effort, there remains a question as to whether they were a priority for SC given its limited expertise in this area and the high levels of assistance available from other sources. The projects were not overtly designed to increase child participation or have wider policy and practice impact, although some positive outcomes were noted in the area of child participation. Objective 4: Child protection and child participation This has been one of the strongest areas of SC’s post-Tsunami work in Thailand with protection and participation approaches infusing much of the work. This work covered a wide range of activities including art, drama and the use puppets. There is evidence of effectiveness from the 2006 SCUK Impact Assessment Report. The projects were appropriate, given the high levels of stress and depression officially recorded. Unfortunately a potentially important child protection situation analysis led by an external consultant in 2005 and carried out by 150 young researchers was never completed. Both SC and UNICEF appear to have been scarred by this bad experience. A draft exists and this evaluator feels that it should be given another look to see if at least some of the work can be used. Objective 5: Child safe organisations training project It is clear from the subsequent evaluation that this project was effective and did have an impact on staff and organisational policies in some of the NGOs that participated.62 Sustainability has been promoted by the production of training materials in Thai and English which have been subsequently used, for example, by ECPAT International and by SC in other countries (China, India, Mongolia and Vietnam cited). The department in the Thai government responsible for the Welfare Promotion, Protection and Empowerment of Vulnerable Groups is reported to be incorporating child protection issues into its staff training. The evaluation noted that its effectiveness was limited by the short time scale of the programme. This work has been highly appropriate, given that many Thai NGO’s lacked orientation, training and internal policies on child protection issues. Coverage has been good with the initial training involving 28 organisations. Objective 6: Case management of separated and orphaned migrant children This has been an effective pilot project which proved that an innovative case-work approach can be used with migrant communities. Impact was limited by the short term nature of the project. Cases of children experiencing health or abuse problems or those in institutions needed longer-term more intensive inputs and staff training. In terms of sustainability it is not certain whether the NGO GHRE will be able to sustain and develop the work in the future. Ideally there should be more follow-up work in conjunction with UNICEF which part funded the project. Project has been documented in a useful internal report. The project was appropriate in the Burmese migrants context, although an initial assessment over-estimated the scale and severity of the problem. 61 Archer. Impact Analysis Report. Undated. 62 Shuey, B.K. Child Safe Organisations Training Project Impact Assessment. SCUK. May – June 2007.

114

Objective 7: Child-led Disaster Risk Reduction This joint pilot project by SCS and SCUK developed as one of the Alliance’s Big Ideas. It appears to be a strong and effective project that has generally achieved its objectives. The programme has strongly demonstrated impact by showing that children had the ability to lead disaster risk reduction showing improved analytical skills, ability to take collective action and the confidence to interact with adults in their communities with a resultant increased recognition by adults. Teachers were trained in using child-focused approaches. In terms of sustainability, schools involved are incorporating the programme into their curricula and there is some hope that the Ministry of Education will adopt the programme nationwide. A training manual has been produced and the programme has been taken up SC in a number of other countries. The internal Lessons Learned document provided some important recommendations for the work, including:

1. The Child-Led DRR programme should have a minimum duration of one year to achieve the desired results and should take account of the school timetable.

2. It is important that understanding of both children’s participation and DRR are equally

emphasised rather than one of them.

3. DRR may not be required by schools that are not seen as vulnerable to disasters.

4. If possible, there needs to increased strategic involvement of adult community representatives in the project.

4. Strategic and management issues Ways of working (direct, with partners, with government etc) In general SC worked through NGO partners in small pilot projects and also worked closely with UNICEF, ECPAT International and various provincial and central government departments. Given the lack of urgent relief needs, SC was able to work in a mainly developmental-type way, taking time to work with partners and support the development of their capacities. Advocacy An interesting example of advocacy work was a campaign with the media around the first anniversary of the disaster to try and avoid inappropriate exploitation of children for news stories. It is possible that SC could have done more to build on its small-scale pilot projects by further advocacy with government departments for mainstreaming, but additional staff resources might have been needed. Recruitment and orientation of staff One of the successes of the SC’s Thailand response appears to have been the recruitment of a small, committed national staff team. Training and capacity building (of staff and of partners) The capacity building of partner NGOs and staff has been an important element of much of the project work across a wide spectrum of topics from child rights to accounting. Having and taking the time to do this has been very important for the success of the programme.

115

Planning, monitoring and evaluation A feature of the response in Thailand has been participatory approaches to planning, monitoring and evaluation exercises, involving partner agencies, but also most importantly children as evaluators in their own communities, often using SCUK’s 5 dimensions of change framework. For example, SCUK’s Tsunami Impact Assessment Report 2006 was produced through a comprehensive process of training and involvement of partners and children and young people. An important feature of the programme has been the documentation of experience and the production of training resource material and guidance. This will have helped to make some of the key programmes more cost efficient. Alliance working Collaboration between SCUK and SCS has generally been good. The DRR project has been a good example collaboration with two-way funding. Both agencies shared the same goal, work methods, and technical resources, but separately managed their own programme and made agreements with each of their local partner organisations under separate budgets. Budgets and human resources for joint activities such as training workshops, meetings, and the development of training manuals were shared by both organisations. 5. Recommendations It would be inappropriate to make any detailed recommendations at this stage, given that nearly all the project work has been completed.

1) As the programme winds down, it might be appropriate for SCUK and SCS to jointly spend some time reflecting on lessons learned and reviewing recommendations from the various evaluations to see if any follow-up work should and can be done at this stage.

2) As indicated above, this process could also involve looking at the draft child protection situation analysis report to see if any of that work can be salvaged.

3) Follow up on the Lessons Learned exercises e.g. SCUK Bangkok May 2007 to see whether further work is needed to implement recommendations.

4) Increase SC’s responsiveness to the situation of children with disabilities.