Designing Sustainability into Health Projects

Designing Sustainability into Health Projects

Frank B. Roseby

BRurSci UNE, MRurSci UNE

March 2003

This project was submitted in partial fulfilment of therequirements for

1

the Degree of Master of Public Health, The University of NewSouth Wales.

Awarded October 2003

2

Declaration

I hereby declare that this submission is my own work and that, to the best of my knowledge, it contains no material previouslypublished or written by another person, nor material that has been accepted for the award of any other degree or diploma by the University, or other institution of higher education, except where due acknowledgement is made in the text.

Signed: Date:

Acknowledgements

I acknowledge the sound support and guidance provided by my supervisor Dr Paul Freeman, despite the challenges presented bythe distance between Seattle and Chongqing. His interesting comments on the final draft underscored the complexity of this topic and the difficulty of conducting a rigorous inquiry into project sustainability with limited resources.

I thank those development practitioners who, despite their own heavy commitments, were willing to assist me with this work: MsLenore Simpson, Project Director, Hassall and Associates International, Canberra; Mr Paul Nichols, IDSS, Community Aid Abroad, Melbourne; Mr John Lyndon, Australian Team Leader, HIV/AIDS Prevention and Care Project, Urumqi, Xinjiang Uygur Autonomous Region, People’s Republic of China.

The support of my wife, Xu Guo Rong has enabled me to complete my course work and to undertake this study while we worked together on a complex poverty alleviation project in rural China.

3

Acronyms and Abbreviations

AIDS Acquired Immune Deficiency SyndromeAUD Australian dollarAusAID Australian Agency for International DevelopmentGOB Government of BangladeshHIV Human Immunosuppressant VirusHNP Health, Nutrition and PopulationIDSS International Development Support ServicesM&E Monitoring and EvaluationNGO Non-government OrganisationNTB Negri Timor BandarOED Operations and Evaluation Department (of World Bank)OECD Organisation for International Cooperation and DevelopmentO&M Operation and MaintenancePDD Project Design DocumentPID Project Implementation DocumentPMM Prevention of Maternal Mortality (Network)PRC People’s Republic of ChinaUSAID United States Agency for International DevelopmentUK United KingdomUNICEF United Nations Children’s FundWCH Women’s and Children’s HealthWHO World Health Organisation

4

Abstract

Most Australian bilateral aid to developing countries is provided in the form of discrete projects. The formulation of these projects is a complex, time consuming and expensive process which is expected to come up with projects that meet the needs of recipient and donor, are cost effective, and are sustainable.

Projects generally, including those in the health field, meet the former two criteria. However, ensuring that a project will continue to provide benefits is more difficult.

Recently, some major donors have recognised this and have encouraged project design teams to address sustainability during the design phase, but typically have provided little useful guidance as to how this should be done. To help guide Australian development aid practitioners when they are designing health projects for implementation in developing countries, this inquiry addressed the question: which particular elements of health project design make the major contribution to project sustainability.

This inquiry explored the development aid literature and various donor data bases to provide lists of those design elements considered most important by three major donors – World Bank, USAID and AusAID. These lists were then collated ina matrix to make commonality and disparity of views between donors apparent. A high level of consensus amongst these donors made it possible to draw up a Donor Consensus list of eleven key criteria impacting on sustainability of health projects.

Three very experienced Australian development aid practitionersthen ranked the Donor Consensus list of criteria. The disparityin their views precluded shortening of the list but verified that each of the eleven was important and therefore should be included in the ‘design criteria for sustainability’ considered in the next phase of this inquiry.

The design documents of three recent and current Australian health projects in the People’s Republic of China (PRC) were

5

then carefully assessed to determine which of the eleven ‘design criteria for sustainability’ had in practice been considered by the design team. The design team of one project was found to have considered all eleven criteria, while the other design teams each considered nine of the criteria, indicating their utility. Possible reasons for the neglect of some criteria were put forward and discussed.

The consensual view of three major donors, the verification by experienced practitioners and the actual consideration by design teams in the field, strongly suggested that the eleven ‘design criteria for sustainability’ identified in this inquiry, all make a major contribution to sustainability of Australian health projects, at least in the PRC. Australia development aid practitioners were therefore encouraged to consider these criteria when designing health projects. By doing so, they will help to ensure that the project not only meets the needs of recipient and donor governments and minimum cost/benefit standards but will continue to provide benefits for an extended period after donor funding has ceased.

6

Table of Contents

DeclarationAcknowledgementsAcronyms and AbbreviationsAbstract

1. Introduction1.1 Background and Context1.2 Objectives

2. Literature review2.1 Sources of Information2.2 What is Sustainability?2.3 Sustainability of Specific Health Projects2.4 World Bank Views2.5 USAID Views2.6 AusAID Views

3. Methods

4. Results4.1 Opinions on Sustainability4.2 Practitioner Ranking4.3 Testing the Design of Recent AusAID Projects

4.3.1 Project A – HIV/AIDS Prevention and Care Project4.3.2 Project B – Bazhong Rural Health Improvement

Project4.3.3 Project C – Ningxia Family Planning, WCH Project

5. Discussion5.1 Consensus of Views5.2 Consideration of Criteria During Design of AusAID

Projects5.2.1 Consideration in Project A5.2.2 Consideration in Project B5.2.3 Consideration in Project C5.2.4 Limits on Extent of Consideration

5.3 Additional Criteria

6. Conclusions and Recommendation

7

7. References and Additional Reading

List of TablesTable 1. Donor Opinions.Table 2. Practitioner Ranking.Table 3. Design Criteria for Sustainability.Table 4. Consideration of Design Criteria for Sustainability in Project A.Table 5. Consideration of Design Criteria for Sustainability in Project B.Table 6. Consideration of Design Criteria for Sustainability in Project C.

8

1. Introduction

This thesis reports on an inquiry into the sustainability of development aid health projects and how the design of these projects can be improved to foster sustainability. It provides background, rationale for the inquiry, the question addressed and specific objectives followed by a comprehensive literature review, incorporating a collation and analysis of aid donor views on factors effecting sustainability. The methods used areoutlined and the results obtained are then documented, discussed and conclusions drawn from the findings.

1.1 Background and Context

Considerable progress has been made by most developing countries towards the goal of ‘health for all’ as envisaged at Alma-Ata in 1978 (WHO and UNICEF, 1978), in many cases with thehelp of significant inputs of aid provided by more developed countries. However, the lack of sustainability of donor supported health projects threatens to undermine public supportfor this transfer of resources, which is so important to developing countries.

Most Australian bilateral aid to developing countries is provided in the form of discrete projects. The formulation of these projects, including their identification, scoping, feasibility testing, design and costing, is a complex, time consuming and expensive process which is expected to come up with projects which:

meet the needs of recipient and donor, are cost effective, and are sustainable.

Projects generally, including those in the health field, meet the former two criteria, or at least appear to. This is not surprising as few if any projects are approved for implementation that do not meet the overtly expressed needs of the recipient and donor governments. Similarly, all proposed projects are tested for economic and financial feasibility and

9

must meet minimum cost/benefit standards before they are approved.

However, the third criterion – sustainability1- appears to havereceived much less attention during project formulation. This is also not surprising because establishing at the design stage, whether or not the improvements to be brought by the project can be sustained is problematic. Thus, while the terms of reference for project design teams usually ask that sustainability be addressed, typically the donor has provided little or no guidance to the team as to how this should or could be done. Even such a major contributor to development assistance as the World Bank ‘has not used its lending portfolio to systematically collect evidence on what works, what does not, and why’ (World Bank, 1999a).

Recently, the Australian Agency for International Development (AusAID) has also recognised this major short-coming and, following an internal study, has encouraged staff to analyse sustainability at each stage of its activity cycle2, from identification through to completion, across its wide portfolioof projects (AusAID, 2000). The extreme complexity of this issue is illustrated by the need for staff to ask 31 ‘guiding questions’ and to use a check-list of 115 indicators3 to determine whether the design team has adequately addressed sustainability.

1.2 Objectives

The purpose of this inquiry was to provide more succinct guidance to Australian development aid practitioners to help them design sustainable health projects for implementation in developing countries. This would be done by answering the question: which particular elements of health project design make the major contribution to project sustainability?

1 The definition of sustainability is addressed in Section 2.2.2 The AusAID activity cycle moves through the country strategy, project identification, project preparation (including design), appraisal, implementation & monitoring and project completion report which feeds back to the country strategy.3 Included as AusGUIDElines 12: Project quality standards by AusAID (2002).

10

To achieve this purpose, the inquiry addressed the following specific objectives:

1. To collect and collate views on which particular elements of project design make the major contribution to sustainability of development aid projects in the health field.

2. To articulate a generic list of ‘design criteria for sustainability’ which need to be addressed when designing health projects.

3. To assess whether these ‘design criteria for sustainability’ are being used by designers of AusAID health projects in China.

11

2. Literature Review

2.1 Sources of Information

The published literature on development aid is very short on primary information and the occasional references to sustainability of interventions are usually secondary sources, heavy on opinion and light on, or completely lacking, evidence.Most of the primary information documented in the development aid field is in consultants’ reports and reviews of specific projects which are usually not published and therefore not accessible. As Garner (1995) succinctly noted: ‘what is needed is more accessible information about the successes and failures’ of development aid projects. He acknowledged that donors constantly collect this information through their own monitoring mechanisms but pointed out that few allow public access to the results of their evaluations and the lessons learnt from them.

An early notable exception was the approval and support provided by the U.S. Agency for International Development (USAID) for publication of a review which drew together information from evaluations of many of its health projects across five countries (Bossert, 1990). Fortunately, in more recent years, some other donors have made aggregated information more readily available so, in conducting this inquiry, the author has had access to the sector and thematic evaluations published on the Internet by the World Bank and theevidence based ‘lessons learned’ data base posted on the Internet by the Australian Agency for International Development(AusAID).

While the emphasis in this inquiry has been on development aid projects in the public health field, information has been drawnfrom donor experience with a wider range of projects, such as those in the related fields of community development and poverty alleviation. Similarly, the findings of this inquiry could be expected to have wider application than just to projects in the health field.

12

2.2 What is Sustainability?

Probably the most widely accepted definition of sustainability as it relates to development aid was that provided by the Organisation for Economic Cooperation and Development (OECD, 1989). Given that development aid projects were ‘specific interventions of donors for assisting a developing country achieve sustainable benefits and maintain supporting activitiesand institutions’ the Organisation suggested that an intervention is sustainable ‘when it is able to deliver an appropriate level of benefits for a extended period of time after major financial, managerial and technical assistance from an external donor is terminated’. Implicit in this definition is that the appropriate level of benefits and the period of time ‘will need to be defined in each instance’ with reference to the prevailing circumstances. However, it is assumed that, while the relationship with (and even limited assistance from) the donor may continue, the developing countrywill provide the resources needed to sustain the intervention.

USAID accepted this general concept of sustainability but, in studies on the sustainability of USAID supported health projects in Central America and Africa, defined a very specificperiod of time for all projects (Bossert, 1990). Thus, projectswere considered to be sustained if the project outputs/outcomescontinued for at least 3 years after completion of the project.

Taking on board the more flexible approach of the OECD, AusAID(2000) suggested that each individual project should prepare its own strategy for maximising ‘the flow of sustainable benefits’ and simplified the OECD definition to ‘the continuation of benefits after major assistance from a donor has been completed’. In this more flexible, case-by-case approach, AusAID staff would be required to take a major role in managing sustainability throughout the project activity cycle and implicitly, the designers of AusAID projects will be required to design in sustainability. In its guidelines4 for staff and contractors, AusAID proposed the series of indicators of project sustainability published by the World Bank in 1990 (Bamberger and Cheema, 1990). These call for the monitoring of project sustainability under four headings:

4 Included as AusGUIDElines 12: Project quality standards by AusAID (2002).13

Continued delivery of services and production of benefits; Maintenance of physical infrastructure; Long-term institutional capacity; and Support from key stakeholders.

Looking at health projects in Tanzania, Andersson-Brolin (1991)extended the OECD definition by couching it in terms of self-reliance. He suggested that ‘a sustainable project is characterised by the developing country having the primary responsibility for the activity, the institutional capacity and commitment to carry it forward, and the ability to mobilise the necessary resources to maintain it’. He noted that this implied a shift in financial and managerial responsibility away from the donor and towards the recipient developing country – a shift often seen reflected in the donor views canvassed in the following review.

2.3 Sustainability of Specific Health Projects

The few empirical studies available on the sustainability of a health intervention generally do not paint a rosy picture. Forexample, in northern Bangladesh where night-blindness is endemic, Hussain and Kvle (1996) found that despite continuing community awareness, the high level of control achieved during the intervention could not be sustained. Thus, the prevalence of night-blindness per thousand children decreased during intervention from 35.5 in 1986 to 17.4 in 1989 but rose again following the intervention to 24.3 in 1992 with falling consumption of [beta]-carotene rich food, protein items and vitamin A capsules, associated with declining household incomes.

Similarly, attempts to change traditional infant feeding practices in Bihar to more ‘baby friendly’ practices (the subject of a major intervention by UNICEF), while having a significant initial impact, have been difficult to sustain without on-going intervention. Prasad and Costello (1995) looked at the impact of a 10 day training program for doctors and nurses on the feeding practices of mothers in their care. Initially, the trainees passed the word along and many mothers responded by changing to more ‘baby friendly’ practices. This was reflected in indicators such as frequency of pre-lacteal feeding which fell from 96% to 43% immediately following the

14

training. However, 6 months after the training, only a third ofmothers were receiving any instruction on feeding practices from the trained clinicians and the frequency of pre-lacteal feeding (77%) reflected this.

A more encouraging observation was made on the sustainability of malaria control programs in Kenya. There was a high level ofacceptance of a simple intervention (insecticide treated bed-nets) in the early 1990s and very good parasitological and clinical outcomes were demonstrated in a controlled study (Beach et al. 1993) conducted during the intervention. In a follow-up study three years later, Kachur et al (1999) found that the original communities were still using the bed-nets originally provided. However, very few households had re-treated these nets, nor had they purchased new nets to protect additions to the family. Obviously, for this intervention to betruly sustainable a better supply system would be needed for the insecticide and the nets. As noted by Lengeler (1999) such a system is likely to come from marketing initiatives by the private sector.

Development aid projects in health and health related fields may be more difficult to implement effectively and less likely to be sustained than those in other sectors. World Bank (1999) noted that only 64% of 107 completed projects in its ‘health, nutrition and population’ (HNP) category were ranked satisfactory at review compared with 79% in all other categories. Only half of the projects in the HNP category were considered likely to be sustained. If this experience is common to other donors/lenders, then the issue of sustainability is obviously one of great importance to development aid project designers and implementers in the health sector and not just a decade-old ‘fad for which researchfunding can no longer be obtained’ (Bossert 2001 personal communication).

2.4 World Bank Views

The recent review of World Bank projects in the ‘health, nutrition and population’ (HNP) category by the Operations Evaluation Department (OED) of the Bank (World Bank 1999a) has identified a number of internal and external factors which it

15

considers determined project effectiveness and/or sustainability. All the internal project factors and most of the external factors that the OED consider are likely to impinge on a project can be allowed for in the design and so are considered relevant to the current inquiry.

In its evaluation, the OED noted that in recent years, responding to lessons from earlier projects, the Bank has placed greater emphasis on ‘client ownership and beneficiary assessments in project design and supervision’. The importance of collecting information directly from potential beneficiariesto ensure accuracy is well recognised and at the same time the contact facilitates beneficiary participation in the planning process, an essential step in establishing client ownership. Realistic objectives which are cognisant of the capacity of theproject are important in permitting positive project outcomes and so fostering sustainability. Thus, an evaluation of the Amazon Malaria Control Program (World Bank, 1998) argued that the decision, by project leadership beginning in late 1992 to change the strategy from malaria eradication to malaria controland case management, was responsible for most of the project's achievements.

Garner (1997) noted that epidemiological objectives and indicators are often inappropriate in the design of health projects, leading to a misallocation of resources with a negative effect on the sustainability of the intervention. He provided in illustration the example of an immunisation projectin a country without routine vital registrations. The project had ‘reduce child mortality’ as an objective, so health workersspent valuable time collecting statistics when they should havebeen immunising children. If a process objective and indicator,such as immunisation coverage, had been chosen instead, the health workers could probably have remained focussed on, and continued to deliver, the intervention.

In the same paper, Garner points out the value of what he terms‘managerial’ objectives. When health authorities in Tanzania moved from epidemiological objectives (such as reducing neonatal mortality), to objectives related to what health staffactually do (such as ensuring a reliable supply of medicines at

16

all health facilities), outcomes and sustainability improved. Lorenz et al (1995) earlier found similar evidence in a value-for-money study of the UK Health and Population Aid Programme which showed that projects using objectives related to activities were better managed and monitored than those with objectives related to broad measures of health status.

In most countries the World Bank has participated in a dialoguewith recipient governments on issues of health financing, the health work force, and civil service reform However, the OED evaluators pointed out that system reform is difficult and time-consuming and may go off the rails when stakeholders outside ministries of health can determine whether reforms succeed or fail. The evaluators suggested that the Bank may have sometimes encouraged overly ambitious institutional and/orsystem reforms.

Focussing on this key issue affecting sustainability, the OED evaluators ranked institutional development as ‘substantial’ inonly 22% of the 107 HNP projects reviewed. They concluded that working with existing institutions and within existing systems was more effective than trying to develop new ones. However, they added that improved success in institutional strengtheningwill require improved Bank capacity for institutional analysis and provision of flexible support to design and implementation teams facing difficult institutional problems.

While recognising the importance of matching designs to the capacity of existing institutions, the OED evaluation unit noted that ‘paradoxically, Bank project designs are usually themost complex - with a greater number of components and organizational units - in countries with weak institutional capacity’.

When institutional capacity in the health sector is very weak, development planners have been disinclined to intervene. However, when Garner et al (1998) argued for a more evidence-based approach to health interventions in developing countries he pointed out that, even if the health system is in disarray and cannot be made to work, it should still be possible to implement and sustain a single health intervention which has been proved to save lives. He quoted the success of Vitamin A

17

supplementation of young children as a good example of such an intervention but cautioned that many “magic bullets” are not based on sound evidence and simply do not work. The importance of developing a learning/information culture in development projects was identified by the OED and linked to effective monitoring and evaluation of the project (World Bank,1999a). Its experience shows that ‘effective M&E design-including the selection of a limited number of appropriate indicators, attention to responsibilities, and capacity for data collection and analysis - enhances the focus on results and increases the likelihood of achieving development impact’. The major benefit here seemed to be that the results from an on-going M&E system will allow the development of a performancebased management system and continue to guide the client institution, long after external funding has ceased.

Garner et al (1998) made the salient point that it is not just the health workers that need information about effectiveness ofthe health interventions; it is members of the community who make the ultimate decision whether to use the health services provided. These customers are busy people with limited time andmoney to invest in their health care and they will not continueusing health services unless they are convinced of their effectiveness. This view was also held by Delclos et al (1999) who advocated information dissemination in the language of the intended audience as an effective tool for enhancing sustainability. The OED noted that the Bank had not placed sufficient emphasis on addressing determinants of health that lie outside the medical care system. They felt that better inter-sectoral coordination was required, such as on work force issues, and suggested that the Bank should encourage communication and collaboration among government ministries, and between government and non-government organisations.

In summary, the OED category evaluation (World Bank 1999a) suggested that the Bank needs to:

work directly with target beneficiaries, establish realistic objectives, demonstrate effective outcomes,

18

work with and strengthen existing health institutions, match design to capacity of target institution(s), emphasise knowledge and information transfer, establish good M&E, address determinants of health that lie outside the health

system, recognise the impact of other sectors on health.

Specific country reviews of Bank projects have also provided useful information, some in support of the above contentions and some not. World Bank (1998) reviewed two major projects in Brazil designed to build and equip health facilities, improve management and train staff to deliver programs of comprehensivecare for women and children. While their effectiveness was difficult to judge because of the absence of an M&E system, it is generally accepted that while they succeeded in expanding access to basic health services they did not transform the modeof basic health care delivery within that system. The evaluators concluded that although the projects were adequatelydesigned given the state of the art and the consensus among health planners at the time, it is apparent that the design team did not work directly with the target beneficiaries whose needs were not appreciated by the health managers.

World Bank (1999b) noted that two quite different nutrition projects implemented in India have challenged the contention that an effective outcome will facilitate sustainability and even lead to replication. The most effective one, the Tamil Nadu Integrated Nutrition Project was an innovative project that focused on changing the way mothers feed themselves and their preschool children, rather than on feeding per se. This project has been well implemented and quite successful in reducing severe malnutrition. However, the less effective project, Integrated Child Development Services, which offered education, supplements, immunization and health checkups for preschool children, and education for pregnant and nursing mothers, has become the template for the government's main program for preschool children. A recent assessment of this project suggest that it was providing only modest benefits. Obviously, other over-riding factors were involved in garneringsupport for its replication.

19

A number of the studies on Indian projects (World Bank 1999b) suggested that despite major inputs by the Bank, the quality ofprimary health services did not improve significantly and that improved income, education, and overall state administration were more important than specific public health interventions in explaining differences in health indicators during the 1980s. The studies found that while the Bank's (mainly infrastructure) inputs expanded capacity, they did not lead to on-going improvements in quality of health services. The evaluators suggested that this was mainly due to poor attentionto monitoring and evaluation and feedback of results. Other suggestions were generally consistent with the OED findings butadded the need during project design:

to take more account of field conditions, not only those affecting the beneficiaries but also those impacting on field staff performance;

to resist the tendency to add new initiatives before older (successful) ones are properly locked-in;

to take full account of rules and incentives which determine how personnel do their job;

to take full account of personalities and managerial style of key managers;

to design into the project good transport and communications;

to work towards introducing performance based budgetinginto projects.

2.5 USAID Views

Bossert (1990) drew together the findings of five country studies on the sustainability of USAID supported health projects in Central America and Africa in an attempt to provideguidelines for USAID staff on how to design projects “so that the activities and benefits achieved during the life of the project will continue after the donor’s funding has ceased, andother sources of funding must be found to maintain them”. Each of these country studies took a systems analysis approach

20

examining the planning and implementation of the project and outputs/outcomes at completion and at least 3 years later. Projects were considered to be sustained if project outputs/outcomes continued for at least 3 years after completion of the project.

A salient finding of Bossert’s review was that projects which were designed with mutually respectful negotiations between donor and recipient were most likely to be sustained. Naturally, those projects designed and driven by the donor withlittle input by the recipients, to either the design or the management of implementation, are less likely to meet their needs and unlikely to be continued. This finding really sets the scene for the process of project design as it proceeds to address what Bossert refers to as ‘context factors’ and ‘project factors’ affecting sustainability.

Context factors

The political, economic and institutional context was found to be an important determinant of sustainability. While Bossert acknowledged that these context factors can seldom be changed by project designers, he correctly pointed out that designs should be responsive to them. Thus, project planners may decidenot to proceed or only proceed with design modifications appropriate to the context.

Context factors of particular importance included the: economic capacity of the recipient country to provide

project funding, administrative capacity of the recipient country to

collect and redistribute funds, institutional strength of the counterpart agency in

planning and implementation.

The projects reviewed had been implemented during the 1980s when USAID policy explicitly rejected the inclusion of institution building in its project designs. (Although this hadbeen common in the 1970s). However, Bossert concluded from analysis of the country studies that stronger institutions weremore likely to sustain project activities and suggested that ‘if donors are serious about sustainability, they may have to

21

begin addressing institution building objectives again and strengthen the institutions expected to sustain the projects’.

Project factors

From the country studies it was apparent that a number of factors directly under the control of project designers and managers related to project sustainability. All the studies identified:

perceived project effectiveness, and integration of project activities into on-going

administrative structuresas important.

With respect to effectiveness, perceptions were more important than fact although real positive results tended to improve perceptions of a project held by key decision makers. Bossert noted that USAID’s continuing emphasis on using measurable objectives on its projects has been important in helping to match perceptions with reality.

While GOB (1989) concluded, from the review of a major programon immunisation in Bangladesh, that integration of interventions into the existing health care system was difficult to sustain, Bossert concluded that, in fact, without integration sustainability was problematic. Thus, he found thatthe projects integrated into existing health institutions, rather than attached to new ones created to implement the project, were more likely to be sustained. While the purpose created, new counterpart agencies may have been more efficient during project implementation, they were more dependent on donor funding and consequently more vulnerable to any winding down in recipient government funding following the project.The importance of establishing a source of non-donor funding during project implementation, and how this was achieved, received considerable attention in the USAID review. Some projects were continued with additional recipient government funding, some raised funds on a fees-for-service basis from thebeneficiaries and others used a combination of these funding methods. Whatever the source, those projects which received a steadily increasing share of funds from local sources and a

22

corresponding decreasing share from the donor during the life of the project were more likely to be sustained.

The inclusion of an effective training component was found to be important in sustaining the project. The level at which the training was targeted was less important than the usefulness ofthe training to the trainee’s current role and chance of winning future jobs. Thus in Guatemala, training that related to improved employment opportunities was most likely to be continued. Training tended to be self-sustaining and facilitated by: trainers themselves, improved managerial capacity, improved staff performance, the positive attitude of trainees and the low cost of on-going training.

Although community participation was expected to have a big impact on sustainability by creating a ‘constituency that woulddemand continuation’, this was not found to be generally the case. That is, the evaluators found that those projects with strong community participation were no more likely to be sustained than those without. However, in situations where costrecovery was based on a user-pays system, such as in Zaire and Senegal, community support for the system and hence, community involvement in its design and management was important.

Bossert concluded from his analysis of the USAID country studies that sustainability of projects was best when they weredesigned and managed with a mutually respectful process of giveand take between the donor and recipient to:(i) demonstrate effectiveness in reaching clearly defined objectives;(ii) integrate their activities fully into established administrative structures;(iii) gain significant levels of funding from national sources (budgetary and cost-recovery) during the life of the project; and(iv) include a strong training component.

2.6 AusAID Views

For some years, the Australian Agency for International Development (AusAID) has, like most bilateral and multilateral aid donors, reviewed individual and/or groups of similar

23

projects (eg. in a single sector) and made attempts to draw conclusions about their overall lasting impact on the recipientinstitution and/or community. From such reviews and the completion reports on individual projects, AusAID has recently established a ‘lessons learned’ data base which is available onits website ( www.ausaid.gov.au ). In its current format (over 70 separate lesson items each based on reviews of one or more projects and each providing one or more lessons) the usefulnessof this data base to busy design practitioners is very limited.However, with patience it has been possible to use it here to identify those factors in project design considered by AusAID, and/or its various independent evaluators, to be important in determining sustainability of its projects in health related fields.

Drawing on the collective experience of many AusAID staff and the Jakarta based field staff of USAID, UNICEF and the World Bank, the AusAID Quality Assurance Group (AusAID, 2000) has recently prepared guidelines for ‘promoting practical sustainability’. These encourage AusAID staff to appraise each stage of a project from identification, through design and implementation to completion, against an extensive list of ‘keyfactors affecting sustainability’. Unfortunately, these ‘key factors’ are generic with respect to sector and are opinion rather than evidence based. However, these guidelines have provided additional material to supplement that drawn from the lessons learned data base and so facilitate collation of AusAIDviews on design factors effecting sustainability of health projects.

Community focus

Reviews of AusAID experience in the 1980s, when it was principally dealing with agricultural and water supply projects, indicated a widespread failure to take account of socio-economic and cultural factors in project design which limited project effectiveness and sustainability. Specifically,failure to:

accurately identify target communities (including with respect to gender),

determine their cultural and socio-economic circumstances,responsibilities, needs and concerns,

24

involve these beneficiaries from the outset in the design process, including problem identification, indicator identification and data collection, and

recognise obstacles posed by negative institutional attitudes towards the poor,

was believed to preclude a community sense of ownership which could have improved project sustainability.

Conversely, in a health sector project (NTB Environmental Sanitation and Water Supply Project) where socio-economic and cultural factors were fully analysed and the community was explicitly involved in the detailed planning of the project, the response from the community was excellent and continuing.

More recent evaluations of HIV/AIDS projects have supported this focus on the community and called for specific emphasis on:

involvement of the community in project design, implementation and review,

helping community based groups to improve their managerialand technical competence in the areas of prevention, care and advocacy,

information dissemination by credible sources (such as HIVpositive patients),

using the community to engender behaviour change in individuals.

The Water Sector Review found that community participation in the construction of water supply and sanitation facilities can reduce capital costs, increase community involvement in management and improve sustainability. In many early projects in this sector, little attention was given to community involvement in design, construction and maintenance of community water supplies. The result was often badly sited or inappropriate facilities which were poorly maintained by government agencies.

These problems were overcome on the Lombok Rural Water Supply and Sanitation Project in Indonesia. Despite recipient government scepticism the project's community development specialist successfully organised water user groups, including women, from the target community. These groups, and the

25

involvement of women in particular, greatly increased local interest in and commitment to the new facilities, improved their design and location, and allowed development of a community-based maintenance program. Thus, AusAID experience suggested that project designers need to focus on the beneficiary community and take into account itssocial, gender and cultural characteristics.

Local participation

Evaluators suggested planning of women's participation should take into consideration the socially and culturally acceptable roles of women, the existing gender/social situation, and the realistic extent of women's participation which can be expected. The project design should clearly state an intention to involve women, include collection of information on women's roles, identify and engage local women's organisations that canfacilitate an active role for women, and identify an explicit role for women in the project.

Lessons from the Water Sector Review in 1990 suggested that involvement of women is crucial to the success of most water projects as women are generally the focus of water use within the household, responsible for family health and food preparation. Women have a vested interest in the establishment and maintenance of reliable water supply and sanitation facilities so ensuring their involvement at all levels is critical.

In the case of the Tung Kula Ronghai Project in Thailand, womenwere only included in the planning of the community developmentcomponent. Project sustainability would have been strengthened by women's participation in the water user groups, with involvement in the selection, design, operation and maintenanceof new systems.

Evaluators have also noted that explicit recognition of women by the project authorities has the capacity to raise their status with direct benefits to family health. This was supported by the observations of Cockcroft (1998) in Nepal where malnutrition in children was seen to be related not only

26

to feeding practices but also to the status of women.

Some AusAID evaluators recommend that project designers supplement the detailed analysis of the target community with acareful analysis of all key individual and organisational stakeholders at the design stage.

In particular, they have recognised the importance of involvingthe lower levels of government administration – those that dealdirectly with the community. They point out that the Nepal-AustForestry Project could have been sustained if emphasis had beenplaced on the local administration as well as the community andnote the success of the Lao Integrated Village Development Project, where providing opportunities for the participation oflocal government community workers minimised implementation problems and helped guarantee higher levels of on-going participation.

In the same way, evaluators found that sustainability of project initiatives was endangered by a lack of counterpart participation in project design and management. They concluded that recipient counterpart organisations must be involved in project design and in management decisions if a project is to have any chance of being sustained after the project period. Inaddition, they noted that such participation is more effective if it occurs within the structure of the organisation, so that counterparting becomes an organisational role.

In the design of the Northern Samar Integrated Rural Development Project there was no significant involvement of Philippines line agencies in the formulation of the project. Assuch, the project was perceived by Philippines counterpart agencies to be an 'Australian Project'. This caused significantmanagement problems and posed serious questions for the overallsustainability of the project initiatives.

Similarly, recipient government agencies involved in implementation of the Thai-Australia Northeast Village Water Resources Project were not involved in project design and did not see the project as part of their routine work but as additional, and definitely not to be continued after project

27

completion.

Thus, AusAID experience indicated that project designers need to involve women, other members of the community, other stakeholders, their counterparts and counterpart agencies in the design process to engender a sense of local ownership to ensure sustainability.

Size and complexity

Some AusAID evaluations of health related projects have argued for a multi-disciplinary and multi-sectoral approach in projectdesign. For example, the Water Sector Review found that where water supply programs were integrated with other health and education programs, the probability of achieving improved community health goals in the long-term was good. However, in its comprehensive review of integrated area development projects (AusAID 1991b) it was recognised that the potential gain from an integrated approach to tackling poverty issues canbe negated by management and co-ordination difficulties. Thus large multi-component projects with numerous objectives and activities, in areas such as engineering, food production and community development, which were often weakly linked, frequently suffered from a loss of project focus and consequentadverse effects on the attainment of project goals. However, the less ambitious approach, in line with local implementation and beneficiary uptake capacities, taken in the South Simbu Rural Development Project was a good example of a small-scale integrated intervention which was appropriate to local circumstances.

AusAID evaluators disagree on the optimum project size for sustainability. The Indonesia Effectiveness Review found that greater success and sustainability was achieved in projects that were less ambitious, highly focussed and localised in impact. Others suggest a need to reach a critical mass to ensure national recognition and continued funding, however, they accepted that a project should not be so large that it is beyond the scope of the recipient institution to sustain it with resources they can reasonably expect to have available.

28

There was general agreement that the level of activity in the counterpart agency should not be built up beyond its capacity to sustain itself after donor funding is withdrawn. In the Indonesian Social Sciences Project, the level of research was substantially increased with Australian funding. However, this increase was dependent on the aid and many of the research initiatives could not be continued after project funding ceased. Similarly, many integrated area development projects have tended to build up social services to a level which could not be sustained. This was especially so in the South Pacific where expectations of local services and standards were often raised well beyond the capacity of the local resource base and economy.

These observations suggested that while an integrated approach can be very effective care is needed to ensure size and complexity do not rise beyond local capacity to manage and coordinate. Counterpart agency selection

The review of integrated area development projects referred to earlier, recommended that regular line agencies should be used for project management, rather than a newly established institution. This applied particularly to infrastructure projects. While the establishment of a separate project implementation agency may maximise the efficiency of project implementation, for the project to be sustained the local line agencies, which will be responsible for long-term management ormaintenance, must be involved and strengthened.

For the Philippines Zamboanga Del Sur Project, a special project office was established to implement this project. Provincial government agencies with line responsibilities for roads and bridges were not involved. As a result, these line agencies had no sense of ownership of project outputs and were administratively and financially unprepared to assume O&M responsibility for the transport infrastructure after the project ended. Within three years of project completion, roads had substantially deteriorated. In addition, most of the technicians trained by the project were unable to find employment to utilise their new skills. Similar problems were

29

avoided in the irrigation component by involving the local agency from the design stage onwards. In the case of the Pichit Rural Development Project in Thailand, selection of the wrong counterpart institution for water supply development resulted in a free-for-all drilling program by village groups and unsustainable water extraction. The counterpart agency had neither the legal power nor staffingcapability to manage aquifer development.

Thus, while implementation may be made more difficult, the project should be managed by (or through) those agencies which shall be responsible for the work in the long-term to facilitate sustainability. Institutional strengthening

Institutional strengthening is a long-term process which is difficult to accomplish within the restricted time frame of a typical three to five-year AusAID project. However, there is general agreement amongst AusAID evaluators that institutional issues must be subjected to rigorous analysis during project design because institutional/administrative factors are seen asthe most common constraints on effective implementation and sustainability. Recipient countries often have highly differentiated and complex institutional structures which must be carefully analysed in order to define successful developmentstrategies.

Evaluators believe the institution strengthening activities should both target and enhance the managerial skills of counterpart managers. Counterpart financial management and personnel management needs should be assessed during project preparation and appropriate training programs established. Technology, training and activities should always be appropriate to the institutional capacity of the recipient to manage and service in the longer term, as well as during the project itself. This is particularly so where the project relies on collaborative efforts from non-government organisations (NGOs) or on a strong participatory approach.

A few highly-focused development projects which strengthened a 30

national institution and enabled it to implement new systems ona national basis have had considerable on-going benefits for the recipient country. However, evaluators again noted that thecapacity of the institution must be carefully assessed during design so the project would not attempt to take it beyond its available resources.

AusAID experience indicated that project designers virtually always need to strengthen the counterpart institution after carefully assessing its needs and capability.

Awareness and training

The AusAID Quality Assurance Group felt that the design processshould include awareness raising activities for the stakeholders and such activities should also be designed into project implementation to mobilise political, administrative and community support for continuation of the project.

The Group noted that training for individuals or groups has often been included in project designs but its positive impact on sustainability was not assured. The training with the most impact built on local capacity to deliver training and was:

given early in the project and repeated, of direct relevance to the trainee’s role, and provided in-country.

Funding

While AusAID evaluators acknowledged that reasonably comprehensive economic appraisals have been prepared for most major projects, some may have been based on optimistic assumptions and/or have not considered in detail the effects ofprojects on government finance and the economy. As a result, inadequate economic rates of return and inappropriate domestic taxes and charges were often considered the principal cause of funding constraints during and after project implementation. Inlight of these findings, evaluators recommended that project designers should always quantify the impact of the project on recurrent government income and expenditure and establish the source and magnitude of funds for ongoing maintenance.

31

Only when soundly based information on financial impact is available, can the recipient government and/or counterpart agency be expected to make a well informed commitment to the on-going funding of a project. The Philippines-Australia Technical and Vocational Education Project evaluation noted that firm recipient government commitments were critical to project sustainability and that more attention should be paid to this area in project design. Sustainability of many large-scale programs, such as the Thai Highland Agricultural and Social Development Project, was foundto be dependent on the continuing ability of the counterpart agency to identify future funding sources and its ability to prepare relevant funding proposals. Some evaluators consider that this ability can be strengthened and sustainability fostered if the design calls on the counterpart agency to meet an increasing proportion of recurrent costs over the life of Australian participation.

Thus, project designers need to quantify the financial impact of the project and establish the source of funds for ongoing maintenance.

Infrastructure maintenance

While most of the infrastructure projects reviewed were successful in engineering terms, benefits were often not sustained because of poor maintenance. This was always a problem when responsibilities and 'ownership' for recurrent budgetary commitments were not clearly established prior to commencement of the project. Evaluators concluded that attention must be given to the infrastructure maintenance capability of recipient agencies/beneficiaries to ensure sustainability and suggested that operation and maintenance responsibilities and funding should be agreed before construction of the infrastructure.

Maintenance of infrastructure was found to be a particular problem in the South Pacific due primarily to funding constraints and lack of technical skills. Here evaluators concluded that designers of infrastructure projects need to consider and influence national policies which determine

32

maintenance funding and commitment. This national commitment, and the funding to support it, must then be transferred down tolower levels of government and, at that local level, skills in maintenance planning and maintenance need to be developed.

More formal consideration of issues which affect the need for maintenance could also improve the sustainability of projects. Thus, the Vila Primary Schools Project in Vanuatu has demonstrated the benefits when project design minimises maintenance of equipment and uses local sources of equipment and technical support. In this Project, procedures aimed at minimising maintenance (eg. procurement of low maintenance equipment) were adopted and staff were trained to recognise thevalue of preventative and corrective maintenance.

The Water Sector Review found that water supply and sanitation facilities were often poorly maintained or operated, with the recipient community leaving responsibility for maintenance to the Government, which could not or would not accept it. A recipient community contribution of labour, money or land engendered a sense of ownership and responsibility for maintenance. However, community operation of water supplies wasonly practical for point source and simple gravity schemes. Thelong term sustainability of major water supply systems will depend on the capacity and capability of the responsible agencyto manage and maintain the system.

Some evaluators believe that cost-recovery mechanisms must be established during the life of the project to ensure sustainability. Support for this approach from the designers of the Mekong Bridge Project led to establishment of a Bridge Management Authority to set tolls to recover costs and has alsoresulted in tariffs becoming a standard feature of water supplyprojects such as the Thai North East Village Water Supply Project.

The Water Sector Review found that water development projects have a greater chance of sustainability if user charges can be levied to provide for operation and maintenance.However, the level of community support for the ‘user-pays’ principle varied with the size of the community, the extent to which the service provided met the community’s stated need for

33

water, and the extent to which communities perceive the benefits of participation to be fair and equitable. The community must also have confidence that the water institution will deliver the services and benefits that they have been promised in terms of quality, quantity and cost of water provided.

Obviously a rigorous assessment must be made of the ability of target groups to pay for the technology, social services and/orinfrastructure being introduced by the project.Thus the Water Sector Review found that water facilities must be financially sustainable at an acceptable cost to communities. Justification based on an internal rate of return was not always appropriate in water supply projects and a cost-effective analysis on humanitarian grounds and recurrent cost-financing approaches may be more appropriate.

Thus, AusAID experience suggested that designers of projects with significant infrastructure must minimise the need for maintenance and build in maintenance capability, both technical(eg. via training) and financial (eg. via cost recovery).

Monitoring and Evaluation

The Water Sector Review in 1990 found that a major problem affecting the implementation of projects was that performance monitoring was generally inadequate. For example, in the case of the Singida Project in Tanzania there was no explicit evaluation undertaken at the end of each phase which would haveallowed a considered decision to be taken on continuation or termination of the project. Fortunately, AusAID’s demands for comprehensive monitoring and evaluation on which to base milestones for payments to the managing contractor, now ensuresthat timely information is always available on how well the project is going.

However, the AusAID Quality Assurance Group noted that feedbackfrom monitoring and evaluation was only really useful on projects which were designed to permit flexibility in implementation, suggesting that designs must be allowed to evolve as experience is gained in the field.

34

3. Methods

The development aid literature and various donor data bases were reviewed to canvass a range of opinions on which elements of design contribute to project sustainability. In particular, the views of the major player in health development aid (World Bank) and two significant bilateral donors (USAID and AusAID) were sourced in the development aid literature and from publicly accessible websites on the Internet.

In the first phase of the inquiry, the opinion of each of the three donors was assessed to provide a list of those design elements considered most important by each donor. Where possible the list was based on the literal opinion of the donor, as found in published material. However, when listing AusAID opinions, the author often found it necessary to deduce a concise statement of opinion from the range of comments in the material reviewed in Section 2.6.

The three resulting opinion lists were then tabulated to make commonality and disparity of views between donors apparent. Asthere was reasonable consensus amongst these donors it was possible to draw up a Donor Consensus list of key design elementsimpacting on sustainability of health projects.

A group of six Australian development aid practitioners with extensive experience in project design and implementation was identified to validate the Donor Consensus list of key design elements and if appropriate refine the list using an iterative approach. Initially, their unprompted views were sought but repeated attempts to obtain a list from each practitioner of the particular elements of health project design which you have found to make the most contribution to project sustainability were unsuccessful. Consequently, a separate list of practitioners views could not be developed.

Practitioners were more forthcoming when asked to rank the Donor Consensus list. The list was presented to practitioners as a generic list of elements of health project design contributing to project sustainability and no attempt was made to impose a definition of sustainability. Three of the

35

practitioners5 responded with their ranking of the eleven criteria, enabling validation of the criteria and an overall ranking to be established. However, none of the criteria received a consistently low ranking by the practitioners so no shortening of the list could be made.

Consequently, with practitioner validation but without short-listing, eleven Design Criteria for Sustainability were identified, whichwere then used in the second phase of the inquiry.

The design documents of three recent and current Australian health projects implemented in the People’s Republic of China were then carefully assessed to determine if these identified Design Criteria for Sustainability had been given explicit, implicit or no consideration during the design of the projects by Australian design teams. The projects assessed were:

Project A: HIV/AIDS Prevention and Care Project (AusAID, 2000a);

Project B: Bazhong Rural Health Improvement Project (AusAID, 2000b); and

Project C: Ningxia Family Planning, Women’s and Children’sHealth Project (AusAID, 1995).

A brief outline of each of the three Projects was provided, including objectives and design features impacting on sustainability noted by the design team. The findings of the assessment were then tabulated and discussed.

Ethical Considerations

The views of individual development aid practitioners were canvassed during the study. Although the views sought were on ‘design elements’ rather than on the approach of various donors, it was recognised that practitioners may respond with views that could be considered critical and that should remain confidential. This need for confidentiality was respected by the author. Views were sought in private on a one-to-one basis by the author so the source of data would be known only to the 5 Two of these practitioners have links to the three Australian health projects assessed. One was the Australian Project Director of Project C andis the Australian Project Director of Project B. Another is the Australian Team Leader of Project A.

36

author. Although the assistance of these practitioners in the inquiry has been acknowledged in this thesis, the individual responses have not been associated with the individual practitioners. Similarly, no material, which could be used to associate names and views have be retained by the author. This approach was explained to each practitioner and his or her informed consent to participate was obtained.

37

4. Results

4.1 Opinions on Sustainability

The opinions of three major donors have been tabulated below. Where possible the opinion of the donor as found in published material has been presented literally. This was often the case with World Bank and USAID views. Where this was not possible, as in the case of most AusAID views, a concise deduction has been made by the author.

Table 1 presents in three columns the opinions, of World Bank, USAID and AusAID, arranged so that similar opinions fall into the same row. The fourth column entitled Donor Consensus presents the consensual view, of at least two of these donors, on the major elements ofproject design contributing to sustainability. The author has attempted to present the consensual views as a concise statement, encapsulating the various similar opinions.

Table 1. Donor Opinions

World Bank Opinion

USAID Opinion

AusAID Opinion Donor Consensus

Take account of local conditions (and capacity).

Address social, gender& cultural characteristics of beneficiaries.

Address local beneficiary situation

Work directly with target beneficiaries

Participation Involve women, other members of the community, other stakeholders, counterparts and counterpart agencies in the design process.

Ensure participation of local women and others in planning

Consider degree of local support from stakeholders

Establish realistic objectives

Clearly defined objectives

Ensure quality objectives

Establish good M&E to provide feedback

Design should be phased and/or allowed to evolve as experience is gained

Allow design to change in responseto feedback duringimplementation

38

in the field by implementers.

Work with and strengthen existing health institutions

Integrate activities fully into established administrativestructures

Manage project by/through those agencies which will beresponsible in the long-term.

Work with and strengthen existing rather than creating new institutions.

Strengthen the counterpart institution.

Match design to capacity of target institution(s)

Ensure size and complexity do not risebeyond local capacity.

Match project to local capacity

Take account of local (conditionsand) capacityEmphasise knowledge and information transfer

Include a strong training component

Raise awareness and provide early, local and relevant training.

Train to raise capacity

Recognise the impact of other sectors on health

Address external political, economic and natural factors

Address impact of external factors

Introduce performance basedbudgeting into projects.

Gain significant and increasingproportion of funding from local sources during the life of the project.

Establish future source of funds, including for recurrent costs.

Availability of local financial resources.

Demonstrate effective outcomes

Demonstrate effectiveness

Demonstrate the financial impact of the project with real benefits


Work within existing systems and avoid major reforms.

Consider economic, administrative, planning andimplementationcontext

Take account of fit with local policy framework and local support

Consider compatibility withlocal context.

Minimise need for infrastructure maintenance and build in maintenance capability.

Ensure good transport and communicationsAddress determinants of health that lie outside the

39

health system.Use appropriate technologyAvoid need for behavioural change

4.2 Practitioner Ranking

The Donor Consensus list as presented in column four of Table 1was ranked by the three responding practitioners from ranking #1 for the criterion with most contribution to project sustainability, to ranking #11 for the criterion with the leastcontribution.

Rank scores were summed to give a Total Score for each criterion and establish an Overall Ranking as presented in Table 2.

Table 2. Practitioner Ranking

Donor Consensus Criteria Practitioner #1 Ranking

Practitioner #2 Ranking

Practitioner #3 Ranking

Total

Overall Ranking

Work with and strengthenexisting rather than creating new institutions

2 3 3 8 1

Match project to local capacity

3 1 6 10 2

Consider compatibility with local context.

1 10 2 13 3

Address local beneficiary situation

4 2 10 16 4

Allow design to change in response to feedback during implementation

5 11 1 17 5

Train to raise capacity 7 7 4 18 6Availability of local financial resources.

8 4 8 20 7

Ensure quality objectives

11 6 5 22 8

Ensure participation of local women and others in planning

10 5 9 24 9


9 9 7 25 10

Address impact of external factors

6 8 11 25 10

40

None of the Donor Consensus Criteria received a consistently low ranking, that would justify exclusion. Thus, it was not possible to refine to a shorter list and consequently the Design Criteria for Sustainability, presented in Table 3 below, are the same as the Donor Consensus Criteria. They are numbered according tothe practitioners’ Overall Ranking.

Table 3. Design Criteria for Sustainability

No. Criteria1 Work with and strengthen existing rather

than creating new institutions2 Match project to local capacity3 Consider compatibility with local context.4 Address local beneficiary situation5 Allow design to change in response to

feedback during implementation6 Train to raise capacity7 Availability of local financial resources.8 Ensure quality objectives9 Ensure participation of local women and

others in planning10 Demonstrate effectiveness11 Address impact of external factors

4.3 Testing the Design of Recent AusAID Projects

The design documents of three recent and current AusAID projects, implemented in the People’s Republic of China (PRC), were assessed to determine whether the Design Criteria for Sustainabilityincluded in Table 3 were explicitly considered, implicitly considered or ignored during the design process. The results ofthe appraisal are tabulated below for Projects A, B and C in Tables 4, 5 and 6 and discussed in Chapter 5.

41

4.3.1 Project A - HIV/AIDS Prevention and Care Project

The HIV/AIDS Prevention and Care Project, supported by AusAID and Government of PRC, was designed in 1999-2000 to reduce the rapid transmission of HIV infection and reduce the impact of the epidemic on the social and economic development of the Xinjiang Uygur Autonomous Region, PRC. The Project commenced inMarch 2002 and shall run for at least five years with an AusAIDcontribution of about AUD 15 million.

The specific objectives were to: improve institutional capacity at the regional, prefecture

and county levels to plan and coordinate multi-sectoral responses to the HIV/AIDS epidemic,

enable the general population and vulnerable groups to adopt healthy behaviours and accepting attitudes related to the transmission and management of HIV/AIDS,

enable health workers and family carers to adopt best practice standards in direct and indirect care environments or at risk of HIV infection,

effectively and efficiently mange the project to achieve desired goals and objectives, within budget and planned timeframe.

The Project Design Document (PDD) included a specific section entitled Strategies for Sustainability (AusAID, 2000a), which highlighted design features considered by the design team to facilitate sustainability. Two of these relate to the Design Criteria for Sustainability as indicated below:

institutional and individual capacity building, with a strong emphasis on strengthening capacity of counterparts to the Australian advisors (Criterion 1),

supporting and working with local strategies and regulatory frameworks (Criterion 3),

using low maintenance, locally supported computer hardwareand software.

The third design feature is consistent with the AusAID opinions(Table 1) that project designs should take into consideration the appropriateness of technology as well as the need for and local capability for maintenance.

42

The design team for Project A gave explicit consideration to nine of the Criteria and implicit consideration to another as shown in Table 4. The comments in the Table are explained and discussed in Chapter 5.

Table 4. Consideration of Design Criteria for Sustainability in ProjectA.

No. Criteria Consideration in PDD CommentExplicit Implicit None

1 Work with and strengthen existing rather than creatingnew institutions

√

2 Match project to local capacity

√

3 Consider compatibility with local context.

√

4 Address local beneficiary situation

√

5 Allow design to change in response to feedback during implementation

√

6 Train to raise capacity √7 Availability of local

financial resources.√

8 Ensure quality objectives √9 Ensure participation of local

women and others in planning√ Due to time

constraints and sensitivity.

10 Demonstrate effectiveness √ After implementation commenced.

11 Address impact of external factors

√

4.3.2 Project B - Bazhong Rural Health Improvement Project

The Bazhong Rural Health Improvement Project, supported by AusAID and Government of People’s Republic of China (PRC) was designed in late 1999. The design team had a detailed proposal from the local health authorities, which they followed closely.The Project goal was to reduce poverty by improving the health and well being of the rural population in Bazhong Prefecture, aremote and poor area in Sichuan Province, PRC. This would be done mainly through improving capacity for health system planning in the Public Health Bureau. Selected interventions would be demonstrated at township and village levels. The AusAID contribution was to be about AUD 14.5 million.

Specific objectives were to:43

improve quality, efficiency and utilisation of the primaryhealth care services in the project townships and villages,

improve health knowledge, attitudes and behaviours in the community,

strengthen the health planning and management capacity at all levels in Bazhong Prefecture to ensure evidence-based policy development and rational resource utilisation,

improve the knowledge, skills and competence of health workers through in-service training,

ensure the project is managed effectively and that activities are undertaken on time and within budget.

The Project commenced in March 2001, to be implemented over 5 years in three phases:Phase 1. Project Planning – one year for baseline surveys, needs assessments and preparation of a plan for implementation.Phase 2. Implementation.Phase 3. Evidenced Based Planning – providing inputs to future planning.

The Project Design Document (PDD) included a specific section entitled Factors in the Design to Promote Sustainability (AusAID, 2000b). This included the following design features considered by the design team to promote sustainability. Most relate directly to one of the Design Criteria for Sustainability as indicated in parenthesisbelow:

assessment of current situation (Criterion 4), stakeholders involved in design (Criterion 9), community consultation (Criterion 9), community management of community interventions (Criterion

1), linked to the system of local five-year planning (Criterion

3), will assist Bazhong achieve its own objectives (Criterion

8), monitoring and feedback system (Criterion 5), focus is on capacity building (Criterion 1), vertically and horizontally integrated into overall health

services (Criterion 1), interventions will demonstrate benefits (Criterion 10), equipment maintenance facility.

44

The design team for Project B gave explicit consideration to nine of the Criteria and implicit consideration to two others as shown in Table 5. The comments in the Table are explained and discussed in Chapter 5.

Table 5. Consideration of Design Criteria for Sustainability in ProjectB.

No.

Criteria Consideration in PDD CommentExplici

tImplicit None

1 Work with and strengthen existing rather than creating new institutions

√

2 Match project to local capacity √ Local institutions involved


√ .


√ And their explicit support for Project.


√ Inception phase anddetailed design prior to implementation.

6 Train to raise capacity √ . 7 Availability of local financial

resources.√

8 Ensure quality objectives √ Consistent with local objectives

9 Ensure participation of local women and others in planning

√ Limited participation of community and women.

10 Demonstrate effectiveness √11 Address impact of external

factors√

4.3.3 Project C – Ningxia Family Planning, Women’s and Children’s Health Project

The Ningxia Family Planning, Women’s and Children’s Health Project was supported by AusAID and Government of People’s Republic of China (PRC) to improve the health of women and children in two poor counties of Ningxia Hui Autonomous Region,in the north-west of PRC. The initial design, prepared in 1993,was for a one year Inception Phase during which the design was refined and presented in a Project Implementation Document (AusAID, 1995). The Project commenced with a one year InceptionPhase in late 1994 and finished in 1999.

45

The specific objectives were to: improve access to quality family planning services, improve women’s and children’s health through better

access to quality essential Women’s and Children’s Health (WCH) services,

improve access to rural water supplies, environmental sanitation and hygiene education,

strengthen institutions involved in health and family planning services at regional, county and township levels.

The Project Implementation Document included a section on sustainability in which the following ‘key issues affecting sustainability’ were identified:

quality of training and linkage to supervision (Criterion 6),

quality of information, education and communication and internalisation of their development (Criterion 6),

provision of operations, repair and maintenance financing by recipient,

acceptance of an increasing share of Project costs by recipient (Criterion 7),

rational use of data for improved health program planning (Criterion 5),

rational use of essential drugs.

The design team of Project C gave explicit consideration to 10 of the eleven Criteria and implicit consideration to the remainingone as shown in Table 6. The comments in the Table are explained and discussed in Chapter 5.

Table 6. Consideration of Design Criteria for Sustainability in Project C.

No. Criteria Consideration in PID CommentExplici

tImplicit None

1 Work with and strengthen existing rather than creatingnew institutions

√

2 Match project to local capacity

√


√


√ Especially during Inception Phase

46


√ If project extendedbeyond two years ofimplementation.

6 Train to raise capacity √7 Availability of local

financial resources.√ Especially during

Inception Phase.8 Ensure quality objectives √9 Ensure participation of local

women and others in planning√ During Inception

Phase.10 Demonstrate effectiveness √11 Address impact of external

factors√

5. Discussion

5.1 Consensus of Views

In recent years, there has been general donor acceptance of theimportance of addressing sustainability during the design of development aid projects, including health projects. This has been paralleled by a commonality in the definition of sustainability used by these donors, which has facilitated a similarity of donor views on the major elements of project design contributing to project sustainability.

This similarity of donor views, reported in Table 1, was observed despite their very different scale of project, basis of funding and approaches to implementation. Thus, World Bank projects are usually larger6 compared with projects supported by the bilateral agencies, make loans rather than grants, and depend upon local personnel to manage implementation. Conversely, the typical bilateral aid agency AusAID uses an Australian managing contractor and in-country implementation

6 For example, currently the World Bank Loan Health Nine Project: HIV/AIDS Prevention and Control is being implemented across four provinces in China. The AusAID HIV/AIDS Prevention and Care Project referred to as Project A in this study is being implemented in one of those provinces (Xinjiang Uygur Autonomous Region).

47

team to manage use of the grant funds it provides. This reinforces the view that sustainability is more about what happens during the design stage than about the scale of the project or how the design is implemented.

The assistance of a group of experience Australian development aid practitioners was sought to lend practical veracity to, andperhaps supplement, the findings from the literature and data bases. Initially, their unprompted views were sought. However, it was found that practitioners were very reluctant to offer their own unprompted views of the particular elements of healthproject design, which they have found to make the major contribution to project sustainability. The author’s isolated location in rural China precluded face-to-face meetings and repeated attempts (via Email, facsimile and telephone) to obtain a list from each practitioner were unsuccessful so a separate list of practitioners views could not be developed.

Some practitioners were more forthcoming when asked to rank theDonor Consensus list of criteria. Three of the practitioners responded with their ranking of the eleven criteria, enabling an Overall Ranking to be established. However, the author’s expectation that practitioners would be fairly consistent in their ranking, enabling the progressive elimination of less favoured criteria was not to be. As evidenced in Table 2, individual practitioners’ rankings were all very different and a ‘short-listing’ process was therefore not considered valid and not pursued.

The overall ranking itself was considered less important than the validation of these eleven criteria, which was provided by the practitioners. All practitioners pointed out the difficultythey had in ranking the criteria, because they considered that all the criteria were important in ensuring project sustainability. It is acknowledged that the generalist approachtaken, to derive the generic list of Design Criteria for Sustainability sought in this inquiry, must sacrifice precision. Thus, the concise statements, encapsulating several similar opinions, in the Donor Consensus list would probably be interpreted differently by the different practitioners. Each could have their own, individual definition of sustainability as their

48

point of reference because no attempt was made to impose a common definition.

Although all were Australian development aid practitioners, their different experiences with different projects would no doubt have also influenced their ranking of the Criteria. More precision would have resulted if the practitioners had been asked to rank the Criteria for a common definition of sustainability and for specified types of project, perhaps one emphasising institutional strengthening and another behaviouralchange in the community.

Bossert (1990) concluded that the projects most likely to be sustained were ‘designed with mutually respectful negotiations between donor and recipient’ and the Prevention of Maternal Mortality Network (Kamara, 1997) similarly concluded ‘collaboration with government and communities enhanced sustainability’ in East Africa. Consistent with these views, in the present study the Overall Ranking of the practitioners indicated they recognise the importance of ‘local knowledge’ and the importance of working with the recipient agencies and communities as embodied in:

Criterion 1 – Work with and strengthen existing rather than creating new institutions;

Criterion 2 – Match project to local capacity; Criterion 3 – Consider compatibility with local context; and Criterion 4 – Address local beneficiary situation.

However, only one of the practitioners gave a high ranking to Criterion 9 - Ensure participation of local women and others in planning. Perhaps this is because the explicit involvement of local people and in particular, local women, in the actual project design process is logistically difficult and often precluded bysocial or cultural factors.

5.2 Consideration of Criteria During Design of AusAID Projects.

The Design Criteria for Sustainability (Table 3) identified in the first phase of this inquiry represents the consensual views of three major supporters of health projects. The importance of these Criteria has been subsequently validated by three experience Australian development aid practitioners. To determine whether

49

these Criteria have actually been used by health project design teams in practice, an assessment was made of three bilateral health projects recently completed or currently being supportedby AusAID in the People’s Republic of China. The results strongly indicate that these Criteria were, in fact, generally considered by the design teams, in most instances explicitly.

5.2.1 Consideration in Project A.

In designing Project A, the team explicitly considered nine of the eleven Design Criteria for Sustainability (Table 4). The team’s emphasis on these nine Criteria and apparent failure to address the others was strongly influenced by how it viewed the Project. Thus, although not directly reflected in the objectives, the design team conceptualised two distinct approaches in this Project: the building of institutional capacity and trials of best-practice approaches to HIV/AIDS care and prevention.

The latter approach was reflected in the team’s attempt to allow design to change in response to feedback during implementation (Criterion 5). An ‘Activity Support Facility’ was included to fund trials and other interventions not specified in the project design. Hence, the team not only acknowledged the importance of Criterion 5 but provided a practical mechanism to allow it to happen.

This ‘trial approach’ may explain why the design team did not ensure participation of local women and other members of the beneficiary community in the design process (Criterion 9). Its failure to do so was surprising in light of the recent AusAID evaluations of other HIV/AIDS projects, which strongly supported this focus on community participation in design (AusAID, 2001). While acknowledging the potential benefit of explicit inclusion of Criterion 9, the design team noted that ‘direct consultation with target populations was limited due toboth time availability and the sensitive nature of HIV/AIDS in Xinjiang’. Given the practical difficulties involved, perhaps the design team felt a subsequent trial could substitute for true community participation in the design.

50

Similarly, perhaps the design team expected this trial of best-practice approaches could also substitute for demonstration of the effectiveness of project objectives (Criterion 10) during the design stage by demonstrating the effectiveness of the health promotion and health care objectives once implementation had commenced. However, Garner et al (1998) would argue that dissemination of information to the community about effectiveness of the objectives cannot be delayed without detriment to beneficiary attitudes to the Project and its interventions.

Consideration of external factors (Criterion 11) was evident throughout the PDD, the design team recognising that major risks to successful implementation and sustainability would be due to religious, political and legal factors outside the health sector. For example, the team noted (AusAID, 2000a) that‘public discussions on sexual behaviour are not generally culturally or religiously acceptable in Xinjiang’ and that ‘drug use and prostitution in China are illegal’, which would severely limit the scope for using internationally accepted strategies for behaviour change or harm minimisation.

5.2.2 Consideration in Project B

In designing Project B, the team explicitly considered nine of the eleven Design Criteria for Sustainability (Table 5). The design team indicated they were strongly influenced by the proposal prepared by the recipient agency7, apparently assuming that it represented the views of local stakeholders and the community. Thus while the team gave consideration to the views of the community beneficiaries (Criteria 4 and 9), this was restricted. Real consultation with women and others in the beneficiary community did not occur during the design and was proposed instead for consideration during implementation. Cockcroft (1998) would no doubt argue that this approach missed an important opportunity to raise the status of women by explicit recognition of them as contributors to family health.

The design team argued that ‘increased awareness of health issues in the population… will lead to sustainable improvements

7 Bazhong Public Health Bureau.51

in health status through improved demand for appropriate healthservices’. However, USAID evaluators in Zaire and Senegal (Bossert, 1990) found that community involvement at the design stage was important in sustaining their user-pays health systems. As cost recovery8 is now a key element of health services in China (Carrin et al, 1999) this finding would suggest that the beneficiary community should have been more involved in the design of Project B.

By taking on the objectives of the recipient government, perhaps the team did not necessarily ensure quality objectives (Criterion 8). While bringing the obvious advantage of compatibility with local context (Criterion 3), the design team risked adopting unrealistic objectives, which Garner (1997) often found caused a misallocation of health sector resources and had a negative effect on sustainability in the projects he studied.

The design of this Project placed emphasis on allowing the design to change in response to feedback (Criterion 5) by including an inception phase of baseline surveys and needs assessment. These were intended to provide the evidence for thedetailed design of the implementation phase and a final planning phase. However, the establishment of a continuing system for evidence collection may have been better than one-off surveys as World Bank (1999) found the major benefit to be when an on-going system was providing feedback to guide the local institution long after external funding had ceased.

While designing a project which would work with and strengthen the existing institutions (Criterion 1) the team overlooked explicit consideration of the need to match the project to local capacity (Criterion 2) thus emulating the fault in many WorldBank projects of using complex designs, with a greater number of components, in situations where institutional capacity is weak (World Bank, 1999).

5.2.3 Consideration in Project C

8 Since the mid-eighties health facilities have received a diminishing shareof their operating budgets from government and staff salaries are no longerfully funded.

52

During the design of Project C, consideration was given to all Design Criteria for Sustainability (Table 6) although, for some, this consideration was not until the design was being finalised during the one year Inception Phase of implementation. For example, Criterion 4 was not fully addressed until the Inception Phase when Australian and Chinese specialists carried out comprehensive assessments of the local beneficiary situation. Similarly, the Inception Phase provided opportunities to consider availability of local financial resources (Criterion 7) and to consult with women and to involve them in defining the services they required (Criterion 9).

The quality and appropriateness of the objectives (Criterion 8) originally identified in the Project Design Document (PDD) wereconfirmed after detailed review during the Inception Phase and included in the Project Implementation Document (PID).

Demonstrating the effectiveness (Criterion 10) of voluntary familyplanning services was included by the design team as a separateOutput in the PDD. Recognising that ‘good data collection requires incentives’ funding for a ‘monitoring fee’ was later included in the PID (AusAID, 1995). However, there was no stated intention in the PID to allow the design to change in response to feedback during implementation (Criterion 5) although,no doubt it was recognised, that the two year extension would provide an opportunity to do so.

Given that all eleven of the Design Criteria for Sustainability identified in this study were considered by this design team, it is instructive to note two major conclusions, which relate to sustainability, made in the Completion Report for Project C (AusAID, 2000c). Firstly, the need for ‘orientation and simplification of activities’ during implementation, which relates to Criterion 5 – Allow design to change in response to feedback during implementation. Secondly, the ‘value of participatory and consultative approaches to the planning and management’, which relates to Criterion 9 – Ensure participation of local women and others in planning.

5.2.4 Limits on Extent of Consideration

53

Assessment of the design documents for Projects A, B and C indicated that the Design Criteria for Sustainability have definitely been considered by recent AusAID design teams, in most instances explicitly. However, the very limited amount of time usually allowed for the design process may have limited the extent of such consideration despite the best intentions of theteam and the requirements of the donor.

Projects A and B both illustrate this conundrum. Both were designed by teams, which spent only a few weeks9 at the proposed project site. This limited both the gathering of firsthand information about the local beneficiary situation (Criterion 4) and the participation of local women and others in the planning of the project (Criterion 9), even though these Criteria are considered essential to sustainability by AusAID (Table 1).Hopefully, the one-year Planning Phase proposed for Project B will provide an opportunity to more fully consider these important Criteria.

This was the case with Project C, which was detailed and finalised during a one-year Inception Phase, providing sufficient time to gain ‘a deeper understanding of the situation and needs of the target populations and the institutions serving them’ (AusAID, 1995). Similarly, the Inception Phase provided opportunities to consult with women and to involve them in defining the services they required.

5.3 Additional Criteria

Infrastructure maintenance was considered important to sustainability by AusAID (see Section 2.6) and two related opinions were included in the ‘AusAID Opinion’ column of Table 1 : (i) Minimise need for infrastructure maintenance and build in maintenance capability; and (ii) Use appropriate technology. As these Opinions were not shared by either of the other two donors studied, theywere not included in the list of Donor Consensus Criteria.

However, the importance of various aspects of ‘maintenance’ in facilitating sustainability was stressed by the design teams of9 For example, Project B design team spent 35 days in Sichuan Province including just 27 days in Bazhong Prefecture and only 13 days in counties, townships and villages.

54

all three of the AusAID health projects appraised, suggesting perhaps that a further criterion should be included in the Design Criteria for Sustainability, such as ‘minimise the need for maintenance and maximise the capability for maintenance’.

This suggestion highlights an important deficiency in this study which, in the absence of unprompted responses from practitioners, has been limited to considering only those elements of health project design put forward by the three major donors – World Bank, USAID and AusAID. However, it also serves as a reminder that a list of Design Criteria for Sustainability could be extended considerably, to the point where it is no longer a practical tool to succinctly guide Australian development aid practitioners when they are designing health projects in developing countries.

It also highlights the need to interpret each of the Design Criteria for Sustainability quite broadly. So that for example, the need to ‘minimise the need for maintenance and maximise the capability for maintenance’ should be considered by a design team within Criteria 2 – Match project to local capacity, Criteria 6 – Train to raise capacity and Criteria 7 – Availability of local financial resources.

55

6. Conclusions and Recommendation

This inquiry has been able to identify the particular elements of project design which are considered to make the major contribution to sustainability of development aid projects in the health field.

The consensual view of three major donors – World Bank, USAID and AusAID – validated by three experienced Australian development aid practitioners has been used to articulate the following generic list of Design Criteria for Sustainability:1. Work with and strengthen existing rather than creating newinstitutions2. Match project to local capacity3. Consider compatibility with local context.4. Address local beneficiary situation5. Allow design to change in response to feedback during implementation6. Train to raise capacity7. Availability of local financial resources.8. Ensure quality objectives9. Ensure participation of local women and others in planning10. Demonstrate effectiveness11. Address impact of external factors

It was recognised that some Criteria may be more important than others in particular projects and/or situations, however, the resources available to this inquiry did not enable this to be clarified. Hence, this list should not be considered as indicating any ranking of importance. All eleven Design Criteria for Sustainability were considered important by both donors and practitioners.

An assessment of the design documents of three AusAID Projects confirmed that Australian project design teams have given consideration to these Design Criteria for Sustainability when formulating health projects in the PRC.

It was concluded that these criteria can provide succinct guidance to Australian development aid practitioners to help

56

them design sustainable health projects, at least in the PRC and probably in other developing countries as well.

Thus, it is recommended that consideration should be given to each of the listed Design Criteria for Sustainability by development aid practitioners when they are designing health projects to maximise the likelihood that the benefits of the project will continue after donor funding has ceased.

57

7. References and Additional Reading

7.1 References

Andersson-Brolin L (1991) The Art of Survival: a study on sustainability in health projects. Swedish International Development Authority (SIDA), Stockholm.

AusAID (1990) Water Sector Review. AusAID, Canberra.

AusAID (1991a) Learning from Experience - A Synthesis of AusAIDEvaluations. AusAID, Canberra.

AusAID (1991b) Review of Integrated Area Development Projects. AusAID, Canberra.

AusAID (1995) Ningxia Family Planning, Women’s and Children’s Health Project. Project Implementation Document, AusAID, Canberra.

AusAID (2000) Promoting Practical Sustainability. Quality Assurance Group, AusAID, Canberra.

AusAID (2000a) HIV/AIDS Prevention and Care Project, Project Design Document, AusAID, Canberra.

AusAID (2000b) Bazhong Rural Health Improvement Project, Project Design Document, AusAID, Canberra.

AusAID (2001) Lessons Learned Data Base. Website: www.ausaid.gov.au with over 70 items posted on various dates.

AusAID (2002) AusGUIDE. Published on Website: www.ausaid.gov.au

AusAID (2002) Australia-China Development Cooperation Program, AusAID, Canberra.

Bamberger M and Cheema S (1990) Case studies of project sustainability, implications for policy and operations from Asian experience. EDI Seminar Series, World Bank, Washington.

58

http://www.ausaid.gov.au/

http://www.ausaid.gov.au/

Beach RF, Ruebush TK, Sexton JD (1993) Effectiveness of permethrin-impregnated bed-nets and curtains for malaria control in a holoendemic area of western Kenya. American Journal of Tropical Medicine and Hygiene 49, 290-300.

Bossert TJ (1990) Can they get along without us? Sustainabilityof donor-supported health projects in Central America and Africa. Soc. Sci. Med. 30: 1015-23.

Bossert TJ (2001) Department of Population and International Health, Harvard University (Personal communication.) 10 March 2001.

Carrin G, Aviva R, Yang H, Wang H, Zhang TH, Zhang LC, Zhang S,Ye YD, Chen JY, Jiang QC, Zhang ZY, Yu J and Li XS (1999) The reform of the rural cooperative medical system in the People’s Republic of China: interim experience in 14 pilot counties. Social Science and Medicine 48: 961-72.

Cockcroft A (1998) From Kilimanjaro to the Himalayas: studies of health determinants in Third World communities. J Royal College of Physicians of London. 32:66-71.

Delclos GL, Felknor SA, Morandi MT, Schulze LJ, Tovar LS and Yanes L (1999) The role of information dissemination in sustainability of international partnerships. International Journal of Occupational & Environmental Health 5(3):203-8.

Garner, P (1995) Is aid to developing countries hitting the spot? BMJ 311:72-73

Garner P (1997) Do objectives of health-aid programmes impair their effectiveness?Lancet 349: 722-23.

Garner P, Kale R Dickson R, Salinas R (1998) Getting research findings into practice: Implementing research findings in developing countries. BMJ 317: 531-5.

GOB (1989) Joint Review of the Expanded Program on Immunisationin Bangladesh: Summary Report of Findings, Key Issues and

59

Recommendations. Government of Bangladesh/World Bank/SISA/USAID/UNICEF/WHO/GUB.

Hussain A and Kvle G (1996) Sustainability of a nutrition education programme to prevent night-blindness. Tropical Medicine & International Health 1: 43-51.

Kachur S P, Phillips-Howard PA, Odhacha AM, Ruebush TK, Oloo AJ, Nahlen BL (1999) Maintenance and sustained use of insecticide-treated bed-nets and curtains three years after a controlled trial in western Kenya. Tropical Medicine and International Health 4: 728-35.

Kamara, A (1997) Lessons learned from the PMM Network experience. International Journal of Gynaecology & Obstetrics 59 Suppl 2: S253-8.

Lengeler C (1999) From Rio to Iragua - sustainability versus efficiency and equity for preventive health interventions. Tropical Medicine and International Health 4 : 409-11.

Lorenz N, Kilima P, Tanner M, Garner P. (1995) The right objectives in health careplanning. World Health Forum 1995; 16: 280-82.

OECD (1989) Sustainability in Development Programs: a compendium of evaluation experience. OECD, Paris, pp. 1-50.

Prasad B and de L Costello AM (1995) Impact and sustainability of a "baby friendly" health education intervention at a district hospital in Bihar, India. BMJ 310: 621-23.

Taylor CE (1995) E.H. Christopherson Lecture: lessons for the United States from the worldwide child survival revolution. Pediatrics 96: 342-6.

WHO and UNICEF (1978) Alma-Ata 1978: Primary Health Care. WHO Health for All Series No. 1. Geneva.

60

World Bank (1998) The Brazil Health System. Operations Evaluation Department Impact Evaluation Study No. 18142, Website: worldbank.org posted 30/6/98. World Bank (1999a) Development Effectiveness in Health, Nutrition, and Population: Lessons from World Bank Experience. Sector study No. 19266 , Website: worldbank.org posted 5/10/99.

World Bank (1999b) Case Study of the World Bank Activities in the Health Sector in India. Operations Evaluation Department Sector Study 19537. Website: worldbank.org posted 29/6/99.

7.2 Additional Reading

Aubel J and Samba-Ndure K (1996) Lessons on sustainability for community health projects. World Health Forum 17: 52-57.

AusAID (1996) Country Environment Profile of China. AustralianAgency for International Development, Canberra, Australia.

Berman PA (1995) (editor) Health System Reform in Developing Countries: making health development sustainable. Harvard Press, Boston, USA.

Bloom G (1998) Health Services in Poor Counties in China: Opportunities for Donor Support. Department of International Development, London, UK.

Bloom G and Shenglan T (1999) Rural health prepayment schemes in China: towards a more active role for government. Social Science and Medicine 48 (7) : 951-960.

Boostrom E (1990) Sustainability of US-supported Health, Population and Nutrition Programs in Guatemala: a review of health services projects (1942-1987). USAID, Washington, D.C., USA.

Carrin G, Ron A, Hui Y, Hong W, Tuohong Z, Licheng Z, Shuo Z, Yide Y, Jiaying C, Qicheng J, Zhaoyang Z, Jun Y and Xuesheng L (1999) The reform of the rural cooperative medical system in

61

http://www.worldbank.org/



the People’s Republic of China: interim experience in 14 pilot counties. Social Science and Medicine 48 (7) : 961-972.

Dietrich C (1994) People’s China. 2nd Edition, Oxford, UK.

Guldner M (1995) Health care in transition in Vietnam: equity and sustainability. Health Policy and Planning 10 (Supplement):49-62.

Gonzalez JL (1998) Development Sustainability Through CommunityParticipation: mixed results from the Philippine health sector.Ashgate Aldershot, Brookfield, USA.

James VU (1996) (editor) Sustainable Development in Third WorldCounties: applied and theoretical perspectives. Praeger, Westport, Connecticut, USA.

La Fond AK (1994) Sustainability and Health Sector Development:a review and analysis of 5 country case studies. Working Paper 9, Save the Children, London, UK.

La Fond AK (1995) Improving the quality in investment in health: lessons on sustainability. Health Policy and Planning 10: 63-76.

OECD (1995) Planning for Sustainable Development: country experiences. OECD, Paris.

Olsen IT (1998) Sustainability of health care: a framework for analysis. Health Policy and Planning 13 (3) : 287-295.

Pelt MJF van (1994) Sustainability oriented appraisal of agricultural projects. Journal of International Development 6 (1) : 57-78.

Prince JS, Neumann AK and North WH (1996) Sustainability of theDanfa Ghana Group of Health Projects: a 26-year overview, in James VU (1996) (editor) Sustainable Development in Third WorldCounties: applied and theoretical perspectives. Praeger, Westport, Connecticut, Chapter 8.

62

Sidel R and Sidel VW (1982) The Health of China. Zed Press, London, UK.

Smithson P (1994) Health Financing and Sustainability: a reviewand analysis of 5 country case studies. Working Paper 10, Save the Children, London, UK.

Stefanini A (1995) Sustainability: the role of NGOs. World Health Forum 16: 42-46.

Tiwari DN (2000) Sustainability criteria and cost-benefit analysis: an analytical framework for environmental decision making at project level. Environment and Development Economics 5 (3): 259-288.

Unschuld PU (1991) Health and medicine, in Hook, B.(Editor), The Cambridge Encyclopedia of China, 2nd Edition, Cambridge University Press, Cambridge, UK. pp120-131.

Wong GC, Li VC, Burris MA and Xiang Y (1995) Seeking women’s voices: setting the context for women’s health interventions intwo rural counties in Yunnan, China. Social Science and Medicine 41 (8) : 1147-57.

World Bank (1992) China: Long-term Issues and Options in the Health Transition. World Bank, Washington, USA.

Word count 15, 557

63

Designing Sustainability into Health Projects

Documents

Transcript of Designing Sustainability into Health Projects