vietnam intervention project to assist persons with disabilities

DISCLAIMER This publication was produced at the request of the United States Agency for International Development and made possible by the support of the American people through USAID. The contents of this publication are the sole responsibility of the Management Systems

International, a Tetra Tech Company, and individual authors, including Dr. Hoang Tran, Dr. Thuy Nguyen, Mr. Lan Dang, Ms. Hanh Ta, Ms. Leah Maxson, and Ms. Susan Eitel (Team Leader. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

MID-TERM WHOLE-OF-PROJECT

EVALUATION

VIETNAM INTERVENTION PROJECT TO

ASSIST PERSONS WITH DISABILITIES

July 2018

ACKNOWLEDGEMENTS

The evaluation team is grateful to all individuals and organizations for their time, candid responses to

questions, and thoughtful insights shared during this evaluation. A special note of thanks to

USAID/Vietnam for their contributions, commitment to learning, and ongoing support for persons with

disabilities.

CONTENTS

ACKNOWLEDGEMENTS 1

ABSTRACT 3

ACRONYMS 4

EXECUTIVE SUMMARY 5

EVALUATION PURPOSE & EVALUATION QUESTIONS 8

PROJECT BACKGROUND 10

EVALUATION METHODS & LIMITATIONS 13

FINDINGS AND CONCLUSIONS 17

RECOMMENDATIONS 41

WOPE LESSONS LEARNED 43

ANNEX I: EVALUATION STATEMENT OF WORK

ANNEX II: WOPE LEARNING

ANNEX III: DATA COLLECTION INSTRUMENTS o Stakeholder interview form

o Beneficiary Survey Form – Direct Assistance

o Beneficiary Survey form - Training

ANNEX IV: SOURCES OF INFORMATION o Disability MEL Framework

o List of Persons Interviewed

o Bibliography of Documents Reviewed

ANNEX V: SUPPLEMENTARY REPORTS DEVELOPED o Desk Review Report

o Beneficiary Survey Report

o Training Survey Report

ANNEX VI: DISCLOSURE OF ANY CONFLICTS OF INTEREST

3 | MID-TERM WHOLE-OF-PROJECT EVALUATION USAID.GOV

ABSTRACT

In early 2018, the United States Agency for International Development (USAID) commissioned a mid-

term whole-of-project evaluation (WOPE) of the Vietnam Disability Project. The purpose of the WOPE

is to determine to what extent the USAID-funded Vietnam Disability Project 2015-2020, with its six

Activities, is progressing toward achieving its purpose: A sustainable model for improving persons with

disabilities’ inclusion developed in USAID target provinces.

The main evaluation questions focus on assessing progress toward achieving the Project purpose, the

contributions of the Project’s six Activities, and the strengths and weaknesses of the Project’s theory of

change. Evaluation methods comprised a desk review, key informant interviews, focus groups, and two

quantitative surveys of beneficiaries receiving rehabilitation care and training from the project.

The Project design centers around the results framework and the sustainable model, neither of which

are fully defined nor unified. Progress has been made in achieving the Project purpose, but measuring the

extent of progress is limited by the lack of clearly defined targets. Implementing partners have exceeded

targets for indicators established at the Project onset. Coordination between implementing partners and

links with former USAID projects and existing rehabilitation initiatives is underdeveloped.

The mid-term WOPE provides an opportunity to assess collaborative efforts toward achieving the

Project purpose. These are not captured in other evaluations. It also highlights the need to further refine

elements of the Project model, as well as the links between the original Project design and the refined

2017 version. To fulfill the Project purpose, USAID should establish benchmarks for sustainability and

inclusion.

ACRONYMS ACDC Action for Community Development Center

ADS Automated Directives System (USAID’s Operational Policies)

AT Assistive Technology

CDCS Country Development Cooperation Strategy

DIS Disability Information System

DMP Disability Monitoring Plan

DOET Department of Education and Training

DOH Department of Health

DOLISA Department of Labor, Invalids and Social Affairs

DPO Disabled Peoples’ Organization

DRD Disability Research and Capacity Development

ECDDI Early Childhood Disability Detection and Intervention

GBV Gender-Based Violence

GVN Government of Vietnam

HCMC Ho Chi Minh City

HI Humanity and Inclusion (formerly Handicap International)

IC International Center

IP Implementing Partner

KII Key Informant Interview

M&E Monitoring and Evaluation

MCNV Medical Committee Netherlands Vietnam

MEL Monitoring Evaluation and Learning

MOET Ministry of Education and Training

MOH Ministry of Health

MOLISA Ministry of Labor, Invalids, and Social Affairs

NCD National Committee on Disability

OT Occupational Therapy

P&O Prosthetics and Orthotics

PAD Project Appraisal Document

PMP Performance Management Plan

POS Physical, Occupational and Speech (Therapies)

PT Physical Therapy

QCA Qualitative Comparative Analysis

RFA Request for Applications

SaLT Speech and Language Therapy

SOW Statement of Work

ST Speech Therapy

TOC Theory of Change

UNCRPD United Nation Convention on the Rights of Persons with Disabilities

USAID United States Agency for International Development

VEMSS Vietnam Evaluation, Monitoring and Survey Services

VNAH Vietnam Assistance for the Handicapped

VND Vietnamese Dong (currency)

WHO World Health Organization

WOPE Whole-of-Project Evaluation

JAN 10, 2017 STATEMENT OF WORK -PAGE 5 OF 184

VIETNAM

EXECUTIVE SUMMARY

The United States Agency for International Development (USAID) has been supporting programs for

persons with disabilities in Vietnam for nearly 30 years. The current USAID-funded Vietnam

Disability Project 2015-2020 is a Project in transition.

An external mid-term whole-of-project evaluation (WOPE) took place during March – June 2018.

WOPEs are a tool to enable USAID to undertake course corrections and learn lessons at the

portfolio level; as of March 2018, approximately five USAID Missions have completed a WOPE.

EVALUATION PURPOSE

The overarching purpose of the evaluation is to determine to what extent the Vietnam Disability

Project 2015-2020, with its six Activities, is progressing toward achieving its purpose: A sustainable

model for improving persons with disabilities’ inclusion developed in USAID targeted provinces.

The aim of the evaluation is to provide USAID/Vietnam, the Government of Vietnam (GVN) and

other in-country stakeholders with objective information on what has been achieved to date, what is

working and what is not. The information will inform decisions related to appropriate modifications

in the remaining period of performance of the Project, and will serve as a reference for the design of

future interventions. USAID/Vietnam also expects to learn and share experience/lessons in

conducting a WOPE.

The main evaluation questions focus on assessing progress toward achieving the Project purpose,

contributions of the six Activities, and the strengths and weaknesses of the Project’s theory of

change (TOC). Additional questions related to project management and cross-cutting issues are also

presented.

PROJECT BACKGROUND

USAID began supporting programs for persons with disabilities in Vietnam in 1989. Areas of

attention include humanitarian response, disability policy/advocacy, inclusive education,

vocational/employment opportunities, and physical rehabilitation. The United States Congress

directive focuses resources for health and disability programs in areas of Vietnam sprayed with

Agent Orange and otherwise contaminated with dioxin. Selection of targeted provinces for this

Project is influenced by this directive, in conjunction with disability prevalence and capacities of

provincial authorities.

In 2014, USAID/Vietnam crafted a Project Appraisal Document (PAD) that provides the background

and roadmap to address the Project purpose at the time: Expanded opportunities for vulnerable

populations. The three key outputs were: 1) disability rights policies, coordination and advocacy

fostered; 2) physical, occupational and speech (POS) therapy services delivered to assist persons

with disabilities to participate in society; and 3) improved local capacity to provide services.

Corresponding results and logical frameworks supported the PAD, an Annual Program Statement

was released, and six implementing partners (IPs) began work in 2015 using this project design.

In late 2017, USAID/Vietnam amended the PAD to adjust the Project purpose to: A sustainable

model for improving persons with disabilities’ inclusion developed in USAID’s target provinces. The

impetus for the amendment was to more clearly define and operationalize the 2014 purpose. The

amended PAD introduces five highly measurable elements that constitute the sustainable model,

refines the previous results framework, expands the geographic coverage of the Project, and extends

the PAD completion date to 2024.

EVALUATION QUESTIONS, DESIGN, METHODS AND LIMITATIONS

The team members for this WOPE included: two international consultants, two Vietnamese

consultants, and staff of the Vietnam Evaluation, Monitoring and Survey Services (VEMSS) mechanism

implemented by Management Systems International, a Tetra Tech Company. The team began

preparation for the evaluation, including document review, in March 2018 and conducted joint

fieldwork and initial analysis in Vietnam from April 2 to May 5, 2018. The evaluation categories and

questions are summarized below.

PERFORMANCE

● To what extent has progress been made in achieving the Project’s purpose?

● To what extent have each of the six component Activities contributed to this?

● What are the key achievements and progress against targets in key program areas?

VALIDATION OF THE PROJECT’S DESIGN

● What are the strengths and weaknesses in the Project’s theory of change?

● Is there a strategy or intervention in initial design that has had an unexpected influence?

PROJECT MANAGEMENT AND CROSS-CUTTING ISSUES

● What are the benefits and challenges of coordinating the Project’s implementation activities

with the Ministry of Labor, Invalids and Social Affairs (MOLISA)?

● To what extent has USAID’s gender policy been implemented, lessons learned?

● What is the evidence of local ownership?

A mixed-method design was used to collect the data needed to answer the evaluation questions.

Data collection comprised an in-depth desk review of Project reports, IP self-assessments, and GVN

disability-related policy documents; two quantitative beneficiary surveys covering aspects of training

and direct assistance; and field-based qualitative research through key informant interviews (KIIs),

focus groups and observation.

One limitation of the evaluation was the lack of a clear and consistent understanding of a WOPE by

USAID/Vietnam, the IPs and the evaluation team, which led to differing expectations for the focus

and outcome of the evaluation. Evaluating the Project design and assessing progress toward the

Project purpose was a challenge as the Project is in transition. Even with limitations, this experience

has provided USAID/Vietnam with information for future programming, as well as lessons learned

from conducting a WOPE for USAID’s Learning Lab.

FINDINGS AND CONCLUSIONS

A. PERFORMANCE

FINDINGS CONCLUSIONS

1. There are signs of progress with all five elements of the sustainable model. However, only two of the five elements have targets, and three have only descriptive/illustrative content.

1. Progress has been made toward achieving the Project purpose, but underdeveloped targets within the sustainable model limit measurement of progress.

2. All six IPs contribute to elements of the sustainable model and the GVN appreciates the support for rehabilitation system strengthening.

2. All IPs contribute to the Project purpose and rehabilitation-related contributions are highly valued by GVN stakeholders.

3. All IPs contribute toward achieving the Project purpose, but not in a systematic or harmonized way with each other, or with other related initiatives.

3. IPs contribute to the Project purpose, but the lack of coordination may limit the impact of some interventions.

4. Available indicator data shows over 100% achievement for the three program areas, but the indicators are based on the 2014 PAD.

4. The Project is overachieving in targets pertinent to the 2014 PAD, but the targets do not fully align with the 2017 sustainable model.

B. VALIDATION OF PROJECT DESIGN


JAN 10, 2017 STATEMENT OF WORK-PAGE 7 OF 184

RECOMMENDATIONS

PERFORMANCE

5. There is a lack of detail in the elements of the

sustainable model, in the Project’s TOC, and in

the sub-components of the results framework.

5. The Project will remain open to misinterpretation until all

Project design aspects are clearly and fully described.

6. Connections between the sustainable model

and the results framework are not fully

developed.

6. A mismatch and the potential for confusion will exist until

the contents of the sustainable model and the results

framework are integrated.

7. USAID has not defined the terms “sustainable”

and “inclusion” as they relate to the Project.

7. Until sustainability and inclusion are defined and benchmarks

are established, efforts to address them will be arbitrary and

difficult to measure.

8. The Project focuses primarily on rehabilitation.

The links between rehabilitation and disability

policies and rights are tenuous. Moreover, the

roles of Disabled Peoples’ Organizations

(DPOs) in the Project are neither well defined

nor resourced.

8. As the Project’s purpose encompasses more than inclusion

of persons with disabilities, DPOs could play a greater role

in driving the demand for rehabilitation and developing

rehabilitation policy.

9. Project IPs have cooperative agreements that

preclude USAID/Vietnam from directing how

interventions are implemented.

9. USAID’s assistance instrument for the Project has resulted

in unintended and innovative partnerships, but has also

created the potential for differing implementation standards.

10. Increasing the health insurance coverage for

rehabilitation techniques has resulted in a high

demand for staff licensed to provide

rehabilitation. However, the methods and

timelines to become licensed are highly

variable.

10. Unless regulations for rehabilitation licensure are

consistently applied, there is a potential for POS reputations

to be damaged and professional standards undermined.

C. PROJECT MANAGEMENT AND CROSS-CUTTING ISSUES


11. Disability and rehabilitation are part of

MOLISA’s mandate. The benefits of

coordinating with MOLISA and its local-level

authorities depend on Project interventions.

11. Disability policy-related efforts supported by the Project are

more likely to be effective if carried out in consultation with

MOLISA’s Legal Department and MOLISA’s local

authorities.

12. Within MOLISA, the National Committee on

Disability (NCD) wants greater ownership and

engagement in the Project, and is taking steps in

this direction.

12. Though the NCD has expressed concern about lack of

engagement with the Project, this committee does not

represent MOLISA in its entirety.

13. Roles of the Ministry of Health (MOH) and

MOLISA related to rehabilitation, as well as the

working relationship between the ministries,

are still evolving. This evolution includes

provision of rehabilitation services, workforce,

and assistive products.

13. MOLISA’s plan to expand rehabilitation-related activities

may either complement or confound the aims of the

Project. Engaging MOLISA’s Health Unit in the Department

of Social Protection is key to promoting synergies and

ensuring effective use of resources.

14. The Project is compliant with the USAID

Gender Policy and ADS 205; Activities address

key gender gaps found in in the 2014 PAD

analysis and 2017 PAD amendment.

14. Operationally, the Project has met the requirements of the

Agency Gender Policy and ADS 205 and has integrated

details within key documents for Project/Activity planning

and implementation. However, the overall the quality and

consistency of common gender-related messaging and

interventions at the Activity level warrants closer attention.

14a. The IPs have varied levels of understanding of

key gender-related gaps and effective responses,

and the overall influence of gender-specific

interventions on beneficiaries is unclear.

15. There is a strong sense of local ownership for

Project Activities that are in line with existing

GVN policies and practices.

15. The GVN is poised to take more responsibility in the

rehabilitation sector. USAID is in a strong position to

facilitate this process.

1. Support rehabilitation systems strengthening and services in Project areas in cooperation with

the relevant responsible ministry/department (departments of health [DOHs] and/or

departments of labor, invalids and social affairs [DOLISAs]) in accordance with MOH policies.

2. Investigate ways to minimize duplication of IP interventions, promote Activity standards, and

capitalize on rehabilitation-related investments outside the Project to effectively realize the

Project purpose.

3. Structure IP coordination meetings to progress beyond information sharing and toward issues-

based engagement, to meet the Project purpose.

4. Continue support for rehabilitation-related policy developments (e.g., MOH Circular #18) with

a vision toward monitoring outcomes of care.

5. Conduct a needs assessment specific to assistive technology in Vietnam. This should include

workforce, products, procurement, and provision.

PROJECT DESIGN 6. Refine the elements of the sustainable model.

7. Integrate the results framework and the sustainable model within the Project design.

8. Update the Project TOC and indicators to include the most current content of the Project

design.

9. Identify unintended consequences of rapidly scaling up rehabilitation services (systems and

personnel) and invest in corrective measures.

CROSS-CUTTING ISSUES

10. Continue coordination with MOLISA to address disability policy and rehabilitation-related

actions.

11. Reinforce rehabilitation content within disability policy, rights and advocacy messaging work.

12. Review with IPs their understanding of, and practices of, integrating gender equality into their

work, as well as the influence upon beneficiaries.

13. Develop a sustainability plan to clearly articulate short- and long-term expectations for the

Project.

EVALUATION PURPOSE & EVALUATION QUESTIONS

USAID’s Automated Directives System (ADS) Chapter 201, Program Cycle Operational Policy,

outlines requirements for evaluations in Section 201.3.5, Monitoring Evaluation and Learning. Within

this section, ADS 201.3.5.13 requires evaluations in three instances. Requirement 3 speaks directly

to a WOPE: each Mission must conduct at least one “whole-of-project” performance evaluation

within their Country Development Cooperation Strategy (CDCS). This applies to CDCSs approved

in January 2015 or later. Although USAID/Vietnam’s CDCS began in 2014, the Mission is committed

to furthering its learning agenda and is in the forefront of USAID’s WOPE process globally.

According to information provided in the Additional Help section of ADS 201, the definition of a

WOPE states that “whole-of-project performance evaluations examine an entire project, including

all of its constituent Activities and progress towards the achievement of the Project purpose.” A

WOPE may be a way to explore how progress toward the Project purpose is greater than the sum

of the constituent parts (i.e., the Activities).

The value of a WOPE is that it examines the relationships among activities working together. A

WOPE might examine the extent to which all Activity-level interventions worked (or are working)

in a complementary and coordinated manner to achieve the stated Project purpose. In articulating

the purpose of the WOPE, it is important to emphasize that the aim of the evaluation is to answer

questions not answered by other means (e.g., monitoring, Activity-level evaluations).


The evaluation purpose and evaluation questions, provided below, are taken directly from the

Statement of Work (SOW) for the mid-term WOPE.

EVALUATION PURPOSE

The purpose of this mid-term WOPE is to determine to what extent the USAID-funded Vietnam

Disability Project 2015-2020, with its six Activities, is progressing toward achieving its purpose: A

sustainable model for improving persons with disabilities’ inclusion developed in USAID targeted provinces.

Specifically, the evaluation serves to:

● Examine the progress made towards the intended purpose and outcomes described in the

Project Appraisal Document (PAD) and six Activity Management Plans (AMPs), and to record

challenges and lessons learned in achieving the Activities’ planned targets;

● Determine whether the Activities, technical interventions and management strategies (as

specified in six program statements of works, and their monitoring and evaluation [M&E] plans)

are leading to the achievement of the changes expected from the whole program design

documents;

● Provide USAID/Vietnam, the Government of Vietnam (GVN), and other in-country

stakeholders with objective information on what has been achieved to date, what is working

and what is not. The information will inform decisions on appropriate modifications in the

remaining period of performance of the project and may also serve as a reference for the

design of future projects; and

● To learn and share gathered experience/lessons in conducting a WOPE.

EVALUATION QUESTIONS

A. PERFORMANCE

● How is the Project making progress toward the achievement of its purpose?

o To what extent has progress been made in achieving the Project’s purpose of developing a

sustainable model for improving persons with disabilities’ inclusion in society (in USAID

target provinces)?

o To what extent have each of the six component Activities contributed to achieving the

project’s purpose?

o What are the achievements and progress against targets in key program areas: fostering

disability rights, policy, coordination and advocacy; provision of physical, occupational, and

speech (POS) therapy services to assist persons with disabilities; and improving local

capacity to provide quality services?

B. VALIDATION OF THE PROJECT’S DESIGN

● What are the strengths and weaknesses (gaps) of the Project’s theory of change?

o Is there any strategy or specific intervention in the initial design that has had an unexpected

influence on the Project’s implementation and/or achieving the project’s purpose?

C. PROJECT MANAGEMENT AND CROSS-CUTTING ISSUES

● What are the benefits of coordinating the Project’s implementation activities with the Ministry

of Labor, Invalids, and Social Affairs (MOLISA) and local authorities to achieve its purpose?

What are identified challenges?

● To what extent has USAID gender policy been implemented in the Project? What are the

lessons learned of what works and what does not?

● What is the evidence of local ownership strengthening? Are the Project’s activities considered

as a part of local development agenda in assisting persons with disabilities?

PROJECT BACKGROUND

USAID began supporting programs for persons with disabilities in Vietnam in 1989. Areas of

attention have included humanitarian response for persons with disabilities, disability

policy/advocacy, inclusive education, vocational/employment opportunities for persons with

disabilities, and physical rehabilitation. The United States Congress focuses resources for health and

disability programs in Vietnam in areas sprayed with Agent Orange and otherwise contaminated with

dioxin.

Although the U.S. Government has adopted a policy of assisting persons with disabilities in Vietnam

regardless of cause, the Project’s geographic focus (Tay Ninh, Binh Phuoc, Quang Nam, Binh Dinh,

Dong Nai, and Hue) and planned expansion to Quang Tri (and possibly Bac Lieu and Kon Tum) are

provinces with a high disability prevalence and those heavily sprayed with Agent Orange.

USAID/Vietnam’s 2014 PAD provides the background and roadmap to address the Project purpose

at the time: Expanded opportunities for vulnerable populations. Three key outputs were: 1) Disability

rights policies, coordination and advocacy fostered; 2) POS therapy services delivered to assist

persons with disabilities to participate in society; and 3) Improved local capacity to provide services.

Consultative meetings with stakeholders in the ensuing years revealed that many of the terms used

in the 2014 document were difficult to measure or operationalize.

The Amended PAD (2017) refines the Project purpose to: A sustainable model for improving persons

with disabilities’ inclusion developed in USAID’s target provinces. It introduces five elements that

constitute the sustainable model, updates the previous results framework, expands the geographic

coverage of the Project and extends the PAD completion date to 2024. The elements of the

sustainable model, with clear definitions and targets, create a pathway to measure progress toward

the Project purpose.

The Project evaluated in this WOPE comprises six Activities summarized in the table below. The

Project fits within USAID’s CDCS for Vietnam (2014-2019) under Development Objective 2:

Capacity strengthened to protect and improve health and well-being, and Intermediate Result 2.3

(formerly) Expanded opportunities for vulnerable populations, (currently) A sustainable model for

improving persons with disabilities inclusion developed in USAID’s target provinces. All Activities are

managed under the Environment and Social Development Office.

* HCMC = Ho Chi Minh City

TABLE 1. SIX ACTIVITIES OF THE USAID/VIETNAM DISABILITY PROJECT

ACTIVITY NAME IMPLEMENTING PARTNER YEARS PROVINCES/ CITIES

Accessibility for Inclusion Disability Research and Capacity Development (DRD)

2015 – 2017

Tay Ninh and Binh Dinh

Moving Without Limits International Center (IC) 2015 – 2020

Thua Thien – Hue; Quang Nam; Binh Dinh

Disability Rights Enforcement, Coordination and Therapy

Vietnam Assistance for the Handicapped (VNAH)

2015 – 2020

Tay Ninh and Binh Phuoc

Disability Integration Services and Therapy Network for Capacity and Treatment

Sustainable Health Development Center (VietHealth)

2015 – 2019

Tay Ninh (and future plan for Binh Phuoc and Dong Nai)

Advancing Medical Care and Rehabilitation Education

Humanity and Inclusion (HI)- formerly Handicap International

2015 – 2020

Hanoi, HCMC,* Dong Nai, and Thua Thien - Hue

Protecting the Rights of Persons with Disabilities

Action to the Community Development Center (ACDC)

2015 – 2018

Thua Thien – Hue, and Binh Phuoc


The map below provides a visual display of each of the Activity locations in the country.

Figure 1: Locations of Disability Project Activities in Vietnam

KEY ASPECTS OF THE PROJECT

The 2014 PAD and the 2017 amended PAD provide the framework for the Project.

DEVELOPMENT HYPOTHESIS

The development hypothesis remains unchanged from the 2014 to the 2017 PAD. “If USAID focuses

on: fostering an enabling environment through the enforcement of disability rights, policies, and

coordination; delivering innovative and quality POS services; and improving local capacity to provide

POS services, these activities will achieve the goal of increasing persons with disabilities’

independence, inclusion and help fulfill their potential in society.”

The 2014 PAD provides additional detail: “Strong GVN commitment and support would in turn

ensure success of the Project. The POS service strengthening program is developed on the argument

that as entry points to services, POS can promote mobility, functional abilities, develop or recover

daily living routines, learning or working skills, or modification of their surrounding environment to

increase persons with disabilities’ participation and performance to be contributing members of

society. The Project will strengthen the capacity of government, civil society, universities, and the

private sector, and is integral to the U.S. Government’s efforts to support Vietnam’s continued

transformation into a responsible, more inclusive partner, and ensure the sustainability of

development interventions. In development of the hypothesis and supporting [programmatic]

activities, USAID assumes that: 1) GVN priorities as outlined in the National Action Plan for

Rehabilitation remain the same through 2018; 2) inter-sectoral coordination remains strong and

effective; and 3) GVN continues to support participation of persons with disabilities in all facets of

life. Success will ultimately depend on USAID’s: 1) collaborative partnership with the GVN, private

sector, universities, civil society, and other development partners; 2) programmatic experience and

expertise in the country; 3) collaborative relationships with disability partners; and 4) strategic and

programmatic choices for implementing and sustaining the interventions described in this PAD.”

RESULTS FRAMEWORK

The differences between the 2014 and the 2017 results frameworks are provided in the Findings

section on page 26 of this report. The results framework currently used by the Project is shown

below.

Table 2. Results Framework (amended in PAD 2017)

SUSTAINABLE MODEL

The sustainable model is a new aspect of the Project. It comprises five interrelated and

complementary elements that contribute to the Project purpose:

● Service Delivery System. The project will set up at least 10 rehabilitation service units at

provincial or district hospitals in each province.

● Human Capital. The project will seek to create well-trained human resources in rehabilitation

for target provinces, starting with training for a minimum of 20 rehabilitation doctors and 45

therapists/therapist assistants to provide services in each target province in the areas of

physical therapy, occupational therapy and speech and language therapy. Additional training on

disability and gender issues, including gender-based violence (GBV), will also be provided.

● Policy Support. The project will support the development and enforcement of national and

provincial policies to support persons with disabilities’ inclusion and/or improve the quality and

accessibility of rehabilitation services.

● Sustainable Finance. The project looks to improve the implementation of health insurance and

local government plans/budgets supporting disability issues to ensure the costs for

rehabilitation services are covered.

● Public Attitude and Self-Awareness: The project will work to improve awareness of the rights of

persons with disabilities and reduce stigma and discrimination towards persons with disabilities.

All activities will be gender sensitive and will try as much as possible to address GBV toward

persons with disabilities.

The elements of the sustainable model are shown in the figure below.


Figure 2. Elements of the Sustainable Model

These five elements are strengthened and

improved through activities in three project

components: 1) provision of direct assistance;

2) service system strengthening; and 3)

support for the enforcement of disability

policies and rights. These three components

are reflected in the current Project results

framework.

THEORY OF CHANGE

The Project’s TOC is: “If USAID builds a

sustainable model that a) delivers direct

assistance to persons with disabilities to

demonstrate interdisciplinary rehabilitation

services that improve persons with disabilities’

daily living, learning, and working functions; b)

strengthens systems to deliver quality services

to persons with disabilities; and c) improves

the enforcement of disability laws/policies,

then persons with disabilities’ inclusion in society will be increased.”

EVALUATION METHODS & LIMITATIONS

1. METHODOLOGY

This WOPE complied with the ADS 201 definition and requirements for examining an entire Project,

including all its Activities and progress towards the achievement of the Project purpose.

The evaluation SOW was designed to address three questions:

● How is the Project making progress toward the achievement of its purpose?

● What are the strengths and weaknesses (gaps) of the Project’s theory of change?

● What are project management and cross cutting Issues?

These broad questions were broken down into more specific sub-questions that formed the basis

for designing the methods and tools used for data collection and analysis

TABLE 3. EVALUATION MATRIX


TYPE OF ANSWER/ EVIDENCE NEEDED

PERFORMANCE EVALUATION DESIGN/ SPECIFIC METHODS FOR DATA COLLECTION

DATA SOURCE(S)

ANALYSIS SPECIFIC METHODS

1. How is the Project making progress

toward the achievement of its purpose?

Yes/No ● Beneficiaries

● IPs

● District officials

● GVN Ministries

● USAID

• Before and after

• Descriptive (content) with

cross-validation

• Desk review

• Key informant interviews

(KIIs)

• Surveys

Description

X Comparison

X Explanation

2. To what extent has progress been made

in achieving the Project’s purpose of

developing a sustainable model for

Yes/No ● Beneficiaries

● IPs


● GVN Ministries

● Descriptive

(content) with

cross-validation

● Desk review

● KIIs

● Surveys

● Observation

X Description

X Comparison

TABLE 3. EVALUATION MATRIX


TYPE OF ANSWER/ EVIDENCE NEEDED

PERFORMANCE EVALUATION DESIGN/ SPECIFIC METHODS FOR DATA COLLECTION

DATA SOURCE(S)

ANALYSIS SPECIFIC METHODS

improving persons with disabilities’ inclusion

in society (in USAID target provinces)?

Explanation ● USAID

3. To what extent have each of the six

component Activities contributed to

achieving the project’s purpose?

Yes/No ● IPs


● GVN Ministries

● USAID

● Contribution,

cross-validation

● Qualitative

Comparative

Analysis (QCA)

● Desk review

● KIIs

● Group

discussion

● Observation

Description

Comparison

X Explanation

4. What are the achievements and progress

against targets in key program areas:

fostering disability rights, policy,

coordination and advocacy; provision of

physical, occupational, and speech (POS)

therapy services to assist persons with

disabilities; and improving local capacity to

provide quality services?

Yes/No ● IPs


● GVN Ministries

● USAID

● Before and after;

● Descriptive

(content) analysis

● Desk review

● KIIs

● Group

discussion

X Description

X Comparison

Explanation

5. What are the strengths and weaknesses

(gaps) of the project’s theory of change?

Yes/No ● IPs


● GVN Ministries

● USAID

● Descriptive

(content) analysis

● QCA

● Desk review

● KIIs

● Group

discussion

X Description

Comparison

X Explanation

6. Is there any strategy or specific

intervention in the initial design that has had

an unexpected influence on the project’s

implementation and/or achieving the

project’s purpose?

X Yes/No ● IPs

● USAID

● Descriptive

(content) analysis

● Case studies

● KIIs

Description

Comparison

X Explanation

7. What are the benefits of coordinating the

project’s implementation activities with the

Ministry of Labor, Invalids, and Social Affairs

(MOLISA) and local authorities to achieve

its purpose? What are identified challenges?

Yes/No ● IPs


● GVN Ministries

● USAID

● Descriptive

(content) analysis

with cross-

validation

● KIIs

● Group

discussion

X Description

Comparison

X Explanation

8. To what extent has USAID gender policy

been implemented in the project? What are

the lessons learned of what works and what

does not?

X Yes/No ● IPs


● GVN Ministries

● USAID

● Descriptive

(content) analysis

with cross-

validation

● Desk review

● KIIs X Description

Comparison

X Explanation

9. What is the evidence of local ownership

strengthening? Are the project’s activities

considered as a part of local development

agenda in assisting persons with disabilities?

Yes/No ● IPs


● GVN Ministries

● USAID

● Descriptive

(content) analysis

with cross-

validation

● Case studies

● Desk review

● KIIs

● Group

discussion

X Description

Comparison

X Explanation

The evaluation was conducted using mixed methods of data collection that included a desk review,

quantitative surveys for beneficiaries and trained partners, and qualitative methods of KIIs, group

discussion and observation.

DESK REVIEW


The evaluation team used the desk review to:

● provide background on the disability and rehabilitation sector in Vietnam;

● develop initial answers to the evaluation questions by collecting, synthesizing, and triangulating

existing data/information from both Project and Activity levels; and

● identify additional data collection needed during the fieldwork phase.

Dropbox and Google Docs were used by the WOPE team to share documents for review. More

than 75 different documents were uploaded and reviewed. Self-assessment reports prepared by IPs

during the desk review process were also a key source of information. (A bibliography is provided as

Annex IV and the Desk Review Report is provided as Annex V).

QUANTITATIVE SURVEYS

Two surveys were designed to capture information about persons with disabilities who i) received

support through the project, and ii) received training provided by the Project. The surveys collected

data from adults and children with disability, caregivers and trained local service providers. Training

participants were selected randomly from lists provided by IPs. Specific methods and findings related

to these surveys are provided as Annexes V.

QUALITATIVE RESEARCH

The qualitative research formed the major part of the information gathering for this evaluation.

Semi-structured KIIs and focus group discussions were conducted as two main methods of data

collection. Direct observation was carried out at rehabilitation centers, in homes of persons with

disabilities, and at assistive product production and distribution sites.

The selection of stakeholders for the qualitative research was based on results of a stakeholder

analysis completed during the WOPE design. The qualitative research involved over 100 people from

over 50 organizations across five provinces and three major cities in Vietnam. Interviews were

conducted at provincial, district and community levels. Broad categories of those interviewed

include.

• Persons with disabilities

• Caregivers of persons with disabilities

• Service providers

• Rehabilitation facility managers

• GVN authorities

• Disabled peoples’ organizations (DPOs)

• University faculty

• IPs

• USAID/Vietnam

A complete list of interviewees, organizations and locations is provided as Annex IV.

A combination of data analysis methods was used to answer the nine specific questions in the

evaluation matrix. These questions served as the framework for designing data collection methods

and analysis.

● Contribution Analysis was applied to assess essential IP contributions in achieving the Project

purpose. The method inferred causality of IP interventions to the five elements of the

sustainable model. This was also used to assess cause-effect relationships of the Project’s TOC.

● Before and After (Comparative) Analysis was used to assess the status of Project progress in

terms of planned targets for performance indicators, the existence of rehabilitation systems in

target provinces and districts, the advances in human capital and policy enforcement.

● Descriptive Analysis was the most common analysis method applied in this evaluation. It was

used to summarize information related to the broad categories of performance, project design,

management and policy compliance (gender). The method was the most relevant way to utilize

both qualitative and quantitative data derived from structured surveys (two surveys of

beneficiaries and trained partners), semi-structured interviews, and project documents.

● Cross Validation (triangulation) and Case Studies were used to compliment other methods to

ensure data quality by checking data consistency from different sources. Case studies added

explanatory narrative factors to reinforce findings.

The evaluation team utilized findings gathered through the above methods in three ways.

● Daily compilation and review of field notes was a standard practice of the WOPE team.

Information from field interviews was typed and circulated among all team members. This

promoted a common understanding of findings, utilization of the interview guides, checks on

information quality, and enabled modification of data collection tools and methods in a timely

fashion.

● Internal consolidation meetings were held for one week at the end of data collection phase.

These team meetings were used to discuss findings related to the evaluation questions, to

share ideas on key messaging for stakeholders, and to delineate tasks for report writing.

● Validation workshops/briefings were conducted with IPs, USAID/Vietnam’s technical team, and

USAID/Vietnam management staff. These three meetings were conducted to share initial

findings, and to respond to questions or concerns that were raised by stakeholders. After each

meeting, the WOPE team conducted further analyses and held further discussions to ensure

data quality in reporting.

II. LIMITATIONS

The main limitations of this evaluation stem from recent adjustments in the Project design and lack

of familiarity with a WOPE by the evaluation team, IPs and USAID/Vietnam.

The sustainable model and the results framework are only partially aligned. The results

framework, with three main components and 12 sub-components, provides the foundation of the

Project. The sustainable model, with five elements, was added in late 2017. There is only partial

alignment between these features of the Project, and this created a barrier to consolidation and

representation of data related to Project performance.

Indicator data are not fully aligned with the Project design. One of the evaluation questions

requires gathering information on progress and achievements against targets. The Project targets and

indicators were created with the original PAD in 2014. Although there have been modifications in

the Project design, there has been limited progress in advancing the indicators in a similar fashion.

This has created a barrier to capturing information about Project progress.

There is a lack of detailed guidance, comparative studies and/or lessons learned for a

WOPE. Globally, only a handful of WOPEs have been conducted for USAID. Although the ADS

201 Additional Help section was useful, this was the first WOPE conducted by the evaluation team

members and the USAID/Vietnam Mission. There were no reports available on completed WOPEs,

best practices or lessons learned. Understanding the details of a WOPE was a process that evolved

over the course of the evaluation.

There is ambiguity between Activity-level performance evaluations and a WOPE.

WOPE team members tried to keep questions at higher levels without delving into programmatic or

technical details. This was a limitation in speaking with different offices within USAID as some staff

understood the WOPE concept while others were expecting that it would focus on results more in

line with Activity-level performance evaluations. The WOPE is also new for IP staff — many

expected the validation workshop to provide information specific to their individual performance. It

also took time for stakeholders to realize that WOPE questions are not focused on progress of a


specific Activity, but more on the interrelationship of multiple Project features toward achieving the

Project purpose.

More details on the lessons learned from the WOPE are provided on page 56 of this report.

FINDINGS AND CONCLUSIONS

FINDINGS

Findings are presented below for each evaluation question under the three headings from the

WOPE SOW.

PERFORMANCE

As noted throughout the document, the Project purpose is: A sustainable model for improving persons

with disabilities inclusion developed in USAID’s target provinces. There are five elements of the sustainable

model: service delivery system, human capital, policy support, sustainable finance, and public

attitudes and self-awareness. The evaluation team focused exclusively on these five elements, as

defined by USAID, when answering questions related to progress, achievement, and contributions

toward the Project purpose.

How is the Project making progress toward the achievement of its purpose? To what

extent has progress been made in achieving the Project’s purpose of developing a

sustainable model for improving persons with disabilities’ inclusion in society (in USAID

target provinces)?

Finding 1. There are signs of progress with all five elements of the sustainable model.

However, only two of the five elements have targets, and three elements have only

descriptive/illustrative content.

The evaluation team measured progress for elements with established targets (i.e., the service

delivery system and human capital). Defining the extent of progress was more difficult for elements

for which only illustrative content is available.

Element 1- Service Delivery System

Progress in achieving this target varies widely across provinces. The most progress was found in

provinces where Vietnam Assistance for the Handicapped (VNAH) is working (Tay Ninh and Binh

Phuoc), as this is one of the Activity’s key focus areas. While services for persons with disabilities

are provided in all six provinces, USAID/Vietnam’s definition of service delivery system strengthening

as “setting up rehabilitation units” lacks definition. For the purpose of the WOPE, this included the

establishment of new units and equipping existing units with rehabilitation equipment or assistive

products.

TABLE 4. PROGRESS RELATED TO ELEMENT 1 - SERVICE DELIVERY SYSTEM

TARGET: TO SET UP AT LEAST 10 REHAB SERVICE UNITS AT PROVINCIAL OR DISTRICT HOSPITALS IN EACH PROVINCE

PROVINCE EVIDENCE OF PROGRESS % PROGRESS

Binh Dinh A set of assistive products were provided to the provincial rehabilitation hospital

and three district health centers.

40%

Element 2 - Human Capital

Stakeholders highly value Project support to train physical, occupational and speech (POS) therapy

professionals and rehabilitation doctors as part of developing an interdisciplinary rehabilitation

service. Provision of artificial limbs and braces requires specialized personnel. The Project’s TOC

recognizes interdisciplinary rehabilitation services, but only POS professionals, nurses and

rehabilitation doctors are supported through the Project.

According to data from USAID’s online performance management tool, AIDtracker+ , the Project

has trained over 6,000 people to provide rehabilitation services. The training has ranged from two

days (on use of assistive products) to five years (students sent to India for master’s degree programs

in OT). Trainees comprise family members or caregivers of persons with disabilities, health workers

at commune levels, teachers, technicians, medical doctors and others. In addition, AIDtracker+

reveals that over 2,000 people have received training related to gender and gender-based violence

(GBV).

These figures indicate extensive progress in developing well-trained rehabilitation specialists in target

provinces. The information in the table below highlights some of the specific figures related to

training provided. The percentage of progress reflects gains against the specific targets of 65

professionals trained (20 rehabilitation doctors and 45 therapists). ble 5. Progress Related to

Element 2 - Human Capi

TABLE 5. PROGRESS RELATED TO ELEMENT 2 - HUMAN CAPITAL

TARGET: CREATE WELL-TRAINED HUMAN RESOURCES IN REHABILITATION FOR TARGET PROVINCES...FOR A MINIMUM OF 20 REHABILITATION DOCTORS AND 45 THERAPISTS...TO PROVIDE POS SERVICES IN EACH TARGET PROVINCE. TRAINING ON GENDER AND GBV WILL ALSO BE PROVIDED.


Binh Dinh

18 doctors and 20 therapists enrolled in rehabilitation orientation training 66%

116 people trained at commune level in assistive technology (AT) use ?

Training provided on GBV 100%

Binh Phuoc

16 doctors and 32 technicians completed rehabilitation orientation training 74%

121 commune staff trained in basic rehabilitation ?


Dong Nai 10 rehabilitation staff attended the Humanity and Inclusion (HI) training ?

Quang Nam 253 people trained at commune level in AT use ?

16 doctors and 26 technicians attended rehabilitation orientation training 65%

120 commune staff trained in basic rehabilitation ?

Binh Phuoc 11 district health centers and 2 provincial hospitals have new rehabilitation units 130%

Dong Nai 3 hospitals received rehabilitation equipment (2 provincial one district); the general

hospital established a speech and language therapy (SaLT) unit

40%

Quang

Nam

A set of assistive products were provided to two district health centers 20%

Tay Ninh 8 district health centers have new rehabilitation units and rehabilitation equipment

was provided to three pre-existing units

80%

Thua Thien

-Hue

3 provincial hospitals added occupational therapy/speech therapy (OT/ST) services;

one new rehab unit in Nam Dong District Health Center

40%


TABLE 5. PROGRESS RELATED TO ELEMENT 2 - HUMAN CAPITAL

TARGET: CREATE WELL-TRAINED HUMAN RESOURCES IN REHABILITATION FOR TARGET PROVINCES...FOR A MINIMUM OF 20 REHABILITATION DOCTORS AND 45 THERAPISTS...TO PROVIDE POS SERVICES IN EACH TARGET PROVINCE. TRAINING ON GENDER AND GBV WILL ALSO BE PROVIDED.


Tay Ninh 41 teachers trained for bachelor’s degree in special education ?

46 staff trained on topics related to childhood disability ?

20 people trained on the function, language and movement education program ?

294 community workers trained on screening, early prevention, detection and intervention

of disabilities

?


Thua Thien -

Hue

Over 40 staff from Hue University Hospital attended HI training ?

383 people trained at commune level in AT use ?


Note: Percent of progress is measured against specific values and categories of professionals provided in the description

of the element. A question mark is used when trainees were not clearly identified as rehabilitation doctors or therapists.

Elements 3 and 4 - Policy Support and Sustainable Finance

Although policy support and sustainable finance are two separate elements within the sustainable

model, they are closely related and the evaluation results for both are reported together in this

section. Sustainable finance looks to improve implementation of health insurance and government

plans/budgets supporting disability issues, to ensure that the costs of rehabilitation services are

covered. Government plans/budgets are part of policy development and Ministry of Health (MOH)

policies specify the rehabilitation techniques approved for national health insurance reimbursement.

Policy highlights achieved with Project support are summarized below.

● The National Action Plan for implementation of the UN Convention on the Rights of Persons

with Disabilities (UNCRPD), Prime Minister’s Decision 1100/QĐ-TTG, 21 June 2016, was

enacted.

● A new policy on health insurance coverage for rehabilitation, MOH Circular No. 18, was

developed and enacted. It greatly expanded the number and type of techniques approved for

reimbursement; the number went from 33 rehabilitation treatment techniques to 265

techniques.

● The provincial disability action plans and rehabilitation plans for Tay Ninh and Binh Phuoc

provinces were developed and enacted.

● Provincial Council on Disability Units were established in Thua Thien - Hue and Binh Phuoc.

● The State Report on the Implementation of the UNCRPD was completed.

● A national indicator system for disability monitoring and evaluation was enacted.

● The Legal Aid Law was revised in 2017 to offer free legal aid for persons with disabilities.

● A national Disability Information System (DIS) was developed and adopted by MOH through

Decision 3815/QD-BYT. The DIS has information on over 400,000 persons with disabilities in

14 provinces in Vietnam and data continues to be added.

Advances in rehabilitation financing, though not directly attributable to the Project, reflect the

GVN’s engagement in the rehabilitation sector.

● Construction of a new rehabilitation hospital in Tay Ninh is planned for October 2018. The

local government is reportedly investing 126 billion Vietnamese Dong (VND) in this effort. Tay

Ninh also allocates approximately 400 million VND annually for rehabilitation services.

● Construction of a new Rehabilitation Hospital in Thua Thien - Hue is underway.

● The Department of Health (DOH) in Binh Dinh has invested 5 billion VND in building and

equipping the rehabilitation department at the Psychiatric Hospital and 1 billion VND in

rehabilitation equipment for health facilities in the provinces.

MOH Circular No. 18 plays a large role in buoying sustainable finance for recognized rehabilitation

providers. Seeing opportunities to generate income from this new funding stream, provincial

hospitals and health centers are keen to be approved, and have staffs licensed, to provide

rehabilitation services.

Element 5 - Public Attitude and Self-Awareness

The Project supports training on disability rights and raising awareness of international conventions

and national laws related to the rights of persons with disabilities. This information is delivered

through information campaigns that include events for large audiences at universities as well as

activities aimed at smaller target groups. The Project also supports the creation and capacity building

of disabled peoples’ organizations (DPOs) to enhance self-advocacy. One of the key findings of the

evaluation is a change in health professionals’ attitudes toward persons with disabilities. This stems

from formal training on disability rights, direct experiences working with IPs, more knowledgeable

health professionals modeling appropriate behaviors, and opportunities to visit persons with

disabilities in their homes.

To what extent have each of the six component Activities contributed to achieving the

Project’s purpose?

The “six component Activities” is synonymous with “the six IPs.” All IPs designed their interventions

in alignment with the results framework. They have partially or wholly contributed to the elements

of the sustainable model and the overall Project purpose by default rather than by design.

Finding 2: All six IPs contribute to elements of the sustainable model and the GVN

appreciates the support for rehabilitation system strengthening.

Figure 3. IPs’ Contribution to the Five Elements of the Sustainable Model

“From my point of view as a student, I always thought a doctor is not someone who can cure all

diseases, but should always be empathetic and supportive to patients in all matters including the

smallest things. I found in rehabilitation work a thoughtfulness and understanding to help patients

overcome barriers to live an ordinary life – this fits with my choice for the future. I am really moved after

listening to your lectures and will try my best in my study on rehabilitation to not betray your trust.”

– Email from medical student to head of the Hue University Department of Rehabilitation

“Bu Dang Health Center in Binh Phuoc Province illustrates this trend. Before the Project started in 2015, no

rehabilitation services were available. The Project supported a new rehabilitation department, and a doctor and

two technicians received 10-months training in rehabilitation. Six months hence, the rehabilitation unit has

served 152 persons, most of them in-patients, and the rehabilitation doctor estimated that income generated by

the rehabilitation unit accounts for 25 percent of the total income of the eight departments in the Health

Center. Most of the rehabilitation units have experienced results like Bu Dang.”

– Representative of one of the Project IPs


Below is a summary of IP contributions to the sustainable model to date:

● USAID/Vietnam describes service delivery system strengthening as setting up rehabilitation

units. Based on this definition, VNAH has greatly contributed to this element. HI has also

provided equipment for many rehabilitation units while IC reportedly equipped a unit in Binh

Dinh.

● All IPs have contributed to building human capital. GBV training is listed as part of the

definition of this element, and all IPs have provided GBV training. If a more restrictive definition

of rehabilitation-related training is applied, DRD and ACDC would not fulfill the criterion

associated with this element.

● All IPs except IC have contributed to policy support. VNAH’s work has focused on

strengthening national and provincial disability policies, HI has focused on strengthening

rehabilitation standards through the development of rehabilitation guidelines, and VietHealth

has contributed to increased access to early childhood disability detection and intervention

(ECDDI). In addition, ACDC and DRD both have worked on enforcing accessible construction

policy (Code 10).

● Two IPs (VNAH and ACDC) have contributed to sustainable finance. VNAH has worked

extensively on health insurance reimbursement for rehabilitation services, while ACDC has

provided counseling and advice on how to access health insurance.

● All IPs have contributed to public attitude and self-awareness. ACDC and DRD have provided

specific training on the UNCRPD and disability rights. IC and VietHealth have provided

information about disability rights in the context of rehabilitation. VNAH and HI have

contributed to awareness raising through their encouraging interactions, behavior modeling for

professionals and ongoing engagement with persons with disabilities.

Sustainable Model for Inclusion

1. Service Delivery System

2. Human Capital

3. Policy Support

4. Sustainable Finance

5. Public Attitude & Self-Awareness

VNAH

1. System

2. Human

3. Policy

4. Finance

5. Awareness

HI

1. System

2. Human

3. Policy

5. Awareness

IC

1. System

2. Human

5. Awareness

Viet Health

2. Human

3. Policy

5. Awareness ACDC

2. Human

3. Policy

4. Finance

5. Awareness

DRD

2. Human

3. Policy

5. Awareness

The GVN’s appreciation for rehabilitation services and training was a common theme during the

WOPE.

Finding 3: All IPs contribute toward achieving the Project purpose, but not in a

systematic or harmonized way with each other, or with other related initiatives.

The contributions of IPs vary by location, target group, and method of intervention. This is in line

with their original Acctivity descriptions, discussions with USAID/Vietnam, and consultations with

GVN partners and stakeholders in the target provinces. The result is that each IP has advanced its

own agenda in meeting established targets; and there is no requirement to build upon past

USAID/Vietnam investments, collaborate with related initiatives outside the Project, or coordinate

with other IPs working toward the same Project purpose.

One of the key aspects of the WOPE is to examine the Project through a lens that focuses on

relationships among IPs working together. As outlined by ADS 201, the WOPE focus is on

complementary and coordinated efforts to achieve the stated Project purpose.

USAID/Vietnam has invested in multiple strategies to facilitate coordination between the IPs. These

include annual coordination meetings (involving IPs and GVN stakeholders), quarterly IP meetings,

monthly conference calls for staff working in specific technical areas, and use of Google calendar to

avoid overlapping schedules and over-taxing GVN partners with competing time demands. The

coordination meeting is valued by IPs as an opportunity to exchange information and Activity

updates. However, these meetings have not been specifically issue-based or framed around

coordinating efforts to meet the Project purpose.

Coordination is needed to reduce duplication or potential inconsistencies in Activity interventions.

Many IPs share common intervention areas in their respective sectors, but do not share common

standards, tools or resources. For example:

● HI, VNAH and IC are providing support to establish or equip rehabilitation units, but there is

no minimum standard on what this entails, standard equipment list, or shared suppliers.

● All IPs are incorporating gender in their training courses, and three IPs have specific training on

GBV, but there is no shared curriculum.

● There are multiple organizations involved in OT training (Curtin University, VNAH, Manipal

University, Medical Committee Netherlands Vietnam [MCNV], and HI), and there is a monthly

call related to OT, but it does not involve all stakeholders – only those directly funded by

USAID/Vietnam.

“We have seen an increase in quantity and quality of services. Before the project rehab was offered only at

provincial level; now we have rehab units at district level. There are staff and equipment; for the quality, the

services seem more professional.”

– Staff of Tay Ninh DOH

“HI invited a foreign trainer to provide information at higher levels… provides more in-depth information.

Previously all hospitals were called ‘nursing and rehabilitation’ hospitals, but in 2015 they changed the name

to rehabilitation hospitals which is positive as it allows them to focus on rehab.”

– Staff at Hue Rehabilitation Hospital

“We have a big appreciation for the way IC works. [Their support] is tailored to individual needs and [IC]

provides follow-up.”

- Staff of Hue DOH


● VNAH and IC both provide assistive products but there are no standard protocols for their

provision, for equipment suppliers, or for beneficiary training tools.

● ACDC and DRD both conduct accessibility audits, but their tools are neither standardized nor

shared.

Continuity of care is greatly reduced by a lack of coordination and/or harmonized services among

the Activities. Rehabilitation is a process requiring multiple and varied interventions to retain or

restore an individual’s function(s). The WOPE team found many of the IP interventions valuable but

disjointed. One example is the Project support for establishing or equipping rehabilitation units. The

Project also supports AT provision at the community level. However, there was no evidence of

purposeful links between these two Activities. Individuals regain function through services provided

in the rehabilitation units. Upon discharge, there are no mobility products available to help them at

home, and as a result many of the functional gains will be lost. At the same time, assistive products

are provided directly to beneficiaries at the community level. This is done outside of the healthcare

system and the interventions are not reinforced by a functioning rehabilitation unit.

Each of the target provinces host IPs and the arrangement of their Activities in each province

influences the number and type of elements of the sustainable model that are supported. The

number of IPs working in a province does not guarantee that all the elements will be addressed or

addressed equally. Evidence of some programmatic synergy is seen through referral services

between VietHealth and VNAH in Tay Ninh. VNAH has provided the bulk of support for

strengthening the rehabilitation service delivery system. This has provided a pillar of support for

other elements of the sustainable model. Although Table 6 presents a robust picture of IP presence

and support for the sustainable model, contributions vary.

TABLE 6. ELEMENTS OF THE SUSTAINABLE MODEL SUPPORTED BY PROVINCE ELEMENTS

BINH DINH

BINH PHUOC

DONG NAI

QUANG NAM

TAY NINH

THUA THIEN -HUE

OTHER

Service Delivery

System

IC VNAH HI IC VNAH HI, IC

Human Capital

IC,

DRD

ACDC,

VNAH,

VietHealth

IC, HI IC DRD,

VNAH,

VietHealth

ACDC,

HI, IC

HI

Policy Support

DRD ACDC,

VNAH,

VietHealth

DRD,

VNAH,

VietHealth

ACDC VNAH

HI

Sustainable Finance ACDC,

VNAH

VNAH ACDC VNAH

Public Attitude/

Self-Awareness

DRD

IC

ACDC,

VNAH,

VietHealth

HI IC DRD,

VNAH,

VietHealth

ACDC,

HI, IC

In addition to capitalizing on current Project Activities, USAID/Vietnam has an opportunity to build

upon investments in the disability sector that have spanned 30 years. These include:

In Thua Thien - Hue, IC provides wheelchairs and assistive products in Phong Dien, Quang Dien and

A Luoi districts. HI supports rehabilitation units in Nam Dong and Hue City hospitals. IC provides

wheelchairs and assistive products, but not in the areas where HI provides support for rehabilitation

units. This appears to be a missed opportunity for the continuity of care.

• VNAH’s extensive investment in accessible construction (1998-2002). This led to the

development of Code 10 (Vietnam’s accessible construction standards). There is a wealth of

information and experience based on this work, but it has not been shared with ACDC or DRD.

• USAID/Vietnam’s Social Work Education Enhancement Program (2013-2016). There were

program indicators that linked to the related case-management system, but it appears that these

were abandoned, and there is no evidence of Project links to the social work sector.

• USAID/Vietnam’s support for prosthetics and orthotics (P&O) training in Vietnam for nearly two

decades. The Project is directly engaged in AT provision, which would benefit from P&O

expertise, but USAID/Vietnam is reluctant to build on previous investments.

• USAID/Vietnam’s investment in inclusive education (2008-2013). The Project supports ECDDI

and kindergarten teacher training, but there is no link to the primary schools supported by the

previous initiative.

Together with opportunities related to current and previous USAID/Vietnam disability-related

investments, achieving the Project purpose could also be supported by linkages with initiatives

outside the Project. Several examples are listed below.

• The MoveAbility foundation’s work to provide individuals in Vietnam with assistive products

(primarily artificial limbs, braces and wheelchairs). The organization influenced MOH Circular

No: 18/2016/TT-BYT (regulating rehabilitation techniques, assistive devices and rehabilitation day

care covered by health insurance) issued on June 30, 2016. The circular provides a list of 20

medical items to be covered by Vietnam’s Health Insurance Fund. These include all types of

orthoses, including all types of cerebral palsy chairs. VNAH mentioned they are working with

MoveAbility, but neither IC nor VNAH staff spoke about the reference to or implications of

health financing for cerebral palsy chairs.

• Wheelchair training guidelines/packages by the World Health Organization (WHO). Although

both IC and VNAH provide wheelchairs, there was no evidence that they adhere to the WHO

guidelines or use the training packages. Conversely, the wheelchair facility in HCMC (Kien

Tuong) provides basic and intermediate training for wheelchair providers in Vietnam using

WHO’s training packages.

• WHO’s Global Cooperation on Assistive Technology. This initiative promotes AT provision, has

developed a list of 50 priority assistive products, and is responsible for the recent adoption of

the UN World Health Assembly’s resolution on AT. There is little mention of this work and

how it may support the Project purpose.

• WHO’s Rehab 2030: A Call for Action. This effort provides extensive resources and

opportunities to strengthen rehabilitation systems and services. Representatives of WHO have

met with MOH, USAID and other Project stakeholders. But there is no information about how

this initiative could be integrated into the Project. IPs have identified lack of time and no

requirement by the donor as barriers to coordination.


• They also noted that they work in different geographical areas, in different sectors (with DOHs;

departments of labor, invalids and social affairs [DOLISAs]; and departments of education and

training [DOETs]), and at different levels (national, provincial, district and commune), and this

limits coordination.

What are the achievements and progress against targets in key program areas:

fostering disability rights, policy, coordination and advocacy; provision of physical,

occupational, and speech (POS) therapy services to assist persons with disabilities; and

improving local capacity to provide quality services?

Finding 4. Available indicator data shows over 100% achievement for the three

program areas, but the indicators are based on the 2014 PAD.

Each of the IPs has their own implementation targets, and they also contribute to USAID’s indicators

for the Project and to higher-level Agency requirements. There are 23 Project indicators. Sixteen

(16) are specific to the disability monitoring plan (DMP), five (5) are higher-level indicators that

contribute to the Mission’s performance monitoring plan (PMP), and two (2) are gender-related

indicators. The WOPE team utilized information from AIDtracker+ related to the 2014 indicators to

answer this evaluation question. Summary information is provided in Table 7 and details for each

indicator are provided in Annex IV.

TABLE 7. ACHIEVEMENTS AND PROGRESS AGAINST TARGETS

KEY PROGRAM AREA DMP INDICATORS TARGET

2016-17

ACHIEVED

2016-17 % ACHIEVED

Provision of POS services to

assist persons with disabilities

Number of persons with disabilities who

have received direct assistance

10,832 14,049 108%

Number of devices produced locally 20 36 180%

Improve local capacity to

provide quality services

Number of organizations or service

delivery systems strengthened

245 294 120%

Number of service providers trained 5,302 6,892 131%

Number of people trained in mitigating

GBV

2,040 2,282 112%

Fostering disability rights,

policy, coordination and

advocacy

Number of GVN laws, policies, or

procedures drafted, revised/issued

33 51 154%

Number of buildings/structures with

newly included accessibility features

20 10 50%

“USAID convenes quarterly partner meetings and now encourages IPs working in the same province to

coordinate activities. However, IPs' priority is to deliver on their expected outputs. They have little time to

collaborate with other IPs.”

– Representative of an IP

“So many people are doing so many things - we’d love to get people in the same room. It’s really important to

bring folks around the table. It would be good to consolidate OT initiatives and convene a national meeting to

decide on a national strategy for OT. Right now, things are very piecemeal. There are very good ideas, but

everyone is in different directions. It would be good to get the main players in the room and develop a

national strategy.”

– Representative of an organization supporting OT in Vietnam

“The roles of DOH, DOLISA and DOET for early identification are not clear. Each is engaged at some level

and how VietHealth and VNAH collaborate in this effort is not clear.”

– Member of Tay Ninh DOH

Number of DPOs participating in

monitoring and evaluations

14 10 71%

Number of GVN units reporting to and

using data from the DIS

311 522 167%

Note: Numbers in the “Target”, “Achieved” and “Percent” columns represent cumulative figures for 2016 and 2017.

The data reflecting achievements by the Project are noteworthy. However, the evaluation team

found no data entered for nine of the 16 DMP indicators. It is not clear if these indicators represent

interventions later in the Project period or if USAID has informally removed them.

The Project indicators and their targets were established in alignment with the three components of

the results framework, not the elements of the sustainable model.

Some indicators may be partially applied to the elements of the sustainable model, but do not

accurately measure achievement of the relevant element. For example, indicator data shows that

6,892 service providers were trained through the Project. This indicator represents many types of

training courses. Therefore, while this data reflects capacity-building efforts, it does not specify how

many rehabilitation doctors and therapists were trained to provide services to persons with

disabilities, as indicated in the human capital element of the sustainable model.

Other indicators capture impressive results on the number of people who received direct assistance

through the Project. This includes rehabilitation treatments, assistive products and other types of

support. The data from 2016-2017 reveals that over 14,000 people received support. The elements

of the sustainable model do not accommodate this data.

VALIDATION OF THE PROJECT’S DESIGN

What are the strengths and weaknesses (gaps) of the Project’s theory of change?

Finding 5. There is a lack of detail in the elements of the sustainable model, the

Project’s TOC, and sub-components of the results framework.

USAID/Vietnam introduced the sustainable model as a key part of the Project design in late 2017.

The five elements of the sustainable model are based on four of the six building blocks for health

systems strengthening, used by WHO1 and USAID’s Bureau for Global Health.2 USAID

supplemented these four building blocks with an element related to public attitude and self-

awareness, and provided one to two statements for each of the elements but with no additional

narrative. Two elements contain Project targets while the others provide a general description of

intent. Two of the elements reference gender and GBV.

The 2014 PAD presents a development hypothesis: If USAID focuses on: fostering an enabling

environment through the enforcement of disability rights, policies, and coordination; delivering

innovative and quality POS services; and improving local capacity to provide POS services, these

activities will achieve the goal of increasing persons with disabilities’ independence, inclusion and help

fulfill their potential in society. Additional information is provided on page 7 of this report to support

this hypothesis.

1 http://www.wpro.who.int/health_services/health_systems_framework/en/

2 https://www.usaid.gov/sites/default/files/documents/1864/HSS-Vision.pdf


The 2017 amended PAD presents a TOC with similar content and adds the sustainable model: If

USAID builds a sustainable model that a) delivers direct assistance to persons with disabilities to

demonstrate interdisciplinary rehabilitation services that improve persons with disabilities’ daily

living, learning, and working functions; b) strengthens systems to deliver quality services to persons

with disabilities; and c) improves the enforcement of disability laws/policies, then persons with

disabilities’ inclusion in society

will be increased. There is no additional information outlining risks, assumptions or other content

that develops in detail the TOC.

The IPs have framed their interventions around the results framework. The 2014 PAD provides

narrative and corresponding indicators for each of the outputs and inputs. The 2017 PAD updated

the content of the results framework, but there is no evidence of corresponding changes in Project

indicators, nor a supplementary narrative to describe new inputs. The table below provides a

comparison of the content with the three components of the results framework highlighted in blue.

The gray shaded cells indicate no clear corresponding link with the specific input.

TABLE 8. COMPARISON OF THE 2014 AND THE 2017 RESULTS FRAMEWORKS

2014 PAD AND RESULTS FRAMEWORK 2017 PAD AND RESULTS FRAMEWORK

Purpose: Expanded opportunities for vulnerable populations. Sub-purpose: Persons with disabilities have expanded opportunities through higher quality and sustainable services, awareness and advocacy strengthening.

Purpose: A sustainable model for improving persons with disabilities’ inclusion developed in USAID’s target provinces.

Output 1 - Disability rights, policies, coordination and advocacy fostered

Disability policies and rights advocacy advanced.

Input 1.1. Enforce national policy, disability information system and interagency cooperation.

Input 1.2. Support the development and implementation of provincial action plans on disabilities

Input 1.3. Strengthen advocacy for disability rights Disability rights advocacy advanced

Positive public attitude toward disability issues

Disability and rehabilitation policies enforced

Output 2 - POS services delivered to assist persons with disabilities to participate in society

Provision of direct assistance to demonstrate inter-disciplinary rehabilitation services for persons with disabilities

Input 2.1. Provides POS services to persons with disabilities Provide POS therapies to persons with disabilities

Input 2.2. Provide services to families and care providers Services provided to family and caregivers (training, finance, other)

Input 2.3. Provide quality and affordable assistive devices (such as hearing aids, prosthetics and wheelchairs)

Humanitarian services provided to persons with disabilities

Input 2.4. Enhance service information Service information increased

Output 3 - Improved local capacity to provide POS services Systems strengthened to sustainably provide services to persons with disabilities

Input 3.1. Develop POS training program for practitioners and technicians

Rehabilitation practitioners trained

Input 3.2. Provide equipment and improve technology for POS facilities at district and lower levels

Equipment and technologies improved in the rehabilitation facilities

Input 3.3. Advocate policies to increase availability and quality of POS services

Input 3.4. Establish effective referral system (case management network) to connect with other health, education and employment services.

Effective mentoring, referral and supervision

Improved health financing for rehabilitation services

Finding 6. Connections between the sustainable model and the results framework are

not fully developed.

The five elements of the sustainable model and the current content of the results framework each

contain content that is relevant to the Project purpose. The results framework was created with the

2014 PAD and was refined in the amended 2017 PAD. The current results framework has three

components and eleven sub-components. The elements of the sustainable model are modified from

USAID’s and WHO’s health system strengthening building blocks, with an additional element on

public attitude and self-awareness. When interviewing representatives of Project IPs about their

contribution to the sustainable model, four of six referenced the results framework – not the

sustainable model.

One of the key challenges related to the elements and sub-components of the results framework is a

lack of detailed descriptions. Without a clear understanding of USAID’s intent, it is difficult to

integrate the subcomponents of the results framework with the elements of the sustainable model.

Figure 4 identifies the links between the each of the elements and sub-components, based on the

information available in Project documents. It highlights the links between seven (7) of the sub-

components with the elements of the sustainable model. It also shows that four sub-components of

the component “Provision of direct assistance to demonstrate inter-disciplinary rehabilitation services for

persons with disabilities” are not captured within the sustainable model. Although one could surmise

that these would be included in the service delivery system element, the definition provided by

USAID — “the project will set up at least 10 rehabilitation service units at provincial or district hospitals in

each province” — does not support this conclusion.

Figure 4. Links between the 2017 Results Framework and the Sustainable Model


Finding 7. USAID has not defined the terms “sustainable” and “inclusion” as they

relate to the Project.

The stated purpose of the Project is a sustainable model for improving persons with disabilities’

inclusion in USAID’s target provinces. However, definitions of sustainability and inclusion were not

found in Project documentation.

Although there is no evidence of sustainability indicators, the Project has two inclusion indicators:

● Number of persons with disabilities with increased social participation, and educational and

working opportunities (disaggregated by sex, level of disability, and province).

● Percentage of beneficiaries reported positive changes in their daily lives since receiving

services.

There was no data available for either of these indicators within IP progress reports or AIDtracker+.

One of the aims of the beneficiary survey was to collect information on how services impacted

inclusion for persons with disabilities (see full survey report provided in Annex V). There were three

questions related to inclusion in the survey: for children who received AT, for adults who received

AT, and for adults who received therapy only. The table below summarizes key findings. Key

Findings from the Ben

TABLE 9. KEY FINDINGS FROM THE BENEFICIARY SURVEY BENEFICIARY SURVEY QUESTIONS RELATED TO SOCIAL INCLUSION

ADULTS + THERAPY

ADULTS +AT CHILDREN

Improved inclusion after receiving assistance 22.3% 59.7% 49.6%

Go out and interact with community after receiving assistance 20.7% 48.8% 45.4%

Increased participation in community activities after receiving assistance 17.4% 36.3% 41%

All interventions (therapies and assistive products) were provided at the community level. The adult

beneficiaries the WOPE team interviewed had long-term impairments. These are individuals for

whom rehabilitation exercise may have minimal impact as they have been living with their disability

for decades. For some, a mobility product such as a wheelchair will have a life changing impact. For

others, improved walking aids may lead to better posture or greater safety, but may make little or

no difference on function.

In discussions with USAID/Vietnam, sustainability was described as the ability for services and

systems to continue after the Project ends and the ability of the GVN to scale up services beyond

the current geographic scope. A sustainability analysis was not seen in the 2014 PAD.

A USAID-funded sustainability assessment in Indonesia3 identified two broad categories of

sustainability “signs” in field-based development projects: process signs (e.g., internalization of

practices by government partners) and results signs (e.g., projects’ lasting impact on policies,

institutions and increased knowledge and skills of counterparts). In addition, five underlying factors

contributing to signs of sustainability emerged:

● Commitment of local governments to pursuing project outcomes;

● Alignment with priorities of national or local policy and regulatory frameworks;

● Stakeholder participation in project planning and implementation;

● Counterpart funding support, including cost-sharing and other forms of in-kind support; and

● Project management by IPs, including field staff relationships with counterparts.

The WOPE team saw evidence of each of these five underlying factors during this evaluation. Some

examples of Project sustainability are provided in the policy highlights section (page 15).

Each of the IPs are required to report on sustainability mechanisms and stakeholder participation

and involvement. Many IPs described capacity building (training) as a sustainability mechanism.

3 Assessment Report: Sustainability of Donor Projects on Decentralization and Governance Reform in Indonesia. February

2018.

“The Project is helping people to be more included in community and family. For example, a man who

previously stayed at home and drank alcohol had his life changed when he received a wheelchair. He began a

business selling lottery tickets and now has his own income. Another person made handicrafts at home and

now with a wheelchair she can deliver these to buyers. A third example is a child [who] used to be carried by

his mother to use the toilet; now with a toilet chair he can be more independent for hygiene.”

– Statement from Quang Dien District Health Center staff member about beneficiary impact from

receiving assistive products

“The Project supports human resource development - a key priority for Government. The fact that the Project

focuses on long-term education and training to produce a new specialty (OT) and improve teaching capacity

in universities, as well as strengthen management in health settings, can be considered as the important

element helping to ensure the long run sustainability.”

- IP staff


Knowledge and skills transfer is a key component of the Project. The WOPE not only looked at the

number of people trained, but also conducted a survey of those who received training through the

Project. One of the key aspects of the training survey was to understand the application of

knowledge and skills in the trainees’ daily work. The survey found that an average of 75% of

respondents confirmed that they applied learning from the Project trainings in their work.

Application rates vary among the different respondent groups, as illustrated in Figure 5 below.

Figure 5. Application of Learning from Project Trainings by Work Categories (%)

● The DPO group had the highest rate of application of learning from trainings; however, the

proportion of those surveyed who were DPO staff was relatively small (only 6%).

● A notably high rate of application of learning (89%) was observed among health staff who work

at district hospitals (a small number of this group works in provincial clinics and universities).

● The lowest application rate (55%) was found among staff who work in government agencies.

Finding 8. The Project focuses primarily on rehabilitation. The links between Project

work on rehabilitation and work on disability policies and rights are tenuous. Moreover,

the roles of DPOs in the Project are not well defined or well resourced.

The results framework lays out several Activity-level interventions. Most Project interventions (73%)

are focused on provision of rehabilitation services. A few interventions (18%) are focused on

improving public attitudes towards persons with disabilities and advancing DPO rights and advocacy

efforts. One intervention (9%) is focused on enforcement of disability and rehabilitation policies.

Figure 6. Project Intervention Areas Leading to Inclusion

96% 89%73%

55%75%

0%

50%

100%

150%

DPO staff Health staff atdistrict & higher

levels

Health staff atcommune and

village

Staff at governmentagencies

All catergories

As illustrated above in Figure 6, USAID’s Project interventions emphasize provision of rehabilitation

services. Approximately 75% of the total Project budget supports rehabilitation. These activities are

well-defined in the results framework and include outcomes such as provision of direct POS

services, a trained rehabilitation workforce, equipped rehabilitation units and improved health

financing for rehabilitation. The other (smaller) part of the Project (budget and Activities) are less

defined in the results framework and have no direct connection to rehabilitation.

Interventions in the areas of enforcement of disability policy, positive public attitudes towards

persons with disabilities, and advancement of DPO rights and advocacy are less defined in the TOC

and the resulting activities have no direct links to the bulk of the Project interventions. These

activities, which represent approximately 11% of the total Project budget, are focused on improving

physical accessibility of the built environment, supporting DPOs, and providing legal aid and

counseling to persons with disabilities on a wide variety of topics, such as land tenure and marital

divorce. These activities were determined by IPs in response to the USAID Project solicitation.

Some IPs working in these areas have adjusted their activities to reinforce the rehabilitation efforts

of the Project. For example, within the scope of promoting accessibility, one IP has focused efforts

on improving access to healthcare centers and supporting peer-to-peer counseling on the use of

health insurance cards. This IP also linked members of local DPOs in need of rehabilitation and

assistive products with other Project IPs focused on providing these services in nearby locations.

Is there any strategy or specific intervention in the initial design that has had an

unexpected influence on the Project’s implementation and/or achieving the Project’s

purpose?

Finding 9. Project IPs have cooperative agreements that preclude USAID/Vietnam

from directing how interventions are implemented.

The original RFA specified target provinces and the types of activities intended to take place under

the Project. Successful applicants who proposed working in the same province were not instructed

to locate their activities in similar districts or communes. Successful applicants who proposed

implementation of initiatives in complementary Project areas, such as physical rehabilitation and

“DPO members are consumers of rehabilitation services, and so DPOs can best advise. We know the

geography and how to work with our own members; we know who needs and uses rehabilitation and we can

provide peer support.”

– Project IP staff


assistive products, were not required to work in proximity, which would allow leveraging of

activities for greater impact. Applicants were able to self-select the district and/or communes for

implementation based on GVN direction and existing IP relationships in the community. Efforts to

coordinate activities with other IPs have taken place after Project approval, and have been limited in

nature. IPs recognize the value of coordinating activities with one another but note that coordination

was not planned at the outset of the Project.

The purpose of the WOPE is to identify how IPs implementing different Activities are purposefully

working together to achieve the Project purpose. The current funding instrument limits

USAID/Vietnam’s capacity to orchestrate these linkages. In addition, the funding instrument also

constrains USAID/Vietnam from controlling how Activities are implemented. The challenges of

coordination and Project standards are addressed under Finding 3. However, one of the advantages

of IPs determining their own path is summarized in the example below.

Finding 10. Increasing the health insurance coverage for rehabilitation techniques has

resulted in a high demand for staff licensed to provide rehabilitation. The methods and

timelines to become licensed are highly variable.

The Project has supported MOH on Circular 18 which has substantially expanded (by a factor of 7.5)

insurance coverage for rehabilitation services. With support from the Project, technical procedures

for 118 additional services have been developed. This represents an 80% increase in overall

rehabilitation procedures eligible for reimbursement (from 145 procedures to 263), which has

created optimism among hospital leadership that rehabilitation services can generate income.

Training and experience needed in Vietnam for rehabilitation licensure (a requirement for health

insurance reimbursement) varies. The minimum course duration for physicians is six months, but

rehabilitation orientation courses, offered in accredited universities throughout Vietnam, vary in

length (i.e., Bach Mai: 6 months, HCMC: 10 months, Hue: 12 months). Course content is not

standardized. In HCMC, with support from the Project, the orientation course now includes six

weeks of training on OT, while other institutions have not yet adopted this. After course

completion, MOH requires18 months of practice to obtain a license, but enforcement differs

between provinces. There is no standard protocol providing guidance on how practice time should

be counted. The evaluation team did not find any written plan to provide technical coaching,

mentoring and supervision to newly-graduated trainees upon return to their hospital.

There are also six-month orientation courses for nurses to become physical therapy technicians.

The curricula for doctors and nurses are different, but the lack of standards is the same. In addition,

short courses (of three months) offered to village health workers has created the perception that

these individuals have capacities equivalent to those who have studied for three to four years.

VNAH engaged Curtin University in Australia to develop the OT content of the rehabilitation

orientation course. The Curtin team realized that simply giving a course in OT was not enough. The

work in Vietnam generated a lot of interest at the university and the team applied for an Australian

Government grant. Curtin received a three-year grant (2018-2020) that supports two teams of

students (6 students for 6 weeks) to travel to Tay Ninh (April) and Binh Phuoc (October) each year

to provide mentoring and follow-up to the two hospitals where the Curtin team are already engaged

in OT (through VNAH support). This work is outside the Project but fully complements the initial

investment and demonstrates a creative relationship that furthers the Project purpose.

An unintended consequence of Project support for increasing the rehabilitation techniques covered

by health insurance and its support for a variety of rehabilitation training courses is that the GVN

has an interest in rapidly scaling up the number of licensed providers without strategically

considering the skill levels needed to deliver services.


What are the benefits of coordinating the Project’s implementation activities with the

Ministry of Labor, Invalids, and Social Affairs (MOLISA) and local authorities to achieve

its purpose? What are identified challenges?

Finding 11. Disability and rehabilitation are part of MOLISA’s mandate. The benefit of

coordinating with MOLISA and its local-level authorities depends on Project

interventions.

From a policy standpoint, MOLISA is the lead agency responsible for formulation and

implementation of disability law. It is mandated by law4 “to assume the prime responsibility for, and

coordinate with other ministries, ministerial-level agencies and provincial-People's Committees in,

materializing legal documents on persons with disabilities; and programs, schemes and plans on

affairs related to their activities.” MOLISA chairs the National Council on Disability (NCD) and is

responsible for reporting on progress towards implementation of the UNCRPD. These

responsibilities for disability policy extend to provincial and district-level departments working under

MOLISA. For this reason, and because the Project is working on general disability policy,

coordination with MOLISA is necessary. An output of the Project resulting from coordination with

MOLISA’s Legal Department is the development of the 2017 Vietnam State Report on the

Implementation of the UNCRPD. This is a noteworthy accomplishment of the Project and one that

could not have happened without close coordination with MOLISA.

The left portion of Figure 7 below illustrates points of oversight authority “crossover” among

relevant ministries in areas that the Project focuses on. The size of each bubble illustrates the

relative investment of Project Activities that fall under the primary mandate of those ministries. The

right portion of Figure 7 distinguishes ministerial responsibilities over each of the Project

interventions, as mandated by the 2011 National Law on Disability.

Figure 7. Responsibilities of Relevant Ministries in Project Intervention Areas

4 Article 50. Responsibilities of ministries, ministerial-level agencies and People's Committees at all levels, Vietnam Law on

Persons with Disabilities, 2011.

“They take a short course and work as a PT... this is not good for the profession. They steal our students’

jobs.”

– Staff at HCMC University

“Rehabilitation orientation training in Vietnam is a challenge. Although MOH Department of Science,

Education and Training is the responsible body for oversight, the curriculum is different from location to

location. There should be a common, uniform standard in the future.”

– Staff at Bach Mai Hospital


Finding 12. Within MOLISA, the NCD wants greater ownership and engagement in the

Project and is taking steps in this direction.

MOLISA has a broad mandate to protect the rights of persons with disabilities, but its primary focus

is on social protection (disability subsidy payments). The NCD, chaired by MOLISA, has expressed a

desire for greater ownership and engagement in the Project. To date, the Project has effectively

coordinated with the Legal Department of MOLISA to carry out national-level disability policy work.

The Project has also effectively coordinated activities with MOLISA’s local authorities at district

levels.

The NCD feels strongly that MOLISA should be more involved in oversight of district-level activities

and that investments in rehabilitation for persons with disabilities should be focused on rehabilitation

and social protection centers at the community level. NCD staff said that USAID would like to work

more directly with provinces, but that the provinces would like to have more direction from

MOLISA in implementing activities. This assertion was not validated by discussions with provincial

and district authorities.

The NCD believes that rehabilitation is the role of MOLISA and that focusing Project efforts on the

health sector under MOH, rather than on community-level rehabilitation and social protection

centers, will result in rehabilitation support being provided for wealthy people only — and thus the

poor will not benefit. This sentiment was not shared by staff in MOLISA’s Health Division of the

Social Protection Unit, who noted plans to coordinate with the MOH on health services for persons

with disabilities.

Figure 7: (left) diagram of overlapping responsibilities of the ministries in relation to person with disabilities; (right) diagram

illustrating levels of responsibilities, lead or supportive, of ministries in relation to USAID Disability Project interventions as outlined in

the 2011 National Law on Disability.

Project

Interventions Lead Ministry

Other

Ministry

Rehabilitation for

adults, children MOH MOLISA

Screening, early

intervention,

inclusive education

for children

MOET MOH

Monitoring,

implementation of

broad disability

policy

MOLISA

Implementation of

accessibility code MOC

Provision of legal aid MOJ

Ministry of Health

(MOH)

Ministry of

Labor, Invalid

& Social

Affairs

(MOLISA)

Ministry of

Education &

Training

(MOET)

Ministry of

Construction

(MOC)

Ministry of

Justice

(MOJ)

Finding 13. The roles of the MOH and MOLISA related to rehabilitation, as well as the

working relationship between the ministries, are still evolving. This evolution includes

provision of rehabilitation services, workforce, and assistive products.

MOLISA and MOH both have rehabilitation-related mandates under the law and are actively working

towards carrying out their mandates. According to the 2011 National Law on Persons with

Disabilities, the MOH has the responsibility “to perform the state management of healthcare for

persons with disabilities,” and “to assume the prime responsibility for, and coordinate with MOLISA

in, specifying activities of functional rehabilitation for persons with disabilities; training in functional

rehabilitation; implementing programs on disability prevention; and guiding community-based

functional rehabilitation for persons with disabilities.” MOLISA has the responsibility “to plan and

manage the system of orthopedic and functional rehabilitation establishments as well as

establishments taking care of persons with disabilities under their management.”

MOLISA is, and continues to be, actively involved in rehabilitation. In September 2017, a Division of

Health was established under the Social Protection Unit of MOLISA. According to staff of this unit,

this development reflects the ministry’s leadership and attention to rehabilitation. The Division

leadership plans to increase its staff of health professionals and work more closely with the MOH to

coordinate health services, including services for persons with disabilities; a cooperation agreement

is to be signed between MOLISA and MOH in May 2018.

MOLISA continues to manage orthopedic and functional rehabilitation interventions through four

specialized orthopedic rehabilitation hospitals in HCMC, Da Nang, Binh Dinh and (newly opened)

Hanoi; the operation of six rehabilitation centers (lower than hospital level) at national and

provincial or district levels; and operation of residential social protection centers that are

responsible for providing rehabilitation for orphans and elderly persons with disabilities at the

community level. In addition, MOLISA manages approximately 10 workshops that produce artificial

limbs and braces at national and provincial or district levels, and provides training to MOH staff on

the production of artificial limbs through the Vietnamese Training Center for Orthopedic

Technologists.

To what extent has USAID gender policy been implemented in the Project? What are

the lessons learned of what works and what does not?

Finding 14. The Project is compliant with the USAID Gender Policy and ADS 205;

Activities address key gender gaps found in in the 2014 PAD analysis and 2017 PAD

amendment.

The Project meets the requirements set forth in ADS 205 for integrating gender equality and female

empowerment in USAID’s program cycle. A project-level gender analysis was completed and the

findings were integrated in the 2014 PAD. As required by agency policy, the PAD-level gender

analysis assessed gender gaps in the status and anticipated levels of participation of women and men

relevant to the Project and highlighted needs and relevant opportunities for female empowerment.

There was no discussion of unintended or negative consequences of Project activities on women and

men. A list of four distinct planned interventions was provided in the PAD that respond to identified

gender gaps. The planned interventions included:

● Address gender as part of analyses that inform policy development (i.e., provincial action plans

on disability and POS strategies);

● Improve POS professionals’ awareness of gender equality, and ensure client-oriented POS

services respond to the needs and preferences of men and women;

● Ensure communication strategies and materials advertising Project-related services reach both

men and women; and


● Promote equal responsibilities between men and women in providing care for adults/children

with disabilities.

The PAD also identified five additional interventions that could be included to address gender gaps.

These interventions focused on approaches such as challenging gender stereotypes, targeted efforts

to support greater social participation of young women with disabilities, increasing the cadre of

women workers to fit other women in need with artificial limbs (noting cultural sensitivities of men

providing this service to women), the promotion of self-help groups for men and women with

disabilities to increase social participation, and promoting gender equality as a key issue of concern

among DPOs.

The resulting solicitation linked to the PAD was not available for review during this evaluation;

however, based on extracts of Activity descriptions, IP self-assessments, IP interviews and

discussions with other Project stakeholders, it is evident that the Project has implemented specific

interventions to address gender gaps found in the PAD gender analysis. Examples of interventions

include: training rehabilitation service providers on appropriate ways to provide care to men and

women, with particular emphasis on provision of care to members of the opposite sex; conducting

workshops and counseling families on the shared responsibility of both men and women as

caretakers of family members with disabilities; and raising awareness of GBV among families. These

interventions are tracked in the Project M&E plan using custom indicators.

As noted elsewhere in this report, the 2017 amended PAD introduces a sustainable model

composed of five interrelated and complementary elements that contribute to the Project purpose.

Two of those elements — human capital, and public attitudes and awareness of disability rights —

specify gender-related requirements. The amended PAD seeks to create a group of individuals well-

trained in rehabilitation and states that additional training on disability and gender issues, including

GBV will also be provided. It also seeks to improve awareness of the rights of persons with

disabilities and to reduce stigma and discrimination, noting that all activities related to this element

will be gender sensitive and will address GBV as possible.

In 2017, USAID/Vietnam conducted a gender literature review and its findings revealed high rates of

GBV against women and girls with disabilities, as well as high rates of violence against men and boys

with disabilities. As a result, the amended PAD describes how new USAID interventions will

respond to that finding; this information was also carried through to the follow-on RFA released in

January 2018 for the Sustainable Model for Improving the Inclusion of Persons with Disabilities

program. Applicants were required to adequately incorporate gender considerations into the

planning and implementation of activities, though gender requirements were not explicitly defined in

the review criteria. Though these changes are still fairly new, IPs with agreements originating from

the 2014 PAD solicitation are already beginning to address issues of GBV within their programming.

Finding 14A. The IPs have varied levels of understanding of key gender-related gaps

and effective responses, and the overall influence of gender-specific interventions on

beneficiaries is unclear.

A close look at the three most common interventions undertaken by IPs to address gender-related

gaps at the Activity level indicates that IPs understand gender-related issues, such as GBV, within the

context of the Project in varying ways. Half of the IPs reported conducting training activities with

beneficiaries on GBV-related issues. However, when asked what this training entailed, the typical

response was “How to treat wives better,” without much more description of what that means. In

one case, an IP explained how a father of a project beneficiary was counseled not to bathe his young

daughter with a disability because this could constitute GBV.

The knowledge and in-house expertise of IP staff related to gender norms, gender equality and GBV

varies widely. Only one IP noted having an in-house gender advisor. Other IPs have relied on

consultants to provide gender-related training as part of Activity implementation. When asked, IPs

consistently reported not having opportunities to learn from one another on contextual gender

issues identified by the Project or on effective strategies for addressing them. One IP admitted

having little in-house expertise on gender-related issues and noted that this as the reason they have

not done more to address gender in their Activity implementation. Another IP pointed out that all

partners are providing gender-related training to beneficiaries, but that it is being done

independently based on IP-created materials.

The influence of gender-specific interventions on Project beneficiaries is unclear. Interviews with

persons with disabilities and caregivers, and with Project partners such as health service providers,

GVN authorities, DPOs and university faculty clearly indicated that “gender” was a topic consistently

embedded in Activity interventions. However, similar to responses from IPs, beneficiaries and

partners demonstrated a limited understanding of the practical relevance of gender-related training

they were receiving, though several commented on having learned new terminology and the

difference between “sex” and “gender.”

What is the evidence of local ownership strengthening? Are the Project’s activities

considered as a part of local development agenda in assisting persons with disabilities?

Finding 15. There is a strong sense of local ownership for Project Activities that are in

line with existing GVN policies and practices.

Local ownership strengthening is not an explicit Project target. IP engagement in this area is

represented by a variety of capacity building activities and information sharing with local partners.

Many of the IP’s Activities are in line with existing GVN policies and practices for assisting persons

with disabilities.

Examples of Project interventions that align with the Government’s development agenda for persons

with disabilities are listed below.

“The ratio of male-female patients we see is fifty-fifty … there are no real signs of inequality or domestic

violence. We received gender training from the project last year. We learned that for children there are no

real differences in treatments for boys or girls, but there are for adults.”

- Staff of a district health center in Hue

“We have received training on GBV and understand that we all have the right to access medical care and

housing … that men and women are equally able and important, violence should not be accepted and that

women with disabilities have the right to childbirth. We can now advise the community when we come across

discrimination.”

- Member of a group of beneficiaries with disabilities

“We attended a five-day training on gender for staff at provincial and district levels [supported by the Project]

… we do not discriminate regarding patient care.”

- Staff at a provincial rehabilitation hospital in Tay Ninh

“We were provided with recommendations on how to address gender issues within our curriculum [from the

Project]; specifically, the curriculum is to include information about maintaining privacy when providing care to

men and women. But this idea does not fit 100% with the local context in Vietnam.”

- Staff of Da Nang University of Medical Technology and Pharmacy


● VietHealth has helped Tay Ninh Province reach the target of screening 90% of children aged 0-

6 by 2020. The provincial Health Department said it could not have reached this goal without

support from VietHealth.

● DRD and ACDC support the implementation of Code 10 (GVN’s Accessible Construction

Policy).

● VNAH’s support to MOLISA’s Legal Department enabled the department to complete the

State Report on the Implementation of the UNCRPD.

● The MOH has identified the Disability Information System (DIS) as the standard data collection

tool for persons with disabilities. VNAH is responsible for the development of this tool.

● The MOH and MOLISA are keenly aware of the reimbursement opportunities for

rehabilitation techniques. VNAH has supported efforts to expand the list of approved

techniques (in both depth and breadth). This has been one of the strongest examples of GVN

ownership related to rehabilitation.

One Project Activity that appears to be outside Vietnam’s development agenda is the provision of

assistive products at the community level. Although this support is highly valued by local authorities

and beneficiaries alike, it is seen as an IP initiative and not the responsibility of the GVN.

CONCLUSIONS

PERFORMANCE

Conclusion 1. Progress has been made toward achieving the Project purpose, but underdeveloped

targets within the sustainable model limit measurement of progress. The WOPE team relied heavily

on the sustainable model in conducting this evaluation. The Project purpose is driven by the

sustainable model, yet many of the elements lack clear definitions or targets. Given this situation, it

was difficult to quantify progress.

Conclusion 2. All IPs contribute to the Project purpose, and rehabilitation-related contributions are

highly valued by GVN stakeholders. Although IPs developed their intervention plans long before the

sustainable model was developed, there were some clear contributions to the Project purpose. Most

evident were the interventions related to establishing rehabilitation units in health facilities, training

related to rehabilitation, and provision of assistive products.

Conclusion 3. IPs contribute to the Project purpose, but the lack of coordination may limit the

impact of some interventions. IP interventions are insular and do not profit from linkages with IPs

conducting the same work (e.g., VNAH and IC both provide assistive products) or complementary

work (DRD’s advocacy work has no direct link with rehabilitation services). In addition, efforts to

“We do not have the budget for this, and also do not have the time. It is not likely that we can continue this

work after IC leaves.”

– Staff of DOH, Thua Thien - Hue

“We are happy with the devices provided by IC. At commune levels the GVN can only give general guidance,

but IC is able to provide assistance and this is 100% from USAID.”

– Staff member of Hue Rehabilitation Hospital

“If the IC project ends, the rehabilitation in the communes will continue, but it won’t be possible to continue

providing devices. The family could buy them and we could advise on how to use them.”

– Staff at Quang Dien District Health Center

capitalize on previous USAID/Vietnam’s disability-related investments or other rehabilitation

initiatives outside the project are negligible.

Conclusion 4. The Project is overachieving in targets pertinent to the 2014 PAD, but the targets do

not fully align with the 2017 sustainable model. Project indicators were designed in 2014 and reflect

Agency priorities as well as targets that are applicable to the current Project design. IPs have

overwhelmingly succeeded in meeting the targets, but the Project has evolved and the indicators no

longer represent the entirety of the Project design.

VALIDATION OF THE PROJECT DESIGN

Conclusion 5. The Project will remain open to misinterpretation until all Project design aspects are

clearly and fully described. USAID/Vietnam has provided partial descriptions or targets for the

elements of the sustainable model. Without further expanding these definitions or specifying targets

it will be increasingly difficult for USAID to bring new partners into the Project and to effectively

manage their performance.

Conclusion 6. A mismatch and the potential for confusion will remain until the content of the

sustainable model and the results framework are integrated. The results framework and the

sustainable model each have valuable content that provide a strong foundation for the Project.

Unfortunately, this content is not mutually reinforcing. If an IP aligns solely with the results

framework, there will be under-utilization of the sustainable model. If an IP focuses solely on the

sustainable model, important interventions from the results framework (such as direct assistance)

will be missed.

Conclusion 7. Until sustainability and inclusion are defined and benchmarks are established, efforts to

address them will be arbitrary and difficult to measure. As USAID/Vietnam uses both terms

(sustainable and inclusion) in the articulation of the Project purpose, it is vital that these terms be

defined and understood in a standard way.

Conclusion 8. As the Project’s purpose encompasses more than inclusion of persons with

disabilities, DPOs could play a greater role in driving the demand for rehabilitation and in developing

rehabilitation policy. Although the majority of Project interventions are focused on rehabilitation, it

is not documented anywhere that the Project is, in fact, a rehabilitation project as its overall purpose

is inclusion of persons with disabilities. With the introduction of the sustainable model there is now

an even greater emphasis on rehabilitation. Persons with disabilities and their representative

organizations (DPOs) can play a more direct role in ensuring availability of rehabilitation services.

Conclusion 9. USAID’s assistance instrument for the Project has resulted in unintended and

innovative partnerships, but has also created the potential for differing implementation standards.

USAID’s funding instrument for the Project limits the extent to which specific guidance can be

provided to IPs on how they implement their Activities. This adversely effects the standards applied

across the project, but also opens up opportunities for partnerships and innovation not previously

envisioned.

Conclusion 10. Unless regulations for rehabilitation licensure are consistently applied, there is a

potential for POS reputations to be damaged and for professional standards to be undermined.

Rehabilitation is relatively nascent in Vietnam and many of the supporting professions (OT and

speech and language therapy [SaLT]) are completely new. If shortcuts in training and licensure

become the norm, this may result in treatments becoming less effective, even harmful, and thus may

destabilize these fledgling efforts in the sector.



Conclusion 11. Disability policy-related efforts supported by the Project are more likely to be

effective if carried out in consultation with MOLISA’s Legal Department and MOLISA’s local

authorities. Implementation of the National Disability Action Plan and related provincial disability

action plans are under the direct purview of MOLISA at the central and local levels. The Legal

Department of MOLISA is well established and has been effective in supporting adoption of the GVN

State Report to the CRPD Committee.

Conclusion 12. Though the NCD has expressed concern about lack of engagement in the Project,

this committee does not represent MOLISA in its entirety. National-level broad disability policy

activities supported by the Project benefit from direct coordination with MOLISA’s Legal

Department. Development and implementation of provincial-level disability action plans supported

by the Project benefit from direct coordination with MOLISA authorities at the provincial and

district levels. The NCD has raised concerns about lack of engagement with the Project, but this

experience is not shared across other Departments in MOLISA.

Conclusion 13. MOLISA’s plan to expand rehabilitation-related activities may either complement or

confound the Project. Engaging the Health Unit in the Department of Social Protection is key to

promoting synergies and to ensuring effective use of resources. The MOH is legally responsible for

state management of healthcare for persons with disabilities. It carries the primary responsibility for

specifying rehabilitation activities, conducting training and guiding community-based rehabilitation for

persons with disabilities — all in coordination with MOLISA. IPs implementing Project Activities

focused on rehabilitation in the health sector would benefit from coordinating directly with the

MOH; at the same time, without further coordinating with MOLISA, the roles and responsibilities

and perceived political challenges with the two ministries in rehabilitation will persist.

Conclusion 14. Operationally, the Project has met the requirements of the Agency Gender Policy

and ADS 205 and has integrated details within key documents for Project/Activity planning and

implementation. However, the overall the quality and consistency of common gender-related

messaging and interventions at the Activity level warrants closer attention. Both this evaluation and

the USAID Vietnam Gender Review 2017 confirm that the USAID Disability Project has met Agency

policy requirements for integrating gender equality throughout the design and planning phase,

implementation phase, and monitoring, evaluation and learning phase. However, as previously noted,

the relevance of gender-specific interventions at the Activity level is yet to be seen. It is clear that

GBV, and in particular its intersections with violence against and neglect of persons with disabilities,

is not fully understood in the context of the Project. IPs have limited knowledge of GBV and how to

effectively integrate it within implementation of Activities. Mixed messaging on what constitutes

GBV, and the potential negative impact this could have on Project interventions, should be

examined.

Conclusion 15. The GVN is poised to take more responsibility in the rehabilitation sector. USAID is

in a strong position to facilitate this process. Rehabilitation is rapidly evolving in Vietnam, and Project

Activities contribute to GVN priorities in the sector. USAID’s historical investments in disability in

Vietnam and the current level of funding for rehabilitation create an opportunity to leverage

discussions with MOH and relevant offices within MOLISA to accommodate growth in the sector in

a strategic way. This includes addressing the impact of rehabilitation interventions and the

sustainability of Activities after the Project ends. In the absence of a clearly articulated sustainability

plan for the Project, local ownership strengthening will likely remain a tangential focus.

RECOMMENDATIONS

PERFORMANCE

1. Support rehabilitation systems strengthening and services in Project areas in

cooperation with the relevant responsible ministry/department (DOH and/or

DOLISA) in accordance with MOH policies. (Finding/Conclusion #13) Although direct

assistance to individuals in the community reaches those outside the healthcare system, support

for rehabilitation (services and systems) within the health system fosters potential for

sustainability.

2. Investigate ways to minimize duplication of IP interventions, promote Activity

standards, and capitalize on rehabilitation-related investments outside of the

Project to effectively realize the Project purpose. (Finding/Conclusion #3) The WOPE

focuses on purposeful synergies to achieve the Project purpose. Actions that would benefit the

Project include a review of tools or curricula (e.g., accessibility audits and GBV training) to avoid

duplication, and a review of Activity standards (such as for equipping rehabilitation units or

providing assistive products). Outside the Project, USAID/Vietnam could benefit from aligning

Project Activities with past investments and/or global rehabilitation initiatives.

3. Structure IP coordination meetings to progress beyond information sharing and

toward issues-based engagement, to meet the Project purpose. (Finding/Conclusion

#3) USAID/Vietnam has invested in multiple strategies to facilitate coordination; in lieu of

increasing the number of meetings, there should be a focus on refining the objectives of the IP

coordination meetings. Coordination can also be strengthened by addressing gaps in IP Activity

standards, and by linking to USAID’s past investments and to other rehabilitation initiatives

outside the Project.

4. Continue support for rehabilitation-related policy developments (e.g., MOH

Circular #18) with a vision toward monitoring outcomes of care. (Finding/Conclusion

#2) The Project has supported broad advances in policy related to rehabilitation services. This

focus has been on services delivery and not on outcomes of service. There is still work to be

done related to health insurance coverage for a wide range of assistive technologies, and a need

for a concentrated effort to recognize outcomes of care.

5. Conduct a needs assessment specific to assistive technology in Vietnam.

(Finding/Conclusion #3) This should encompass workforce, products, procurement, and

provision. The assessment should complement previous work in this area (by IC) and existing

assistive technology assessments (by MoveAbility), and should ideally align with WHO’s Global

Cooperation on Assistive Technology.

PROJECT DESIGN

6. Refine the elements of the sustainable model. (Findings/Conclusions #1, #5 and #7)

These elements must capture POS services provided in health centers and communities, as well

as assistive technologies (products, personnel, provision and policy), and provide clear

descriptive content and benchmarks for sustainability.

7. Integrate the results framework and the sustainable model within the Project

design. (Finding/Conclusion #6) Explicit links between the sustainable model and the results

framework are required if both aspects of the Project are to be fully utilized. This can be

achieved by combining the results framework with elements of the sustainable model within one

table and creating direct linkages.

8. Update the Project’s TOC and indicators to include the most current content of the

Project design. (Finding/Conclusion #4) After USAID has refined the elements and integrated

these with the results framework, the TOC and indicators should be updated to provide clear

alignment with the Project purpose. These newly crafted or updated measures should be

relevant to the updated Project design.

9. Identify unintended consequences of rapidly scaling up rehabilitation services

(systems and personnel) and invest in corrective measures. (Finding/Conclusion #10)


The GVN (MOH and MOLISA) are keen to capitalize on reimbursement opportunities for

rehabilitation techniques and orthoses. There are many training and licensing initiatives needed

for individuals/institutions to access these reimbursement opportunities. The Government’s zest

for the rehabilitation sector is encouraging, but USAID should exercise caution in supporting all

initiatives without consideration of potential negative consequences.

CROSS-CUTTING ISSUES

10. Continue coordination with MOLISA to address disability policy and rehabilitation-

related actions. (Finding/Conclusion #11) MOLISA continues to have a role in the Project. It

is essential to coordinate with the relevant offices (the Legal Department for policy work and

the Health Unit in the Department of Social Protection for rehabilitation) to promote synergies

and ensure effective use of resources.

11. Reinforce rehabilitation content within disability policy, rights and advocacy

messaging work. (Finding/Conclusion #8) Interventions to consider include leveraging the

expertise of DPOs to advise on appropriate service provision, serve as consumer advocates, and

help mobilize people with disabilities in rural communities.

12. Review IPs’ understanding of and practices in integrating gender equality into

Project interventions, including knowledge of how gender-informed interventions

influence beneficiaries. (Finding/Conclusion #14) Although the Project complies with

USAID’s Gender Policy and ADS 205, there is value in exploring the understanding of IPs of how

different training activities have made a difference for beneficiaries, and how each of them are

interpreting their work in this area.

13. Develop a sustainability plan to clearly articulate short- and long-term expectations

for the Project. (Finding/Conclusion # 15) USAID should invest time and resources to create a

clear pathway toward sustainability. The foundation of this sustainability plan would be a clear

definition of sustainability related to the rehabilitation sector and well-defined benchmarks for

sustainability. From this point, USAID, together with the GVN and relevant stakeholders, could

develop content that builds upon this foundational framework.

WOPE LESSONS LEARNED

OVERALL TAKE-AWAY

ADS 201 (September 2016) outlines a WOPE and provides guidance on how a whole-of-project

performance evaluation differs from an Activity-level evaluation. Even with this guidance, it is

challenging to find and retain the balance of identifying synergistic contributions of Activities toward

the Project purpose versus a mere compilation of achievements of the constituent parts (Activities).

The biggest challenge is knowing enough about the Activities to see the linkages or contributions to

the Project purpose (or lack thereof) without getting drawn into implementation issues, technical

details or performance discussions.

PREPARATION FOR A WOPE

To frame the evaluation and keep it focused on progress of all the constituent Activities toward

achieving the Project purpose, specific attention to the following preparatory actions could be

beneficial.

1. Evaluation Questions

ADS 201 provided illustrative questions/sub-questions to guide the WOPE in three key areas:

● Examining the contribution from all constituent parts to the Project purpose.

● Examining strengths and weaknesses of the Project’s TOC; and

● Examining the interaction among activities as they contribute to the Project purpose.

Suggestions:

● Focus on WOPE-level content to avoid asking questions that could be answered by other

types of evaluations.

● Ensure there are questions focused on Activity coordination, as this is fundamental to a

WOPE.

● Consider expanding the coordination questions to include the evolving context of those

involved in the general sector.

● In addition to looking at current partner engagement, it would be useful to reflect on past

USAID investments in the same sector to see what synergies or opportunities exist to support

the current Project.

2. Implementing Partner Self-Reports

Having each of the IPs submit a self-report is a good idea and could be highly informative. IP report

guidance sets a 20-page limit and requests content similar to what typically is found in a progress

report.

Suggestions:

● Require all partners to complete this activity; and complete it during the desk review phase.

● Ensure questions are relevant to the WOPE with a focus on coordination, interaction among

activities, and contributions specific to the Project purpose.

● Limit the number of questions to about five, and a maximum length of 3-4 pages.

3. Orientation to a WOPE for USAID staff, evaluation team and others

Since Activity-level performance M&E is more prevalent than a WOPE, expectations for a WOPE

may drift toward familiar areas of individual Activity evaluation, and this may confound the process.

Suggestions:

● Explain/review the intent of a WOPE to USAID staff and to the evaluation team prior to,

during, and at the end of a WOPE to ensure all have an equal understanding of the purpose.

● Ensure IPs and other partners understand what a WOPE is and how if differs from other

evaluations.

TIMING OF A WOPE

The USAID/Vietnam Mission refined the Project Purpose six months prior to this WOPE, and there

are plans to continue activities until 2023. New content supporting the revised Project Purpose is

not well integrated into the lexicon used by IPs. Conversely, the previous Purpose (and results

framework) are similar in scope, and conducting this WOPE created space to bring these

contributing parts together.

ACQUISITIONS OR ASSISTANCE MECHANISMS

The WOPE looks toward how activities under the Project have clearly articulated synergies, highly

interdependent implementing mechanisms, and/or important coordination points across activities.

Encouraging these relationships through contracts can be relatively straightforward. Assistance

mechanisms (used in Vietnam) are not controlled in the same way by the Mission, and this can create

challenges to encouraging all Activities/IPs to work synergistically toward the Project Purpose.


ADDENDUM

ANNEX I: EVALUATION STATEMENT OF WORK

WHOLE – OF – PROJECT EVALUATION OF VIETNAM INTERVENTION PROJECT TO ASSIST PERSONS WITH DISABILITIES

I.PURPOSE OF THE EVALUATION This Mid-term WOPE will primarily determine to what extent the USAID-funded Vietnam Disability

Project 2015-2020, with its six constituent activities, is progressing toward achievement of its

purpose: “A sustainable model for improving persons with disabilities’ inclusion developed in USAID

targeted provinces”.

Specifically, the evaluation will serve to:

● Examine the progress made towards the intended purpose and outcomes described in the

Project Appraisal Document (PAD) and six Activity Management Plans (AMPs), and to record

challenges and lessons learned in achieving the activities’ planned targets;

● Determine whether the activities, technical interventions and management strategies (as

specified in six program statements of works, and their M&E plans) are leading to the

achievement of the changes expected from the whole program design documents;

● Provide USAID/Vietnam, the Government of Vietnam (GVN), and other in-country stakeholders

with objective information on what has been achieved to date, what is working and what is not.

The information will inform decisions to make appropriate modifications in the remaining period

of performance of the project and may also serve as a reference for the design of future

projects; and

● To learn and share gathered experience/lessons in conducting a whole of project evaluation.

II.SUMMARY INFORMATION

Activity Name USAID Office

Implementer TEC Years Active Regions

Mission DO

CA or Contract #

Accessibility for

inclusion ESDO5

Disability Research

and Capacity

Development

310,000 2015 –

2017

Tay Ninh and

Binh Dinh DO – 26

Moving without

limits ESDO

International

Center 3,000,000

2015 –

2020

Thua Thien –

Hue; Quang

Nam

DO – 2

Disability right

enforcement,

Coordination and

Therapy

ESDO

Vietnam Assistance

for the

Handicapped

5,889,249 2015 –

2020

Tay Ninh and

Binh Phuoc DO – 2

Disability

integration

services and

therapy network

for capacity and

treatment

ESDO

Sustainable Health

Development

Center

1,800,000 2015 –

2019 Tay Ninh DO – 2

Advancing medical

care and

rehabilitation

education

ESDO Handicap

international 5,400,000

2015 –

2020

Hanoi,

HCMC, Dong

Nai, and Thua

Thien - Hue

DO – 2

Protecting the

rights of persons

with disabilities

ESDO

Action to the

Community

Development

Center

1,200,000 2015 –

2018

Thua Thien –

Hue, and Binh

Phuoc

DO – 2

5 Environment and Social Development Office

6 Development Objective 2: Capacity Strengthened to Protect and Improve Health and Well-being


BACKGROUND

A. DESCRIPTION OF THE PROBLEM, DEVELOPMENT HYPOTHESIS (ES), AND THEORY OF

CHANGE

In 2014, USAID Vietnam developed a project to assist persons with disabilities in Vietnam

(hereafter referred as the USAID Disability Project). This project was built on the foundation of

past USAID assistance for persons with disabilities, and focuses on improving quality of life and

inclusion of persons with disabilities by improving disability policy, advocacy, and coordination, and

strengthening the caliber and availability of physical, occupational, and speech therapy services.

The Vietnam Disability Project is a significant component of USAID/Vietnam’s Country

Development Cooperation Strategy (CDCS) Development Objective 2: Capacity strengthened to

protect and improve health and well-being through Intermediate Result 2.3: Expanded

opportunities for vulnerable populations.

PROJECT PURPOSE:

“A Sustainable Model for Improving Persons with Disabilities’ Inclusion Developed in USAID’s Target

Provinces”.

The sustainable model is composed of five interrelated and complementary elements that contribute

to the Project Purpose:

● Service Delivery System: The project will set up at least 10 rehabilitation service units at

provincial or district hospitals in each province.

● Human Capital: The project will seek to create well-trained human resources in rehabilitation for

target provinces, starting with training for a minimum of 20 rehabilitation doctors and 45

therapists/therapist assistants to provide services in each target province in the areas of physical

therapy, occupational therapy and speech and language therapy. Additional training on disability

and gender issues, including gender-based violence (GBV), will also be provided.

● Policy Support: The project will support the development and enforcement of national and

provincial policies to support persons with disabilities’ inclusion and/or improve the quality and

accessibility of rehabilitation services.

● Sustainable Finance: The project looks to improve the implementation of health insurance and

local government plans/budgets supporting disability issues to ensure the costs for rehabilitation

services are covered.

● Public Attitude and Self-Awareness: The project will work to improve awareness of the rights of

persons with disabilities and reduced stigma and discrimination towards persons with disabilities.

All activities will be gender sensitive and will try as much as possible to address GBV toward

persons with disabilities.

These five elements are strengthened and improved through activities in three project

components: 1) provision of direct assistance; 2) service system strengthening; and 3) support for

the enforcement of disability policies and rights.

Development Hypothesis:

If USAID focuses on: fostering and enabling environment through the enforcement of disability

rights, policies, and coordination; delivering innovative and quality POS services; and improving local

capacity to provide POS services, these activities will achieve the goal of increasing persons with

disabilities’ independence, inclusion and help fulfill their potential in society.

Theory of Change:

If USAID builds a sustainable model that a) delivers direct assistance to persons with disabilities to

demonstrate interdisciplinary rehabilitation services that improve persons with disabilities’ daily

living, learning, and working functions; b) strengthens systems to deliver quality services to persons

with disabilities; and c) improves the enforcement of disability laws/policies, then persons with

disabilities’ inclusion in society will be increased.

Figure 1: Results Framework

B. SUMMARY STRATEGY/PROJECT/ACTIVITY/INTERVENTION TO BE EVALUATED

Project activities work in three areas: policy advocacy and coordination, direct assistance including

occupational and physical therapy and providing assistive devices, and health systems strengthening

to provide quality disabilities-related care and treatment. Persons with disabilities have been more

readily integrated into society by improving their access to health, education and social services.

USAID programs support local governments in implementing the national disability law and the U.N.

Convention on the Rights of Persons with Disabilities and influencing public policies that affect the

lives of persons with disabilities.

C. SUMMARY OF THE PROJECT/ACTIVITY MONITORING, EVALUATION, AND LEARNING (MEL)

PLAN

The project’s monitoring and evaluation (M&E) framework includes 23 performance indicators

measuring different levels of expected outcomes (development objectives, project purpose,

outcomes and outputs). Specific performance indicators by component are provided in Annex 1.

M&E indicator data is reported from six implementing mechanisms and from a baseline survey

conducted in early 2016, and will be made available to the evaluation team. In addition, other

reports of the assessments and/or evaluations that were previously conducted by USAID/Vietnam

and implementing partners are also other sources of information that can be useful for the

evaluation.



A. Performance

How is the project making progress toward the achievement of its purpose?

• To what extent has progress been made in achieving the project’s purpose of developing a

sustainable model for improving persons with disabilities’ inclusion in society (in USAID target

provinces)?

● To what extent have each of the six component Activities contributed to achieving the project’s

purpose?

● What are the achievements and progress against targets in key program areas: fostering disability

rights, policy, coordination and advocacy; provision of physical, occupational, and speech (POS)

therapy services to assist persons with disabilities; and improving local capacity to provide quality

services?

This section should include an analysis of performance monitoring data and analysis of why targets

are or are not being met, as well as an assessment of the performance of the overall project.

B. Validation of the Project’s Design

What are the strengths and weaknesses (gaps) of the project’s theory of change?

● Is there any strategy or specific intervention in the initial design that has had an unexpected

influence on the project’s implementation and/or achieving the project’s purpose?

C. Project Management and Cross- Cutting Issues

● What are the benefits of coordinating the project’s implementation activities with the Ministry of

Labor, Invalids, and Social Affairs (MOLISA) and local authorities to achieve its purpose? What

are identified challenges? Recommendations for the management section should include how the

project design, management, and implementation can become more efficient, effective and

relevant toward achieving the overall project objectives -- in consideration of any changing

contextual dynamics.

● To what extent has USAID gender policy been implemented in the project? What are the

lessons learned of what works and what does not?

● What is the evidence of local ownership strengthening? Are the project’s activities considered as

a part of local development agenda in the area of assisting persons with disabilities?

III.EVALUATION DESIGN AND METHODOLOGY

It is expected that a mixed methods methodology will be used in this WOPE. The evaluation team,

together with VEMSS, will develop a Getting to Answers Matrix including sub-evaluation questions,

data/information needed, and data sources (further details on the use of evaluation design templates

and matrices can be found here: https://usaidlearninglab.org/library/evaluation-design-matrix-

templates

Tentatively, the mixed method evaluation will include the following key components:

1. Collect, synthesize, and triangulate existing data/information from both the project and

activity levels.

https://usaidlearninglab.org/library/evaluation-design-matrix-templates

https://usaidlearninglab.org/library/evaluation-design-matrix-templates

This task will serve several purposes: 1) to provide background on the disability sector in Vietnam;

2) to initially answer the evaluation questions through collecting, synthesizing, and triangulating

existing data/information from both project and activities levels; and 3) to identify additional data

collection needed during the fieldwork phase.

The evaluation team will conduct a comprehensive literature review of pertinent documents

including studies and assessments on disability in Vietnam; GVN strategies and plans related to

persons with disabilities; and USAID project and activity documents, including but not limited to:

● The Disability Project Appraisal Document (PAD)

● Activity cooperative agreements/contracts, modifications, annual work plans and

subcontracts;

● Performance reports, and performance monitoring, evaluation and learning plans (MEL

Plans);

● Performance monitoring data (from USAID database, such as Aid Tracker+, and from

implementing partners);

● Formative research, surveys, assessment reports and presentations, which were conducted

by Activities;

● Annual provincial work plans, if available and accessible;

● Other relevant documents, as available.

A summary of the desk review, developed by the evaluation team, will be completed prior to the

team’s full deployment to Hanoi.

In addition, VEMSS will structure a data collection and synthesis guide for implementing partners to

complete prior to the initiation of the evaluation. This will include a request to provide summary

data and analysis on key aspects of the performance of implementing partner activities and will

provide basic data sets for the evaluation team’s use.

2. Quantitative beneficiary survey.

A baseline survey was conducted in 2015 in four provinces (Thai Binh, Thue Thien-Hue, Binh Phuoc,

and Tay Ninh) based on a cross-sectional design. About 900 persons with disabilities were recruited

for the survey. Local stakeholders were also invited to participate in a small survey using a

structured interview process. (More details about the baseline survey will be provided).

In the WOPE, a follow-up survey will be designed to measure and monitor potential changes in

the:

Quality of life of persons with disabilities;

● Capacity of disability service providers;

● Quality and availability (accessibility) of local services for persons with disabilities; and

● Enabling environment for persons with disabilities.

Several factors will be considered when designing the follow-up survey (to ensure comparability

with the baseline):

● Survey sites. The evaluation team will provide recommendations on the optimal selection of

project sites for this evaluation, given limited time and available resources.

● Sampling strategy. The ability to make scientifically valid conclusions on the changes that are

attributable to the USAID assistance is required; e.g., can observed change in persons with

disabilities’ quality of life be attributed to USAID assistance?

● Data collection tools (structured questionnaires).


3. Qualitative research:

Qualitative research methods will be used in addition to the beneficiary survey. These may include:

1) in-depth interviews with key stakeholders; 2) group discussions; and/or 3) direct observation.

Key stakeholder groups will include:

● USAID Vietnam

● Activity implementing partners

● Persons with disabilities and caregivers

● Government ministries (Ministry of Health [MOH], MOLISA, and provincial line

departments)

● Hospital health workers (at national, provincial and district levels)

● University faculties (from universities in Da Nang, HCMC and Hai Duong)

● Local civil society organizations (persons with disabilities organizations, and/or associations)

● Other international and national partners who work in the disability sector

To utilize the advantage of a mixed-method design, the evaluation sequence will be: 1) desk review;

2) beneficiary survey; and 3) qualitative data collection. The desk review and preliminary data analysis

will help to inform the design of both quantitative and qualitative surveys. Quantitative results will

help to interpret observed results from previous phases (e.g., explain quantitative data more deeply).

DELIVERABLES AND REPORTING REQUIREMENTS

1. Evaluation Work plan: Within 6 weeks of the approval of the SOW, a draft work plan for the

evaluation shall be completed presented to the Agreement Officer’s Representative/Contracting

Officer’s Representative (AOR/COR). The work plan will include: (1) the anticipated schedule

and logistical arrangements; and (2) a list of the members of the evaluation team, delineated by

roles and responsibilities.

2. Evaluation Design: Within 4 weeks of approval of the work plan, the evaluation team must

submit to the Agreement Officer’s Representative/Contracting Officer’s Representative

(AOR/COR) an evaluation design (which will become an annex to the Evaluation report). The

evaluation design will include: (1) a detailed evaluation design matrix that links the Evaluation

Questions in the SOW to data sources, methods, and the data analysis plan; (2) draft

questionnaires and other data collection instruments or their main features; (3) the list of

potential interviewees and sites to be visited and proposed selection criteria and/or sampling

plan (must include calculations and a justification of sample size, plans as to how the sampling

frame will be developed, and the sampling methodology); (4) known limitations to the evaluation

design; and (5) a dissemination plan.

USAID offices and relevant stakeholders are asked to take up to 5 business days to review and

consolidate comments through the AOR/COR. Once the evaluation team receives the

consolidated comments on the initial evaluation design and work plan, they are expected to

return with a revised evaluation design and work plan within 5 days.

3. In-briefing / inception report. Within 5 days of arrival in Hanoi, the evaluation team will have an

in-briefing with the USAID Mission for introductions and to discuss the team’s understanding of

the assignment, initial assumptions, evaluation questions, methodology, and work plan, and/or to

adjust the Statement of Work (SOW), if necessary.

4. Validation workshop with implementing partners. The key purpose of this workshop will be to

present preliminary findings, conclusions and recommendations to the Mission and implementing

partners, and to seek inputs and comments for validating or challenging findings. The validation

workshop will be conducted before the team begins drafting the evaluation report. The

validation workshop also provides an opportunity for key stakeholders to provide ideas on

appropriate recommendations. This will help to ensure that recommendations are feasible (in

consideration of priorities, context and budget) and will help to build ownership for the

implementation of recommendations by key stakeholders.

5. Final Exit Briefing. The evaluation team is expected to hold a final exit briefing to the Mission

prior to leaving the country to discuss the status of data collection and preliminary findings and

conclusions. This presentation will be scheduled as agreed upon during the in-briefing.

6. Draft Evaluation Report: The draft evaluation report should be consistent with the guidance

provided in Section IX: Final Report Format. The report will address each of the questions

identified in the SOW and any other issues the team considers to have a bearing on the

objectives of the evaluation. Any such issues can be included in the report only after consultation

with USAID. The submission date for the draft evaluation report will be determined in the

evaluation work plan. Once the initial draft evaluation report is submitted, USAID ESDO/PDO

and Implementing Partners will have 10 business days in which to review and comment on the

initial draft, after which point the AOR/COR will submit the consolidated comments to the

evaluation team. The evaluation team will submit a revised final draft report 15 business days

hence, and again the ESDO and PDO will review and send comments on this final draft report

within 5 business days of its submission.

7. Final Evaluation Report: The evaluation team will be asked to take no more than 10 business

days to respond/incorporate the final comments from the ESDO and PDO. VEMSS will then

submit the final report to the AOR/COR. All project data and records will be submitted in full

and should be in electronic form in easily readable format, organized and documented for use by

those not fully familiar with the intervention or evaluation, and owned by USAID.

EVALUATION TEAM COMPOSITION

1. Team Leader

Roles and Responsibilities

● Serve as the primary manager of the evaluation team.

● Responsible for the design of the evaluation’s methodology, team management, and for

producing expected deliverables (see Deliverables section).

● Lead the preparation and presentation of the key evaluation findings, conclusions and

recommendations to the USAID/Vietnam team and key partners.

Qualifications

● An advanced degree in public health, social work or another field related to disabilities

programming.

● Must be a senior consultant with multiple years of experience in leading and conducting

USAID program evaluations.


● Be familiar and comfortable with a range of qualitative and quantitative data collection and

analysis techniques.

● Must have led at least three independent performance evaluations. Experience in conducting

disability-specific evaluations/assessments is highly desired.

● Have at least 10 years senior-level experience working in a developing country.

● Prior knowledge of/experience in Vietnam is an asset or, alternatively, experience in working

in similar countries in Southeast Asia.

● A solid understanding of USAID-funded project implementation, administration, financing,

and management procedures.

● Be knowledgeable of Gender- related issues and USAID gender policy.

● Excellent oral and written skills are required.

2. International Evaluation Expert

Role and Responsibilities

● Serve as a team member.

● Contribute to the design of the evaluation’ methodology, analysis and report writing.

● Participate in field visits and stakeholder consultations.

Qualifications

● Have an advanced degree in social sciences or area related to research and evaluation.

● Secondarily, knowledge and experience in evaluating social services program is an asset.

● Demonstrated knowledge and skills in a range of qualitative and quantitative data collection

and analysis techniques.

● Must have 5 years of experience conducting USAID program evaluations.

● Experience working in a developing country.

● Prior knowledge of/experience in Vietnam is an asset.

● Excellent oral and written skills are required.

3. National Disability Experts (three positions)


These experts will serve as full team members and will perform specific tasks assigned by the Team

Leader that may include, but are not limited to:

● Conducting a literature review and summarization.

● Contributing to the design of WOPE.

● Participating in field visits and stakeholder consultations.

● Providing expert analysis of the program’s operations and performance.

● Contributing to report writing.

Qualifications:

The National Disability Experts should have:

● A degree public health, social work or other field related to disabilities programming, and at

least 5-10 years of experience in disabilities programming.

● Knowledge of USAID programs and context is highly desirable or, alternatively experience

with similar programs managed by other donors, or work with relevant GVN programs.

● A strong understanding of the challenges facing such programs in Vietnam or Southeast Asia.

● Experience in program evaluation and knowledge of how to conduct surveys, key informant

interviews and focus group discussions; experience and understanding of policy

development, capacity building and/or training initiatives and approaches to strengthen

government ownership will be an advantage.

● In-depth knowledge and experience in gender mainstreaming is an advantage

● Strong English language and writing skills are required.

4. National Research Specialist


● Design the beneficiary survey, in consultation with the team leader, including the survey

protocol, data collection instruments and plan.

● Organize and oversee the fieldwork of the beneficiary survey.

● Perform data analysis and prepare summary of key results (prior to the qualitative phase).

● Contribute to overall WOPE design.

● Participate in field visits and stakeholder consultations as needed.

● Provide expert analysis of the program’s operations and performance.

● Contribute to report writing.

Qualifications:

● Advanced degree in survey methodology.

● At least 10 years of experience conducting community-based surveys in Vietnam; experience

with disability surveys is preferred.

● Demonstrated skills in survey data analysis and report writing.

● Strong English language and writing skills are required.

In addition, VEMSS will provide research support (Ms. Hanh Dang, MEL specialist), and logistics

coordination and interpretation (Ms. Linh Tran) during the evaluation process. All team members

will be required to provide a signed statement attesting to a lack of conflict of interest or describing

any existing conflict of interest.

The evaluation team shall demonstrate familiarity with USAID’s evaluation policies and guidance

included in the USAID Automated Directive System (ADS) in Chapter 200.

EVALUATION SCHEDULE

Timeline Proposed Activities Important Considerations/

Constraints

● December 15, 2017 ● Finalization of the SOW ●


Timeline Proposed Activities Important Considerations/

Constraints

● January 22 ● Evaluation plan shared

with provinces ●

● January 26, 2018

● Finalization of the work

plan, including proposed

team members

● VEMSS will develop the

work plan

● February 2 ● USAID Approval of the

work plan ●

● February 21

● Finalization/submission of

the evaluation design,

including data collection

instruments

●

● February 27 ● USAID Approval of the

evaluation design ●

● February 28 ● Desk review starts ●

● March 12 ● Data collection for

beneficiary survey starts ●

● March 23 ● Completion of the desk

review ●

● March 29 ● Completion of survey

data collection ●

● March 26 – March

30

● In-Briefing

● Final adjustments to work

plan/design submitted for

USAID approval

●

● April 2 – April 20 ● Evaluation fieldwork ●

● April 23 – April 27 ● Data Analysis ●

● May 2 – May 4 ● Validation workshop/

Final exist briefing

● There will be a 4-day

national holiday from

April 28 to May 2

● May 7 – June 8 ● Report

writing/submission ●

● June 11 – June 22 ● USAID review of Draft

Report ●

● July 6

● Incorporate USAID

comments and prepare

Final Report

●

Estimated LOE in days by position

Pre-field

Work Field Work Report Writing

Position

Pre

para

tio

n/D

esk

Revie

w

Meth

od

olo

gy

develo

pm

en

t

Tra

ve

l to

/fro

m

Co

un

try

In-C

ou

ntr

y D

ata

Co

llecti

on

In-

Co

un

try

An

aly

sis

Rep

ort

wri

tin

g

Fin

alizati

on

of

Rep

ort

To

tal L

OE

in

days

Expat Team Leader 12 5 4 20 5 15 5 66

Expat Evaluation Specialist 3 2 4 20 5 14 2 50

Local Disability Specialist 1 7 2 20 5 5 39



National

Research/Evaluation

Specialist

5

10

20

5

10

2

52

FINAL REPORT FORMAT

The evaluation final report should include an abstract; executive summary; background of the local

context and the strategies/projects/activities being evaluated; the evaluation purpose and main

evaluation questions; the methodology or methodologies; the limitations to the evaluation; findings,

conclusions, and recommendations. For more detail, see “How-To Note: Preparing Evaluation

Reports” and ADS 201mah, USAID Evaluation Report Requirements. An optional evaluation report

template is available in the Evaluation Toolkit.

The executive summary should be 2–5 pages in length and summarize the purpose, background of

the project being evaluated, main evaluation questions, methods, findings, conclusions, and

recommendations and lessons learned (if applicable).

The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation

shall be disclosed in the report, with particular attention to the limitations associated with the

evaluation methodology (e.g., selection bias, recall bias, unobservable differences between

comparator groups, etc.)

http://usaidlearninglab.org/library/evaluation-report-template


The annexes to the report shall include:

• The Evaluation SOW;

• Any statements of difference regarding significant unresolved differences of opinion by

funders, implementers, and/or members of the evaluation team;

• All data collection and analysis tools used in conducting the evaluation, such as

questionnaires, checklists, and discussion guides;

• All sources of information, properly identified and listed; and

• Signed disclosure of conflict of interest forms for all evaluation team members, either

attesting to a lack of conflicts of interest or describing existing conflicts of.

• Any “statements of difference” regarding significant unresolved differences of opinion by

funders, implementers, and/or members of the evaluation team.

• Summary information about evaluation team members, including qualifications, experience,

and role on the team.

In accordance with ADS 201, the contractor will make the final evaluation reports publicly available

through the Development Experience Clearinghouse within three months of the evaluation’s

conclusion.

CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION

REPORT

Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation

reports will be evaluated against the following criteria to ensure the quality of the evaluation

report.7

• Evaluation reports should represent a thoughtful, well-researched, and well-organized effort

to objectively evaluate the strategy, project, or activity.

• Evaluation reports should be readily understood and should identify key points clearly,

distinctly, and succinctly.

• The Executive Summary of an evaluation report should present a concise and accurate

statement of the most critical elements of the report.

• Evaluation reports should adequately address all evaluation questions included in the SOW,

or the evaluation questions subsequently revised and documented in consultation and

agreement with USAID.

• Evaluation methodology should be explained in detail and sources of information properly

identified.

• Limitations to the evaluation should be adequately disclosed in the report, with particular

attention to the limitations associated with the evaluation methodology (selection bias, recall

bias, unobservable differences between comparator groups, etc.).

• Evaluation findings should be presented as analyzed facts, evidence, and data and not based

on anecdotes, hearsay, or simply the compilation of people’s opinions.

7 See ADS 201mah, USAID Evaluation Report Requirements and the Evaluation Report Review

Checklist from the Evaluation Toolkit for additional guidance.

http://usaidlearninglab.org/library/sample-disclosure-conflict-interest-form

• Findings and conclusions should be specific, concise, and supported by strong quantitative or

qualitative evidence.

• If evaluation findings assess person-level outcomes or impact, they should also be separately

assessed for both males and females.

If recommendations are included, they should be supported by a specific set of findings and should

be action-oriented, practical, and specific.

OTHER REQUIREMENTS

All quantitative data collected by the evaluation team must be provided in machine-readable, non-

proprietary formats as required by USAID’s Open Data policy (see ADS 579). The data should be

organized and fully documented for use by those not fully familiar with the project or the evaluation.

USAID will retain ownership of the survey and all datasets developed.

Modifications to the required elements of the SOW of the contract/agreement, including technical

requirements, evaluation questions, evaluation team composition, methodology, or timeline, should

receive formal approval from the COR. Any revisions should be updated in the SOW that is

included as an annex to the Evaluation Report.

LIST OF ANNEXES

The project MEL framework

SOW Approval Process:

Drafted by: VEMSS

Date submitted to USAID for review: November 22, 2017

Date approved: Dec 18 [by USAID]

Date modified: [by USAID]

Current version: 2

Change log:

Version 2: [JAN 26/2018] – The modification is to adjust timeline after consultation with the project

provinces and implementing partners

DISABILITIES MEL FRAMEWORK

Narrative Summary Indicators

Project’s Evaluation Question

To what extent has the project

increased persons with disabilities’

access to health services, education,

social, and economic opportunities?

Percentage of beneficiaries reporting improvement in their

accessibility to health services, education, social and economic

opportunities

Percentage of beneficiaries reporting improvement in their quality of

life

59

Non-Project CDCS Goal (DO2):

Capacity strengthened to protect and

improve health and well- being

DO 2. PM6: Number of persons with disabilities who have received

direct assistance provided by USAID-funded programs (disaggregated

by sex, level of disability, and province)

DO 2. PM7: Number of USG-assisted organizations and/or service

delivery systems strengthened that serve vulnerable populations

(disaggregated by type: government institutions, registered NGOs,

non-registered NGOs, private sector)

Project Purpose (I.R.2.3.):

Expanded opportunities for

vulnerable populations

IR 2.3. PM1: Number of GVN laws, policies, or procedures drafted,

revised, and/or issued to support inclusion of vulnerable populations

IR 2.3. PM2: (or PPR 3.3.2-13): Number of service providers

(individuals) trained who serve vulnerable persons (disaggregated by

sex and province)

IR 2.3 PM3: Level of satisfaction among male and female persons with

disabilities receiving social service or prosthetics and orthotics (P&O),

ST in targeted areas

Sub-Purpose (Sub-I.R. Disability):

Persons with disabilities have

expanded opportunities through

higher quality and sustainable

services, awareness, and advocacy

strengthening

DMP 1: Percentage of beneficiaries who report increased

independence as a result of the project

DMP 2: Number of persons with disabilities with increased social

participation, and educational and working opportunities

(disaggregated by sex, level of disability, and province)

Output 1:

Disability rights, policies, advocacy

and coordination fostered



(or an indicator on implementation)

IR 2.3. PM2: (or PPR 3.3.2-13) Number of service providers


sex and province)

DMP 3: Percentage of persons with disabilities in targeted geographic

areas who experience discrimination (disaggregated by sex, level of

disability, and province)

DMP 16: Number of buildings/structures with newly included

accessibility features

DMP 13: Percentage of GVN officials who are aware of disability

rights, laws and policies (disaggregated by sex, province and

administrative level)

DMP G2: Number of people trained in mitigating gender-based

violence (individuals should be disaggregated by sex, level of disability,

and province)

1.1

Enforce national disability policies,

disability information system and

inter-agency cooperation

DMP 4: Number of GVN units reporting to and using data from the

Disability Information System (DIS) (disaggregated by province)



1.2 DMP 5: Number of recommended actions under Provincial Action

Plans (PAPs) implemented

Support the development and

implementation of provincial action

plans on disabilities

DMP 6: Number of persons with disabilities who have benefited from

PAPs (disaggregated by sex, severity of disability, and province)

DMP 7: Number of provincial action plans on disabilities developed to

promote gender equality (e.g., to address specific needs of males and

females with disabilities)



1.3.

Strengthen advocacy activities of

persons with disabilities organizations

and their representatives (such as

parents)




DMP 8: Number of disability advocacy messages/plans proposed by

national NGOs/DPOs reflected in PAPs (disaggregated by province)

DMP 9: Number of DPOs and PAs participating in monitoring and

evaluations of local (provincial) program for persons with disabilities

DMP G1: Percentage of males and females who provide home- based

cares to persons with disabilities (disaggregated by province)


violence (disaggregated by sex, severity of disability, and province)

Output 2

Direct Assistance - quality POS

services delivered to persons with

disabilities in focus provinces

DO2. PM6: Number of persons with disabilities receiving direct

assistance provided by USAID-funded programs (disaggregated by

sex, severity of disability, and province).



sex, level of disability, and province)

2.1.

POS services provided to persons

with disabilities to strengthen

independent living skills, learning,

working, and other social functions

(including screening, early detection

and early intervention)


independence as a result of the project (disaggregated by sex, level of




disabilities and province)

2.2.

Provide services to families and

caregivers




DMP 10: Percentage of persons trained using learnt skills after 1 year

(disaggregated by sex, level of disability, and province)




violence (disaggregated by sex, level of disability, and province)

2.3.

Provide quality and affordable

assistive devices

DO 2. PM6: Number of persons with disabilities who have received

direct assistance provided by USAID-funded programs (disaggregated

by sex, level of disabilities and province).


independence as a result of the project (disaggregated by sex, level of

disabilities and province)

DMP 11: Number of devices produced locally by USAID-supported

innovation

61

2.4.

Enhance service information

DMP 12: Percentage of persons with disabilities and caregivers who

are aware of availability of services (what, when and how to access)

(disaggregated by sex and province)

DMP 13: Percentage of GVN officials aware of disability rights, laws

and policies (disaggregated by sex and province)

Output 3

Improved local

capacity to provide quality POS

services

IR 2.3. PM2 (or PPR 3.3.2-13): Number of service providers


sex, level of disabilities and province)

DMP 10: Number of persons trained using learnt skills after 1 year





3.1.

Develop POS training programs for

practitioners and technicians

DMP 14: Number of institutional POS training programs

updated/developed (disaggregated by province)

3.2.

Provide equipment and improve

technology for POS facilities at

district and lower levels

DMP 15: Number of facilities equipped (disaggregated by province)

3.3.

Advocate policies to increase

availability and quality of POS services



3.4.

Establish effective referral system to

connect with other health, education,

and employment services (case

management network)

Number of persons with disabilities who received education, social,

and employment support as a result of referral system (disaggregated

by sex, level of disability, and province)

Number of persons with disabilities who received POS services as a

result of case management and social services (disaggregated by sex,

level of disabilities and province)

Number of case managers/social workers trained at MOLISA

commune level (disaggregated by sex)

Number of CM/SW supervisors trained at MOLISA district level

(disaggregated by sex)

Number of CM/SWs in medical setting trained (disaggregated by sex)

3.5.

Improve support from DPOs, PA and

local NGOs to POS service

IR 2.3. PM2 (or PPR 3.3.2-13): Number of service providers



Cross-cutting: Gender DMP G1: Percentage of male and female who provide home- based




Impact on the lives of Beneficiaries Percentage of beneficiaries reported positive changes in their daily

lives since receiving services

63

ANNEX III: DATA COLLECTION INSTRUMENTS

QUESTIONNAIRE FOR PERSONS WITH DISABILITIES

Province Code District Code Commune Code Patient Code

Gender:

Male 1

Female 2

Age

Contact information (Name and telephone number of the MAIN caregiver):

...........................................................................................................................................................................................

...........................................................................................................................................................................................

...........................................................................................................................................................................................

...........................................................................................................................................................................................

...........................................................................................................................................................................................

...........................................................................................................................................................................................

...........................................................................................................................................................................................

INTRODUCTION

Hello, my name is ………. We are collaborating with Department of Health to conduct a survey on

persons with disabilities. We would like to collect information on how you feel about your life,

including your health, and needs of assistance in your daily life. We would like to also record your

opinion on the services/interventions or other assistance you have received. There is no guarantee

that you will receive the services mentioned in the interview; however, the information you provide

will be very helpful for people who are designing the program supporting persons with disabilities in

your province. We randomly selected you and your household, and about 300 other persons with

disabilities in your province, to participate in this survey, based on the list provided by the local

authorities.

We will ask you questions about yourself. The interview will take about 30- 45 minutes to complete.

Your participation is entirely voluntary, and you can stop the interview whenever you want.

All your personal information will be kept confidential. Your name, address and any other personal

information will not be shown and presented in any report. Very few people who are responsible for

the survey can access the database where your information will be stored.

In appreciation of your contribution and your time, you will receive 100,000 VND at the end of the

interview.

Are you willing to participnate in the survey?

Yes 1 Proceed with the interview

No 2 Ask for reason and stop the interview

Reasons: ……………………………………………………………………………

65

PART I: SOCIO-DEMOGRAPHIC INFORMATION

NO. QUESTION RESPONSE SKIP

LOGIC

1 Ethnic group? Kinh

Khmer

Cham

Pako

Ta Oi

Katu

Van Kieu

Other (please specify)

……………………………………

1

2

3

4

5

6

7

8

2 Level of education:

What is the highest level of

schooling that you have

completed?

(Select the highest level of

schooling completed. For ex,

if the respondent completed

grade 7, select Primary

school)

Illiterate

Do not attend school

No qualification

Primary school

Lower secondary school

High school

College/university and higher

1

2

3

4

5

6

7

Q,4

3 What type of school have

you attended?

Special school

Regular school

1

2

4 What is your household’s

living standard as officially

classified by the

Government?

Poor

Near poor

Non-poor

1

2

3

5 Do you have a disability

certificate?

Yes

No

I don’t know

1

2

3

6 What is your degree of

disability severity?

Very severe

Severe

Mild

Unidentified

1

2

3

4

7 When was your disability

identified?

Indicated at birth

1 year ago

2-3 years ago

3-4 years ago

Over 5 years ago

1

2

3

4

5


LOGIC

8 What is your current

employment status?

(full-time: work 8 hours/day

or more; part-time: work 4

hours/day or less)

I have a job to generate income

I do not work

1

2

9 Do you have monthly

income?

Yes

No

1

2

Q.12

10 What is your major source

of income?

(Multiple choice)

Social allowance

Employment

Family support (from relatives, children,

grandchildren)

Savings

Pension


……………………………………

1

2

3

4

5

6

11 To which level does your

income cover your personal

needs?

Abundant

Sufficient

Slightly insufficient

Significantly insufficient

1

2

3

4

12 Do you have an active

health insurance card?

Yes, I have an active health insurance card

No, I don’t

I don’t know

1

2

3

Q. 14

Q. 14

13 In the past year, did you use

your health insurance card?

Yes

No

I don’t remember

1

2

3

14 Which member of your

family is regularly taking care

of you?

Father

Mother

Elder brother

Elder sister

Grandfather

Grandmother

Wife

Husband

I don’t need a care giver

Other (specify relationship and gender of the

caregiver)

……………………………………

1

2

3

4

5

6

7

8

9

10

15 In your opinion, what is the

role of male and female

members in your family in

giving care to person with

disability?

In my family, male and female members are

equally involved in the care-giving

I want male members to be more involved

I want female members to be more involved

My family has no other male/female member

1

2

3

67


LOGIC

4

16 Are you aware of the

existence of any DPO in

your locality?

Yes

No

1

2

Part II

17 Are you a member of a local

DPO?

Yes

No

1

2

18 Have you received any

service(s)/assistance from a

local DPO?

Yes

No

1

2

Part II

19 What assistance did you

receive?

(Multiple choice)

Provision of legal knowlege

Opportunities to share with others

Referral to medical examination

Assistive device


……………………………………

1

2

3

4

5

PART II: LEVEL OF SATISFACTION


LOGIC

As far as we know, you have received some assistance to improve your health conditions. We would like

to ask you about that assistance.

1 Do you think your

condition can be improved?

Absolutely possible

Partially possible

Absolutely impossible

I don’t know

1

2

3

4

2 What assistance(s) have

you received in the past 2

years?

[MULTIPLE CHOICE]

Home-based rehabilitation

Hospital-based rehabilitation

Assistive device (wheelchairs, canes, etc.)

Legal consultancy

Toilet improvement

1

2

3

4

5

Sect. A

Sect. A

Sect. B

Sect. C

Sect. D

A-FOR RESPONDENTS WHO RECEIVED REHABILITAION SERVICES (HOME-BASED

AND HOSPITAL BASED) - THOSE WHO CHOSE OPTION 1 OR 2 IN QUESTION 2

1.1 How often do you receive

guidance for rehabilitation

exercises from health

workers?

Every day

Every week

Every month

Once every 2-3 months

Less often

Never

1

2

3

4

5

6


LOGIC

1.1a Are your family members

instructed to help you

practice the exercises?

Yes

No

I’m not sure/I don’t know

1

2

3

1.2 Are the exercises suitable

for you?

Absolutely suitable

Unsuitable (too difficult, too heavy, painful)

Mixed

1

2

3

1.3 How often do you practice

the exercises as guided by

health workers?

3 times/day or more

1-2 times/day

Less than 1 time/day

I don’t practice at all

1

2

3

4

Q. 1.5

Q. 1.5

Q. 1.4

Q. 1.4

1.4 If you only practice less

than 1 time/day, what is the

reason?

(Multiple choice)

I don’t have time

I don’t have anybody to help me

The exercises are not suitable

The exercises are unnecessary (impossible to

recover from my condition)

Other (specify)

………………………………………….

1

2

3

4

5

1.5 How do you rate the

quality of the guidance for

rehabilitation exercises that

you received?

(Use showcard 5)

Very useful

Relatively useful

Neutral (I’m fine with or without the

guidance)

Not very useful

Not useful at all

1

2

3

4

5

1.6 Did the rehabilitation

exercises meet your needs?

Fully met my needs

Partly met my needs

Did not match my needs

1

2

3

1.7 What do you think about

the health workers who do

medical examination and

guide you to practice

rehabilitation exercises?

Very good

Good

Neither poor nor good

Mixed

Not good at all

1

2

3

4

5

1.8 How satisfied are you with

the quality of the guidance

for rehabilitation exercises

that you received?

(Use showcard 2)

Very satisfied

Satisfied

Neither satisfied nor dissatisfied

Dissatisfied

Very dissatisfied

1

2

3

4

5

69


LOGIC

1.9 Will you continue to

practice the rehabilitation

exercises?

Yes

No

I don’t know yet

1

2

3

B-FOR RESPONDENTS WHO RECEIVED ASSISTIVE DEVICE (WHEELCHAIR,

SUPPORT BELTS, ETC.) - THOSE WHO CHOSE OPTION 3 IN QUESTION 2

2.1 Have you ever possessed a

similar assistive device?

No, I haven’t

Yes, I have

1

2

Q. 2.3

Q. 2.2

2.2 If you had a similar assistive

device before, why do you

need a new one?

(Multiple choice)

The device is old

The old device was broken

The old device was unsuitable

Health worker recommended I use new

device

Other (specify) ……………………….

1

2

3

4

5

2.3 Did you receive any

instruction from health

workers when receiving the

new device?

Yes

No

I don’t remember

1

2

3

2.4 Did health workers come

to provide support and

instructions to you in the

process of using the device?

Yes

No

I don’t remember

1

2

3

2.5 Do you use the device

regularly?

Always

Sometimes

Rarely

Never

1

2

3

4

Q. 2.7

2.6 If you don’t use your

equipment regularly, what

is the reason?

(Multiple choice)

The device is not suitable for persons with

disabilities

(too big, too small, to difficult to use, etc.)

I don’t need to use the device regularly

I don’t have anybody to help me


………………………………………….

The device is broken

I feel pain when using the device

1

2

3

4

5

6

2.7 Without the device, what

difficulty/difficulties do you

encounter?

I have difficulty when lying

I cannot or have difficulty sitting

I cannot or feel difficulty standing

I cannot or have difficulty moving around

1

2

3

4


LOGIC

(Multiple choice) I can only go around inside my house

I have difficulty doing daily activities (e.g.,

toilet use, eat and drink)

I feel self-conscious about my appearance

Other (Specify):

….…………………………………….

5

6

7

8

2.8 How does the device you

received help you in your

life?

(Multiple choice)

It does nothing to my life

I can sit

I can move around inside my house

I can go around more and further

I feel more comfortable doing daily activities

I feel more confident about my appearance


………………………………………

1

2

3

4

5

6

7

Q. 2.9.

Do not

choose any

other

options


quality of the assistive

device that you received?

(Use showcard 5)

Very useful

Relatively useful


assistance)

Not very useful

Not useful at all

1

2

3

4

5

2.10 Did the device meet your

needs?

Fully met my needs

Partly met my needs


1

2

3


the health workers who

have conducted medical

examination and provided

the assistive device to you?

Very good

Good


Mixed

Not good at all

1

2

3

4

5


the device that you

received?

(Use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

2.13 Will you continue to use

the device?

Yes

No

I don’t know yet

Other (specify)

1

2

3

4

71


LOGIC

…………………………………………..

2.14 Do you know where to go

for repair of the deveice

that you received

Yes

No

Other (specify)

…………………………………………..

1

2

3

C-FOR RESPONDENTS WHO RECEIVED LEGAL CONSULTANCY - THOSE WHO CHOSE

OPTION 4 IN QUESTION 2

3.1 How many times have you

received legal consultancy

services (on the phone or in

person) over the past 2

years?

Number of times [__|__]

3.2 How does the legal

consultancy help you?

[Multiple choice]

My problems are addressed and tackled

I gain more knowledge about the legal

framework

I feel more confident

Other (specify)

…………………………………………..

1

2

3

4


quality of the legal advice

that you received?

(Multiple choice)

Very useful

Relatively useful


assistance)

Not very useful

Not useful at all

1

2

3

4

5

3.4 Did the assistance meet

your needs?

Fully met my needs

Partly met my needs


1

2

3


the staff who provided legal

advice to you?

Very good

Good


Mixed

Not good at all

1

2

3

4

5


the legal consultancy

service that you received?

(Use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5


LOGIC

3.7 Will you continue to seek

legal assistance?

Yes

No

I don’t know yet

1

2

3

Q. 3.9

Q. 3.8

Q. 3.9

3.8 If not, why? My problems have been solved

Such assistance is not useful

Other (specify):

….……………………………………….

1

2

3

3.9 Are you willing to refer

other persons with

disabilities to legal

consultancy services when

they have a need?

Yes

No

Not sure

1

2

3

D-FOR RESPONDENTS WHO RECEIVED TOILET IMPROVEMENT ASSISTANCE -

THOSE WHO CHOSE OPTION 5 IN QUESTION 2

4.1 How does the improved

toilet help you in your life?

(Multiple choice)

It is more convenient for me

I can use the toilet by myself without

assistance

It is safer for me

It helps me keep my personal hygiene better

Other (specify)

…………………………………………..

1

2

3

4

5


quality of this assistance?

(Use showcard 5)

Very useful

Relatively useful


assistance)

Not very useful

Not useful at all

1

2

3

4

5

4.3 Did the improved toilet

meet your needs?

Fully met my needs

Partly met my needs


1

2

3


the assistance that you

received?

(Use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

73

PART III: CHANGES IN QUALITY OF LIFE

NO. QUESTION RESPONSE

1 How would you rate your health status?

(use showcard 1)

Very good

Good


Poor

Very poor

1

2

3

4

5

2 How satisfied are you with your current health

status?

(use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

3 How would you rate your quality of life?

(use showcard 1)

Very good

Good


Poor

Very poor

1

2

3

4

5

4 How satisfied are you with your quality of life?

(use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

5 Did the assistance you received help you improve

your quality of life?

(use showcard 3)

Not at all

A little

A moderate amount

Very much

A huge amount

1

2

3

4

5

6 Did the assistance you received help you improve

your health status?

(use showcard 3)

Not at all

A little

A moderate amount

Very much

A huge amount

1

2

3

4

5

PART IV: CHANGES IN LEVEL OF INDEPENDENCE


1a In the past (before receiving assistance), did

you have difficulty eating or drinking?

I was unable to eat or drink by myself

I needed help from another person or

needed to change how I eat or drink

I could eat or drink by myself

0

5

10

1b Currently do you have difficulty eating or

drinking?

I am unable to eat or drink by myself

I need help from another person or

need to change how I eat or drink

I can do eat or drink by myself

0

5

10


you have difficulty bathing?

I needed help from another person

(dependent)

I could do it by myself (independent)

0

5

2b Currently do you have difficulty bathing? I need help from another person

(dependent)

I can do it by myself (independent)

0

5


you have difficulty taking care of your face,

hair, brushing your teeth, or shaving?

I needed help with personal care


0

5

3b Currently do you have difficulty taking care

of your face, hair, brushing your teeth, or

shaving?

I need help with personal care


0

5


you have difficulty getting dressed?

I could not do it by myself (dependent)

I needed help but could do about half

unaided


0

5

10

4b Currently do you have difficulty getting

dressed?

I can not do it by myself (dependent)

I need help but can do about half

unaided


0

5

10


you have difficulty with bowel movements?

I always had difficulty (incontinent)

I had occasional accidents

I could control my bowel movements

(continent) to call for help

0

5

10

5b Currently do you have difficulty with bowel

movements?

I always have difficulty (incontinent)

I have occasional accidents

0

5

10

75


I can completely manage my bowel

movements (continent)


you have difficulty with bladder control?

I always had difficulty (incontinent)

I had occasional accidents

I could completely manage bladder

control (continent)

0

5

10

6b Currently do you have difficulty with bladder

control?

I always have difficulty (incontinent)

I have occasional accidents

I can completely manage bladder

control (continent)

0

5

10


you have difficulty using the toilet?

I could not go to the toilet

I needed some help, but could do some

of it alone

I could go to the toilet by myself

0

5

10

7b Currently do you have difficulty using the

toilet?

I cannot go to the toilet

I need some help, but can do some of it

alone

I can go to the toilet by myself

0

5

10


you have difficulty transferring from bed to

chair and back?

I could not do it by myself (no sitting

balance)

I needed a lot of help

I needed a little bit of help


0

5

10

15

8b Currently do you have difficulty transferring

from bed to chair and back?

I cannot do it by myself (no sitting

balance)

I need a lot of help

I need a little bit of help


0

5

10

15


you have difficulty moving (on level

surfaces)?

I was immobile or could move < 50m

I was wheelchair independent and

could move > 50m

I could walk with help of one person >

50m

I could move (may use aid such as

stick) > 50m by myself (independent)

0

5

10

15

9b Currently do you have difficulty moving (on

level surfaces)?

I am immobile or can move < 50m

I am wheelchair independent and can

move > 50m

I can walk with help of one person >

50m

0

5

10


I can move (may use aid such as stick)

> 50m by myself (independent)

15


you have difficulty climbing stairs?

I was unable to do it

I needed help


0

5

10

10b Currently do you have difficulty getting

dressed?

I am unable to do it

I need help


0

5

10

11 Since receiving assistance, what do you think

about the need for help from your family

members in your daily life?

My family members do not need to

help me as before

I still need my family members to help

me, but less than before

No change compared to the past

1

2

3

PART V: ENABLING ENVIRONMENT


LOGIC

1 Are you aware of the Vietnam

Law on Disability?

Yes

No

1

2

Q. 4

2 If yes, where did you hear about

it?

(Multiple choice: Enumerator lets

respondents speak by themselves

and selects the respective option)

Local loud speaker system

Mass media (TV, newspaper, etc.)

Village/commune meeting

Government staff at village and

commune level

DPOs

Non-governmental organizations

Family members

Friends and neighbors

Other (specify)

………………………………

1

2

3

4

5

6

7

8

9

3

Do you know the contents of the

Vietnam Law on Disability?

I don’t know about the content

I know some of it

I know a fair amount of it

I fully understand the contents

1

2

3

4

4 Are you aware of the rights of

persons with disabilities?

Yes

No

1

2

Part VI

77


LOGIC

5 What right(s) of persons with

disabilities are you aware of?




Right to equality and non-

discrimination

Right to access (accessibility)

Right to liberty and personal

security

Right to live independently and

being included in the community

Right to education

Right to healthcare

Right to work

Right to participate in political and

public life

Right to participate in cultural life,

recreation, leisure and sport


………………………………

1

2

3

4

5

6

7

8

9

10

6 From which source(s) do you get

information about the rights of









commune level

DPOs


Family members


Other (specify)

………………………………

1

2

3

4

5

6

7

8

9

PART VI: SOCIAL INCLUSION


LOGIC

1 Since receiving assistance, have

you had opportunities to go

out in the community more

often than before?

More often

Unchanged

Less often

1

2

3


LOGIC

1b

Since receiving assistance, have

you had more opportunities to

go out and participate in

activities in your village

(visiting friends and relatives,

joining in community activities,

etc.) more often than before?

More often

Unchanged

Less often

1

2

3

Q. 3

Q. 3

2 If yes, how did other people

treat you?

(Multiple choice)

They cared for me more

They helped me more

They talked to me more

Unchanged


………………………………

1

2

3

4

5

Q. 3. Do

not choose

any other

options


you found yourself accepted

and cared for by your family

members more than before?

(Use showcard 4)

Much more

A little more

Unchanged

Less

Much less

1

2

3

4

5


you helped with household

chores (cleaning, cooking,

etc.)?

(Use showcard 4)

Much more

A little more

Unchanged

Less

Much less

1

2

3

4

5


you felt what you do is valued

more by others?

(Use showcard 4)

Much more

A little more

Unchanged

Less

Much less

1

2

3

4

5

6 Since receiving assistance,

have you felt more confident

(to express yourself, go out or

do something)?

(Use showcard 4)

Much more

A little more

Unchanged

Less

Much less

1

2

3

4

5

79


LOGIC

7 If you have the need for

education (vocational

training), how do you rate the

change in your learning ability

since receiving assistance?

Better

Unchanged

I do not have the need for education

1

2

3

8 If you have the need for

employment, how do you rate

the change in your ability to

find a job since receiving

assistance?

Better

Unchanged

I do not have the need for

employment

1

2

3

9 Have you faced any

discrimination?

Discrimination: the acts of

treating a person with a

disability differently (often in a

worse way than the way in

which other people are

treated).

Persons with disabilities face more

discrimination

Unchanged

Persons with disabilities still face

discrimination, but less than before

No discrimination against persons

with disabilities

Persos with disabilities have never

faced discrimination

1

2

3

4

5

10 Have you noticed any changes

in the discrimination against

persons with disabilities

compared with the past?

Unchanged



No discrimination against persons

with disabilities

Persos with disabilities have never

faced discrimination

I don’t know

1

2

3

4

5


you felt more included in the

daily life?

Yes

No

I don’t know or I don’t understand

inclusion concept

1

2

3

Part VII

Part VII

12 If yes, why do you think that

you are more included?

(Multiple choice)

I can go out socially with other

people

I can do things I wanted but couldn’t

do before

My family members are happier

Members of the community treat me

better

1

2

3

4


LOGIC

I face discrimination less often than

before

Other (specify)

………………………………

5

6

13 Did you notice any gender-

related risks (gender based

violence, sexual abuse) faced

by persons with disabilities in

your community?

Yes

No

I don’t know

1

2

3

Part VII

Part VII

14 If yes, how have the gender-

related risks faced by persons

with disabilities changed in the

past 2 years?

Unchanged

Much less

More common

1

2

3

15 Do you know any measure to

prevent the gender-related

risks faced by persons with

disabilities?

Yes

No

1

2

Part VII

16 If yes, please specify: …………………………………….

…………………………………….

…………………………………….

…………………………………….

PART VII: SUPPORTING SERVICES FOR PERSONS WITH

DISABILITIES

2. We are going to name some services for persons with disabilities; please tell us if you know

where each service is provided. (at commune, district or provincial level)

1a. Do you

know

where the

service is

provided?

1b. Can you

access and use

the service?

1c. Compared with the

past, have you noticed any

changes in accessibility to

the service?

81

Yes

No

Yes

No

Not su

re

Unch

ange

d

Min

or ch

ange

More

accessib

le

I don’t k

now

2 Skip

2b, 2c

Monthly social allowance 1 2 1 2 3 1 2 3 4

Regular/Routine health check-up 1 2 1 2 3 1 2 3 4

Specialized health check-up for


1 2 1 2 3 1 2 3 4

Provision of assistive devices 1 2 1 2 3 1 2 3 4

Home-based rehabilitation 1 2 1 2 3 1 2 3 4

Rehabilitation at commune health

center

1 2 1 2 3 1 2 3 4

Legal assistance (e.g., identification

of disability status)

1 2 1 2 3 1 2 3 4

NO. QUESTION

3 Have you noticed any

changes in the ability to

come and get inside public

area by persons with

disabilities compared with

the past?

(Public area: People’s

committee, hospital, health

center, community house,

etc.)

No improvement. But persons with disabilities can get

inside the area

No improvement. Persons with disabilities have

difficulty/are unable to get inside

Minor improvements but still difficult for persons with

disabilities

Major improvements, persons with disabilities can get

inside easily

No comment/I do not go to public areas

1

2

3

4

5

4 Have you noticed any

changes in the ability to use

public transportation by



No improvement. But persons with disabilities can use

public transportation


difficulty/are unable to use public transportation


disabilities

Major improvements, persons with disabilities can use

public transportation easily

No comment/I do not use public transportation

1

2

3

4

5

FORM B: QUESTIONNAIRE FOR CAREGIVERS OF CHILDREN WITH DISABILITIES

[__] [__] [__]

Province Code District Code Commune/Ward Code Child Code

Name of the child:

Gender of the child:

Male 1

Female 2

Age of the child [__|__]

Are you the main caregiver of the child?

Yes

No Stop the interview

What is your relationship with the child?

Mother 1

Father 2

Elder brother 3

Elder sister 4

Grandfather 5

Grandmother 6

Other (specify relationship and gender) 7

…………………………………………..

Contact information (Name of person to contact, telephone #, etc.):

...........................................................................................................................................................................................

...........................................................................................................................................................................................

...........................................................................................................................................................................................

............................................................................................................................................................................................

83

INTRODUCTION

Hello, my name is ………. We are conducting a survey on children with disabilities or their

caregivers. We would like to collect information on how persons with disabilities feel about their

life, including their health, and level of satisfaction with the assistance they received from the project

conducted by USAID, Department of Health, Department of Education and Training, VNAH,

VietHealth, and IC.

We will ask you questions about your child. The interview will take about 40- 45 minutes to

complete. Your participation is entirely voluntary, and you can stop the interview whenever you

want.

All personal information about the child will be kept confidential. Name, address and other personal

information will not be shown or presented in any report. Very few people who are responsible for

the survey can access the database where the information will be stored.

0. Are you willing to join the survey?

Yes Proceed with the interview

No Ask for reason and stop the interview

Reasons: ……………………………………………………………………………

PART I. SOCIO-DEMOGRAPHIC INFORMATION

A. INFORMATION OF CAREGIVER

NO

.

QUESTION RESPONSE SKIP

LOGIC

1 Ethnic group? Kinh

Co Tu

Van Kieu

Khmer

Cham


……………………………

1

2

3

4

5

6

2 What is the highest level of schooling

that you have completed?

(Select the highest level of schooling

completed. For example, if the respondent

completed grade 7, select Primary school)

Illiterate

Do not attend school

No qualification

Primary school

Lower secondary school

High school

1

2

3

4

5

6

3 What is your household’s living

standard as officially classified by the

Government?

Poor

Near poor

Non-poor

1

2

3

4 How many people are there in your

household?

[__|__]

5 Are you currently employed? Yes, part-time

Yes, full-time

Yes, I am working on our

family business

No, I am not working

I work when I have free time

1

2

3

4

5

85

B. INFORMATION OF THE CHILD WITH DISABILITY

NO

.

QUESTION RESPONSE SKIP

LOGIC

8 What is the child’s form(s) of disability?

(Multiple choice)

Mobility and movement disability

Visual disability

Hearing/speech disability

Intellectual disability

Unidentified


..................................................

1

2

3

4

5

6

9 What is the child’s degree of disability

severity?

Very severe

Severe

Mild

Unidentified

I don’t know, I’m not sure

1

2

3

4

5

10 What is the cause of the child’s

disability?

(Multiple choice)

Indicated at birth

After a severe disease

Accident

Agent Orange (dioxin)


………………………………

Unidentified

1

2

3

4

5

6

11 Have you received an activated health

insurance card for the child??

Yes

No

I don’t know

1

2

3

PART II: LEVEL OF SATISFACTION


LOGIC

As far as we know, the child has received some assistance to improve his/her health conditions and life

in general. We would like to ask you about that assistance.

1 Do you think the child’s

conditions can be

improved?

Absolutely possible

Partially possible

Absolutely impossible

I don’t know

1

2

3

4

2 What assistance has the

child received in the past 2

years?

(Multiple choice)

Home-based rehabilitation by caregivers

Hospital-based rehabilitation

Assistive device or guidance to make assistive

device (wheelchairs, canes, etc.)

Educational intervention by kindergarten

teachers

Referral to health examination

Toilet improvement

1

2

3

4

5

6

Sect. A

Sect. A

Sect. B

Sect. C

Sect. D

Sect. E

A-FOR CHILDREN WHO RECEIVED REHABILITATION SERVICES (HOME-BASED

AND HOSPITAL-BASED) - THOSE APPLICABLE FOR OPTION 1 OR 2 IN QUESTION 2

1.1 Before receiving guidance

for rehabilitation exercises,

do you know how to help

you child practice the

exercises?

Yes

No

1

2

1.2 Before receiving guidance

or assistance, did you help

your child practice


Regularly

Occasionally

Rarely

Never

1

2

3

4

1.3 How often do you receive

guidance for rehabilitation

exercises from health

workers?

Once in every 1-2 months



Less often

Never

1

2

3

4

5

1.4 Are the exercises suitable

for the child?

Absolutely suitable

Unsuitable (too difficult, too heavy, painful)

Mixed

1

2

87


LOGIC

3

1.5 How often does the child

practice the exercises as

guided by health workers?

3 times/day or more

1-2 times/day

Less than 1 time/day

He/She does not practice at all

1

2

3

4

Q. 1.7

Q. 1.7

1.6 If the child only practices

under 1 time/day, what is

the reason?

[MULTIPLE CHOICES]

The child feels pain and does not want to

practice

The child does not have anybody helping

him/her

The exercises are not suitable

The exercises are unnecessary (impossible to

recover from the disability)

Other (specify)

………………………………………….

1

2

3

4

5


the training classes which

provided instruction on

how to help the child

practice rehabilitation

exercises?

(Use showcard 5)

Very useful

Relatively useful

Neutral (The child is fine with or without the

assistance)

Not very useful

Not useful at all

1

2

3

4

5

1.8 Did the training classes

meet your needs to help

the child practice


Fully met my needs

Partly met my needs

Did not match my need

1

2

3


the benefits of

rehabilitation exercises to

your child?

(Use showcard 5)

Very useful

Relatively useful


assistance)

Not very useful

Not useful at all

1

2

3

4

5


the health workers who do

medical examination and

instruct you to practice

rehabilitation exercises

with the child?

Very good

Good


Mixed

Not good at all

1

2

3

4

5


LOGIC


the quality of the guidance

for rehabilitation exercises

for the child?

(Use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

1.12 Will you continue to help

the child practice


Yes

No

I don’t know yet

1

2

3

B-FOR CHILDREN WHO RECEIVED AN ASSISTIVE DEVICE (WHEELCHAIR, SUPPORT

BELTS, ETC.) - THOSE APPLICABLE FOR OPTION 3 IN QUESTION 2

2.1 Has the child ever used a

similar assistive device?

No

Yes

1

2

Q. 2.3

2.2 If yes, why does the child

need a new one?

The device is old

The old device was broken

The old device was unsuitable

Health worker recommended to use new

device

Other (specify) ……………………….

1

2

3

4

5

2.3 Did you receive any

instruction from health

workers when receiving the

new device?

Yes

No

1

2

2.4 Did health workers come

to provide support and

instructions to you in the

process of using the device?

Yes

No

I don’t remember

1

2

3


use the device?

Always

Occasionally

Rarely

Never

1

2

3

4

Q. 2.7

2.6 If your child does not use

the equipment regularly,

what is the reason?

(Multiple choice)

The device is not suitable for the child (too

big, too small, to difficult to use, etc.)

The child does not need to use the device

regularly

The child does not have anybody to help

him/her use it

1

2

3

89


LOGIC


………………………………………….

4

2.7 Without the device, what

difficulty does the child

encounter?

[MULTIPLE CHOICES]

The child cannot sit or has difficulty sitting

The child cannot stand or has difficulty

standing

The child cannot move around or have

difficulty moving around

The child can only move around inside the

house

The child cannot do or has difficulty doing

daily activities (e.g., toilet use, eat or drink)

The child feels self-conscious about his/her

appearance

Other (specify):

….…………………………………….

1

2

3

4

5

6

7

2.8 How does the device

received help the child in

his/her life?

It does nothing to his/her life

The child can sit

The child can move around inside the house

The child can go around more and further

The child feels more comfortable doing daily

activities

The child feels more confident about his/her

appearance


………………………………………

1

2

3

4

5

6

7


quality of the assistive

device that the child

received?

(Use showcard 5)

Very useful

Relatively useful


assistance)

Not very useful

Not useful at all

1

2

3

4

5

2.10 Did the device meet the

child’s needs?

Fully met his/her needs

Partly met his/her needs

Did not match his/her needs

1

2

3


the health workers who

have conducted medical

examination and provided

Very good

Good


1

2

3


LOGIC

the assistive device to the

child?

Mixed

Not good at all

4

5


the device that the child

received?

(Use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

2.13 Will you continue to help

the child use the device?

Yes

No

I don’t know yet

Other (Specify)

………………………………………

1

2

3

4

C-FOR CHILDREN WHO RECEIVED AN EDUCATIONAL INTERVENTION - THOSE

APPLICABLE FOR OPTION 4 IN QUESTION 2


receive support from

his/her teachers?

Every day

Every week (2-3 times/week)

Every month (4 times/month or less)


Less often

1

2

3

4

5

3.2 Did the teachers instruct

you to teach your child at

home?

Yes

No

I don’t remeber

1

2

3

3.3 How often do you teach

the child as instructed by

the teachers?

Every day

Every week (2-3 times/week)

Every month (4 times/month or less)


Less often

Never

1

2

3

4

5

6


support provided by the

teachers?

(Use showcard 5)

Very useful

Relatively useful


assistance)

Not very useful

Not useful at all

1

2

3

4

5

91


LOGIC

3.5 Does the support help

improve the child’s

conditions?

Much better

Somehow better

Unchanged

1

2

3


his/her teachers?

Very good

Good


Mixed

Not good at all

1

2

3

4

5


the support from the

teachers?

(Use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

3.8 Do you want the child to

continue to receive support

from his/her teachers?

Yes

No

I don’t know yet

1

2

3

Part III

or Section

D (those

applicable

for option 5

in Question

2)

3.9 If not, why? The child has improved very much

Such assistance is not really useful

Other (specify):

….……………………………………….

1

2

3

D-FOR CHILDREN WHO RECEIVED REFERRAL TO HEALTH EXAMINATION - THOSE


4.1 Did you take the child to

health facility for

examination when

referred?

Yes

No

1

2

Q. 4.3

4.2 If no, what was the reason?

(Multiple choice)

I did not have time

The health facility is far from my house

I don’t have money

I felt that it was unnecessary

Other (specify)

….……………………………………….

1

2

3

4

5

Q. 4.8

Q. 4.8

Q. 4.8

Q. 4.8

Q. 4.8


LOGIC

4.3 If yes, did the child receive

any treatment?

Yes

No

1

2

Q. 4.5

4.4 If no, what was the reason? The child didn’t need any treatment

The treatment was expensive

I felt that it was unnecessary

Other (specify)

….……………………………………….

1

2

3

4

Q. 4.8

Q. 4.8

Q. 4.8

Q. 4.8

4.5 Did you have to pay for

your child’s examination

and treatment?

Yes

No

1

2

4.6 Currently is your child

continuing the treatment?

Yes

No

1

2

Q. 4.8

4.7 If no, what was the reason? The treatment is expensive

The child has fully recovered

The treatment does not work

Other (specify)

….……………………………………….

1

2

3

4


referral to health

examination assistance?

(Use showcard 5)

Very useful

Relatively useful


assistance)

Not very useful

Not useful at all

1

2

3

4

5


the referral to health

examination assistance?

(Use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

E-FOR CHILDREN WHO RECEIVED TOILET IMPROVEMENT ASSISTANCE - THOSE


5.1 How does the improved

toilet help your child in

his/her life?

It is more convenient for the child

The child can use the toilet by myself without

assistance

1

2

93


LOGIC

[MULTIPLE CHOICE]

It is safer for the child

It helps the child keep his/her personal

hygiene better

Other (specify)

…………………………………………..

3

4

5


quality of this assistance?

(Use showcard 5)

Very useful

Relatively useful

Neutral (the child fine with or without the

assistance)

Not very useful

Not useful at all

1

2

3

4

5

5.3 Did the improved toilet

meet the child’s needs?

Fully met his/her needs

Partly met his/her needs

Did not match his/her needs

1

2

3


the assistance?

(Use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

PART III: CHANGES IN THE QUALITY OF LIFE


1 How would you rate the child’s health status?

(Use showcard 1)

Very good

Good


Poor

Very poor

1

2

3

4

5

2 How satisfied are you with the child’s current

health status?

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

3 How would you rate the child’s quality of life? Very good 1


(Use showcard 1)

Good


Poor

Very poor

2

3

4

5

4 How satisfied are you with the child’s quality of

life?

(Use showcard 2)

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

1

2

3

4

5

5 What do you rate the change in the child’s quality

of life compared with before receiving assistance?

Much better

Somehow better

Unchanged

Somehow worse

Much worse

1

2

3

4

5

6 What do you rate the change in the child’s health

status compared with before receiving assistance?

Much better

Somehow better

Unchanged

Somehow worse

Much worse

1

2

3

4

5

7 In general, how does the assistance affect your

family’s daily life?

Much better

Relatively better

Unchaged

Worse

Much worse

1

2

3

4

5

95

PART IV: CHANGES IN THE CHILD’S DEVELOPMENT

In the past (before receiving

assistance), what difficulties did the

child have?

(ASK CAREGIVERS TO LIST THE

DIFFICULTIES)

Does the child encounter any

difficulties?

How have the child’s conditions changed since

receiving interventions?

No difficulty - 1

A little - 2

A moderate amount - 3

A huge amount - 4

The child cannot do it - 5

Improved a huge amount (no longer has

difficulties)

Improved a lot (but still has some difficulties)

Improved a moderate amount

Improved a little (still has a lot of difficulties)

Unchanged (the child cannot do it)

1

2

3

4

5

MO

VEM

EN

TS

REQ

UIR

ING

LIT

TLE F

INESS

E Lie (lifting his/her head when she/he is

on his/her stomach), change positions

(rolling)

Flip (to the side, from his/her back to

his/her tummy and vice versa...)

Sit

Crawl

Stand

Walk

Go up and down the stairs

Run

Jump from a high position

Throw a ball

Catch a ball

Ride a bike M

OV

EM

EN

T

S REQ

UIR

ING

FIN

ESS

E

Grip with two fingers

Move objects

CO

GN

ITIO

N &

LEA

RN

ING

Stack blocks

Play jigsaw puzzle

Know how to identify big-small, long-

short, tall-short

Know how to count

Know how to add within 5

Recognize basic colors and shapes

Recognize and remember simple

numbers and letters

LA

NG

UA

GE -

CO

MM

UN

ICA

TIO

N

Play attention to sound and speech

Turn in the direction of the sound of

talking

Respond when called by name (look at

the caller)

Point to an object when she/he wants it

Understand simple commands

97

Distinguish between family members and

strangers

Say single words

Say a sentence with 2-3 words

Say a sentence with more than 3 words

Communicate with family members

Communicate with people other than

family and friends

Pronounce words

Response to questions

Recount an event

Express emotions (get angry, sulky)

BEH

AV

IOR

Lack of concentration

Lack of attention

Be hyperactive

Not maintain eye contact

Like to do things his/her own way

Throw a tantrum, scream

Be aggressive/violent

Hurt himself/herself

Speak continuously and repeatedly

Play with only one toy, object

Be shy and fearful

SELF-C

AR

E

Express toileting needs

Suck

Drink from a cup

Eat with a spoon

Chew food (soft and hard)

Go to the toilet by himself/herself

Get dressed and undressed

Wash his/her face

Put on shoes

Brush his/her teeth

Wash himself/herself after using the

toilet

Other Specify:

………………………

………………………..

1 | WHOLE-OF-PROJECT DESK REVIEW USAID.GOV

PART V: ENABLING ENVIRONMENT

NO. QUESTION RESPONSE SKIP LOGIC

1 Are you aware of the Vietnam

Law on Disability?

Yes

No

1

2

Q. 1.4

2 If yes, where did you hear about

it?



and selects the respective options)





commune level

DPOs


Family members


Other (specify)

………………………………

1

2

3

4

5

6

7

8

9

3

Do you know the content of the

Vietnam Law on Disability?

I don’t know about the content

I know some of it

I know a fair amount of it

I fully understand the content

1

2

3

4

4 Are you aware of the rights of


Yes

No

1

2

Part VI

5 What right(s) of persons with

disabilities are you aware of?

(Multiple choices. Enumerator lets



Right to equality and non-

discrimination

Right to access (accessibility)

Right to liberty and personal security

Right to live independently and being

included in the community

Right to education

Right to healthcare

Right to work

Right to participate in political and

public life

Right to participate in cultural life,

recreation, leisure and sport


1

2

3

4

5

6

7

8

USAID.GOV WHOLE-OF-PROJECT EVALUATION DESIGN | 2


……………………………… 9

10

6 From which source(s) do you get

information about the rights of









commune level

DPOs


Family members


Other (specify)

………………………………

1

2

3

4

5

6

7

8

9

PART VI: SOCIAL INCLUSION


1

Since receiving assistance, has

your child had opportunities to

go out in the community more

often than before?

More often

Unchanged

Less

1

2

3

2 Since receiving assistance, has the

child had more opportunities to

go out and participate in activities

in your village more often than

before?

More often

Unchanged

Less

1

2

3

3 Since receiving assistance, do you

think that the child play inside

your house more often than

before?

(Use showcard 4)

Much more

A little more

Unchanged

Less

Much less

1

2

3

4

5




think that members in your family

know how to play with the child

better than before?

(Use showcard 4)

Much more

A little more

Unchanged

Less

Much less

1

2

3

4

5


think that the child feel more

confident (to express

himself/herself)?

(Use showcard 4)

Much more

A little more

Unchanged

Less

Much less

1

2

3

4

5

6 What do you think about the

change in the child’s ability to

attend school?

Much better

Unchanged

The child does not need to attend

school

The child is not in school age

1

2

3

4

7 Have the child and your family

faced any discrimination because

of the child’s disability?

Discrimination: the acts of treating

a person with disability differently

(often in a worse way than the

way in which other people are

treated).

Persons with disabilities face more

discrimination

Unchanged



No discrimination against persons with

disabilities

Persos with disabilities have never faced

discrimination

1

2

3

4

5

8 Have you noticed any changes in

the discrimination against



Unchanged



No discrimination against persons with

disabilities

Persos with disabilities have never faced

discrimination

1

2

3

4




think that the child is more

included in the daily life?

Yes

No

I don’t know or I don’t understand

inclusion concept

1

2

3

Part VII

Part VII

10 If yes, why do you think that you

are more included?

(Multiple choice)

The child can go out socially with other

people

The child can do things she/he wanted

but couldn’t do before

My family members are happier

Members of the community treat

him/her better

The child faces discrimination less often

than before

Other (specify)

………………………………

1

2

3

4

5

6

11 Did you notice any gender-

related risks (gender based

violence, sexual abuse) faced by

persons with disabilities in your

community?

Yes

No

I don’t know

1

2

3

Part VII

Part VII

12 If yes, how have the gender-

related risks faced by persons

with disabilities changed in the

past 2 years?

Unchanged

Much less

More common

1

2

3

13 Do you know any measure to

prevent the gender-related risks

faced by persons with disabilities?

Yes

No

1

2

Part VII

14 If yes, please specify: …………………………………….

…………………………………….

…………………………………….

…………………………………….


PART VII: SUPPORTING SERVICES FOR PERSONS WITH

DISABILITIES

2. We are going to name some services for persons with disabilities; please tell us if you know where each

service is provided? (at commune, district or provincial level)

1a. Do

you know

where the

service is

provided?

1b. Can you

access and use

the service?

1c. Compared with

the past, do you

notice any changes

in the accessibility

to the service?

Yes

No

Yes

No

Not su

re

Unch

ange

d

Min

or

chan

ge

More

accessib

le

2 Skip

2b, 2c

Vaccinations 1 2 1 2 3 1 2 3

Monthly social allowance 1 2 1 2 3 1 2 3

Regular/Routine health check-up 1 2 1 2 3 1 2 3

Specialized health check-up for persons with

disabilities

1 2 1 2 3 1 2 3

Provision of assistive device 1 2 1 2 3 1 2 3

Home-based rehabilitation 1 2 1 2 3 1 2 3

Rehabilitation at commune health center 1 2 1 2 3 1 2 3

Legal assistance (e.g., identification of disability

status)

1 2 1 2 3 1 2 3

NO. QUESTION


in the ability to come and get

inside public area by persons

with disabilities compared

with 2 years ago?

(Public area: People’s

committee, hospital, health

No improvement. But persons with disabilities can get

inside the area


difficulty/are unable to get inside


disabilities

Major improvements, persons with disabilities can get

inside easily

1

2

3

4

5


NO. QUESTION

center, community house,

etc.)

No comment/persons with disabilities do not go to

public areas


in the ability to use public

transportation by persons

with disabilities compared

with 2 years ago?

No improvement. But persons with disabilities can use

public transportation


difficulty/are unable to use public transportation


disabilities

Major improvements, persons with disabilities can use

public transportation easily

No comment/persons with disabilities do not use public

transportation

1

2

3

4

5


QUESTIONNAIRE FOR TRAINING PARTICIPANTS

MEDICAL STAFF

Information about the interview: (to be filled in by the enumerator after the interview)

Name of enumerator: ______________________________________________

Date of interview: _________________________________________________

Questionnaire code (only used by data entry clerk)

Name of training provider (to be filled by the enumerator by checking against the provided lists)

HI VNAH Viet Health IC ACDC DRD

Part I - Introduction:

Hello Mr./Mrs. (Name of the selected respondent).

My name is (name of interviewer), a member of the mid-term evaluation team of the Disability Inclusion

Project supported by USAID in Vietnam..

You are contacted because you have (been completing) completed training courses by the project. The

interview will take maximum 15 minutes to complete. Information you provided will be kept

confidentially. Your name, address and any other personal information will not be shown and presented

in any report. Very few people, who are responsible for the survey, can access database where your

information is stored.

Are you willing to join the survey?

Yes Process the interview


Reason: ……………………………………………………………………………

__________________________________________________

General Information of respondent

1. Full name: _______________________________________________


2. Sex:

1. Male 2. Female

3. Year of birth: ____________________________

4. Where do you work in terms of geographical area

4.1. Province: ___________ 4.2. District: ___________

5. Where are you working? (type of organization)?

1. Hospital, health center at district level or

higher

2. Commune district health center

3. Private business

4. Disabled People's Organisation (DPO)

5. Union/Mass organizations of all levels (Women

Union, Youth Union, etc.)

6. Education instutions

7. Government agencies

8. I work at home

9. Other, specify _________________

6. Please select the profession that most accurately describes your work?

1. Doctor of rehab./ traditional medicine

6. General medical doctor

2. Physical therapist

7. Health worker/Nurse

3. Speech therapist 8. Gov. staff in charge of rehab

4. Occupational therapist

9. Others

(specify)______________


5. Orthopedic and

rehabilitation technician (+

physiotherapist)

7. For the training(s) you attended, what was/were the topic(s) of the training(s)

o Direct services (health and rehabilitation) to persons of disability

o Legal support services to caregivers and persons with disability

o Capacity of teaching and training (TOT +BOT)

o Awareness raising on rights of persons with disability

o Finance and management

o DPO

o Gender

o Accessibility and social inclusion

o Others

1

2

3

4

5

6

7

8

9


PART II – QUESTIONS

(read out these questions as they are and wait for the answer without any explanation. As

the respondents may not understand read the questions again. Only after that an attempt

to rephrase questions is possible).

# Question Option Instructions

8 How many training (directly

supported by VNAH, Viethealth, HI,

etc.) have you attended?

Write the number

of course.

“I don’t

remember” - 99

9 Did the training you attended provide

you with new knowledge and skills?

Yes

No

No comment

1

2

99

Ask for example

to verify the new

knowledge/skill

10 What are your daily

responsibilities? (Multiple choice)

Examine, prescribe rehab. for patients

Help patients practice rehab. exercises

Examine and treat other disease

Take care of patients

In charge of rehab. program

Other activities at health center

1

2

3

4

5

6


Health activities at village/commune

Others

No comment

7

8

99

11 Have you applied what you learned in

your work?

Yes

No

No comment

1

2

99

If “No”, skip Q 12,

Q 13

12 Can you provide one example of how

you applied what you learned from

the training(s)?

Able to recall the patient’s name

Able to recall where the skills were

applied

Able to name the disease

Able to describe the intervention(s)

Able to describe changes in patient’s

conditions

Tick the applicable

cell. If 3 out of 5

criteria are met,

choose “Yes”:

1 = Yes

2 = No

3 = Others

13 How often do you apply what you

learnt from the course(s)?

Everyday

Every week

Every month

Once in a few months

I don’t apply what I learnt

No comment

1

2

3

4

5

99


14 How do you rate the relevance of

the training contents to your daily

work?

Very relevant

Relevant

Mixed

Irrelevant

Very irrelevant

No response

1

2

3

4

5

99

15 What has prevented you from

applying the information or skills you

have learned?

(Open-ended question)

Code:

16. To what extent are you satisfied with

the training course(s)?

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

No comment

1

2

3

4

5

99

17 Please rank your level of agreement

with the following statement:

“The training(s) I have received have

resulted in more inclusion of persons with

disability in society.”

Strongly agree

Agree

Neither agree nor disagree

Disagree

Strongly disagree

No comment/I don’t know

1

2

3

4

5

99

18 Please give 1-2 reasons for your

opinion in the above question? (open-

ended question)

Code:


19 Do you need a professional license to

apply what you learnt in practice?

Yes

No

I already have a license

Not applicable

1

2

3

99

Applied for

doctors and

technical staff

only

Non applicable

– skip 19

20 Do you have a plan to apply for a new

license, or upgrade the existing one in

near future?

Yes

No

I have a license

Not applicable

Don’t know

1

2

3

4

99

21 What are some training topics that

are necessary for your work?

Code:

___________

___________


QUESTIONNAIRE FOR TRAINING PARTICIPANTS

SOCIAL ISSUE-RELATED TRAININGS

Information about the interview: (to be filled in by the enumerator after the interview)

Name of enumerator: ______________________________________________

Date of interview: _________________________________________________

Questionnaire code (only used by data entry clerk)

Name of training provider (to be filled by the enumerator by checking against the provided

lists)

1. VNAH 2. Viet Health 3. ACDC 4. DRD

Part I - Introduction:

Hello Mr./Mrs. (Name of the selected respondent).

My name is (name of interviewer), a member of the mid-term evaluation team of the Disability Inclusion

Project supported by USAID in Vietnam..

You are contacted because you have (been completing) completed training courses by the project. The

interview will take maximum 15 minutes to complete. Information you provided will be kept

confidentially. Your name, address and any other personal information will not be shown and presented

in any report. Very few people, who are responsible for the survey, can access database where your

information is stored.

Are you willing to join the survey?

Yes Process the interview


Reason: ……………………………………………………………………………

__________________________________________________

General Information of respondent

1. Full name: _______________________________________________


2. Sex:

1. Male 2. Female

3. Year of birth: ____________________________

4. Where do you work in terms of geographical area

6.1. Province: ___________ 4.2. District: ___________

1. Where are you working? (type of organization)?

1. Disabled People's Organisation (DPO)

Union/Mass organizations of all levels (Women Union,

Youth Union, etc.)

2. State management agencies

3. Health-related organization/agency (of all levels)

4. Education-related organization (schools, state

management agency in education, etc.)

5. Service provider/private business

6. Others: (Specify) _________________

6. For the training(s) you attended, what was/were the topic(s) of the training(s)

o Accessibility and social inclusion 1

o DPO development 2

o Accessibility and social inclusion 3

o Social works 4

o Gender 5

o Others 6


PART II – QUESTIONS

(read out these questions as they are and wait for the answer without any explanation. As

the respondents may not understand read the questions again. Only after that an attempt

to rephrase questions is possible).


8 How many training (directly

supported by VNAH, Viethealth,

ACDC, DRD) have you

attended?

Write the

number of

course.

“I don’t

remember” - 99

9 Did the training you attended

provide you with new

knowledge and skills? (Ask for

example to verify the new

knowledge/skill)

Yes

No

No comment

1

2

99

10 What are your daily

responsibilities? (open

question)

10a How is your work related to

support for persons with

disabilities?

11 Have you applied what you

learned in your work related to

support for persons with

disabilities?

Yes

No

No comment

1

2

99

If “No”, skip Q

12, Q 13

12 Can you provide one example

of how you applied what you

learned from the training(s)?

Able to recall the situation

Able to recall where the skills

were applied

Able to name the new

skill/knowledge applied

Able to describe how it was

applied

Able to describe changes in their

work thanks to the new

knowledge/skills

Tick the

applicable cell. If

3 out of 5

criteria are met,

choose “Yes”:

1 = Yes

2 = No

3 = Others



13 How often do you apply what

you learnt from the course(s)?

Everyday

Every week

Every month

Less often

I don’t apply what I learnt

No comment

1

2

3

4

5

99

14 How do you rate the

relevance of the training

contents to your daily work?

Rất phù hợp

Very relevant

Relevant

Mixed

Irrelevant

Very irrelevant

No response

1

2

3

4

5

99

15 What has prevented you

from applying the information

or skills you have learned?

(Open-ended question)

Code:

16 To what extent are you

satisfied with the training

course(s)?

Very satisfied

Satisfied


Dissatisfied

Very dissatisfied

No comment

1

2

3

4

5

99

16a Was there any gender-related

content in the course(s) you

attended?

Yes

No

No comment/I don’t remember

1

2

99

16b Was the gender-related content

helpful

Very helpful

Helpful

Neither helpful nor unhelpful

Unhelpful

Very unhelpful

No comment

1

2

3

4

5

99

17 Please rank your level of

agreement with the following

statement:

“The training(s) I have received

have resulted in more inclusion of

persons with disability in society.”

Strongly agree

Agree

Neither agree nor disagree

Disagree

Strongly disagree

No comment/I don’t know

1

2

3

4

5

99



18 Please give 1-2 reasons for your

opinion in the above question?

(open-ended question)

Code:

21 What are some training topics

that are necessary for your

work?

INTERVIEW GUIDE (CARE GIVER AND PERSON WITH

DISABILITY)

INTRODUCTION

Thank you for your time in meeting with us today. We are a team that is meeting with different people in

Vietnam over the next two weeks to learn about what is happening related to disability and rehabilitation.

(Introduce Team members)

The US government has provided money to support different projects in Vietnam related to

rehabilitation and disability.

This evaluation is independent from USAID.

We are asking some standard questions to many people in order to hear your opinions and learn from

your experience.

We have 5-10 questions that will take about an hour. Is this okay for you? Do you have any questions

before we begin?

Question #1:

(Provide the one sheet with the table the rates the five elements). Provide pen if needed. If person cannot read,

then go through the table with them and mark the reply.


On this paper there is a list of five different topics related to rehabilitation and disability. In the last two

years, please describe any improvements you have seen in these areas (0= no improvement, 5= big

improvement.. If you don’t know about the topics, you can select I don’t know.

Give a few minutes to complete the form. Then collect it from them.

Question #2: (For the elements where the interviewee has selected 3-4-5 in term of the amount of change)

You identified _________ as having a lot of changes. Can you describe the changes you have observed

over the last 2 years?

Question #3:

How have you personally benefitted from the change? If yes how?

Question #4:

For the areas where you have not seen much change, are these important to help you (or the person

with disability you are caring for) participate more in the family or community? If yes, what could be

improved?

Question #5:

What assistance have you received from the project? Has this helped you (or the persons with disability

you are caring for) to participate more in the family or community? If yes, how?

Question #6:

What are the biggest challenges that prevent you (or the person with disability you are caring for) from

participating in the family or community?

Question #7: (For care givers only)

What are your biggest challenges for you as a caregiver?

What is needed to help reduce these challenges?

Question #8:

What do you think is needed to help you (or the person with disability you are caring for,) participate

more in the family or community, or be able to do thing things you like?

Thank you for taking time to answer these questions – do you have any questions for us?


INTERVIEW GUIDE (GENERAL)

INTRODUCTION

Thank you for your time in meeting with us today. We are a team that is meeting with different people in

Vietnam over the next two weeks to learn about what is happening related to disability and rehabilitation.

(Introduce Team members)

The US government has provided money to support different projects in Vietnam related to

rehabilitation and disability. This evaluation is not to look at each individual project, but the progress of

all of them combined.

We are asking some standard questions to many people in order to hear your opinions and learn from

your experience. We are independent from USAID and really value your opinions and would greatly

appreciate any recommendations you have related to each questions.

We have about 10 questions that will take about an hour. If you have any presentation for us, that would

be great, but it would be great to keep it a bit short so that we have time for discussion.

Do you have any questions before we begin?

(ELEMENTS)

Question #1:

(Provide the one sheet with the table the rates the five elements). Provide pen if needed. If person cannot read,

then go through the table with them and mark the reply.

On this paper there is a list of five different topics related to rehabilitation and disability. These are areas

that are important to USAID and may also be supported by others. In the last two years, please describe

improvements you have seen in these areas (0= no improvements, 5= big improvement). If you don’t

know about the topics, you can select I don’t know.

Give a few minutes to complete the form. Then collect it from them.

Question #2: (For the elements where the interviewee has selected 3-4-5 in term of the amount of change)

Can you please describe the changes you have observed with __________ over the past 2 years?

Question #3:

How do you think the USAID project has contributed to the change(s) you have identified?

Question #4:

Outside of the USAID project, has your organization, or any other organization, contributed to this

change? If yes, how?


Question #5:

Do you have any plans of your own to continue this action?

Is the USAID project helping to further your plan?

Question #6:

You noted that _______ are areas that haven’t had much change in the past two years. How can this be

improved? Any recommendations?

(INCLUSION and THOERY OF CHANGE)

The next few questions are to help us to understand how the project has helped people with disability.

Question #7:

Is the project helping persons with disabilities participate more in family and community? If yes, how?

If no, what can be improved? Any recommendations?

Question #8:

(Show the inclusion diagram that has physical rehabilitation services, training, policy and advocacy influencing

inclusion)

This diagram shows that physical rehabilitation services, HR,/Equipment/Finance, and policy strengthening

will help persons with disabilities be more included in society. Do you agree? Why or why not?

(MANAGEMENT COORDINATION)

The last part of the interview is about management and coordination.

Question #9:

How is the USAID project managed in the Province? What is the coordination between government

ministries, government and implementing partners, and between government and USAID?

Can you identify any benefits of the coordination? Challenges? Recommendations?

(DIRECT ASSISTANCE AND GENDER)

Question #10:

What is your opinion about direct assistance provided through the project? Advantages/disadvantages?

Do you have any recommendations?


Question #11:

Has the USAID Project helped to make any changes related to gender? If yes, please describe. Do you

have any recommendations related to gender?

Thank you for taking time to answer these questions – do you have any additional information you would

like to add or questions for us?

ANNEX IV: SOURCES OF INFORMATION

DISABILITY LOGICAL FRAMEWORK

Narrativ

e

Summar

y

Indicators Means of

Verificati

on

Assumptions

Non-Project CDCS Goal (DO2): Capacity strengthened to protect and improve health and well- being

DO2 PM5: # of persons

(disaggregated by sex) with

disabilities with increased

access to health services

and access to education,

social, and economic

opportunities

Ministry

reports

Provinces’

reports

NCCD reports

Data available

MOLISA continues its

support to service

program to persons

with disabilities

Project Purpose (I.R.2.3.): Expanded

opportunities for vulnerable populations

IR 2.3. PM1: Number of

GVN laws, policies, or

procedures drafted, revised,

and/or issued to support

inclusion of vulnerable

populations

IR 2.3. PM2: (or PPR 3.3.2-13)

Number of service

providers (organizations

or individuals) trained

who serve vulnerable

persons

IR 2.3 PM3: Level of

satisfaction among male and

female persons with

disabilities receiving social

(POS) services in targeted

areas

IR 2.3 PM4: Participatory

Action plan developed for

persons with disabilities in

all target provinces by

2018

MOLISA

reports

NCCD reports

Project reports

Provinces’

reports

Beneficiary

satisfaction

survey

MOLISA

demonstrates

commitment and

support to the

development and

enforcement of

disabilities policies

MOLISA continues

its support to

National Action Plan

on Disabilities 2011-

2020

MOF’s commitment

in resource

allocation


Sub-Purpose

(Sub-

I.R. Disability):

Persons with

disabilities have

expanded

opportunities

through higher

quality and

sustainable

services,

awareness, and

advocacy

strengthening

1. # of persons with

disabilities who report

increased independence as a

result of the project

(disaggregated by sex)

2. # of persons with

disabilities with increased

social participation,

educational, and working

opportunities (disaggregated

by sex)

Ministry reports

NCCD reports

Provincial

reports Project

M&E records

MoH

demonstrates

commitment to

POS development

KEY INFORMANT INTERVIEWS

BINH DINH

NO NAME SEX POSITION ORGANIZATION

Provincial level

1 Le Quang Hung M Director DOH

2 Ha Anh Thach M Head of Health care service DOH

3 Phan Dinh Hoa M Vice Director DOLISA

4 Tran Ngoc Vy F Head of Social Protection DOLISA

5 Vo Duc Viet M Director Quy Nhon Orthopedic & Rehabilitation Hospital

6 Nguyen Tien Luan M Orthotist Quy Nhon Orthopedic & Rehabilitation Hospital

7 Vo Ngoc Phai M Vice director Hospital of Rehabilitation

8 Dang Van Loc M Vice director Hospital of Rehabilitation

9 Thu F Staff Hospital of Rehabilitation

10 Dua F Staff Hospital of Rehabilitation

11 Huong F Staff Hospital of Rehabilitation

12 Trong M Staff Hospital of Rehabilitation

13 Luong M Staff Hospital of Rehabilitation

14 Ly F Staff Hospital of Rehabilitation

District level


BINH DINH


15 Huynh Ba Thinh M Director Tay Son Preventive Medicine

16 Pham Van Nghiem M Disability and rehabilitation officer

Tay Son Preventive Medicine

17 Nguyen Thuan M Planning department officer Tay Son Preventive Medicine

18 Truong Van Ky M Vice Director Tuy Phuoc District Health Center

19 Nguyen Thi Giang F Traditional medicine unit Tuy Phuoc District Health Center

20 Nguyen Thi Hoa F Rehabilitation unit Tuy Phuoc District Health Center

Commune level

21 Nguyen Thi Lan F Head Binh Tan Health Station

22 Ho Duc Do M In charge; persons with disabilities Binh Tan Health Station

23 Ho Thi Ngoc Anh F Vice Head Phuoc Loc Health Station

People with disabilities (PwDs) and Caregivers

Number of PwDs/ Caregivers Sex Beneficiary from IP Working place

8 PwDs 5 F

3 M Beneficiaries from IC Binh Tan Commune and

Phuoc Loc Commune

3 Caregivers 3 F Binh Tan Commune

BINH PHUOC


Provincial level

1 Le Anh Tuan M Vice Director Provincial General Hospital

2 Hoang Manh Hoa M Head of health care service Provincial General Hospital

3 Trinh Quoc Tuan M Head of General Planning Provincial General Hospital

4 Vo Van Mang M Director DOLISA

5 Le Xuan Manh M Social Protection Unit DOLISA


BINH PHUOC


6 Nguyen Thu Dung F Social Protection Unit DOLISA

7 Tran Thi Loan F Head of General Admin Department Center of Legal Aids

8 Pham Minh Hoang M Vice head of Traditional Medicine and Rehabilitation Unit

Provincial General Hospital

9 Bui The Anh M Rehabilitation doctor Provincial General Hospital

10 Truong Huu Nhan M Director Hospital of Traditional Medicine

11 Luong Xuan Viet M Vice Director Hospital of Traditional Medicine

12 Pham Thanh Duy M Head Int. Medicine and Pediatrics Hospital of Traditional Medicine

13 Dinh Thi Phuong F Doctor Int. Medicine & Pediatrics Hospital of Traditional Medicine

District Level

14 Nguyen Van Thanh M Vice Director Bu Dang Health Center

15 Nguyen Thanh Hoi M Department of Traditional Medicine and Rehabilitation

Bu Dang Health Center

16 Bien Thi Hoa Tho F Department of Traditional Medicine and Rehabilitation

Bu Dang Health Center

17 Ho Duc Anh M Social protection unit Bu Dang Division of Labor, Invalid and Social Affair

18 Nguyen Thi Thao F Center Director Dong Xoai Health Center

19 Phan Thi Ngoan F Planning Department Dong Xoai Health Center

20 Vu Thi Thuy F Doctor of Traditional Medicine Dong Xoai Health Center

21 Duong Thị Thanh F Accountant Dong Xoai Health Center

22 Nguyen Thi Nhan F Planning Department Dong Xoai Health Center

23 Mai Giang Hai M Social protection staff Dong Xoai Division of Labor, Invalid and Social Affair

DA NANG


Provincial level

1 Do Van Thanh M Director of hospital Orthopedic and Rehabilitation Hospital


DA NANG


2 Nguyen Van Tien M Director of O&P workshop Orthopedic and Rehabilitation Hospital

3 Loc M Staff Orthopedic and Rehabilitation Hospital

4 Nga F Staff Orthopedic and Rehabilitation Hospital

5 Cao Bich Thuy F Head of rehabilitation faculty Da Nang University of Medical Technology and Pharmacy

HAI DUONG


1 Pham Thi Cam Hung F Head of Rehabilitation Department Hai Duong Medical Technical University

2 Tran Quang Canh M Vice Director of The University Hai Duong Medical Technical University

3 Dinh Thi Xuyen F Head of Dept. Int. Coop. & Research Hai Duong Medical Technical University

4 Vu Dinh Tuyen M Department of Training Management Hai Duong Medical Technical University

5 Le Thi Kim Phuong F Physical Therapy Hai Duong Medical Technical University

6 Le Thi Ha F Rehabilitation doctor Hai Duong Medical Technical University

7 Le Duc Thuan M Department of Training Management Hai Duong Medical Technical University

8 Dinh Thi Hoa F Vice Head of Rehabilitation Department

Hai Duong Medical Technical University


HANOI


1 Phan Hoai Chung M Director, National Center Transportation Development and Strategy Institute, Ministry of Transport

2 Nguyen To Ha F Vice Director, National Center Transportation Development and Strategy Institute, Ministry of Transport

3 Dang Xuan Khang M Head of Medical Division Social Protection Department, MOLISA

4 Dang Huynh Mai F President Viet Nam Federation of Disability

5 Nguyen Van Binh M Deputy Director General Legal Department, MOLISA

6 Nguyen Thi Ngoc Yen F Head of International Law Unit Legal Department, MOLISA

7 Tran Thi Thanh Y F Administrator, Urban Planning Institute of Urban Planning and Development

8 Tran Trong Hai M President Vietnam Rehabilitation Association

9 Le Tuan Dong M Head of Rehab/Medical Division MOH

10 Nguyen Trong Khoa M Deputy Director Medical Service Admin MOH

11 Luong Tuan Khanh M Director Rehabilitation Center - Bach Mai Hospital

12 Dinh Thi Thuy F Deputy Chief Officer NCD

HO CHI MINH CITY


1 Do Thi Bich Thuan F In charge of Rehabilitation Department Pediatric Hospital #3

2 Dinh Quang Thanh M Chief of Physiotherapy Unit HCMC hospital of rehabilitation and professional diseases

3 Nguyen Thi Hong F Chief Nurse HCMC hospital of rehabilitation and professional diseases

4 Nguyen Thi Trang F Head of Nursing Department HCMC hospital of rehabilitation and professional diseases

5 Pham Thu Hoan F Vice Director PhaNa Trading Company


6 Nguyen Tien Toan M Founder and CEO Kien Tuong Wheelchairs

7 Nguyen Oanh F Communications/daughter Kien Tuong Wheelchairs

8 Le Thanh Van F Head of PT Program Ho Chi Minh City Medicine and Pharmacy University

9 Nguyen Thanh Duy M Deputy Head of PT Program Ho Chi Minh City Medicine and Pharmacy University

10 Tran Thi Bich Hanh F Lecturer Ho Chi Minh City Medicine and Pharmacy University

TAY NINH


Provincial Level

1 Nguyen Van Qua M Vice Director DOLISA

2 Tran Van Hoang M Social Protection Unit. DOLISA

3 Luu Thi Xuan F Officer of Planning Unit DOH

4 Tran Tuong Quoc M Vice Director Department of Construction

5 Le Tan Nhac M Head of Construction management unit Department of Construction

6 Nguyen Van Phuoc M Vice Director DOET

7 Nhan Hoa Phuong F Head of Kindergarten Education DOET

8 Tran Huynh An M Doctor Traditional Medicine Hospital

9 Nguyen Thi Hanh F Doctor Traditional Medicine Hospital

10 Tran Thi Nhan F Nurse Traditional Medicine Hospital

11 Thai Thi Hieu F Nurse Traditional Medicine Hospital

12 Nguyen Thi Thuy F Assistant doctor Traditional Medicine Hospital

13 Ho Thanh Su M Assistant doctor Traditional Medicine Hospital

14 Le Thi Mai Linh F Assistant doctor Traditional Medicine Hospital

15 Thai Thanh Tan M Assistant doctor Traditional Medicine Hospital

16 Mai Ngoc Phuc M Assistant doctor Traditional Medicine Hospital

17 Huynh Tran Cong Hien M Deputy Director Tay Ninh Rehabilitation Hospital


TAY NINH


18 Nguyen Thi Cam Nhung F Deputy head of planning unit Tay Ninh Rehabilitation Hospital

19 Nguyen Minh Viet M Department of Health Officer Tay Ninh Rehabilitation Hospital

20 Huyen Trang F PT Tay Ninh Rehabilitation Hospital

21 Vo Hoai Thuong F OT Tay Ninh Rehabilitation Hospital

22 Yen Trinh F OT Tay Ninh Rehabilitation Hospital

23 Tran Van Yen M Director Center for Children with Visual Impairment

District level

24 Le Thi Ha Phuong F Head of Rehabilitation Department Go Dau Division of Labor, Invalid and Social Affair

25 Tran Anh Nhan F Deputy Head Go Dau Division of Labor, Invalid and Social Affair

26 Bui Ngoc Thuy F Social Protection Unit Go Dau Division of Labor, Invalid and Social Affair

27 Ho Thi Thom F Social Protection Unit Go Dau Division of Labor, Invalid and Social Affair

28 Nguyen Chi Sang M Deputy Head Tan Bien Division of Labor, Invalid and Social Affair

Commune level

29 Huynh Ngoc Thu F Acting head Go Dau health center

30 Huynh Thi Phuoc F Vice head Go Dau health center

31 Nguyen Le Quyen F Teacher Preschool of May 19

32 Ha Phuong Anh F Teacher Preschool of May 19

33 Truong Ngoc Duyen F Teacher Preschool of Hoa Thanh

34 Cao Thi Lung F Teacher Trung Vuong preschool

35 Nguyen Thi Huong F Teacher Trung Vuong preschool

36 Lam Thi Lan Trinh F Teacher Trung Vuong preschool

PwDs and caregivers

Number of PwDs/ Caregivers Sex Beneficiary from IP Working Place

5 PwDs 3 F

2 M Beneficiary from VNAH

Go Dau District, Tan Bien District


TAY NINH


4 Children with disabilities 3 F

1 M Beneficiary from VH

Trung Vuong school, Chau Thanh District

5 Caregivers 5 F Preschools in Hoa Thanh District; Go Dau District; Tan Bien District

THUA THIEN HUE


Provincial level

1 Nguyen Thi Mung F Head of Rehabilitation Unit Hue Central Hospital

2 Ha Chan Nhan M Head of Rehabilitation Department Hue University and

Medicine Pharmacy

3 Ho Dan M Deputy Director DOLISA

4 Nguyen Ngoc Huy Hoang M Official, Social Protection DOLISA

5 Vo Thanh Quyet M Official, Social Protection DOLISA

6 Duong Quang Minh M Deputy Director DOH

7 Truong Phan Thuy Dung M Deputy Director Legal Aid Center,

Department of Justice

8 Tran Thi Hong Minh F Program Officer Legal Aid Center,


9 Dang Ba Quat M Program Officer Legal Aid Center,


10 Nguyen Thi Be M Program Officer Legal Aid Center,


11 Ho Thi Ly F Program Officer Legal Aid Center,


12 La Thach M Chairman DPO Thua Thien Hue Province

13 Truong Thi Ngoc Anh F Administrator DPO Thua Thien Hue Province

13 Thu Giang F Book keeper DPO Thua Thien Hue Province

14 Members of Management Board DPO Thua Thien Hue Province


District level

15 Vo Phi Long M Vice director Nam Dong Health Center

16 Nguyen Trong Tan M Rehab doctor Nam Dong Health Center

17 Le Duy Lo M Chairman DPO of Hue city

18 Nguyen Luu M Vice-chairman DPO of Hue city

19 Nguyen Huu Thanh M Member DPO of Hue city

20 Dang Van Ngoc M Member DPO of Hue city

21 Vo Thi Huyen F Member DPO of Hue city

THUA THIEN HUE

PWDs and caregivers

Number of PwDs/ caregivers Sex Beneficiary from IP Organization

8 PwDs 2 F

6 M

4 Beneficiaries from ACDC

4 Beneficiaries from IC

Nam Dong District and Quang Dien District

IMPLEMENTING PARTNERS

NO NAME OF INTERVIEWEES SEX POSITION ORGANIZATION

1 Bui Van Toan M Country Director VNAH

2 Tong Thu Trang F Project Coordinator VNAH

3 Le Hai Anh M Project Officer VNAH

4 Nguyen Thanh Son F M&E Officer VNAH

5 Nguyen Van Kinh M Project Officer in Tay Ninh MCNV

6 Pham Dung M Country Director MCNV

7 Tran Quynh Trang F Project Officer ACDC

8 Nguyen Thi Lan Anh F Country Director ACDC

9 Nguyen Hoang Kha Tu F Project Coordinator ACDC

10 Nguyen Thi Hong Minh F Project Coordinator ACDC

11 Nguyen Thi Quy F Project Coordinator ACDC

12 Didier Demey M Country Director HI

13 Truong Hien Anh F Project Coordinator HI


IMPLEMENTING PARTNERS

NO NAME OF INTERVIEWEES SEX POSITION ORGANIZATION

14 Nguyen Thi Mai Hien F Project Director IC

15 Hoang Cam Linh F Project Coordinator IC

16 Nguyen Thu Huyen F Project Officer IC

17 Luu Thi Anh Loan F Former Director DRD

18 Truong Huy Vu M Project Officer DRD

19 Le Quang Duong M Director VIETHEALTH

USAID/VN


1 Van Le F Disability Program USAID/VN

2 Chris Abrahms M Director, ESDO USAID/VN

3 Le NguyenThi Hoa F Disability Program USAID/VN

4 Jeremiah Carew M Director, Program Office USAID/VN

SKYPE CALLS


1 Lindy McAllister F Staff Trinh Foundation

2 Sue Woodward F Founder Trinh Foundation

3 Kirrily Manning F Project Manager, OT school Curtin University

4 Joel Nininger M Regional Manager Movability, Vietnam


ANNEX V: SUPPLEMENTARY REPORTS DEVELOPED

DESK REVIEW

INTRODUCTION

The United States Agency for International Development (USAID) Vietnam has an over-arching project

to assist persons with disabilities in Vietnam. This is within the Country Development Cooperation

Strategy (CDCS) Development Objective (DO) 2, Capacity strengthened to protect and improve health

and well-being. Within the 2014 Project Appraisal Document (PAD), the Project Purpose states

Expanded opportunities for vulnerable populations, and Sub- Purpose, Persons with disabilities have

expanded opportunities through higher quality and sustainable services, awareness, and advocacy

strengthening. Three key outputs and an extensive number of indicators support the Purpose and Sub-

Purpose.

Based on consultations with stakeholders8, USAID modified the PAD in October 2017. The revised

Project Purpose reads, “A sustainable model for improving persons with disabilities’ inclusion developed

in USAID targeted provinces”. The sustainable model comprises five interrelated and complementary

elements9 that contribute to the Project Purpose:

• Service delivery

• Human capacity building

• Policy

• Financing

• Public awareness

These five elements are strengthened and improved through activities in three project components: 1)

provision of direct assistance; 2) service system strengthening; and 3) support for the enforcement of

disability policies and rights. Elements and components are addressed through six implementing

activities along with six Implementing Partners (IPs).

USAID is supporting a Mid-term Whole of Project Evaluation (WOPE) for the 2015-2020 Disability

Project. The three phases of the evaluation include:

• Desk Review (collect, synthesize, and triangulate existing data/information from both the

project and activity levels).

• Beneficiary Survey

• Qualitative Research

8 These include: all Mission offices, national ministries, provincial authorities, technical institutions in health and

social sectors, and persons with disabilities and their organizations. 9 The elements are loosely based on World Health Organization’s (WHOs) six building blocks for health systems

strengthening.


The evaluation team accessed relevant documents through a Drop Box (created by VEMSS), USAID’s

Google Docs, and shared email attachments or links to additional resources.

OBJECTIVES

The Desk Review will serve several purposes:

1) To provide background on the disability sector in Vietnam;

2) To initially answer the evaluation questions through collecting, synthesizing, and triangulating

existing data/information from both project and activities levels; and

3) To identify additional data collection needed during the fieldwork phase.

Specific to the project and activities the Desk Review will serve to:

• Gain understanding on the design and implementation of the six activities;

• Assess the progress toward achievement of the project purpose; and

• Identify challenges and lessons learned from implementation of the six activities.

Desk Review documents/resources include:

• USAID’s PAD, 2014 and revised 2017.

• USAID’s performance monitoring data.

• USAID’s Gender Policy; and Vietnam’s Gender Assessment Report.

• Documentation on SWEEP (Social Work Education Enhancement Program)

• USAID 2018 Assessment Report: Sustainability of Donor Projects in Indonesia

• IP’s performance reports (annual reports).

• IP’s work plans, and monitoring, evaluation and learning (MEL) plans.

• IP’s self-assessment (due to VEMSS by March 30, 2018); see Annex 1 for outline.

• WHO’s Building Blocks for Health Systems Strengthening.

• WHO’s Rehabilitation 2030: A Call for Action.

• WHO’s Global Cooperation on Assistive Technology (GATE).

• WHO Resolution EB142.R6 (Jan 2018) on Improving Access to Assistive Technology

• United Nations Convention on the Rights of Persons with Disabilities (UNCRPD)

• 2015 National Action Plan on Rehabilitation (NAPR).

• Second National Action Plan on Disability (NAP).

• Code and Standards on Accessible Construction & National Technical Regulation on

Construction for Disabled Access to Buildings and Facilities, introduced in 2014.

• Disability Information System (DIS); Decision 3815/QD-BYT

• National Law on Disability (2011?)

• GVN Disability Survey 2017.

• CRPD documents: Action Plan (Plan 1100), report on CRPD progress, and shadow report.

• MoH Circular #18

• NCD/MOLISA's national disability indicators.

• MOET Circular guiding education for children with disabilities.

• National Strategy to promote education support to children with disabilities (2017).


RESULTS

A. KEY DESIGN INCLUDING THEORY OF CHANGE FOR EACH ACTIVITY

As presented in the introduction, the project design is multi-layered and complex.

The overall theory of change for the project states that, “If USAID builds a sustainable model that

a) delivers direct assistance to persons with disabilities to demonstrate interdisciplinary rehabilitation services that improve persons with disabilities’ daily living, learning and working functions; b) strengthens

systems to deliver quality services to persons with disabilities; and c) improves the enforcement of

disability laws/policies, then persons with disabilities’ inclusion in society will be increased.”

The table below presents DRD’s theory of change and paraphrases other IP’s theories of change.

IP Theory of Change

ACDC If DPOs and legal aid services are established or strengthened, then persons with disabilities will be

supported in their journey toward an integrated life.

DRD If physical accessibility rights of persons with disabilities are fully respected, persons with disabilities

will have equal opportunities in life.

HI If the gaps in treatment of brain lesions (lack of standard procedures, technical capacities, service

availability) are addressed, then quality of life of persons with brain lesions will be improved.

IC-VVAF If persons with disabilities have appropriate devices and improved mobility, then they will have

greater capacity for independent living and greater participation in the broader community.

VietHealth If early detection and early intervention activities for children with disabilities are established, then

there will be a reduction in the rate and severity of disability in children from 0-6 years old.

VNAH If health and rehabilitation service systems, and national coordination mechanisms are improved,

then quality of life and social integration of persons with disabilities will be improved.

Desk Review Findings/Observations

• USAID’s 5 elements of a sustainable model are loosely based on WHO’s building blocks10 for

strengthening health systems; containing four of the building blocks, but omitting health

information systems and medical products and technologies. In addition to four WHO-linked

topics, the model also identifies public attitude and self-awareness.

10 Building blocks include service delivery, health workforce, health care financing, governance (policies), medical

produtcs/technologies (assistive devices), and health information system.


• IPs seemingly use “integration” and “inclusion” synonymously – though they mean different

things.

• The three component activities involve both the MoH and MOLISA as the key GVN partners.

• Only one IP (DRD) explicitly stated a theory of change.

• Although five of the IPs obliquely reference inclusion (through “quality of life”, greater

participation, social integration, equal opportunities, integrated life), none of them specifically

reference inclusion or sustainability in their theory of change.

• Although laws are mentioned in IP reports, content is not related to enforcement of laws.

B. INTERVENTIONS AND TARGET AREAS BY ACTIVITIES

The table below summarizes interventions and geographic areas according to the three key components.

In addition, a map in Annex 4 provides a visual representation of Activities per province.

Project Output Activities/Interventions IPs Provinces

Output 1

Disability rights,

policies, advocacy

and coordination

fostered

International and national conferences with

participation from GVN and DPOs to promote

disability rights

ACDC,

VNAH

Ha Noi, and National

level

Contributing to amendments/changes of national

policies on disability rights

ACDC,

VNAH

Hue, Binh Phuoc

Developing Provincial Disability Action Plans ACDC Tay Ninh

Promoting DIS for improved monitoring and

networking for persons with disability

VNAH Tay Ninh, Binh Phuoc

Output 2

(Direct

Assistance)

Quality POS

services delivered

to persons with

disabilities in

focus provinces

Rehabilitation services provided to persons with

disability

VNAH, HI

VietHealth

Tay Ninh, Binh Phuoc

Assistive devices provision and training VNAH, IC Tay Ninh, Binh Phuoc,

Hue, Binh Dinh, Quang

Nam

Promote ECDDI (Early Childhood Disability

Detection and Intervention)

VietHealth

IC (?)

Tay Ninh

Public building accessibility (with accessibility

elements)

DRD

ACDC

Tay Ninh, Binh Dinh

Hue, Binh Phuoc

Mobility impairment screening IC Hue, Binh Dinh, Quang

Nam

Consultancy on rights and assistive devices IC Hue, Binh Dinh, Quang

Nam

Legal aid and consultancy (Legal field clinics and

online legal counseling)

ACDC Hue, Binh Phuoc

Output 3 Build capacity of rehabilitation centers VNAH, HI Tay Ninh, Binh Phuoc


Project Output Activities/Interventions IPs Provinces

Improved local

capacity to

provide quality

POS services

Train community rehabilitation practitioners VNAH

VietHealth,

Tay Ninh, Binh Phuoc

Training for medical staff social workers, parents/

caregivers, and teachers on ECDDI

VietHealth Tay Ninh

Train on legal aid and consultancy ACDC Hue, Binh Phuoc

Accessibility training & Awareness Raising Events ACDC,

DRD

Tay Ninh, Binh Dinh,

Hue, Binh Phuoc

DPO establishment & capacity development ACDC,

DRD

Hue, Binh Phuoc, Binh

Dinh

Training for practitioners at district and

communes

IC Hue, Binh Dinh,

QuangNam

Developing professionals, educational

curriculums, and country -level guidelines for

medical schools and clinical hospitals in

rehabilitation, PT, OT, ST and other related

services for persons with disability

HI,

VietHealth

HCMC, Dong Nai, Hue,

Danang, Hai Duong,

Hanoi

Cross-Cutting

Gender

Gender awareness raising and training activities

in/out hospitals

HI Hue

Training on gender for service providers and

persons with disability

VNAH Tay Ninh, Binh Phuoc

Desk Review Findings/Observations • There are a number of organizations working on different activities in the same province.

• There are a number of organizations working on similar activities in different provinces.

• There are some topic areas (e.g., SaLT) that are addressed by different IPs (VietHealth and HI

and VNAH) with different focus – professional training, equipping hospitals, community level

training.

• Only one IP (HI) references completing a gender analysis and using the findings to address gender

gaps within programming. Other IP activities are less clear about gender gaps and associated

interventions. Specific references are mostly related to “gender training on GBV” and

disaggregation of people-level indicators by sex.

• All IPs are required to report on sustainability mechanisms in their annual reports, but most are

directly linked to their project activities and not the overall project purpose.


C. ACHIEVEMENTS AGAINST TARGETS BY ACTIVITIES

USAID’s Disability Project in Vietnam is complex. It comprises six Activities (often synonymous with IPs)

working on three components11 (direct service, system strengthening, and enforcement of policies and

rights) in seven provinces and three cities to address five elements that contribute to a sustainable model

to increase inclusion for persons with disability in society.

Each of the IPs has their own project targets; these include targets specific to their own activities as well

as those that contribute to USAID’s overall project targets. For the purposes of the WOPE the targets

are the indicators established in USAID’s original 2014 PAD12. There are a total of 24 (twenty-four)

indicators relating to the project. These are summarized below:

:

• Five indicators relating to the Mission’s Performance Management Plan (PMP)

• 16 (sixteen) indicators relating to the Disability Monitoring Plan (DMP)

• Two DMP Gs (gender-related indicators)

• One indicator to capture the impact on the lives of beneficiaries

The table in here provides details on achievements to date by Activity and following the structure in the

MEL for the overall project.


• Although a number of indicators require disaggregation by “level of disability”, this information is

not available as most people do not know their level, some have not been screened, and there is

an evolving framework for identification of disability based on the ICF.

• Disaggregating data by province is problematic as USAID data tracker does not disaggregate this

information, and IPs that work in more than one province consolidate data without

disaggregating.

• There have been notable achievements (and targets exceeded) in four main areas: number of

service providers trained, number of people trained in mitigating gender based violence, number

of people who have received direct support, and number of people who have benefitted from

Provincial Action Plans (PAPs).

• Case management and referral systems (links with social workers) have gone largely un-reported.

• Figures on people who face discrimination and people who report increased independence as a

result of the project have received little attention.

• There are no specific sustainability targets identified through the project.

• The indicators used in the WOPE are from the 2014 PAD; indicators for the revised PAD (2017)

have not yet been developed.

D&E. PRELIMINARY ASSESSMENT OF SUCCESSES BY ACTIVITY / IDENTIFIED

CHALLENGES/LESSONS LEARNED


11 In the 2014 PAD, these are identified as Outputs. 12 Indicators for the updated PAD (2017) have not yet been completed.

https://www.dropbox.com/s/fppx0n7jht7heps/Achivevement%20against%20target%20-%20by%20IP%20-%2020%2003%2018.xlsx?dl=0


The successes identified in this section stem primarily from review of IP’s 2017 annual reports.

Please see the summary table provided in Annex 5.

F. EVALUATION QUESTIONS

PERFORMANCE

How is the project making progress toward the achievement of its purpose? ● To what extent has progress been made in achieving the project’s purpose of developing a

sustainable model for improving persons with disabilities’ inclusion in society (in USAID target

provinces)?

● To what extent have each of the six component Activities contributed to achieving the project’s

purpose?

● What are the achievements and progress against targets in key program areas: fostering disability

rights, policy, coordination and advocacy; provision of physical, occupational, and speech (POS)

therapy services to assist persons with disabilities; and improving local capacity to provide quality

services?


• Unable to determine inclusion measures through desk review.

• Although sustainability has not been specifically defined, USAID has multiple resource documents

that provide proxy indicators for sustainability and these could be applied to the project – either

in process sustainability or impact.

• All of the IPs are required to report on “sustainability mechanisms” in their annual report; the

focus is on their own interventions and not reference the overall project purpose or sustainable

model.

• Some IPs have reduced their targets from original plan; unclear documentation of these changes

with USAID approval. Many achievements are now exceeding targets, while others go

unreported

• Most of the achievements are outputs only; there are a limited number of outcome measures and

even fewer impact statements (though project life is still early).

Project Design

What are the strengths and weaknesses (gaps) of the project’s theory of change?

● Is there any strategy or specific intervention in the initial design that has had an unexpected influence

on the project’s implementation and/or achieving the project’s purpose?

Desk Review Findings/Observations ● The Theory of Change together with the five elements and three components is complex and the

pathway to achieve the project purpose is difficult to follow.

● Some content in the design is duplicative, and language used is not always consistent.

● Interventions are scattered – not comprehensive or consistent across provinces. Different places

receive different interventions; each organization has it’s own activities and it is not clear how these


organizations are working to have consistent messaging on same topics (like building accessibility), or

how organizations are combining their expertise to achieve the project purpose. In addition, some

organizations are working at national level while others are at provincial or district levels. This may

result in incomplete coverage of some topics and difficulty to scale others.

● An unexpected result of the design is that IPs may be targeting the same government staff to attend

trainings and implement activities – this may create an increased burden on these staff especially in

relation to others duties they may have outside the USAID project.

Project Management and Cross-Cutting Issues

● What are the benefits of coordinating the project’s implementation activities with the Ministry of

Labor, Invalids, and Social Affairs (MOLISA) and local authorities to achieve its purpose? What are

identified challenges? Recommendations for the management section should include how the project

design, management, and implementation can become more efficient, effective and relevant toward

achieving the overall project objectives -- in consideration of any changing contextual dynamics.

● To what extent has USAID gender policy been implemented in the project? What are the lessons

learned of what works and what does not?

● What is the evidence of local ownership strengthening? Are the project’s activities considered as a

part of local development agenda in the area of assisting persons with disabilities?


• Coordinating with MOLISA is essential as this is the responsible ministry providing overall

guidance and social welfare support to persons with disabilities. It is also essential to be

coordinating with the MoH , particularly on issues related to physical rehabilitation, as this

sends the correct message that rehabilitation is for everyone who needs it – not just people

with long-term impairments. A potential unanticipated consequence of coordination only

with MOLISA on issues of rehabilitation is the messaging it could send that rehabilitation is

for persons with disabilities only.

• Most of the project partners have included gender-related training in their activities and all

have disaggregated beneficiaries by sex, but there is a general lack of clarity on the gender-

related gaps or issues pertinent to the individual activities and little reporting that points to

meaningful results of gender-related training to date.

• There is extensive capacity building in the projects, government engagement in developing

training guidelines, and reference to GVN policy documents; all contribute to local

ownership, which may be considered a proxy measure for a sustainable model.

RECOMMENDATIONS

• The desk review has provided a foundation of information related to the enabling environment in

Vietnam (through various policies and national guidance documents) as well as the six key

Activities and progress in their work.


• Additional areas to explore that have not yet been captured in the Design:

• Review sustainability factors; how each of the IPs conceptualize and address sustainability within

their programming – progress or impact.

• Consider the role of WHO and current global initiatives related to rehabilitation and assistive

technology and how this project could be more aligned with and benefit from WHO’s actions.

• Coordination amongst partners and consistency/coherence with their approaches and alignment

with government policy and guidelines.

• Relationship or complementarity of the project purpose and project design (activities/target

provinces) with broader context (i.e. are there other non-USAID-funded activities happening in

the same place, and if so, how do they and USAID activities related or complement each other or

not.

REPORT OUTLINE FOR IMPLEMENTING PARTNER’S SELF-

ASSESSMENT

Overall Instruction: - The report should NOT exceed 20 pages. There is no limitation for the appendix, so please

include as much information as needed in the appendix.

- The goal of this report is to explore how each of USAID Project Activity (hereafter referred as

the program) contributes to progress to achieve the project purpose. Therefore, it is

recommended that only the discussion on the high level of result should be included in the

report (i.e.; at sub-purpose or higher level).

Suggested Outline of the IP’s Self-Assessment Report I. Program Description

1. Purpose/sub-Purpose

2. What does Success look like? (Highest-level result the program can achieve)

3. Approach/Theory of Change

• Partners, context, trends

• Relationship of sub-purposes to purpose (What are your sub-purposes and how do they

relate to your purpose)? What are relationships between inputs and expected outcomes?

What should be adjusted in the coming time?

4. Key assumptions

• What assumptions are needed for the program to achieve expected results?

• What are assumptions that still hold and what do not?

II. Assessment of Progress

• What is the evidence of progress towards achievement of the program purpose?

(indicators, other forms of data, evaluation results, milestones, anecdotal, etc.)

• What should be adjusted and amended in the coming time to achieve the expected

results?


• Host country ownership: What's been the experience with local ownership (GVN,

CBOs, sub-national) and private sector involvement? What is working well? What could

be improved upon?

• What gender gaps were identified early on in relation to your project and what progress

has been made to date to address them accordingly?

III. Lessons Learned

• What are included in the program-learning plan?

• What are the most important lessons in the last two years?

SUMMARY OF OVERALL PROJECT INDICATORS

Regarding USAID’s Performance Management Plan (PMP)) there are five indicators:

PM 1: Number of GVN laws, policies or procedures drafted, revised, and/or issued to support

inclusion of vulnerable populations.

PM 2: Number of service providers (individuals) trained who serve vulnerable populations

(disaggregated by sex and province).

PM 3: Level of satisfaction among male and female persons with disabilities receiving social service or

prosthetics and orthotics (P&O)13, ST in targeted areas.

PM 614: Number of persons with disabilities receiving direct assistance provided by USAID-funded

programs (disaggregated by sex, level of disability and province).

PM 7: Number of USG-assisted organizations and/or service delivery systems strengthened that serve

vulnerable populations (disaggregated by type: government institutions, registered NGOs, non-

registered NGOs, private sector).

For the Disability Monitoring Plan (DMP) there are sixteen indicators.

DMP 1: Percentage of beneficiaries who report increased independence as a result of the project

(disaggregated by sex, level of disability and province)

DMP 2: Number of persons with disabilities with increased social participation, and educational and

13 The intent of this indicator may have been physical and occupational (P&O) rather than prosthetics and orthotics. 14 There were no indicators listed for PM4 or PM5.


working opportunities (disaggregated by sex, level of disability and province).

DMP 3: Percentage of persons with disabilities in targeted geographic areas who experience

discrimination (disaggregated by sex, level of disability, and province).

DMP 4: Number of GVN units reporting to and using data from the Disability Information System

(DIS) (disaggregated by province).

DMP 5: Number of recommended actions under Provincial Action Plans (PAPs) implemented.

DMP 6: Number of persons with disabilities who have benefitted from PAPs (disaggregated by sex,

severity of disability, and province).

DMP 7: Number of provincial action plans on disabilities developed to promote gender equality (e.g.,

to address specific needs of males and females with disabilities).

DMP 8: Number of disability advocacy messages/plans proposed by national NGOs/DPOs reflected in

PAPs (disaggregated by province).

DMP 9: Number of DPOs and Pas participating in monitoring and evaluations of local (provincial)

program for persons with disabilities.

DMP 10: Percentage15 of persons trained using learnt skills after one year (disaggregated by sex, level of

disability, and province).

DMP 11: Number of devices produced locally by USAID-supported innovation.

DMP 12: Percentage of persons with disabilities and caregivers who are aware of availability of services

(what, when, how to access) (disaggregated by sex and province).

DMP 13: Percentage of GVN officials who are aware of disability rights, laws and policies (disaggregated

15 Also presented as “number”.


by sex, province and administrative level).

DMP 14: Number of institutional POS training programs updated/developed (disaggregated by province)

DMP 15: Number of facilities equipped (disaggregated by province).

DMP 16: Number of buildings/structures with newly included accessibility features.

Regarding gender (G), there are two indicators:

DMP G 1: Percentage of males and females who provide home-based care to persons with

disabilities (disaggregated by province)

DMP G 2: Number of people trained in mitigating gender-based violence (disaggregated by sex,

level of disability, and province).

Impact on the lives of beneficiaries is captured by:

Percentage of beneficiaries reported positive changes in their daily lives since receiving services.


MAP OF VIETNAM WITH IMPLEMENTING PARTNER LOCATIONS


IMPLEMENTING PARTNER SUCCESSES, CHALLENGES, LESSONS

LEARNED

IP SUCCESSES CHALLENGES LESSONS

ACDC New DPOs were established in

districts where there had been no

DPOs before activity implementation,

and capacity building was provided to

all DPOs to engage in disability

advocacy in events and platforms

with the participation of GVN

representatives, service providers,

and DPOs/PwDs.

Language in legal aid law regulations

changed from “no one to rely on” to

“people who face difficulties in

finance”, which is a result of the

project and also positive direction.

Direct and online-based legal

consultancy services were to over

1,000 PwDs, which led to about 30%

of them understand and benefited

from having their rights exercised.

Capacity building events were held on

accessibility for PwDs to local

officials, service providers and DPOs.

A toolkit to audit the accessibility of

public buildings was developed and

used in Hue and Binh Phuoc.

Local bureaucracy,

bottlenecks by DOLISA,

and lack of government

approval in Binh Phuoc have

resulted in project delays

and restricted DPO

registration.

Network of DPOs were

mentioned, but not

operational (provincial,

regional and national?).

Continuation of legal

consultation service (on-line

or in-person) after project

ends.

Use of Women’s

Union in Phuoc

Long (where

project is

delayed) to

house a “PWD

Family Club” that

provides

opportunities for

PwDs to meet.

Use of

“mobilizing

committee” in

lieu of DPOs to

raise voices?

Use of on-line

consulting seems

cost effective.

DRD A task force group with the

participation of PwDs was established

and conducted accessibility audit for

50 public buildings, resulting in

approval of local authorities to build

22 ramps in 10 public buildings in Tay

Ninh and Binh Dinh.

DRD solicited small grant proposals

from local authorities to improve

accessibility, received 15 and

Awareness activities are

one-off and may not be

effective to promote lasting

change.

The strategy does not

include a variety of methods

nor any practice.

No mention of

collaboration with ACDC

Short-term

trainings need to

be

complemented

by other

strategies to

effect more long-

term change.

Targeting

colleges and

USAID.GOV MID-TERM WHOLE-OF-PROJECT EVALUATION | 2


approved 13. Delay in

implementation due to discussion on

construction policy.

Six rounds of One World For All

(OWFA) events held at universities in

Tay Ninh and Binh Dinh – raising

awareness of college students on

disability.

Has secured additional resources

through public private partnership

(PPP) to improve accessibility in

structures.

who is also focused on

accessible structures and

has developed a tool kit.

universities may

be an interesting

model to build-

upon.

HI 14 rehabilitation guidelines have been

drafted in English on four topic areas

(stroke, traumatic brain injury (TBI),

cerebral palsy (CP) and spina bidifia).

Stroke and TBI are finalized and are

to be translated in 2018. Process

was participatory, evidence-based,

and in-line with MoH vision.

Hospitals have been provided with

basic equipment to improve

rehabilitation services – including

SaLT equipment in 5 hospitals.

World Confederation for Physical

Therapy evaluated three Bachelor-

level PT Programs at three

universities in late 2017.

Recommendations will be

forthcoming.

The project supports foundational

training in occupational therapy (OT).

5 students are studying for a Masters

Degree in OT in India. Two

universities are now offering

Participatory process was

time-consuming and has

created delays in the

original project scheduling.

Guidelines related to

nursing and SaLT for

stroke, TBI and CP are not

yet developed.

In Hue, the rehabilitation

and university hospitals do

not have adequate space to

keep rehabilitation

equipment.

Project delays in the first

year led to packed

scheduling for CMEs – this

was a burden for GVN staff.

As the project is new, there

is limited opportunity to

evidence impact of the

CMEs, and OTs have not

GVN involved at

all levels of

guideline

development

may increase

likelihood of

use/adoption.

When looking at

multi-disciplinary

planning, all fields

must be

considered

(SaLT, nursing,

SW) in the

original design –

not only OT/PT.



bachelors program in OT, Hai Duong

(HTMU) and HCMC (HUMP).

HUMP is a 3-year bridging course and

HTMU is 1.5 years.

HUMP has 24 students and started in

March 2017.

HTMU expects ~30 students and is

to begin November 2018.

Continuing medical education (CME)

is provided and content is based on

the modules HI has developed

through this project. CME duration is

1-3 weeks. It seems CME for seven

curricula have been validated at sub-

national level.

1724 adults and children with brain

lesions directly benefitted from

rehabilitation and medical care

services.

yet completed their

coursework.

If CME curricula are not

nationally

recognized/validated, are

there implications for use in

other provinces, what is

link to health insurance

reimbursement?

Unclear linkages with

VietHealth – supporting the

SaLT training.

IC-VVAF 1890 people with mobility

impairment screened and 882

received assistive devices (Hue 283;

QNam 240; BDinh 359).

Monitoring and follow-up for assistive

devices reveals 86% of beneficiaries

surveyed report high rates of

satisfaction, and 85% of those

surveyed note increased

independence with their assistive

devices. (No mention of

disaggregation by type of device).

59 provincial/district staff received 5-

day ToT courses; includes content on

Over 50% of people coming

to outreach for screening

did receive assistive device

– assume they did not need

assistive devices?

Provincial trainers are not

keen to use participatory

approaches in conducting

training and do not appear

to be active/motivated.

No mention of VietHealth

regarding early detection

training/collaboration.

Pre-screening

communication

activities are key

to attracting

appropriate

beneficiaries.

Outreach may be

an effective way

to provide

assistive devices

to remote areas.

ToT /cascading

model may not

be effective for



the International Classification of

Functioning (ICF).

15 2-day courses for 564 commune

staff and village health workers

(VHW) on assistive device

management and use; early detection.

1,974 people trained on “appropriate

topics related to assistive devices”

and 3,024 received 1-day training on

“rights of PWDs and assistive

devices”. (Effectiveness of the training

is unknown.)

Nine sets of assistive devices have

been provided to DHCs (standard

and CP wheelchairs, crutches, waking

frame, sticks). Quantities of sets

unknown.

Project plans to provide a 6-month

training basic rehab including AT for

technicians and 12-month training

course(s) on rehab including AT for

doctors – cooperation with Da Nang

University of Medical Technology and

Pharmacy.

Lack of rehab doctors limit

screening activities.

Poor motivation and low

quality P&O devices from

Hue Central Hospital; this

resulted in project change

to Da Nang Orthopedic

Rehab Hospital.

Lack of infrastructure and

human resources

postponed creation of local

device production

workshops.

50% of P&O devices are

poor quality.

Sustainability of assistive

device provision is unclear.

Does MoH Circular #18

cover assistive devices?

Outreach services?

No mention of

collaboration with VNAH in

relation to training health

professionals.

training on

assistive devices.

VIET-

HEALTH

Over 62,000 children have been

screened; less than 500 received

intervention: ~100 children

rehabilitation-related services, ~350

special education, and remainder are

Parents hesitate to have

their children screened for

disabilities either because

they don’t understand the

purpose of their program,

Clear

information

messaging is

needed to

overcome stigma

that may be



referrals. Outcomes of interventions

are not well reported.

41 kindergarten teachers in Tay Ninh

are attending a 3-year bachelor

degree in special education. (Who is

providing this?)

Extensive training provided for

various stakeholder groups on

classifying and evaluating disabilities,

special education intervention, and

rehabilitation.

VH recognizes the value in

coordinating with other USAID

disability project partners and noted

the kick off conference as being

particularly useful, as well as targeted

coordination with VNAH for

activities in the target locations

where they are both working.

A pool of local personnel (community

health (?) workers and teachers)

established for ECDDI

implementation in six districts of Tay

Ninh.

or due to misconception of

“disability”.

Who is responsible for

ECDDI in Vietnam? Whose

budget? Project partners

with DoLISA, DoET, DoH

to conduct screening,

examination, classification

and diagnosis for CWDs.

Who does what? Is this the

same outside project areas?

Gender imbalance in

service providers trained;

only 75 men (30% of target)

of 650 total.

No reporting on SaLT

capacity building (bachelor

of rehabilitation specialized

in SaLT in Da Nang), SaLT

trainers in 4 medical

universities, and awareness

of SaLT among

rehabilitation professionals.

associated with

screening.

VNAH A number of legal documents have

been enacted/revised to be more

conducive for PwD in different

aspects.

Government and line ministries have

committed more funding for disability

related issues.

Local government to

allocate adequate funding

for CRPD implementation.

It is difficult to assess the

level of contribution and

attribution of the activity to

the series of laws and legal

documents the project lists

Capacity building

for local staff is

important and

effective for long

lasting outcome

and sustainability.

VNAH seems to

have a good



MoH adopted Disability Information

System for use in national health

system promise improved data

collection, monitoring and follow up.

17 Rehabilitation units at province

and district levels have been set up in

the two provinces.

323 rehabilitation practitioners

including doctors, technicians and

community health workers have been

trained and obtained understanding

and knowledge to practice OT

2225 people in received rehabilitation

service, 960 receive assistive devices.

VNAH seems to have deep

understanding of the GVN policy

framework for disability and has been

able to positively influence the

development of several policies that

they note as having social inclusion

benefits to persons with disabilities.

as having been approved or

are pending approval.

OT is new in Vietnam and

as such not easy to be put

into practice due to limited

knowledge and skills among

practitioners.

Lack of human resources at

local level to carry out

rehab services. Of the two

provinces where the

DIRECT project is taking

place, the Binh Phuoc

province may be

experiencing slower

implementation of their

CRPD action plan. It was

noted that the local

government did not have

funds to support

rehabilitation, health checks

and vocational training. In

comparison, Tay Ninh

province did budget modest

funds.

There seem to be ongoing

challenges with the health

insurance scheme that are

affecting where individuals

seek services (at home or

in a center) and whether or

not certain services are

provided.

Local bureaucracy

continues to be a major

system in place

with multiple

levels of

monitoring by

varied

stakeholder that

allows to

Some cost

information is

available in

relation to

income

generated by

rehabilitation

services (p. 65).

This could be

useful for

informing budget

and resource

allocation by

provincial

governments.



challenge for the IP,

especially in Binh Phuoc,

which has very rigid

regulations on having

foreigners working in the

province.

1

Disclosure of Real or Potential Conflict of Interest for USAID Evaluations

Instructions:

Evaluations of USAID projects will be undertaken so that they are not subject to the perception or reality of

biased measurement or reporting due to conflict of interest.1 For external evaluations, all evaluation team

members will provide a signed statement attesting to a lack of conflict of interest or describing an existing

conflict of interest relative to the project being evaluated.2

Evaluators of USAID projects have a responsibility to maintain independence so that opinions, conclusions,

judgments, and recommendations will be impartial and will be viewed as impartial by third parties. Evaluators

and evaluation team members are to disclose all relevant facts regarding real or potential conflicts of interest

that could lead reasonable third parties with knowledge of the relevant facts and circumstances to conclude that

the evaluator or evaluation team member is not able to maintain independence and, thus, is not capable of

exercising objective and impartial judgment on all issues associated with conducting and reporting the work.

Operating Unit leadership, in close consultation with the Contracting Officer, will determine whether the real or

potential conflict of interest is one that should disqualify an individual from the evaluation team or require

recusal by that individual from evaluating certain aspects of the project(s).

In addition, if evaluation team members gain access to proprietary information of other companies in the

process of conducting the evaluation, then they must agree with the other companies to protect their

information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the

information for any purpose other than that for which it was furnished. 3

Real or potential conflicts of interest may include, but are not limited to:

1. Immediate family or close family member who is an employee of the USAID operating unit managing the

project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated.

2. Financial interest that is direct, or is significant/material though indirect, in the implementing

organization(s) whose projects are being evaluated or in the outcome of the evaluation.

3. Current or previous direct or significant/material though indirect experience with the project(s) being

evaluated, including involvement in the project design or previous iterations of the project.

4. Current or previous work experience or seeking employment with the USAID operating unit managing

the evaluation or the implementing organization(s) whose project(s) are being evaluated.

5. Current or previous work experience with an organization that may be seen as an industry competitor

with the implementing organization(s) whose project(s) are being evaluated.

6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and

organizations being evaluated that could bias the evaluation.

1 USAID Evaluation Policy (p. 8); USAID Contract Information Bulletin 99-17; and Federal Acquisition Regulations (FAR) Part 9.5, Organizational Conflicts of Interest, and Subpart 3.10, Contractor Code of Business Ethics and Conduct. 2 USAID Evaluation Policy (p. 11) 3 FAR 9.505-4(b)

2

x

x

x

Disclosure of Conflict of Interest for USAID Evaluation Team Members

Name Thuy Nguyen TM Title MD, PhD, Assoc. Prof. Organization MSI - VEMSS Evaluation Position Team Leader Team member Evaluation Award Number (contract or other instrument)

USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable)

Name of project: USAID Disability Project

Name of Implementor: MSI

I have real or potential conflicts of interest to disclose.

Yes No

If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the

USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated.

2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation.

3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project.

4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated.

5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated.

6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation.

I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished.

Signature

Thuy Nguyen TM

Date February 11, 2018

200 12th Street South, Suite 1200 Arlington, VA 22202 USA

+1 703 979 7100

msiworldwide.com

USAID Prime Contract Number AID-440-C-14-00003

NON-DISCLOSURE AGREEMENT

USAID Vietnam Evaluation, Monitoring, and Survey Services

I am an employee of MSI Inc. assigned to work for the USAID Vietnam Evaluation, Monitoring, and Survey

Services (herein after referred to as the "PROGRAM") under contract number AID-440-C-14-00003. In

consideration of my being provided access under the PROGRAM to confidential, business-sensitive or

proprietary information that may belong to the Government, MSI, other contractors or prospective contractors

who have submitted such information in confidence to the Government, I hereby agree that during the period of

my employment on this PROGRAM or thereafter, I shall not disclose any such information except in compliance

with this Agreement or at the direction of the MSI Directors of Operation for USAID Vietnam Evaluation,

Monitoring, and Survey Services, with expressed approval of the Government.

Information subject to the nondisclosure obligations of this Agreement (“Protected Information”) includes

information, such as programs, evaluations, plans, policies, reports, studies, financial plans, internal data protected

by the Privacy Act of 1974 (Public Law 93-579), data that has not been released or otherwise made available to

the public; and "Source Selection Information" and "Proprietary Information" of third party Contractors, as those

terms are defined in Section 27 of the Office of Federal Procurement Policy Act (41 U.S.C. 423), the Procurement

Integrity Act. Such Protected Information includes, but is not limited to information submitted to the Government

on a confidential basis by other persons and covers such information whether or not in its original form (for

example where the information has been included in subcontractor generated work or where it is discernible from

materials incorporating or based upon such information.)

I agree that I shall not release, disclose or use in any way that would permit or result in disclosure to any party

outside the Government any Protected Information provided to me during or as a result of my performance on the

PROGRAM without permission from the MSI Corporate Oversight, and the expressed approval of the

Government. This prohibition applies to release of Protected Information to or between any affiliate of my

employer or any other subcontractor, consultant, or employee of my employer or any joint venture involving my

employer. In addition, Protected Information shall not be released, duplicated, used or disclosed, in whole or in

part, for any purpose other than in the performance of the PROGRAM, unless so directed by MSI and with the

expressed approval of USAID.

I acknowledge that the MSI Directors of Operation is the only person who is authorized by MSI to direct me to

release or disclose Protected Information.

I agree not to participate in any manner in the preparation of any proposal or bid to be submitted by any person or

organization involving Protected Information, including disclosing such information, to which I was exposed in

my work on the PROGRAM.

I agree to use and examine Protected Information exclusively in the performance of work required to carry out my

duties within the PROGRAM, and agree to take suitable steps to prevent the disclosure of such information to any

parties other than those authorized to have access to such Protected Information under the PROGRAM. At the

conclusion of my performance on the PROGRAM, or at the request of the MSI Corporate Oversight, I agree to

surrender all Protected Information in my possession or control. I will have no ownership or right to possess such

Protected Information except to fulfill my specific PROGRAM work assignments.

http://www.msiworldwide.com/

msiworldwide.com

2

I further agree that I will report to the OCI Board any known or suspected violations of the spirit or the intent of

the procedures established for the protection of sensitive information.

I, the undersigned, having read and fully understood this Agreement, agree to abide by the provisions of this

agreement.

Name: Ta Thuy Hanh

Date: 13 February 2018

http://www.msiworldwide.com/


ANNEX X, BENIFICIARY SURVEY REPORT

SURVEY OF PERSONS WITH DISABILITIES WHO RECEIVED DIRECT

ASSISTANCE FROM THE USAID-SUPPORTED DISABILITIES PROJECT

BACKGROUND

USAID disability interventions in Viet Nam have a long history, beginning in early 1990. A wide range of

interventions have been provided for persons with disabilities in the country, including assistance to

develop disability legal frameworks, inclusive education, health and medical support, and assistance in

other areas such livelihoods/employment for persons with disabilities.

In late 2015, USAID/Vietnam designed a new project with the purpose of establishing sustainable models

for improving persons with disabilities’ inclusion in targeted provinces. The Vietnam Intervention Project

to Assist Persons with Disabilities has three key components, which are expected to contribute to the

Project purpose: direct assistance focusing on disciplinary rehabilitation for persons with mobility

disability and children with development problems; health system strengthening; as well as advocacy and

policy implementation. The direct assistance has been provided directly by implementing partners.

The USAID-supported Project has been implemented in six provinces: Tay Ninh, Binh Phuoc, Thua

Thien - Hue, Binh Dinh, Quang Nam, and Dong Nai. In addition, several medical universities/schools

have been involved in the development of long-term training in rehabilitation specialties including

physical therapy (PT), occupational therapy (OT), and speech and language therapy (SaLT).

In 2018, USAID/Vietnam Evaluation, Monitoring and Survey Services (VEMSS), implemented by

Management Services International (MSI), was asked to conduct a whole-of-project evaluation (WOPE).

To gather evidence of the Project’s effectiveness among the target populations, the evaluation design

included a cross-sectional survey of persons with disabilities who have received direct assistance

through the USAID Disabilities Project. The following sections provide details of the survey design and

key results. A detailed description of the Project can be found in the main evaluation report.

OBJECTIVES

The objective of the survey is to evaluate the effect of the Project on persons with disabilities, using the

following indicators:

● self-perceived quality of life of persons with disabilities who have received USAID direct

assistance (beneficiaries);

● self-reported physical and functional improvements among beneficiaries;

● observed improvement in beneficiaries’ independence; and

● self-perceived improvement in persons with disabilities’ social inclusion.


SURVEY DESIGN

SURVEY SITES

Province selection. In consultation with USAID/Vietnam, the survey team selected Tay Ninh and Thua

Thien - Hue for the beneficiary survey in 2018. There were several key reasons for this:

● the interventions being implemented are comprehensive in these provinces, including direct

assistance, health system strengthening, advocacy efforts and policy implementation;

● direct assistance has been provided for a sufficient period (more than a year), allowing

observation of potential improvements among beneficiaries who have received rehabilitation-

related services (see the participant’s inclusion criteria for more details); and

● local Government of Vietnam partners were willing to conduct the survey.

District selection. In Tay Ninh the survey covered all districts where USAID direct assistance has

been provided, except Trang Bang as there were only four beneficiaries in this district who met the

inclusion criteria. In Thua Thien - Hue the survey was undertaken in three districts — Quang Dien,

Phong Dien and A Luoi — where both Action for Community Development Center (ACDC) and

International Center (IC) have interventions. The survey did not cover other districts (where ACDC

has provided assistance) due to the lack of approval from the Thua Thien - Hue Department of Labor,

Invalids and Social Affairs (DOLISA) for the survey protocol.

SURVEYED POPULATIONS AND INCLUSION CRITERIA

In Tay Ninh there were two groups of beneficiaries: adults with disabilities who have received direct

assistance from VNAH; and children who have received assistance from VietHealth (under 6 years of

age) and VNAH (from age 7 to 16). In Thua Thien - Hue, direct assistance has been provided by IC

(assistive devices) and ACDC (legal assistance) for both adults and children (under 16 years of age).

The inclusion criteria was persons with mobility disabilities who:

● have received any kind of assistance (rehabilitation therapies, assistive products or legal support)

from Vietnam Assistance for the Handicapped (VNAH), IC, ACDC, or VietHealth, and those

who received rehabilitation therapies had to have been in the program for three months or

longer;

● had at least one round of follow-up by project staff after initiation of the services (only for those

who received rehabilitation-related services); and

● were willing and agreed to participate in the survey.

Persons with disabilities who stopped or dropped out of the program were excluded from the survey.

SAMPLING

In Tay Ninh

By December 2017, a total of 349 children aged 0 to 6 had been assessed, screened and received

interventions (home-based rehabilitation or special/inclusive education) by VietHealth in Tay Ninh. A


“take all” sampling approach was applied using the list of beneficiaries provided. A total of 333 children

who fully met the survey criteria were selected. The sample did not reach the desired size because: 1)

some children left the program; 2) some children and their families migrated to other provinces; and 3)

some caregivers refused to participate in the survey.

All kindergarten teachers who were trained by Viet Health and who provided education interventions

for the children in the sample were recruited and invited to participate in the survey.

There were 381 beneficiaries aged 7 years or older in the list of people with disability provided by

VNAH. The “take all” approach was also used to recruit survey participants. The sample included 270

adults with disabilities and 78 children aged 7 to 16 who were qualified and available for interviewing.

There were 411 children (from 0 to 16 years of age) and 270 adults in the final samples in Tay Ninh.

In Thua Thien - Hue

IC provided lists that consisted of 467 people who had received assistive products through the Project.

There were only 43 adults with disabilities who received legal assistance by ACDC and who agreed to

participate in the survey (based on screening by ACDC). A simple random sampling scheme was used to

recruit 350 participants from the provided lists. The sample included 320 adults and 30 children with

disabilities.

In the final sample for data analysis (590 adults and 441 children aged under 16 years) 681 survey

participants were from Tay Ninh and 350 were from Thua Thien - Hue.

DATA COLLECTION

Survey data was collected from different sources, as described below.

BENEFICIARY INTERVIEWS

A total of 1,031 individual interviews were completed with persons with disabilities recruited for the

survey. The interviews were conducted using a structured questionnaire that covered:

● socio-demographic characteristics of the survey participants;

● self-perceived overall quality of life;

● client (beneficiary) satisfaction;

● level of independence;

● self-perceived degree of social inclusion;

● awareness of disability law and policies; and

● access to public services.

All interviews were carried out by trained enumerators. The data collection team in Tay Ninh was

formed by drawing on the MSI/VEMSS network of survey enumerators. In Thua Thien - Hue, all

interviewers were from the Public Health Department of the Hue University of Pharmacy and Medicine.


Prior to the fieldwork, both data collection teams underwent a two-day training course that covered the

introduction of the survey, the survey questionnaires, interview skills (including role-playing), and field

planning.

Members of the data collection team visited beneficiaries’ homes to conduct the interviews. If the

selected persons with disabilities were unable to answer the interview questions, their main caregivers

(those taking care of the child on a daily basis) were invited for the interview instead. Parents or main

caregivers of the children (under 16 years of age) with disabilities in the survey gave their informed

consent and provided information about their children.

TEACHER INTERVIEWS

In addition to beneficiaries and caregivers, kindergarten teachers of the children with disabilities who

received special/inclusive education interventions were also interviewed. These interviews were used to

record their observations of the children’s improvements after receiving interventions.

In total, 45 kindergarten teachers in Tay Ninh were interviewed in order to collect information/data

about the 135 children participating in the survey.

DATA EXTRACTION FROM CLINICAL ASSESSMENTS

Additional data was extracted from the clinical records of the beneficiaries of the VNAH and VietHealth

Activities. In those programs, persons with disabilities were assessed at baseline (when they were

screened and joined the programs), and again every three months after treatment initiation. For this

survey, the results from the baseline and the last assessment were collected for 681 participants from

these two implementing partners.

DATA ANALYSIS

The data collected were cleaned and analyzed using STATA 13 software. The data on adults and

children were analyzed separately. Data analysis employed tabulation, cross-tabulation and comparisons

(where appropriate) of interested variables.

The changes in quality of life, independence level, physical condition and social inclusion were analyzed at

the individual level (not aggregated for the whole sample). In other words, measurements of physical

function before receipt of intervention services were compared with measurements from the follow-up

assessments (after receiving the intervention services) to identify the level of change in each participant.

For example, if a man was enrolled in the program who could not walk even with help, and after

receiving rehabilitation therapy he was able to do so with assistance, then he was counted as a person

with improved mobility function. The survey results summarize the percentage of beneficiaries who

were improved, remained the same or got worse in mobility function after receiving intervention

services.

Depending on the interested variables, the data were also disaggregated by sex, age group, level of

disability, type of service, and/or duration of service. Disaggregated data were only presented in the

report if they showed a difference between sub-groups or supported meaningful interpretation of the

results.


Throughout the analysis, logistic regression models were developed to identify and test for factors

associated with interested outcomes. Based on prior knowledge, all variables that were potentially

associated with interested outcomes were included in the models. A p-value of 0.05 (or smaller) was

used as the level of statistical significance in the model.

RESULTS

Table 1 summarizes the characteristics of the adults and children with disabilities who participated in the

survey. The mean age of adults with disabilities was 56, with a range from 17 to 96 years of age. Notably,

about 50 percent of adults in the survey were 60 years of age or older. Participating children were 6.4

years old on average, with 40 percent aged under 6. Two-thirds of both adults and children in the

survey were male, which is consistent with data from the annual reports of the implementing partners.

The proportion of participants with very severe disabilities was 17.3 percent among the adults and 19.0

percent among the children. The proportion of adults with mildly severe disabilities was only 6.6

percent, while the rate was 40 percent among children. One explanation for this is that while

interventions for adults focus on people with severe or very severe mobility disabilities, the programs

for children aim at early detection of their disabilities or difficulties. In addition to the 63.2 percent of

survey participants with mobility disabilities, other difficulties identified among children related to

learning (47.4 percent), language and communication (66.7 percent), behavior control (46 percent), and

self-care (48.5 percent).

Table 1: Participants’ Characteristics

ADULTS CHILDREN

% 95% CI* % 95% CI

Age

Mean (year) 56.3 54.8 - 57.8 6.4 6.1 -6.7

Min - Max 17 - 96 - 1 - 16 -

Sex

Male 66.6 62.8 - 70.4 65.5 61.1 - 70.0

Female 33.4 29.6 - 37.2 34.5 30.0 - 38.9

Ethnicity

Kinh 93.6 91.6 - 95.6 94.8 92.7 - 96.9

Ethnic Minority 6.4 4.5 - 8.4 5.2 3.1 - 7.3

Household Economic Status

Poor 23.4 20.0 - 26.8 12.0 9.0 - 15.1

Near Poor 13.4 10.6 - 16.1 9.3 6.6 - 12.0

Neither 63.2 59.3 - 67.1 78.7 74.8 - 82.5

Disability Severity


Very Severe 17.3 14.2 - 20.3 19.0 15.4 - 22.7

Severe 61.0 57.1 - 64.0 32.4 28.0 - 36.8

Mild 6.6 4.6 - 8.6 40.1 35.5 - 44.7

Undetermined 15.1 12.2 - 18.0 4.5 2.6 - 6.5

* CI = Confidence Interval

Table 2 shows the types of Project interventions received by the survey beneficiaries from 2016 through

the end of 2017. Rehabilitation services and provision of assistive products were the most common

interventions for adults with disabilities. There was a difference between Tay Ninh and Thua Thien -

Hue (due to the project design): 75.2 percent of the sample population in Tay Ninh received

rehabilitation services from VNAH, while the rate was 8.4 percent in Thua Thien - Hue (IC only

provides assistive products). In Thua Thien - Hue, 90.6 percent of respondents received assistive

products while the rate was 59.6 percent in Tay Ninh.

Among children, education interventions reached 31 percent of the sample population, and

rehabilitation services were provided to 58.7 percent of them.

The mean time interval between when people received and started using provided services to the time

of the survey was 13 months for adults with disabilities and 9 months for children with disabilities.

Table 2: Type of Assistance Respondents Received in Past Two Years

ADULTS CHILDREN

% 95% CI % 95% CI

Home-based rehabilitation 33.6 29.7 - 37.4 28.6 24.3 - 32.8

Facility-based (hospitals or commune health

centers) rehabilitation 15.8 12.8 - 18.7 45.4 40.7 - 50.0

Assistive products 76.4 73.0 - 79.9 17.7 14.1 - 21.3

Legal assistance (for adults only) 8.0 5.8 - 10.2 - -

Special education (children only) - - 31.3 26.9 - 35.6

Reference given for other specialized health

services (for children only) - - 16.3 12.9 - 19.8

LEVEL OF SATISFACTION WITH PROVIDED SERVICES

More than half of beneficiaries who received home-based or facility-based rehabilitation said they were

satisfied or very satisfied with the services. Approximately 78 percent of those who received assistive

products and 83 percent of those who received legal assistance expressed a high level of satisfaction

with the products, significantly higher than the rate of satisfaction among people who received

rehabilitation services (see Figure 1 below).


About 20 percent of beneficiaries were not satisfied with the rehabilitation services they received.

Qualitative information collected during the fieldwork revealed some of the key reasons for the

dissatisfaction: 1) the participants’ condition did not improve as they expected (survey data show that 39

percent of beneficiaries of rehabilitation services reported that the services did not meet their needs); 2)

there was no helper available for people with serious conditions who received home-based

rehabilitation services; and 3) people with long-term disabilities did not believe that their condition could

be improved by rehabilitation (almost 60 percent of respondents said they do not believe that their

disability can be improved).

Approximately 12 percent of adults who received assistive products were not satisfied with them. Key

reasons included: 1) the perceived low quality of the products; 2) the product, especially wheelchairs,

was not useable in the area where the beneficiary lives; 3) the size of the product (either too big or too

small); and 4) the beneficiary felt pain when using some of the products (e.g., artificial legs).

Figure 1: Adult Beneficiaries’ Satisfaction with Provided Services

The proportion of parents who were satisfied or very satisfied with the services provided for their

children ranged from 66 percent for rehabilitation services to 77 percent for assistive products and 85

percent for education interventions, as shown in Figure 2. While the reasons for not being satisfied

were similar to those mentioned by adult beneficiaries, slightly more than 80 percent of parents whose

children received education interventions reported that their child’s functions had greatly (33 percent)

or partially (50 percent) improved.

13.9%

42.6%

22.6%

16.5%

4.3%

43.5%

34.6%

4.4%

11.5%

0.6%

38.3%

44.7%

17.0%

0.0% 0.0%

Very satisfied Satisfied Neither satisfied nor

dissatisfied

Dissatisfied Very dissatisfied

Rehabilitation services Assistive products Legal assistance


Figure 2: Parent’s Satisfaction with Services Provided to their Children

IMPROVEMENT IN SELF-PERCEIVED QUALITY OF LIFE

More than 50 percent of respondents reported improvement in both overall health and quality of life

following the project interventions. From one-fourth to one-third experienced no change in their quality

of life and their health status. And about 12 percent of respondents reported that their health worsened

after receiving assistance. All of these individuals were in a serious and progressive condition at the time

the survey team visited. There was no difference between males and females regarding improvements in

health and quality of life.

Figure 3: Changes in Self-Perceived Overall Health and Quality of Life among Adult

Beneficiaries

24.8%

41.1%

16.7%

7.8%9.7%

44.9%

32.1%

7.7%

11.5%

3.8%

37.0%

48.6%

8.7%

5.1%

0.7%

Very satisfied Satisfied Neither satisfied nor

dissatisfied

Dissatisfied Very dissatisfied

Rehabilitation services Assistive products Education

3.1%

50.0%

34.7%

10.8%

1.4%6.9%

59.5%

25.1%

7.6%

0.8%

Much better Better No change Worse Much worse

Overall health Overall quality of life


Regarding the children in the survey, 38 percent of parents/caretakers interviewed said that their

children’s health has improved since receiving the Project’s interventions. Among those parents who

said that their children’s health has not changed following assistance (60 percent of interviewed parents),

many of them assessed the physical health of their child as normal or good before the intervention; they

said their children just had problems with communication/language and/or cognitive ability. In only a few

(six) cases had the children’s health worsened; these were children with cerebral palsy or

hydrocephalus.

Figure 4: Children’s Overall Health and Quality of Life after Intervention, as Reported by

Parents

INDEPENDENCE IN DAILY ACTIVITIES

11.3%

27.0%

60.3%

0.7% 0.7%

17.5%

37.9%

43.5%

0.7% 0.5%

Much better Better No change Worse Much worse

Overall health Overall quality of life

Mr. LXT, a 72 year-old man in Tay Ninh, had a stroke in 2013. After several months of hospital

treatment in Tay Ninh and HCMC, he almost recovered and was able to function normally.

However, he had additional strokes in 2016 and 2017. Currently Mr. LXT is unable to perform any

of his daily living activities by himself. His physical condition had gotten steadily worse. Over the last

several months he stopped communicating. His wife and his daughter were provided with some

guidance on how to better care of him, to avoid and prevent possible consequences (such as skin

ulcers or pneumonia) from lying in bed for long periods. Although all family members highly

appreciated the assistance from the Project, they did not observe any improvement in the patient’s

physical condition.


Overall, 32 percent of beneficiaries said that the assistance provided has helped to reduce their

dependence on family members for conducting their daily activities (6 percent of beneficiaries no longer

need any assistance from family members).

For the sample of adults with disabilities, the level of independence was measured by the Barthel Index

of Activities of Daily Living.16 The scale includes 10 items, with a maximum score of 100, measuring the

patient’s performance in basic daily activities: bowels, bladder, grooming, toilet use, feeding, transfers,

mobility, dressing, stairs, and bathing. Each item assesses respondent’s ability to perform a specific

activity, with a performance range for each activity from being unable or dependent on a helper to fully

independent. If the total Barthel Index score is less than 45, a person is considered fully dependent. A

total score of between 50 and 85 indicates that a person is partially dependent. If the score is 90 or

greater, the assessed person is considered fully independent.

Before enrolling in the intervention program, 40 percent of people interviewed reported that they were

fully dependent, and 35 percent said that they were partially dependent. Figure 5 shows the percentages

of respondents who have challenges with specific activities. More than half of respondents reported

difficulties in using stairs and in walking independently for more than 50 meters.

Figure 5: Percentage of Adult Respondents who Could not Perform Daily Activities

Independently

16 COLLIN C, WADE DT, DAVIES S, HORNE V. THE BARTHEL ADL INDEX: A RELIABILITY STUDY. INT DISABIL STUD. 1988; 10(2):61-63.


Figure 6 shows significant changes in independence among beneficiaries following the interventions. The

percentage of respondents who were fully independent in their living activities increased from 39

percent17 prior to receiving services to 51 percent after receiving them.18

Over a quarter (26 percent) of the group of people who were fully or partially dependent at baseline

showed improvement in their level of independence after receiving intervention services. Only 3 percent

(6 people) had a decline in level of independence following the intervention services.

Among the 229 people who were fully independent before receiving the intervention services, only 6 (3

percent) became dependent on a helper.

Figure 6: Adult Beneficiaries’ Improvement in Independence following Intervention

Services

17 95% CI: 34.9% - 42.8%. 18 95% CI: 46.8% - 54.9%.

9.2%

10.0%

24.6%

30.7%

31.7%

38.8%

43.2%

44.7%

54.4%

68.5%

Bladder

Bowels

Feeding

Grooming

Toilet use

Dressing

Transfer

Bathing

Mobility

Stairs

26.6%

34.6%38.8%

25.1% 24.1%

50.8%

Fully Dependent Partially Dependent Fully Independent

Before After


Figure 7 shows levels of improvement following program interventions for adult respondents who have

experienced difficulties with specific daily activities. Approximately 41 percent of respondents who

previously had difficulties with mobility — the ability to walk independently — reported an

improvement. More than 20 percent of respondents reported an improvement in transfers (from bed to

chair) and in toilet use.

Use of stairs was the most common difficulty among all respondents in the sample. However, only 16

percent reported an improvement in this area following the Project inventions. In addition, less than 15

percent of respondents reported an improvement in other activities including feeding themselves,

bathing and grooming. However, very few reported that their performance in daily activities worsened

after receiving assistance from the project. The responses given by the surveyed persons with disabilities

were consistent with clinical assessments of them performed by Project staff (doctors and therapists).

Figure 7: Improvements in Specific Daily Living Activities following Program Interventions

9.9%

14.4%

15.0%

15.2%

16.3%

17.0%

18.5%

22.4%

29.0%

41.1%

0.9%

3.0%

1.7%

1.9%

1.6%

2.8%

1.3%

90.1%

84.7%

85.0%

84.8%

80.7%

81.4%

79.6%

76.0%

68.2%

57.6%

Grooming

Dressing

Bathing

Feeding

Stairs

Bowels

Bladder

Toilet use

Transfer

Mobility

Improved Worse Remained the same

Mr. NTL, a 65 year-old man in Tay Ninh, experienced a stroke in 2013 after living with

hypertension for a long time. He was discharged from the hospital after a month in the extensive

care unit and traditional medicine department (where he received some rehabilitation services). He

has suffered from hemiparesis since being discharged. His wife assists with all of his daily living

activities.

Project staff showed Mr. NTL and his wife in early 2017 how to carry out physical exercises at

home to improve his condition. After almost a year, his most visible improvement is his ability to

feed himself without any help. His communication with others has also been improved. However,

he still needs support or help from others for other functional activities.


There were differences between the groups of persons with disabilities that received different types of

intervention services (rehabilitation therapies, assistive products, or both). A higher proportion of those

who received assistive products reported improvements in their mobility and transfer independence

(from bed to chair) compared to the persons with disabilities who received rehabilitation therapies: 52

percent versus 18 percent, respectively.

Surprisingly, respondents who had received services (rehabilitation or assistive products) within the last

six months prior to the survey were more likely to report improvements in their independence than

those who received the services for a longer period. This might be due to:

● Reporting bias — People who recently received the service were more aware of the assistance and

thus were more likely to notice and report their improvement than those who were in the program

for a long time.

● Composition of the sample — The majority of persons in this group (those who received the

service for less than six month) were provided with assistive products only (persons with disabilities

who received rehabilitation therapies for three months or less were excluded), and therefore

persons in this group were more likely to report independence improvement than those who

received other services.

There was no difference found in improvement in mobility independence due to sex, age, or different

levels of disability severity.

Clinical Assessments of 270 Persons with Disabilities Enrolled in VNAH Program in Tay Ninh

Persons with disabilities’ mobility — both indoors and outdoors — was assessed using a four-item

scale (no difficulty was coded as zero, while levels of difficulty were coded from 1 to 4, with 4

indicating the person is unable to perform the activity). At baseline (before intervention) 69 percent

of beneficiaries had difficulty walking and moving inside their house and 79 percent had mobility

difficulties moving around their community.

Improvements were found following the VNAH Program interventions:

• Among those who had difficulty moving around indoors at baseline, 52 percent

improved their mobility inside their house, while 39 percent showed no change.

Mobility decreased for 9 percent of the individuals in this group.

• Among those who had difficulty with mobility outdoors at baseline, no change was

observed in 49 percent of this group. Improvement was observed in 43 percent, and 8

percent reported less mobility.

There were also 16 people who had no difficulties with mobility at baseline but who developed

problems after receiving intervention services.


Table 3: Improvements in Mobility, Transfer and Toilet Activities by Service Type and

Duration of Intervention

MOBILITY TRANSFER TOILET

% 95% CI % 95% CI % 95% CI

Type of intervention service

Rehabilitation therapies only* 18.3 8.3 - 28.4 18.0 7.0 - 29.0 19.0 8.6 - 29.4

Assistive products only 51.8** 44.2 - 59.4 37.9** 29.2 - 46.6 26.1 17.8 - 34.4

Rehabilitation and assistive products 39.8** 29.0 - 50.5 21.9 12.2 - 31.6 20.0 10.4 - 29.6

Duration of intervention

<=6 months* 58.8 48.8 - 68.7 50.7 38.8 - 62.6 37.7 25.2 - 50.2

6 - 12 months 31.8** 17.5 - 46.1 17.9** 5.3 - 30.6 20.5 7.3 - 33.8

>12 months 35.9** 28.6 - 43.1 21.9** 14.9 - 28.9 17.3** 11.0 - 23.7

* Reference group

** p-value was less than 0.05 (compared with reference group)

A logistic regression model was developed to identify potential factors related to any self-reported

improvements in the ability to perform daily activities independently (the interested outcome) among

beneficiaries. Age, sex, disability severity, time with disability, type of received service, and duration of

intervention (time interval from date of initiation to the interview date) were included in the model.

Adjusted odd ratios (ORs) were estimated with a p-value of 0.05 or less as a statistically significant level.

The final results suggest that the following beneficiaries are more likely to report independence

improvement in any of 10 daily activity domains: persons with disabilities who received assistive

products or assistive products plus rehabilitation therapies; persons who have had disabilities for less

than four years; and persons who have received services for six months or less. The data are presented

in Table 4 below.

Table 4: Factors Associated with Independence Improvements

REPORTED IMPROVEMENT ADJUSTED OR* 95% CI P-VALUE

Type of intervention service

- Rehabilitation therapies only 1

- Assistive products only 2.2 1.1 - 4.2 0.018

- Rehabilitation and assistive products 3.5 1.8 - 6.7 0.000

Time with disability

- At birth or 5 years or longer 1

- 3 – 4 years 2.8 1.4 - 5.5 0.003

- 1– 2 years 2.1 1.0 - 4.3 0.041

- Less than a year 2.5 1.1 - 5.8 0.030

Duration of intervention

- 6 months or less 1

- 6 – 12 months 0.4 0.2 - 0.9 0.017


- 12 months or longer 0.5 0.3-0.8 0.002

CHILDREN’S DEVELOPMENT

In the survey, the development of children was assessed in six domains: gross motor skills (12

movements that involve using large muscles of the body for activities such as crawling, sitting, standing

or walking); fine motor skills (two movements that involve using the small muscles in children’s hands

and forearms); learning (seven skills); language and communication (15 skills); behavior control (11

abnormal behaviors); and self-care (11 skills). In the interviews, children were identified with the various

domains if they had difficulty performing any specific activity or were unable to control abnormal

behaviors.

A five-item scale was used to evaluate children’s ability: not difficult (normal), slightly difficult, quite

difficult, very difficult, and unable to perform or control the activity. A child may have issues with more

than one item in the same domain, and/or in more than one domain.

Figure 8 shows the percentage of children surveyed who had development impairment in the six

domains. About two-thirds of children in the sample had difficulties with language/communication and

gross motor skills. About one-third had challenges with fine motor skills, and nearly 50 percent had

problems with their ability to take care of themselves, learn new things, or control their behaviors.

Figure 8: Percentage of Children with Impairments in Six Domains of Development

Parents were then asked to report observed changes in their children after receiving services

(rehabilitation, education, and assistive devices) from the program. Figure 9 below summarizes the

percentage of children whose condition/ability was improved (or got worse) in specific domains,

according to their parents. Since a child may have challenges with more than one skill in the same

domain, if there was improvement in one of those skill areas then he/she was counted as “had

improvement.” The red bar represents the percentage of children who experienced greater difficulty

with at least one functional skill after receiving services. It is possible that a child saw improvements with

some specific skills/abilities while also having greater difficulties with others.

32.7%

46.0%

47.4%

48.5%

61.5%

66.7%

Fine Motor Skills

Behaviors

Learning

Self-care

Gross Motor Skills

Language and Communication


Figure 9 simply provides a visual overview of changes in children’s functional ability as observed and

perceived by their parents; the changes were not based on results from clinical assessments, which can

provide a more accurate picture of the children’s conditions.

Figure 9: Reported Change in Children’s Development following Program Interventions

Parents reported the most improvement in their children

in the domains of gross motor skills (68 percent) and

behavior control (65 percent). They also observed nearly

60 percent improvement among their children in

communication and language-related skills. The least

improvement was in the domain of self-care: parents

reported that 43 percent of children who had difficulty

taking care of themselves experienced an improvement

after receiving program services.

On the other hand, the ability to perform functional

skills/activities was reportedly worse after intervention

for a few children (about six percent or less in all six

domains).

Figure 10: Improvements in Development Domains by Severity of Disabilities

42.5%

47.9%

48.3%

58.5%

64.5%

67.5%

5.1%

2.1%

0.5%

4.4%

5.4%

5.9%

Self-care Skills

Fine Motor Skills

Learning Skills

Language and Communication Skills

Behavior Control

Gross Motor Skills

Got worse Improved (to some extent)

Results from Clinical Assessments of

Children who Received Rehabilitation

Services from VietHealth

Gross motor function, measured using

the Gross Motor Function

Classification System (GMFCS), was

improved in 47.6 percent of children.

Manual ability, measured using the

Manual Ability Classification System

(MACS), was improved in 56.4

percent of children.


There was no significant difference between male and female children, but there were differences

between groups of children with different severity levels. More improvement was observed and

reported in the group of children with mild disabilities compared to children with severe disabilities. For

example, parents observed improvement with gross motor skills in about 76 percent of children with

less severe disabilities, while the improvement rate was 47 percent among children with very severe

disabilities.

To test for the factors that may be related to observed improvements in children’s skills/abilities, a

logistic regression model was used in each specific development domain to analyze children’s condition

after intervention (improved or not). Variables included in the model included age, sex, disability severity

level, and intervention time. Type of intervention (rehabilitation therapy, assistive products, and

education) were also added to the model. The results are presented in Table 5 below. Cell values

indicate the estimated AOR for the variables found to be factors significantly associated with the

interested outcome (with p-value <0.05). For example: the OR is 2.8 for the severe disability group

under the gross motor function domain. This can be interpreted to mean that improvement in gross

motor function was 2.8 times more likely to be observed in the group of children with severe disabilities

than in the group of children with very severe disabilities.

The results show that, regardless of service type, age was a factor in two domains (gross motor function

and behavior), severity of disability was related to intervention outcome in all domains, and intervention

length was associated with the intervention outcome in two domains (gross motor function and

51.6%

63.6%

59.3%

68.0%

70.0%

75.9%

28.8%

27.6%

13.5%

30.6%

44.7%

46.7%

Self-care Skills

Fine Motor Skills

Learning Skills

Language and Communication Skills

Behavior Control

Gross Motor Skills

Very Severe Severe/Mild


learning).

Table 5: Factors Associated with Improvements in Children’s Development

INTERESTED OUTCOME

GR

OSS M

OT

OR

FU

NC

TIO

N

FIN

E M

OT

OR

FU

NC

TIO

N

LEA

RN

ING

LA

NG

UA

GE A

ND

CO

MM

UN

ICA

TIO

N

BEH

AV

IOR

S

SELF-C

AR

E

Age 0.84 - - - 0.87 -

Sex - - - - - -

Level of severity (compared with very severe

group)

Severe 2.8 4.5 4.9 - 3.7 -

Mild 3.2 23.4 13.4 3.4 2.3 2.7

Intervention time (compared with children who

received intervention for less than 3 months)

3 – 6 months 2.8 - - - - -

6 months or longer 3.3 - 6.1 - - -

SOCIAL INCLUSION

There is no clear definition of social inclusion in the Project document. There is also no validated and

standardized measurement scale for social inclusion for any vulnerable group in Vietnam. In this survey,

respondents were asked if they felt they were more included in society, without any definition given by

the interviewers. Almost 50 percent of the survey respondents reported that their level of social

inclusion (or that of their children) has improved, as shown in Figure 11.

Figure 11: Self-Perceived Changes in Social Inclusion Following Interventions

Social Participation

46.7% 49.6%44.1%

40.5%

Adults Children

Yes - My social inclusion has been improved No Don't know


Overall, about 50 percent of all respondents said (perceived) that they were more included in society

since being enrolled in the intervention program. The interviews also included questions about specific

aspects of social inclusion, including participation in family and community activities, and experience of

stigma and/or discrimination.

At a minimum, social inclusion in this survey was defined as the extent in which persons with disabilities

have more opportunity to be exposed to and participate in community activities. Figure 12 shows the

percentage of adults and children who reported changes in their opportunities to go out and socialize in

the community after they received intervention services.

Figure 12: Changes in Social Participation

Around one-third of adults with disabilities said they had more

opportunity to go out and interact with members of their community thanks to improvements in their

mobility function. Remarkably, 45.9 percent of adult beneficiaries felt more confident when they went

out or shared their thoughts, and 28.5 percent perceived that what they did and shared was more

appreciated by others.

Forty-one percent of parents reported that their children’s ability to go out and play with others

increased after they received program services — not only due to improvement in their mobility

function but also because of improvement in their communication skills. Over 45 percent of children

said they were more confident and open to sharing and expressing their thoughts with family members

and others.

There was no difference between male and female beneficiaries for these estimates.

Family Life

Another aspect of social inclusion is one’s relationship with other family members and the ability to join

in family activities (including housework). In this survey, 24 percent of persons with disabilities said they

received more attention, sharing of thoughts and sympathy from their family members after they

received program services, and 16 percent started doing some simple housework to help the family. It

should be noted that although 75 percent of respondents reported no change in the way family

35.9%

27.6%

45.4%

41.0%

61.9%

70.0%

54.4%

58.8%

2.2%

2.4%

0.2%

0.2%

Adults: Opportunity to socialize with the

community

Adults: Participation in community

activities

Children: More open to share

thoughts/opinion

Children: Participation in community

activities (playing with other children)

More No change Less

Of the 590 adults interviewed, 138

(23.4 percent) said they desire to

work. Among those who want to

work, 29 (21 percent) thought that

their opportunity to find a job had

increased after they received

program services.

Of the 441 children participating in

the survey, 397 (90 percent) were

in kindergarten or wanted to go to

school. About 47 percent of

parents of the children assessed

said that their children’s learning

ability and opportunity to study

had increased since they received

program services.


members treated them, this does not necessarily mean their family members did not take care of them

well.

After participating in the project, 55.6 percent of parents reported that their knowledge and skills

relating to helping and playing with their children improved. Almost 53 percent said that their children

became more open and played and communicated more with other family members after they received

program services.

Figure 13: Changes in Family Life for

Adults

Figure 14: Changes in Family Life for

Children

Stigma and Discrimination

Stigma and discrimination are other important aspects of social inclusion for persons with disabilities.

They potentially prevent persons with disabilities from joining in community activities. In this survey, 25

percent of adults and 42 of children with disabilities, respectively, said they still experienced stigma and

discrimination.

When asked for their overall impressions about the level of stigma and discrimination against persons

with disabilities in their communities, 10 percent of adults with disabilities and 23 percent of parents of

children with disabilities said that stigma and discrimination have decreased (i.e., the situation has

improved) in the past two years, while about 50 percent of respondents said they have never

experienced these problems.

14%

8%

11%

9%

75%

83%

Care, attention and

sympathy from family

member

Doing housework

Much more More No change Less Much less

26%

28%

30%

25%

44%

47%

Parent's skills to care for and

play with children

Playing and communicating

with others in the family

Much more More No change Less Much less


Access to Services

During the interviews, respondents were asked questions

about accessibility to services for persons with disabilities, in

addition to the services they received from the Project (i.e., “In

addition to what you are currently receiving, do you know if

the following services are available in your community

[province, district or commune]?” and “Have you been able to

receive or access these services?”). The services listed included

social welfare (monthly allowance) and health insurance as

these are benefits included in government policies for persons

with disabilities (who are certificated by a local authority).

The results to these questions are presented in Figure 15

below. Note that these figures do not represent the whole

population of persons with disabilities in the survey site; the respondents to these questions were

derived from the group of persons with disabilities who have received Project services , and thus are

more likely to have information on services available locally than others who were not included in the

Project.

Figure 15: Accessibility of Support Services

In addition to a monthly allowance, more than 50 percent of respondents reported that health check-

ups are accessible for them. This might be due to the high coverage of health insurance (more than 90

percent of the surveyed population) and the availability of health services in the healthcare system,

which exists from commune to province levels. The survey results indicate that 13 percent of children

and 23 percent of adult beneficiaries knew about and were able to access locations where assistive

products are sold. One-third of adults said that hospital/clinic-based rehabilitation was accessible for

them. Fewer parents of children with disabilities knew about and were able to access rehabilitation

services for their children.

76.8%

65.6%

8.3%

23.2%

11.7%

29.8%

26.8%

66.4%

51.7%

46.0%

13.4%

10.9%

13.2%

14.5%

Monthly allowance

Routine health check-up

Health check-up specialized for persons with disabilities

Assitive products

Home-based rehabilitation

Facility-based rehabilitation

Legal assistance

Adults Children

“My daughter was diagnosed

with cerebral palsy. We have

taken her around the country

over the past 10 years. Last year

we were informed that our

district hospital opened a unit

that provides therapy for her. It

was a great news for us as it

only takes us five minutes to

bring our daughter there…”

Father of an11-year-old

girl with cerebral palsy


These percentages are much higher than those reported in a 2015 survey conducted in the same area

among persons with mobility disabilities. In the 2015 survey, about one percent (or less) of persons with

disabilities said they were aware of rehabilitation services, and less than six percent said they were

aware of locations to access assistive products. The estimates from the more recent survey were all

higher, possibly because:

● as noted above, beneficiaries were given information about the services as part of the program

intervention; and/or

● more services have become available locally since 2015 in the survey sites.

GENDER

With regards to satisfaction with intervention services, quality of life, independence improvement and

social inclusion, the survey data show no difference between male and female participants (both adults

and children).

Nevertheless, the survey found some potential gender-related problems. First, as shown in Table 1, male

participants accounted for two-thirds of the sample. Since the sampling approach aimed to obtain a

representative sample, the results should reflect the fact that more males were included in the

intervention programs. Cross-checking with project monitoring data reaffirmed this. The number of

male beneficiaries is about twice the number of female beneficiaries. In the scope of this survey there is

not enough evidence to conclude that males with disabilities are more likely (or have more opportunity)

to receive project services than females with disabilities. However, it would be worth re-visiting this

question as part of future Project implementation.

Secondly, as shown in Figure 16, the majority of caregivers of persons with disabilities in the survey

were female. There are many possible reasons for this. Among them is the traditional division of labor

within families; men are more likely to go out to work, and women are more likely to stay home and

take care of family member with disabilities. Another possible reason is that women perceive themselves

as better in taking care of people with disabilities. However, these reasons should not necessarily result

in an imbalanced role of men and women in taking care of persons with disabilities.

Figure 16: Main Caregivers of Persons with Disabilities

14.5%

18.6%

85.5%

65.3%

16.1%Adults with disabilities

Children with disabilities

Male caregiver Female caregivers Don't have (or need) caregiver


Lastly, very few respondents were aware of the issue with gender-based violence. Less than six percent

of persons with disabilities have ever heard about violence or abuse related to gender. This is important

as the project has provided significant training on gender-based violence to persons with disabilities in

the survey site. It is possible that the training targeted different groups of persons with disabilities than

those who received direct assistance and were recruited for participation in this survey.

CONCLUSIONS

The following conclusions summarize the key results from the survey data. The survey represents the

population of persons with disabilities in Tay Ninh and Thua Thien - Hue who have received direct

assistance (rehabilitation therapies, assistive products, and legal assistance) from the USAID-supported

Vietnam Intervention Project to Assist Persons with Disabilities. These conclusions take into account

the fact that the Project has been implemented for only about two years, and, on average, each

beneficiary has been involved in the intervention for about nine to thirteen months.

1) The Project has reached a group of adults with disabilities with the following characteristics: mean

age of 56 years old; mostly male (67 percent); primarily ethnic majority (94 percent); majority with

severe disabilities (61 percent); and most identified with disability at birth or more than five years

ago (80 percent). The group of children involved have a mean age of 6 years; consist of more boys

(66 percent) than girls (35 percent); are mostly Kinh ethnic majority (95 percent); and most have a

severe or mild disability (73 percent).

2) The majority of beneficiaries say they are satisfied with the services provided by the Project:

depending on the type of service they received, from 56 percent (rehabilitation services) to 85

percent (education intervention) said they were satisfied or very satisfied with the Project services.

Assistive products and educational interventions (for children) were appreciated more than

rehabilitation therapies.

3) Key reasons for not being satisfied were: 1) the therapies did not meet expectations (i.e., the

condition of persons with disabilities did not improve as they expected); and 2) the low quality of

the assistive products and/or inappropriateness of them for use in local areas.

4) A total of 53 percent of adult beneficiaries reported improvements in their overall health and quality

of life, while 38 percent of interviewed parents reported improvements in their children.

5) The survey provides strong evidence of intervention impacts on persons with disabilities’

independence in daily living activities:

● The percentage of beneficiaries able to perform their daily living functions independently

increased from 39 percent to 51 percent (from baseline to the time of the survey).

Specifically, the greatest improvement was observed and reported in mobility: 41 percent of

people who had problems with mobility function reported an improvement following

Project assistance. Improvements were also reported in all other domains of daily living

activities. The estimates derived from self-reporting by beneficiaries were all consistent with

results from clinical assessments conducted by the implementing partners.

● Individuals in the group that received assistive products reported greater improvement in

independence compared to the group benefitting from rehabilitation therapies. Either

rehabilitation therapies or assistive products could have great impact on people’s

independence if provided within five years after individuals are identified with a disability.

6) The Project interventions also had positive impacts on children’s development.


● Positive changes were reported in 43-68 percent of children involved in the Project in the

six domains of children’s development (gross motor function, fine motor function, learning,

language and communication, behavior control, and self-care).

● Worsened abilities were reported in less than six percent of the children with disabilities.

● Factors associated with reported improvements in children’s development included younger

age, less severe disability, and being in the intervention program for three months or longer.

This reaffirms the effectiveness of early identification and intervention for children with

disabilities.

7) About half of both adults and children with disabilities reported being more included in society after

receiving intervention services. More specifically:

● One-third of adults with disabilities said they had more opportunities to participate in

community activities after receiving Project services. This figure was about 40 percent for

children with disabilities.

● Persons with disabilities participated more in their family life after receiving Project services;

including more respect, care and attention from other family members. Children had more

opportunities to play with other family members, and their parents learned more about the

skills needed to support play and care for children with disabilities. Nearly half of

interviewed parents thought that the intervention services increased their children’s

opportunities to attend school.

● Access to disability-related services among beneficiaries was also greatly improved (based on

comparison with results from the survey conducted in 2015 in the same areas).

On the other hand, one-fourth of adults in the survey reported personal experience with stigma and

discrimination against persons with disabilities. Experience with stigma and discrimination was

reported by 40 percent of the parents of children with disabilities.

8) There was no difference between male and female beneficiaries with regards to perceived impacts of

the intervention services. However, the survey data flagged a few potential gender-related issues

that would be worth considering in future Project implementation:

● more males than females were included in the intervention programs;

● the majority of caregivers of persons with disabilities were women; and

● awareness of gender-related violence among persons with disabilities was low.

RECOMMENDATIONS

1) USAID should continue providing the intervention services including rehabilitation therapies and

assistive products, as they are highly valued and appreciated by persons with disabilities.

2) Recommendations to consider for future implementation include the following:

● Identify persons and provide them with interventions early (soon after they are diagnosed

or acquire a disability), as interventions started early have greater impact in improving

physical capacities and quality of life.


● Provide an intervention package that includes both rehabilitation therapies and assistive

products.

● Provide more detailed information about rehabilitation therapies to patients in order to

manage their expectations.

● Provide interdisciplinary services — including physical, occupational, and speech therapies —

as these will have greater impacts on beneficiaries. Significant numbers of persons with

disabilities in the survey receive home-based rehabilitation that includes only physical

therapy.

● Develop a quality control procedure to ensure that assistive products are suitable and useful

for persons with disabilities. Resources for maintenance and repair of the assistive products

should be in place. Those receiving the assistive products should be monitored closely to

make sure the products are being used properly.

3) There is a need for an assessment of the potential impacts of current gender-focused interventions.

The assessment should include verification of the male-to-female ratio among persons with

disabilities.

4) USAID and implementing partners should develop standard patient assessment tools and set up a

system able to provide clinical monitoring data, not only for reporting but also for clinical use.

LIMITATIONS

There were a number of limitations that need to be taken into account when interpreting the data

obtained in this survey.

1) The survey was designed to be a cross-sectional representation of beneficiaries. While respondents

were asked to report their status before and after they received intervention services, the

responses might be subject to recall bias with unknown direction.

2) The process for selection of persons with disabilities who received legal assistance has limitations.

For confidential reasons, the survey team could not make direct contact with these beneficiaries.

The screening was done by the project team and only 43 people (out of hundreds) volunteered to

participate. Therefore, this portion of the sample might not fully represent for the group of those

who had received legal assistance.

3) The study lacked a control group; all participants received intervention services. It is not statistically

feasible to make any causal inference if the services lead to self-reported changes among persons

with disabilities. For example, if a man reported improvement in his independence after he received

the service, it could be due to the rehabilitation service or the fact that his condition would have

progressed naturally even without the intervention.

4) During the design phase, the survey team planned to use a significant amount of clinical data from

the implementing partners; it was expected that patient files would include periodic patient

assessments to monitor patients’ clinical progression and improvement. However, existing data from

beneficiaries’ clinical records did not meet the survey team’s expectations due to: 1) incomplete

assessments (missing data); 2) the fact that clinical assessments were not conducted periodically due

to the lack of a standard protocol (some patients were reassessed every three months but others

were not); and 3) lack of a standard assessment tool used by all implementing partners. Therefore,

the use of clinical data was very limited in the survey.


ANNEX 1X: REFERENCES

Action to the Community Development Center. (2016). Protecting the Rights of Persons with Disabilities -

Annual Report. Reporting Period: October 2015 - September 2016. Hanoi: USAID.


Annual Report. Reporting Period: October 2016 - September 2017. Hanoi: USAID.


Monitoring & Evaluation Plan. Hanoi: USAID.


Monitoring & Evaluation Plan. Hanoi: USAID.

Action to the Community Development Center. (2015). Year 10/2015 – 9/2016 Work Plan. Hanoi:

USAID.


USAID.


USAID.

Disability Research and Capacity Development. (2016). Accessibility for inclusion - Annual Report. Reporting

Period: 1st October, 2015 until 30th September, 2016. Hanoi: USAID.

Disability Research and Capacity Development. (2017). Accessibility for inclusion - Annual Report. Reporting

Period: 1st October, 2016 until 30th September, 2017. Hanoi: USAID.

Disability Research and Capacity Development. (2017). Accessibility for inclusion - Midterm Report.

Reporting Period: 1st October, 2016 until 31st March, 2017. Hanoi: USAID.

Disability Research and Capacity Development. (2015). Accessibility for inclusion - Monitoring and Evaluation

Plan. Hanoi: USAID.

Disability Research and Capacity Development. (2015). Accessibility for inclusion - Year 1 Work plan. Hanoi:

USAID.

Disability Research and Capacity Development. (2016). Accessibility for inclusion - Year 2 Work plan. Hanoi:

USAID.

Handicap International. (2016). Advancing Medical Care and Rehabilitation Education - Annual Report.

Reporting Period: 1st October, 2015 until 30 September, 2016. Hanoi: USAID.

Handicap International. (2017). Advancing Medical Care and Rehabilitation Education - Annual Report.

Reporting Period: 1st October, 2016 until 30 September, 2017. Hanoi: USAID.


Handicap International. (2015). Advancing Medical Care and Rehabilitation Education - Monitoring and

Evaluation Plan. Hanoi: USAID.

Handicap International. (2017). Advancing Medical Care and Rehabilitation Education - Monitoring and

Evaluation Plan. Hanoi: USAID.

Handicap International. (2016). Advancing Medical Care and Rehabilitation Education - Work Plan Year

1(2015 - 2016). Hanoi: USAID.

Handicap International. (2017). Advancing Medical Care and Rehabilitation Education - Work Plan Year 2

(2016 - 2017). Hanoi: USAID.

Handicap International. (2018). Advancing Medical Care and Rehabilitation Education - Work Plan Year 3

(2017 - 2018). Hanoi: USAID.

International Center. (2016). Moving Without Limits - Annual Report. Reporting Period: July 1st – Sep 30th,

2016. Hanoi: USAID.

International Center. (2017). Moving Without Limits - Annual Report. Reporting Period: July 1st – Sep 30th,

2017. Hanoi: USAID.

International Center. (2016). Moving Without Limits - Monitoring and Evaluation Plan. Hanoi: USAID.

International Center. (2015). Moving Without Limits - 2016 Work Plan. Hanoi: USAID.



Mary, H., Hoang, T., Tra, N., Long, T., & Chang, L., & Management Systems International (2015).

Evaluation of Vietnam Disabilities Program. Hanoi: USAID

Ministry of Construction. (2014). QCVN 10:2014/BXD - National Technical Regulation on Construction for

Disabled Access to Buildings and Facilities. Hanoi.

Ministry of Education and Training. (January 29, 2018). Circular 3/2018/TT-BGDDT - Inclusive education for

the disabled. Hanoi.

Ministry of Health. (December 11, 2013). Circular 43/2013/TT-BYT - Detailing Levels of Technical Service

delivery applicable to Health Facilities. Hanoi

Ministry of Health. (December 26, 2014). Circular 50/2014/TT-BYT - Classification of Surgeries, Medical

Procedures and Personnel Norms applied to Operations, Medical Procedures. Hanoi.

Ministry of Health. (June 30, 2016). Circular 18/2016/TT-BYT. The list of rehabilitation techniques that are

covered by Health Insurance. Hanoi.


Ministry of Health. (May 10, 2017). Circular 21/2017/TT-BYT. Rehabilitation techniques recognized were

added. Hanoi.

Ministry of Health. (August 21, 2017). Decision 3815/QD-BYT. Implementing information systems about

rehabilitation. Hanoi.

Ministry of Health. (October 06, 2014). Decision 4039/QD-BYT. Approving the National Plan for

Rehabilitation Service Development for the 2014 – 2020 period. Hanoi.

Ministry of Health and Ministry of Finance. (October 29, 2015). Inter-Circular 37/2015/TTLT-BYT-BTC -

Unifying Prices for Medical Examination and Treatment Services covered by Medical Insurance among hospitals

of the same level across the country. Hanoi.

The Economic and Social Commission for Asia and the Pacific, The United Nations. (2018). Building

Disability - Inclusive Societies in Asia and the Pacific. Assessing Progress of the Incheon Strategy. Retrieved from

http://www.unescap.org/sites/default/files/publications/SDD%20BDIS%20report%20A4%20v14-5-E.pdf

The National Assembly. (June 17, 2010). Law 51/2010/QH12 - Law on Persons with Disabilities. Hanoi.

The Prime Minister. (August 05, 2012). Decision 1019/QD-TTg - Approving the Scheme for Assisting the

Disabled in the 2012 – 2020 period. Hanoi.

The Prime Minister. (June 21, 2016). Decision 1100/QD-TTg. Approving Action Plan for The Implementation of

The United Nations Convention on The Rights of Persons with Disabilities. Hanoi.

The United Nations. (2010). Analyzing and Measuring Social Inclusion in a Global Context. Retrieved from

http://www.un.org/esa/socdev/publications/measuring-social-inclusion.pdf

The United Nations. (2016). Convention on the Rights of Persons with Disabilities and Optional Protocol.

Retrieved from http://www.un.org/disabilities/documents/convention/convoptprot-e.pdf

The United Nations. (2016). Leaving no one behind: the imperative of inclusive development. Report on the

World Social Situation 2016. Retrieved from http://www.un.org/esa/socdev/rwss/2016/full-report.pdf

The World Health Organization (2010). Monitoring the Building Blocks of Health Systems: A Handbook of

Indicators and Their Measurement Strategies. Retrieved from

http://www.who.int/healthinfo/systems/WHO_MBHSS_2010_full_web.pdf?ua=1

The World Health Organization (2016). Priority Assistive Products List. Retrieved from

http://apps.who.int/iris/bitstream/handle/10665/207694/WHO_EMP_PHI_2016.01_eng.pdf?sequence=1.

The World Health Organization (2017). Rehabilitation 2030 – A call for action – Meeting Report. Retrieved

from http://www.who.int/disabilities/care/Rehab2030MeetingReport2.pdf?ua=1

The World Health Organization (2017). Rehabilitation in health systems. Retrieved from

http://apps.who.int/iris/bitstream/handle/10665/254506/9789241549974eng.pdf;jsessionid=CE42A0BC12

AA95A3890B6A21EF6F01D4?sequence=1

http://www.un.org/disabilities/documents/convention/convoptprot-e.pdf

http://www.un.org/esa/socdev/rwss/2016/full-report.pdf


The World Health Organization (2018). Resolution EB142.R6 Improving access to assistive technology.

Retrieved from http://apps.who.int/gb/ebwha/pdf_files/EB142/B142_R6-en.pdf

The World Health Organization (n.d.). Health services development – The WHO Health Systems Framework.

Retrieved from http://www.wpro.who.int/health_services/health_systems_framework/en/#.

USAID (2012). Gender and Female Empowerment Policy. Retrieved from

https://www.usaid.gov/sites/default/files/.../GenderEqualityPolicy_0.pdf

USAID (2015). USAID’s Vision for Health Systems Strengthening. Retrieved from

https://www.usaid.gov/sites/default/files/documents/1864/HSS-Vision.pdf

Viet Health. (2016). Disabilities Integration Services and Therapies Network for Capacity and Treatment - FY

2016 Annual Report. Hanoi: USAID.

Viet Health. (2017). Disabilities Integration Services and Therapies Network for Capacity and Treatment - FY

2017 Annual Report. Hanoi: USAID.

Viet Health. (2017). Disabilities Integration Services and Therapies Network for Capacity and Treatment -

Monitoring, Evaluation and Learning Plan. Hanoi: USAID.

Vietnam Assistance for the Handicapped (2016). Disability Rights Enforcement, Coordination and Therapies -

Annual Report. Reporting Period: November 1st, 2015 to September 30th, 2016. Hanoi: USAID.


Annual Report. Reporting Period: October 1st, 2016 to September 30th, 2017. Hanoi: USAID.


Monitoring and Evaluation Plan. Hanoi: USAID.


Monitoring and Evaluation Plan. Hanoi: USAID.


2016 Work Plan. Hanoi: USAID.





Vietnam Evaluation, Monitoring and Survey Services Project. (2017). USAID/Vietnam Gender Review 2017-

Final Report. Hanoi: USAID.

http://www.wpro.who.int/health_services/health_systems_framework/en/

vietnam intervention project to assist persons with disabilities

Documents

Transcript of vietnam intervention project to assist persons with disabilities