Which reading methods give the best value for money?

The use of “all” as a performance criterion

Helen AbadziGlobal Partnership for Education

Oxford, UKApril 4, 2012

Many reading methods are promoted

in low-income countries

If piloted most of them show some increase above a baseline

E.g. from 10% to 15% compared to controls

We use statistical significance, magnitude of effect

But practically how big is the improvement?

E.g. Is an increase from 1 to 3 words per minute satisfactory?

What criteria should be used to advise governments on which methods to adopt?

Education documents often refer to “all”

Education for All

Learning for All

All Children Reading

Is all a euphemism or a real goal?

If so, governments should look for means to make ALL students literate

i.e. adopt strategies, methods, activities, that achieve this

10 pilot schools grades 1 to 3; 5 comparison schools

a 9-module teacher training on explicit reading instruction, conducted in monthly

cycles;

development of Pashto reading primers and alphabet charts;

community-based Book banks with Urdu storybooks and Pashto reading primers;

a Reading Buddy system, older children read to younger children;

conducting weekly Reading Camps run by trained Reading Camp Leaders

conducting regular community reading awareness sessions with parents

conducting Story Time activities in the community.

All 15 schools included:

Health & Nutrition: deworming, Vitamin A, water, sanitation facilities, health

education;

Provision of teaching-learning materials for classrooms,

libraries-in-a-box with Urdu books, wall charts in Urdu and English, math

manipulatives, and Urdu alphabet cards;

A 6-day teacher training on basic pedagogy and active learning strategies, 27

topics.

Results on 190 children with baseline and endline (out of 243 baseline) gr. 2 end

to gr. 3 end

Save the Children - Literacy Boost Teacher TrainingPakistan (circa 2012)

Pakistan Literacy Boost (circa 2012)Zero scores reduced to 7-13%

potentially meeting the 83% criterion !

Nepal – Literacy Boost Kailali 2009-10, grade 2

big difference from controls,

but far from meeting 83% criterion

Nepal, Kailali 2009-10

28% of students read all letters

10% read all words

therefore far from meeting 83% criterion

Nepal – Literacy Boost Kailali 2009-10

Results highly statistically significant

But far from meeting 83% criterion

Literacy Boost - Chichewa

Malawi grade 2

Far from meeting 83% criterion

Malawi – Chichewa grade 2results are statistically but not practically

significant

Malawi grade 2

means must be found to instruct

Olinga Foundation – Ghana

seems below the 83% criterion

Remediation program reportedly makes

literate about 52% of its studentshttp://www.onecountry.org/story/ghana-innovative-literacy-program-

produces-dramatic-results

What standards should be applied to

various reading programs?

Value for money

Lower-income countries should implement methods shown to teach the vast majority of students

Is the 83% success a reasonable standard?

Sampling, definition of success

Literacy programs falling below this could be strengthened

But data often are unavailable

Some reading methods are

clearly more efficient than others

- For the middle classes all methods work

- For low-income students there are no

degrees of freedom

To use money well, programs should

abide by research

Scientific chain of causality shows these variables effective for low-income students

- Teaching letters one by one, one every 1-2 days

- Local languages rather than English, French, Portuguese

- Pattern detection (ka ke ki ak ek ik)

- Ensuring that students actually see blackboard print

- Fat textbooks for all, about 4000 words for practice

- Respecting critical print size and spacing

- Intensive practice to reduce reaction time

- Activate visual word form area sufficiently

- even15 seconds of feedback for all

- Effective teacher training (e.g. through videos)

- Time use for instruction and practice

Few reading programs include all of these activities !!

Are donors inadvertently financing

and promoting inefficient practices?

Currently there is no quality control of

NGO outcomes

Hard data needed, not just personal

opinions on how to teach

NGO public relations campaigns may

downplay poor outcomes

Governments must understand clearly to

what they commit money and time

Need for donor agencies to decide:

Should reading be taught according

to science or philosophy?If philosophy, “best practices” are ok

If science, evidence and standards needed

Reliable and valid numerical standards come from cognitive neuroscienceVisual word form area activation at 45-60 words per minute -

related to 80% in many studies

But reading organizations and ed. colleges rarely talk to reading neuroscientists

Literacy NGOs may disparage this work

Ineffective reading instruction

damages student opportunities

The poor may drop out early, spend little

time in school

Students’ limited opportunity may get

wasted

They may never have time to recuperate

e.g. Cambodia – whole language program

Illiterate students had to repeat grade 1

in the afternoon while in grade 3

Need for financial effectiveness,

disclosure

- How well have the various CSOs and NGOs

performed financially?

- How many $$$ do they spend per child?

- per successful reader?

- E.g. boutique operations spending $400 per

child and getting 30% success rates vs.

governments spending $10 per child and

getting 15% success?

- How transparent are the finances of these

'partners'?

Donor agencies face more hard decisions

Should they fund “best practices” or NGOs with great

public relations that may teach inefficiently?

- Friendly organizations may promise results, but their

beneficiaries may remain illiterate

Should donors raise the bar and risk alienating their

political supporters?

Should they bypass most NGOs and hope that

governments can deliver on their own?

Need for candor and specificity to avoid wastage

and disappointments

GPE benchmarks vague: Students should

read ‘fluently enough to comprehend text’

Governments may expect 80-100%

comprehension

May instead get low speeds and 20%

comprehension

- students unable to make sense of text

Evaluations may give some

answers

But implementation and evaluations often

take 5+ years

Inefficient investments likely by then

Need for independent evaluations

But which organizations can be really

considered independent?

Accountability issues for advice

on literacy

Who will be held accountable for the

billions to be spend on literacy?

Partnerships divide accountability to

multiple partners

Risk that no one may be held accountable

for poor advice to partner countries!

But setting standards likely to create

sensitivities

Much money available, no tradition of

scrutiny

Providers should have rates of success

Perhaps NGOs should be certified

Who will “certify” methods, providers?

Special interests involved in NGO fundraising

Can evaluations be truly independent?

Sampling problems currently significant

Please consider these issues

Propose how standards can be set

Propose how the political issues can

be resolvedThank you for your attention