Post on 23-Apr-2023
Effectiveness of health-promoting media literacyeducation a systematic review
Lynda J Bergsma1 and Mary E Carney2
Abstract
Media literacy education to promote healthamong youth involves them in a critical exam-ination of media messages that promote riskybehaviors and influence their perceptions andpractices Research on its effectiveness is in itsinfancy Studies to date have been conductedwith more or less rigor and achieved differingresults leaving many questions about effective-ness unanswered To elucidate some of thesequestions we conducted a systematic reviewof selected health-promoting media literacy ed-ucation evaluationresearch studies guided bythe following research question What are thecontext and process elements of an effectivehealth-promoting media literacy education in-tervention Based on extensive analysis of 28interventions our findings provide a detailedpicture of a small 16- to 17-year (1990 to July2006) body of important research including ci-tation information health issue target popula-tionNage research design intervention lengthand setting conceptsskills taught who deliv-ered the intervention and ratings of effective-ness The review provides a framework fororganizing research about media literacy edu-cation which suggests that researchers should bemore explicit about the media literacy core con-
ceptsskills they are including in their interven-tions and should more carefully address whodelivered the intervention with what fidelity inwhat setting for how long and utilizing whatpedagogical approach
Introduction
In recent decades health professionals have in-
creasingly recognized that the media have a signif-
icant influence on the health of young people In
their review of research on the mediarsquos influence on
health the Committee on Public Education of the
American Academy of Pediatrics stated lsquoResearch
has shown primary negative health effects on vio-
lence and aggressive behavior sexuality academic
performance body concept and self image nutri-
tion dieting and obesity and substance use and
abuse patternsrsquo [1 p 423]
Some researchers estimate that youth spend 33ndash
50 of their waking hours with some form of
mass media [2] According to the Kaiser Family
Foundation [3] children and teens are spending
an increasing amount of time using lsquonew mediarsquo
like computers the Internet and video games with-
out cutting back on the time they spend with lsquooldrsquo
media like television print and music Their Gen-eration M Media in the Lives of 8-18 Year-oldsstudy found that the total daily media exposure of
young people increased from 729 to 833 hours
between 2000 and 2005 counting time as double
when multi-tasking with two different forms of me-
dia The large amount of time youth spend with
media makes it critical to address related health
concerns
1Health Promotion Sciences Division Mel and Enid
Zuckerman College of Public Health University of Arizona
Tucson AZ 85724 USA and 2Department of Health
Sciences New Mexico State University Las Cruces NM
88003 USA
Correspondence to L J Bergsma
E-mail lbergsmauarizonaedu
The Author 2008 Published by Oxford University Press All rights reservedFor permissions please email journalspermissionsoxfordjournalsorg
doi101093hercym084
HEALTH EDUCATION RESEARCH Vol23 no3 2008
Pages 522ndash542
Advance Access publication 17 January 2008
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Public health professionals have used many strat-
egies to address the effects of media on health
Regulating media content limiting childrenrsquos me-
dia use and social marketing are approaches that
have been used traditionally but media literacy ed-
ucation has emerged in the last 20 years as a prom-
ising alternative to the censorship of regulating
lsquounhealthyrsquo programming or limiting media use
[4] Media literacy has been defined as lsquothe ability
to access analyze evaluate and create media in
a variety of formsrsquo [5 p 21] Rather than trying
to protect youth from potentially harmful messages
media literacy education to promote health involves
them in a critical examination of media messages
that influence their perceptions and practices It is
designed to give youth the critical thinking skills
necessary to ameliorate the influence of these mes-
sages and make healthy choices Media literacy has
been recommended as an effective health pro-
motion strategy by a number of respected orga-
nizations including the American Academy
of Pediatrics the Office of National Drug Control
Policy and the Centers for Disease Control
Even though the field of media literacy educa-
tion effectiveness research is in its infancy several
studies have tested the ability of various media lit-
eracy curricula to teach young people how to ana-
lyze media messages and to improve their choices
on a variety of health topics Because these studies
have been conducted with more or less rigor and
achieved differing results however many ques-
tions remain lsquoIn spite of research policy and ad-
vocacy centers Web sites and a movement largely
among educators there is little evidence to suggest
that media literacy efforts are either widespread or
effectiversquo [6 p 205] To date there has been no
comprehensive review of the research literature on
media literacy interventions with a health promo-
tion purpose nor has there been an attempt to ana-
lyze the research to determine components that
make such interventions effective
For these reasons we conducted a systematic re-
view of recent publications about health-promoting
media literacy interventions with the intent of de-
veloping a framework for analysis of components
that may make such interventions more or less ef-
fective The conference report on Setting ResearchDirections for Media Literacy and Health Educa-tion stressed the need to document the different
contexts processes and outcomes of health-focused
media literacy education [7] To initiate this we
analyzed selected interventions to identify four de-
pendent variables relating to context and process
(intervention setting intervention length con-
ceptsskills taught and who taught the intervention)
and assigned a rating of effectiveness based on out-
comes Our review was guided by the following
research question What are the context and process
elements of an effective health-promoting media
literacy education intervention
Methods
The methodology used was a systematic review in
which we examined the evidence regarding health-
promoting media literacy education using explicit
methods to identify appraise and select relevant
primary research and to extract and analyze data
from the studies selected for review lsquoSystematic
reviews provide information about the effectiveness
of interventions by identifying appraising and
summarizing the results of otherwise unmanageable
quantities of researchrsquo [8 Phase 1 p4] While sta-
tistical methods (meta-analysis) may be part of
a systematic review the heterogeneity of the studies
reviewed particularly with regard to outcome
measures precluded statistical comparison [9] al-
though we used rudimentary quantitative measures
such as percentages where appropriate
Literature search
Given the multi-disciplinary nature of the media
literacy field we searched the following indexes
CINAHL Plus Communications Abstracts Com-
munication and Mass Media Complete Disserta-
tion Abstracts ERIC Expanded Academic
ASAP Health and Safety Science Abstracts MED-
LINE PsycINFO and Social Sciences Abstracts
Studies were identified by using the following key-
words media literacy media education television
Effectiveness of health-promoting media literacy
523
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
education and media analysis For indexes not
health related the keyword lsquohealthrsquo was also used
Studies were also located by reviewing the refer-
ences of primary studies considered in addition to
some articles and reports published about media
literacy in recent years [7 10 11] To further locate
appropriate studies a preliminary list of sources
being considered for inclusion was circulated to
several experts in the field resulting in some addi-
tional recommendations
The literature search was conducted initially by
the first author All publications found were given
to the second author who did her own search and
found several additional studies for a total pool of
65 publications that were reviewed and considered
for inclusion Only 26 met the study selection cri-
teria detailed below
Study selection
The criteria we used to determine study inclusion or
exclusion were as follows
1 Studies were published in English from 1990
through July 2006
2 Studies were peer reviewed (including journal
articles conference proceedings and disserta-
tions) and were publicly available through
a searchable index
3 The sample study design intervention evalua-
tion measures and analysis were described
4 Studies included an experimental media literacy
intervention of 25 or more minutes in length
designed to improve the health of youth
5 The study focused primarily on teaching critical
media literacy skills and did not use media liter-
acy as a small part of a larger curriculum
The time frame for study inclusion was selected
because there have been no comprehensive reviews
of media literacy interventions since 1990 [12]
although the research has continued to advance
Studies that were available to us but were in
press as of July 2006 were excluded from the
analysis [13]
The sample was limited to peer-reviewed publi-
cations due to concerns that non-peer-reviewed
sources would describe less rigorous experimental
procedures and data We debated this decision
however because the youth of the field of media
literacy education research and the small number of
studies that met our criteria made it seem useful to
include sources that had not undergone peer review
but were readily available publicly and seemed to
contribute to the field [14ndash16] including a study
[17] that was reported in a chapter of an edited
book
Some studies were excluded because they did not
describe a rigorous experimental evaluation of
a specific intervention [18 19] One article was
excluded because while it provided information
about the qualitative evaluation conducted none
of the quantitative evaluation measures were de-
scribed and results were not reported [20] Another
study was excluded because the media literacy in-
tervention it used was only 3ndash5 minutes in length
[21] Interventions less than 25 minutes in length
were deemed to be too short to teach critical media
literacy skills and allow students to practice them
using an inquiry approach which is an essential
pedagogical tenet of media literacy education
[22] While most of the selected studies focused
the intervention directly on youth one study was
selected in which the intervention was given to
adult caregivers with a focus on improving the
home nutrition environment to promote the health
of preschoolers [23]
As stated in the fifth criterion above interven-
tions needed to focus on teaching lsquocritical media
literacy skillsrsquo which were operationalized as the
following abilities based on the Center for Media
Literacyrsquos definition [5 p21]
1 To access useful information in the media
2 To analyze media messages using critical think-
ing skills (being able to identify bias and credi-
bility of a source differentiate fact from opinion
determine if a message is unrealistic understand
a messagersquos purpose)
3 To evaluate a message (to determine its truth
applicability)
L J Bergsma and M E Carney
524
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
4 To create media messages which achieve spe-
cific goals
To be included in this review the intervention
needed to teach at least one of the four skills listed
above Although critical media literacy skills may
have been taught in three studies reviewed they
were excluded because it was not clear that this
was the case [24ndash26] Several studies that focused
on teaching youth the skills to resist advertising
pressure to smoke cigarettes were excluded for
the same reason Criterion 5 above also stated that
the intervention must focus lsquoprimarilyrsquo on teaching
critical media literacy skills This distinction was
made because some studies incorporated a small
media literacy component in their intervention as
part of a larger curriculum that taught other knowl-
edge or skills which may have influenced effective-
ness findings [27 28]
Some studies were excluded because they fo-
cused mainly on reducing television use instead
of teaching critical media literacy skills [29ndash31]
Although one of the sources selected did include
the concept of reduced television use in its interven-
tion it was included because it allowed participants
to make informed decisions about media use rather
than simply teaching them that media use should be
avoided [32] Most of the study inclusion or exclu-
sion decisions were made independently by the sec-
ond author based upon the criteria established by
both authors and reviewed by the first author
Where there was any question the authors dis-
cussed their conclusions and a final decision was
made by consensus
During peer review of the manuscript five addi-
tional studies were recommended for inclusion
Upon review we found that one study [33] did
not meet selection criterion 3 and two studies did
not meet criteria 4 and 5 [34 35] Two of the rec-
ommended studies were included [36 37]
Twenty-three studies in 26 publications met the
inclusion criteria [23 32 36ndash59] The same data
from two of the 23 studies were reported in two
publications and two articles reported on different
measures and analyses extracted from the same re-
search study Data from these publications were
grouped and considered as one study
Data extraction and analysis
To enable extraction and analysis of a large quantity
of data we developed an extensive database of in-
formation for each of the selected studies The fol-
lowing sections detail first general descriptive data
and second data that were extracted and analyzed
for each measure
General descriptive data
Table I contains descriptive data about the selected
sample Twenty-eight interventions were detailed
in the 23 studies because two studies employed
multiple experimental groups that experienced dif-
ferent media literacy interventions Austin and
Johnson [40] (two interventions) and Byrne [43]
(three interventions) In addition two studies had
multiple interventions but only some of them were
judged to be media literacy interventions Wilsch
Tiggemann and Wade [37] three of five and Irving
and Berel [48] one of three
Health issues included nutrition (two studies)
nutritioneating disorders prevention (nine) body
image (one) substance abuse prevention (five)
and violence prevention (six) Eighteen studies
recruited participants from schools two from both
schools and community groups and three from non-
school locations Participant sample size ranged
from 17 to 723
Studies were categorized according to target age
as follows children (aged 11 and belowGrade
6 and below) adolescents (aged 12ndash19grades
7ndash12) and college students (mean age 22 and below)
When a study targeted youth in different categories
it was categorized according to the larger number
of participants Thirteen studies targeted children
seven adolescents and three college students
Each studyrsquos research design is included in Table
I While in most cases the research design was
stated explicitly in some instances it was inferred
from the methodology description Because all the
studies used a quasi-experimental design this de-
scriptor was omitted from Table I Most of the stud-
ies selected utilized quantitative measures a few
Effectiveness of health-promoting media literacy
525
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table I Descriptive data concerning systematic review sample
Study Health issue Target population (n) Target age group Research design Number of interventions
Austin and Johnson
[38 39]
Substance abuse (alcohol) Third grade students from
three schools in a northern
mid-western town (246)
Children Solomon four-group
design
1
Austin and Johnson [40] Substance abuse (alcohol) Third grade students from
Roseville MN (225)
Children Solomon four-group
design with two
experimental groups
2
Austin et al [41] Substance abuse (tobacco) Adolescents from 76
Washington state schools
and community groups
mean age 15 (723)
Adolescents Post-test-only design 1
Austin et al [42] Substance abuse (tobacco) Adolescents from
community groups in
three western WA
communities average age
16 (119)
Adolescents Pre-testndashpost-test design 1
Byrne [43] Violence Fourth and fifth grade
students from three west
coast schools (199)
Children Pre-testndashpost-testndashre-test
design with three
experimental groups
3
Coughlin and Kalodner
[44]
Nutrition (eating
disorders)
Female undergraduate
students in introductory
womenrsquos studies course at
a rural university average
age 20 (92)
College students Pre-testndashpost-test design 1
Evans et al [45] Nutrition Fourth and fifth grade
students from two South
Carolina schools (39) and
their parents
Children Pre-testndashpost-test design 1
Fuller et al [36] Body image Fourth grade girls from an
urban school (17)
Children Qualitative focus groups
pre-testndashpost-test design
1
Gonzales et al [46] Substance abuse (tobacco) 10th grade students from
four Los Angeles high
schools (609)
Adolescents Pre-testndashpost-test design 1
LJBerg
smaandMECarn
ey
526
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Hindin [23] and Hindin
et al [47]
Nutrition Low-income mothers
(33) a father and
a grandmother of
preschoolers (aged 3ndash6)
from four NYC Head Start
sites aged 22ndash62 (35)
Children Pre-testndashpost-test
comparison conditionndash
intervention condition
design with no control
group
1
Irving and Berel [48] Nutrition (eating
disorders)
Female college students
from introductory
psychology classes at
Washington State
University average age
19 (110)
College students Post-test-only design with
three experimental groups
and a control group
1
Irving et al [49] Nutrition (eating
disorders)
Female 10th grade
students from two English
classes in a WA high
school average age
15 (41)
Adolescents Post-test-only design 1
Kusel [50] Nutrition
(eating disorders)
Female fourth fifth and
sixth grade students from
two suburban NY school
districts aged 9ndash13 (172)
Children Solomon four-group
design
1
Neumark-Sztainer
et al [51]
Nutrition (eating
disorders)
Female fifth and sixth
grade students from 24
mid-western Girl Scout
troops average age
11 (226)
Children Pre-testndashpost-testndashre-test
design
1
Rabak-Wagener
et al [52]
Nutrition (eating
disorders)
Undergraduate students
from healthful living
classes at a medium-sized
metropolitan university in
the mid-west mainly aged
18ndash23 (105)
College students Pre-testndashpost-test design 1
Rosenkoetter et al [32] Violence First second and third
grade students from 13
classrooms in four schools
in or near a community of
population 50 000 (177)
Children Pre-testndashpost-test design 1
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
527
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Scharrer [53] Violence Sixth grade students from
five classrooms in three
northeast towns (93)
Children Pre-testndashpost-test design
with no control group
1
Sprafkin et al [54] Violence Emotionally disturbed and
learning disabled children
from 29 classes in two
Long Island schools aged
6ndash12 (167)
Children Pre-testndashpost-testndashre-test
design with two
experimental and two
control groups
1
Vooijs and van der
Voort [55]
Violence Fourth fifth and sixth
grade students from 18
classes in six Dutch
schools (437)
Children Pre-testndashpost-testndashre-test
design
1
Vooijs and van der
Voort [56]
Violence Fifth and sixth grade
students from 12 classes in
six Dutch schools (300)
Children Pre-testndashpost-test design 1
Wade et al [57 58] Nutrition (eating
disorders)
Eighth grade students
from four private high
school classes in
Australia average age
134 (86)
Adolescents Post-test-only design with
two experimental groups
Pre-testndashpost-testndashre-test
design with two
experimental groups
1
Wilksch et al [37] Eating disorders (media
internalization)
Eighth grade students
from 11 classes at three
metropolitan private
schools in Australia mean
age 139 (237 100 girls
and 137 boys)
Adolescents Pre-testndashpost-test design
with 3 media literacy
education experimental
groups
3
Wolf-Bloom [59] Nutrition (eating
disorders)
Female adolescents from
two Girl Scout Councils
and a private school in
Northeast Ohio aged 10ndash
15 (70)
Adolescents Pre-testndashpost-testndashre-test
design
1
LJBerg
smaandMECarn
ey
528
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
utilized some qualitative measures as well and one
utilized qualitative measures only [36] Nineteen
studies had one control group one study employed
two different control groups [52] and three studies
had no control group [37 39 50]
Intervention setting
Intervention settings were grouped into the follow-
ing categories (i) in-class (intervention took place
in class during regularly scheduled class time) (ii)
in-school (intervention took place at a school out-
side of the regular classroom) (iii) community
groups or (iv) location not specified (Table II)
Analysis revealed that 19 interventions took place
in-class one both in-class and in community
groups three in school outside of a regular class
or after class three in community groups and two
studies did not specify the intervention setting
Intervention length
This measure was defined as the number of minutes
allotted to the intervention excluding time for pre-
tests post-tests and re-tests When the intervention
took place in a specified number of class sessions
based on analysis of K-12 and college class length
we assigned an average value of 45minutes to a class
session When it was clear that the pre-test or post-
test had been given during the time designated for
the intervention but the pre-testpost-test time was
unspecified we assumed that each evaluation mea-
sure took 15 minutes When a range was provided
for the intervention length we computed an average
Analysis revealed that intervention length ranged
from 25 minutes to 24 hours To better compare
these lengths with intervention effectiveness we
grouped them into the following categories short
(60 minutes or less) average (1ndash5 hours) or long
(more than 5 hours) There were 11 short interven-
tions nine average and eight long (Table II)
Conceptsskills taught
Because the Center for Media Literacyrsquos lsquoCore
Concepts of Media Literacyrsquo [5] has been used
widely in the United States where most of the se-
lected studies were published we used this frame-
work to assess concepts and associated skills
taught Determining core concepts and associated
skills taught required reviewing the publication
multiple times and making many interpretations be-
cause most studies did not specifically state whether
they taught any core concepts The following
guidelines were used to categorize interventions
1 All media messages are lsquoconstructedrsquo Interven-tion taught about how the media differs from
reality evaluating what is shown compared with
real life experiences or the producerproduction
of media messages
2 Media messages are created using a creativelanguage with its own rules Intervention taught
about recognizing advertisingproduction tech-
niques or creatingproducing media messages
3 Different people experience the same messagedifferently Intervention explored how media af-
fect people what people can do to avoid nega-
tive effects of media or that people can take
action to change the media
4 Media have embedded values and points of viewIntervention taught how to identify stereotypes
myths biases values lifestyles andor points
of view represented in or omitted from media
messages
5 Most media messages are constructed to gainprofit andor power Intervention taught about
the purpose of advertising or marketing strate-
gies skepticism toward advertising or creating
counter-advertising
Analysis revealed that the core concepts and as-
sociated skills were taught with the following fre-
quency one (in all 28 interventions) two (17) three
(19) four (28) and five (13) (Table II)
Because our decision to investigate the presence
of the core concepts in the selected studies was
based on prior knowledge of media literacy rather
than a selection of themes that would emerge from
analysis of each text we analyzed the publications
for additional types of knowledge taught Table II
documents the two themes that emerged from this
analysis how media affects health (17) and knowl-
edge about the targeted health issue (14)
Effectiveness of health-promoting media literacy
529
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table II Comparison of intervention setting length core concepts taught and who delivered intervention with effectiveness ratings
Intervention Effectiveness ratingIntervention information
Short
term
Long
term
Setting Length Conceptskills
taughtaWho delivered
interventionb
Fuller et al [36] NR NR School outside class Average 1 2 3 4 5 a b Researchers
Wilksch et al [37] NR NR In-class Short 1 2 4 a Researcher with teacher
present
Byrne [43]mdashbasic Condition 1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashcognitive
activity condition
1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashthird person
condition
1 1 In-class Short 1 3 4 a Unspecified
Irving et al [48]mdashexternally
oriented condition
2 NA Location not specified Short 1 3 4 a Researcher
Irving et al [49] 2 NA School outside class Short 1 2 3 4 5 Teen peer
Scharrer [53] 2 NA In-class Average 1 3 4 a Undergraduate
college student
Wade et al [57 58] 2 1 In-class Average 1 2 3 4 5 a b Teacher assisted by
graduate student
Neumark-Sztainer
et al [51]
2 2 Community groups Long 1 2 3 4 a b Girl Scout troop
leaders
Sprafkin et al [54] 2 3 In-class Long 1 2 4 5 Teacher
Austin et al [42] 3 NA Community groups Average 1 2 3 4 5 b Teen peers
Evans et al [45] 3 NA School outside class Long 1 2 3 4 a b Unspecified
Austin and Johnson
[39 40]3 2 In-class Short 1 2 4 5 Researcher
Kusel [50] 3 2 In-class Average 1 2 3 4 5 a b Researcher
Wolf-Bloom [59] 3 4 Location not specified Average 1 2 3 4 5 a Senior Girl Scout
peers
Austin et al [41] 4 NA In-class and community
groups
Average 1 2 3 4 5 b Teen peers (2)
Coughlin and Kalodner [44] 4 NA In-class Average 1 2 3 4 a b Researcher
Rabak-Wagener et al [52] 4 NA In-class Long 1 2 4 5 a b Researcher and
instructor (assisted)
Rosenkoetter et al [32] 4 NA In-class Long 1 2 3 4 5 a b Researcher
Gonzales et al [46] 5 NA In-class Long 1 2 3 4 5 a b Unspecified
Hindin [23] and Hindin
et al [47]
5 NA Community groups Long 1 2 3 4 5 a b Unspecified
Vooijs and van der
Voort [56]
5 NA In-class Average 1 3 4 b School television
broadcast series
and teacher
Vooijs and van der
Voort [55]
5 5 In-class Long 1 3 4 b Researcher
Austin and Johnson
[40]mdashalcohol-specific
condition
5 5 In-class Short 1 4 5 Researcher
Austin and Johnson
[40]mdashgeneral condition
5 5 In-class Short 1 4 5 Researcher
NR = not rated NA = not applicableaNumbers refer to media literacy core concepts taught and letters refer to (a) how media affect health and (b) knowledge about healthissuebAssumed to be researcher (research project staff) if unspecified
L J Bergsma and M E Carney
530
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Who delivered the intervention
Analysis of each intervention revealed that the per-
son who most frequently delivered the intervention
was the researcher (11) In one study the researcher
was assisted by the class instructor In six studies
where the person who delivered the intervention
was not specified we assumed it was a member
of the research staff Other modes of delivery in-
cluded teen peers (four) some by themselves and
some in teams of two Two interventions involved
the use of college students one utilized a trained
undergraduate student and another utilized the
classroom teacher assisted by a graduate student
Classroom teachers were the primary person deliv-
ering the intervention in three cases although one
intervention utilized a series of school television
broadcasts facilitated by teachers using a guide In
one study the intervention was delivered by Girl
Scout troop leaders (Table II)
Intervention effectiveness
lsquoIntervention effectivenessrsquo was difficult to assess
due in part to the many different health issues
addressed and research designs theoretical models
and outcome measures used Effectiveness was de-
fined very differently in each of the studies which
made it impossible to impose a common framework
upon them In an attempt to find elements by which
to compare the studies the following information
was compiled for each intervention health issue
research design theoretical approach intervention
outcomes (Table III) results at pre- and post-test
results at re-test (delayed post-test) and elaborated
results In addition we recorded how much signif-
icant change the intervention achieved on its meas-
ures at post-test and re-test
This method of standardizing each studyrsquos eval-
uation excluded considerable information For ex-
ample in the research by Austin and Johnson
[38ndash40] analyses were conducted to compare each
measurersquos success in predicting outcomes on other
measures as hypothesized in their Message Inter-
pretation Process Model By choosing to focus only
on the media literacy interventionrsquos direct effect on
each of the selected measures a great deal of useful
information was excluded For this reason we in-
cluded an elaborated description of each studyrsquos
results in the database that was developed from
which to determine the effectiveness ratings
As we reviewed the outcome evaluation meas-
ures to identify ways in which they might be
grouped we found that all of them fell into one of
the following categories knowledge attitudes
behaviors risk factors for the health problem inten-
tions skills or process evaluation Then we re-
reviewed the measures and their results and
recorded which measures corresponded with each
category and the amount of statistically significant
change that the intervention achieved for each cat-
egory at post-test and re-test
Using all of the information compiled about the
results of each study as well as background knowl-
edge about each theoretical approach and the
strengthslimitations of each research design we
attempted to make an unbiased rating of each inter-
ventionrsquos effectiveness Given the diversity of the
data available for comparison we determined that
each studyrsquos rating of effectiveness would primarily
be a function of whether the intervention appeared
to achieve a significant change on evaluated mea-
sures that were of importance according to the
selected theoretical approach Interventions were
rated for short-term effectiveness (outcomes at
post-test) and long-term effectiveness (outcomes
at re-test) on a scale of one to five with five being
the highest effectiveness All 24 interventions rated
included a post-test and 12 included a re-test yield-
ing a total of 36 assigned ratings (Table II) Note
that the two studies recommended for inclusion
during peer review could not be assessed using
the same procedures and raters due to the fact that
the research team was no longer intact These stud-
ies are included in Table II but are marked NR for
not rated
Several measures were taken to insure the reli-
ability of effectiveness assessment First each in-
tervention was labeled with a number and
interventions were sorted randomly so that each re-
viewer got a differently ordered data set Second
the two authors independently made their ratings at
different times with little conversation about how to
Effectiveness of health-promoting media literacy
531
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Public health professionals have used many strat-
egies to address the effects of media on health
Regulating media content limiting childrenrsquos me-
dia use and social marketing are approaches that
have been used traditionally but media literacy ed-
ucation has emerged in the last 20 years as a prom-
ising alternative to the censorship of regulating
lsquounhealthyrsquo programming or limiting media use
[4] Media literacy has been defined as lsquothe ability
to access analyze evaluate and create media in
a variety of formsrsquo [5 p 21] Rather than trying
to protect youth from potentially harmful messages
media literacy education to promote health involves
them in a critical examination of media messages
that influence their perceptions and practices It is
designed to give youth the critical thinking skills
necessary to ameliorate the influence of these mes-
sages and make healthy choices Media literacy has
been recommended as an effective health pro-
motion strategy by a number of respected orga-
nizations including the American Academy
of Pediatrics the Office of National Drug Control
Policy and the Centers for Disease Control
Even though the field of media literacy educa-
tion effectiveness research is in its infancy several
studies have tested the ability of various media lit-
eracy curricula to teach young people how to ana-
lyze media messages and to improve their choices
on a variety of health topics Because these studies
have been conducted with more or less rigor and
achieved differing results however many ques-
tions remain lsquoIn spite of research policy and ad-
vocacy centers Web sites and a movement largely
among educators there is little evidence to suggest
that media literacy efforts are either widespread or
effectiversquo [6 p 205] To date there has been no
comprehensive review of the research literature on
media literacy interventions with a health promo-
tion purpose nor has there been an attempt to ana-
lyze the research to determine components that
make such interventions effective
For these reasons we conducted a systematic re-
view of recent publications about health-promoting
media literacy interventions with the intent of de-
veloping a framework for analysis of components
that may make such interventions more or less ef-
fective The conference report on Setting ResearchDirections for Media Literacy and Health Educa-tion stressed the need to document the different
contexts processes and outcomes of health-focused
media literacy education [7] To initiate this we
analyzed selected interventions to identify four de-
pendent variables relating to context and process
(intervention setting intervention length con-
ceptsskills taught and who taught the intervention)
and assigned a rating of effectiveness based on out-
comes Our review was guided by the following
research question What are the context and process
elements of an effective health-promoting media
literacy education intervention
Methods
The methodology used was a systematic review in
which we examined the evidence regarding health-
promoting media literacy education using explicit
methods to identify appraise and select relevant
primary research and to extract and analyze data
from the studies selected for review lsquoSystematic
reviews provide information about the effectiveness
of interventions by identifying appraising and
summarizing the results of otherwise unmanageable
quantities of researchrsquo [8 Phase 1 p4] While sta-
tistical methods (meta-analysis) may be part of
a systematic review the heterogeneity of the studies
reviewed particularly with regard to outcome
measures precluded statistical comparison [9] al-
though we used rudimentary quantitative measures
such as percentages where appropriate
Literature search
Given the multi-disciplinary nature of the media
literacy field we searched the following indexes
CINAHL Plus Communications Abstracts Com-
munication and Mass Media Complete Disserta-
tion Abstracts ERIC Expanded Academic
ASAP Health and Safety Science Abstracts MED-
LINE PsycINFO and Social Sciences Abstracts
Studies were identified by using the following key-
words media literacy media education television
Effectiveness of health-promoting media literacy
523
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
education and media analysis For indexes not
health related the keyword lsquohealthrsquo was also used
Studies were also located by reviewing the refer-
ences of primary studies considered in addition to
some articles and reports published about media
literacy in recent years [7 10 11] To further locate
appropriate studies a preliminary list of sources
being considered for inclusion was circulated to
several experts in the field resulting in some addi-
tional recommendations
The literature search was conducted initially by
the first author All publications found were given
to the second author who did her own search and
found several additional studies for a total pool of
65 publications that were reviewed and considered
for inclusion Only 26 met the study selection cri-
teria detailed below
Study selection
The criteria we used to determine study inclusion or
exclusion were as follows
1 Studies were published in English from 1990
through July 2006
2 Studies were peer reviewed (including journal
articles conference proceedings and disserta-
tions) and were publicly available through
a searchable index
3 The sample study design intervention evalua-
tion measures and analysis were described
4 Studies included an experimental media literacy
intervention of 25 or more minutes in length
designed to improve the health of youth
5 The study focused primarily on teaching critical
media literacy skills and did not use media liter-
acy as a small part of a larger curriculum
The time frame for study inclusion was selected
because there have been no comprehensive reviews
of media literacy interventions since 1990 [12]
although the research has continued to advance
Studies that were available to us but were in
press as of July 2006 were excluded from the
analysis [13]
The sample was limited to peer-reviewed publi-
cations due to concerns that non-peer-reviewed
sources would describe less rigorous experimental
procedures and data We debated this decision
however because the youth of the field of media
literacy education research and the small number of
studies that met our criteria made it seem useful to
include sources that had not undergone peer review
but were readily available publicly and seemed to
contribute to the field [14ndash16] including a study
[17] that was reported in a chapter of an edited
book
Some studies were excluded because they did not
describe a rigorous experimental evaluation of
a specific intervention [18 19] One article was
excluded because while it provided information
about the qualitative evaluation conducted none
of the quantitative evaluation measures were de-
scribed and results were not reported [20] Another
study was excluded because the media literacy in-
tervention it used was only 3ndash5 minutes in length
[21] Interventions less than 25 minutes in length
were deemed to be too short to teach critical media
literacy skills and allow students to practice them
using an inquiry approach which is an essential
pedagogical tenet of media literacy education
[22] While most of the selected studies focused
the intervention directly on youth one study was
selected in which the intervention was given to
adult caregivers with a focus on improving the
home nutrition environment to promote the health
of preschoolers [23]
As stated in the fifth criterion above interven-
tions needed to focus on teaching lsquocritical media
literacy skillsrsquo which were operationalized as the
following abilities based on the Center for Media
Literacyrsquos definition [5 p21]
1 To access useful information in the media
2 To analyze media messages using critical think-
ing skills (being able to identify bias and credi-
bility of a source differentiate fact from opinion
determine if a message is unrealistic understand
a messagersquos purpose)
3 To evaluate a message (to determine its truth
applicability)
L J Bergsma and M E Carney
524
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
4 To create media messages which achieve spe-
cific goals
To be included in this review the intervention
needed to teach at least one of the four skills listed
above Although critical media literacy skills may
have been taught in three studies reviewed they
were excluded because it was not clear that this
was the case [24ndash26] Several studies that focused
on teaching youth the skills to resist advertising
pressure to smoke cigarettes were excluded for
the same reason Criterion 5 above also stated that
the intervention must focus lsquoprimarilyrsquo on teaching
critical media literacy skills This distinction was
made because some studies incorporated a small
media literacy component in their intervention as
part of a larger curriculum that taught other knowl-
edge or skills which may have influenced effective-
ness findings [27 28]
Some studies were excluded because they fo-
cused mainly on reducing television use instead
of teaching critical media literacy skills [29ndash31]
Although one of the sources selected did include
the concept of reduced television use in its interven-
tion it was included because it allowed participants
to make informed decisions about media use rather
than simply teaching them that media use should be
avoided [32] Most of the study inclusion or exclu-
sion decisions were made independently by the sec-
ond author based upon the criteria established by
both authors and reviewed by the first author
Where there was any question the authors dis-
cussed their conclusions and a final decision was
made by consensus
During peer review of the manuscript five addi-
tional studies were recommended for inclusion
Upon review we found that one study [33] did
not meet selection criterion 3 and two studies did
not meet criteria 4 and 5 [34 35] Two of the rec-
ommended studies were included [36 37]
Twenty-three studies in 26 publications met the
inclusion criteria [23 32 36ndash59] The same data
from two of the 23 studies were reported in two
publications and two articles reported on different
measures and analyses extracted from the same re-
search study Data from these publications were
grouped and considered as one study
Data extraction and analysis
To enable extraction and analysis of a large quantity
of data we developed an extensive database of in-
formation for each of the selected studies The fol-
lowing sections detail first general descriptive data
and second data that were extracted and analyzed
for each measure
General descriptive data
Table I contains descriptive data about the selected
sample Twenty-eight interventions were detailed
in the 23 studies because two studies employed
multiple experimental groups that experienced dif-
ferent media literacy interventions Austin and
Johnson [40] (two interventions) and Byrne [43]
(three interventions) In addition two studies had
multiple interventions but only some of them were
judged to be media literacy interventions Wilsch
Tiggemann and Wade [37] three of five and Irving
and Berel [48] one of three
Health issues included nutrition (two studies)
nutritioneating disorders prevention (nine) body
image (one) substance abuse prevention (five)
and violence prevention (six) Eighteen studies
recruited participants from schools two from both
schools and community groups and three from non-
school locations Participant sample size ranged
from 17 to 723
Studies were categorized according to target age
as follows children (aged 11 and belowGrade
6 and below) adolescents (aged 12ndash19grades
7ndash12) and college students (mean age 22 and below)
When a study targeted youth in different categories
it was categorized according to the larger number
of participants Thirteen studies targeted children
seven adolescents and three college students
Each studyrsquos research design is included in Table
I While in most cases the research design was
stated explicitly in some instances it was inferred
from the methodology description Because all the
studies used a quasi-experimental design this de-
scriptor was omitted from Table I Most of the stud-
ies selected utilized quantitative measures a few
Effectiveness of health-promoting media literacy
525
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table I Descriptive data concerning systematic review sample
Study Health issue Target population (n) Target age group Research design Number of interventions
Austin and Johnson
[38 39]
Substance abuse (alcohol) Third grade students from
three schools in a northern
mid-western town (246)
Children Solomon four-group
design
1
Austin and Johnson [40] Substance abuse (alcohol) Third grade students from
Roseville MN (225)
Children Solomon four-group
design with two
experimental groups
2
Austin et al [41] Substance abuse (tobacco) Adolescents from 76
Washington state schools
and community groups
mean age 15 (723)
Adolescents Post-test-only design 1
Austin et al [42] Substance abuse (tobacco) Adolescents from
community groups in
three western WA
communities average age
16 (119)
Adolescents Pre-testndashpost-test design 1
Byrne [43] Violence Fourth and fifth grade
students from three west
coast schools (199)
Children Pre-testndashpost-testndashre-test
design with three
experimental groups
3
Coughlin and Kalodner
[44]
Nutrition (eating
disorders)
Female undergraduate
students in introductory
womenrsquos studies course at
a rural university average
age 20 (92)
College students Pre-testndashpost-test design 1
Evans et al [45] Nutrition Fourth and fifth grade
students from two South
Carolina schools (39) and
their parents
Children Pre-testndashpost-test design 1
Fuller et al [36] Body image Fourth grade girls from an
urban school (17)
Children Qualitative focus groups
pre-testndashpost-test design
1
Gonzales et al [46] Substance abuse (tobacco) 10th grade students from
four Los Angeles high
schools (609)
Adolescents Pre-testndashpost-test design 1
LJBerg
smaandMECarn
ey
526
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Hindin [23] and Hindin
et al [47]
Nutrition Low-income mothers
(33) a father and
a grandmother of
preschoolers (aged 3ndash6)
from four NYC Head Start
sites aged 22ndash62 (35)
Children Pre-testndashpost-test
comparison conditionndash
intervention condition
design with no control
group
1
Irving and Berel [48] Nutrition (eating
disorders)
Female college students
from introductory
psychology classes at
Washington State
University average age
19 (110)
College students Post-test-only design with
three experimental groups
and a control group
1
Irving et al [49] Nutrition (eating
disorders)
Female 10th grade
students from two English
classes in a WA high
school average age
15 (41)
Adolescents Post-test-only design 1
Kusel [50] Nutrition
(eating disorders)
Female fourth fifth and
sixth grade students from
two suburban NY school
districts aged 9ndash13 (172)
Children Solomon four-group
design
1
Neumark-Sztainer
et al [51]
Nutrition (eating
disorders)
Female fifth and sixth
grade students from 24
mid-western Girl Scout
troops average age
11 (226)
Children Pre-testndashpost-testndashre-test
design
1
Rabak-Wagener
et al [52]
Nutrition (eating
disorders)
Undergraduate students
from healthful living
classes at a medium-sized
metropolitan university in
the mid-west mainly aged
18ndash23 (105)
College students Pre-testndashpost-test design 1
Rosenkoetter et al [32] Violence First second and third
grade students from 13
classrooms in four schools
in or near a community of
population 50 000 (177)
Children Pre-testndashpost-test design 1
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
527
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Scharrer [53] Violence Sixth grade students from
five classrooms in three
northeast towns (93)
Children Pre-testndashpost-test design
with no control group
1
Sprafkin et al [54] Violence Emotionally disturbed and
learning disabled children
from 29 classes in two
Long Island schools aged
6ndash12 (167)
Children Pre-testndashpost-testndashre-test
design with two
experimental and two
control groups
1
Vooijs and van der
Voort [55]
Violence Fourth fifth and sixth
grade students from 18
classes in six Dutch
schools (437)
Children Pre-testndashpost-testndashre-test
design
1
Vooijs and van der
Voort [56]
Violence Fifth and sixth grade
students from 12 classes in
six Dutch schools (300)
Children Pre-testndashpost-test design 1
Wade et al [57 58] Nutrition (eating
disorders)
Eighth grade students
from four private high
school classes in
Australia average age
134 (86)
Adolescents Post-test-only design with
two experimental groups
Pre-testndashpost-testndashre-test
design with two
experimental groups
1
Wilksch et al [37] Eating disorders (media
internalization)
Eighth grade students
from 11 classes at three
metropolitan private
schools in Australia mean
age 139 (237 100 girls
and 137 boys)
Adolescents Pre-testndashpost-test design
with 3 media literacy
education experimental
groups
3
Wolf-Bloom [59] Nutrition (eating
disorders)
Female adolescents from
two Girl Scout Councils
and a private school in
Northeast Ohio aged 10ndash
15 (70)
Adolescents Pre-testndashpost-testndashre-test
design
1
LJBerg
smaandMECarn
ey
528
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
utilized some qualitative measures as well and one
utilized qualitative measures only [36] Nineteen
studies had one control group one study employed
two different control groups [52] and three studies
had no control group [37 39 50]
Intervention setting
Intervention settings were grouped into the follow-
ing categories (i) in-class (intervention took place
in class during regularly scheduled class time) (ii)
in-school (intervention took place at a school out-
side of the regular classroom) (iii) community
groups or (iv) location not specified (Table II)
Analysis revealed that 19 interventions took place
in-class one both in-class and in community
groups three in school outside of a regular class
or after class three in community groups and two
studies did not specify the intervention setting
Intervention length
This measure was defined as the number of minutes
allotted to the intervention excluding time for pre-
tests post-tests and re-tests When the intervention
took place in a specified number of class sessions
based on analysis of K-12 and college class length
we assigned an average value of 45minutes to a class
session When it was clear that the pre-test or post-
test had been given during the time designated for
the intervention but the pre-testpost-test time was
unspecified we assumed that each evaluation mea-
sure took 15 minutes When a range was provided
for the intervention length we computed an average
Analysis revealed that intervention length ranged
from 25 minutes to 24 hours To better compare
these lengths with intervention effectiveness we
grouped them into the following categories short
(60 minutes or less) average (1ndash5 hours) or long
(more than 5 hours) There were 11 short interven-
tions nine average and eight long (Table II)
Conceptsskills taught
Because the Center for Media Literacyrsquos lsquoCore
Concepts of Media Literacyrsquo [5] has been used
widely in the United States where most of the se-
lected studies were published we used this frame-
work to assess concepts and associated skills
taught Determining core concepts and associated
skills taught required reviewing the publication
multiple times and making many interpretations be-
cause most studies did not specifically state whether
they taught any core concepts The following
guidelines were used to categorize interventions
1 All media messages are lsquoconstructedrsquo Interven-tion taught about how the media differs from
reality evaluating what is shown compared with
real life experiences or the producerproduction
of media messages
2 Media messages are created using a creativelanguage with its own rules Intervention taught
about recognizing advertisingproduction tech-
niques or creatingproducing media messages
3 Different people experience the same messagedifferently Intervention explored how media af-
fect people what people can do to avoid nega-
tive effects of media or that people can take
action to change the media
4 Media have embedded values and points of viewIntervention taught how to identify stereotypes
myths biases values lifestyles andor points
of view represented in or omitted from media
messages
5 Most media messages are constructed to gainprofit andor power Intervention taught about
the purpose of advertising or marketing strate-
gies skepticism toward advertising or creating
counter-advertising
Analysis revealed that the core concepts and as-
sociated skills were taught with the following fre-
quency one (in all 28 interventions) two (17) three
(19) four (28) and five (13) (Table II)
Because our decision to investigate the presence
of the core concepts in the selected studies was
based on prior knowledge of media literacy rather
than a selection of themes that would emerge from
analysis of each text we analyzed the publications
for additional types of knowledge taught Table II
documents the two themes that emerged from this
analysis how media affects health (17) and knowl-
edge about the targeted health issue (14)
Effectiveness of health-promoting media literacy
529
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table II Comparison of intervention setting length core concepts taught and who delivered intervention with effectiveness ratings
Intervention Effectiveness ratingIntervention information
Short
term
Long
term
Setting Length Conceptskills
taughtaWho delivered
interventionb
Fuller et al [36] NR NR School outside class Average 1 2 3 4 5 a b Researchers
Wilksch et al [37] NR NR In-class Short 1 2 4 a Researcher with teacher
present
Byrne [43]mdashbasic Condition 1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashcognitive
activity condition
1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashthird person
condition
1 1 In-class Short 1 3 4 a Unspecified
Irving et al [48]mdashexternally
oriented condition
2 NA Location not specified Short 1 3 4 a Researcher
Irving et al [49] 2 NA School outside class Short 1 2 3 4 5 Teen peer
Scharrer [53] 2 NA In-class Average 1 3 4 a Undergraduate
college student
Wade et al [57 58] 2 1 In-class Average 1 2 3 4 5 a b Teacher assisted by
graduate student
Neumark-Sztainer
et al [51]
2 2 Community groups Long 1 2 3 4 a b Girl Scout troop
leaders
Sprafkin et al [54] 2 3 In-class Long 1 2 4 5 Teacher
Austin et al [42] 3 NA Community groups Average 1 2 3 4 5 b Teen peers
Evans et al [45] 3 NA School outside class Long 1 2 3 4 a b Unspecified
Austin and Johnson
[39 40]3 2 In-class Short 1 2 4 5 Researcher
Kusel [50] 3 2 In-class Average 1 2 3 4 5 a b Researcher
Wolf-Bloom [59] 3 4 Location not specified Average 1 2 3 4 5 a Senior Girl Scout
peers
Austin et al [41] 4 NA In-class and community
groups
Average 1 2 3 4 5 b Teen peers (2)
Coughlin and Kalodner [44] 4 NA In-class Average 1 2 3 4 a b Researcher
Rabak-Wagener et al [52] 4 NA In-class Long 1 2 4 5 a b Researcher and
instructor (assisted)
Rosenkoetter et al [32] 4 NA In-class Long 1 2 3 4 5 a b Researcher
Gonzales et al [46] 5 NA In-class Long 1 2 3 4 5 a b Unspecified
Hindin [23] and Hindin
et al [47]
5 NA Community groups Long 1 2 3 4 5 a b Unspecified
Vooijs and van der
Voort [56]
5 NA In-class Average 1 3 4 b School television
broadcast series
and teacher
Vooijs and van der
Voort [55]
5 5 In-class Long 1 3 4 b Researcher
Austin and Johnson
[40]mdashalcohol-specific
condition
5 5 In-class Short 1 4 5 Researcher
Austin and Johnson
[40]mdashgeneral condition
5 5 In-class Short 1 4 5 Researcher
NR = not rated NA = not applicableaNumbers refer to media literacy core concepts taught and letters refer to (a) how media affect health and (b) knowledge about healthissuebAssumed to be researcher (research project staff) if unspecified
L J Bergsma and M E Carney
530
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Who delivered the intervention
Analysis of each intervention revealed that the per-
son who most frequently delivered the intervention
was the researcher (11) In one study the researcher
was assisted by the class instructor In six studies
where the person who delivered the intervention
was not specified we assumed it was a member
of the research staff Other modes of delivery in-
cluded teen peers (four) some by themselves and
some in teams of two Two interventions involved
the use of college students one utilized a trained
undergraduate student and another utilized the
classroom teacher assisted by a graduate student
Classroom teachers were the primary person deliv-
ering the intervention in three cases although one
intervention utilized a series of school television
broadcasts facilitated by teachers using a guide In
one study the intervention was delivered by Girl
Scout troop leaders (Table II)
Intervention effectiveness
lsquoIntervention effectivenessrsquo was difficult to assess
due in part to the many different health issues
addressed and research designs theoretical models
and outcome measures used Effectiveness was de-
fined very differently in each of the studies which
made it impossible to impose a common framework
upon them In an attempt to find elements by which
to compare the studies the following information
was compiled for each intervention health issue
research design theoretical approach intervention
outcomes (Table III) results at pre- and post-test
results at re-test (delayed post-test) and elaborated
results In addition we recorded how much signif-
icant change the intervention achieved on its meas-
ures at post-test and re-test
This method of standardizing each studyrsquos eval-
uation excluded considerable information For ex-
ample in the research by Austin and Johnson
[38ndash40] analyses were conducted to compare each
measurersquos success in predicting outcomes on other
measures as hypothesized in their Message Inter-
pretation Process Model By choosing to focus only
on the media literacy interventionrsquos direct effect on
each of the selected measures a great deal of useful
information was excluded For this reason we in-
cluded an elaborated description of each studyrsquos
results in the database that was developed from
which to determine the effectiveness ratings
As we reviewed the outcome evaluation meas-
ures to identify ways in which they might be
grouped we found that all of them fell into one of
the following categories knowledge attitudes
behaviors risk factors for the health problem inten-
tions skills or process evaluation Then we re-
reviewed the measures and their results and
recorded which measures corresponded with each
category and the amount of statistically significant
change that the intervention achieved for each cat-
egory at post-test and re-test
Using all of the information compiled about the
results of each study as well as background knowl-
edge about each theoretical approach and the
strengthslimitations of each research design we
attempted to make an unbiased rating of each inter-
ventionrsquos effectiveness Given the diversity of the
data available for comparison we determined that
each studyrsquos rating of effectiveness would primarily
be a function of whether the intervention appeared
to achieve a significant change on evaluated mea-
sures that were of importance according to the
selected theoretical approach Interventions were
rated for short-term effectiveness (outcomes at
post-test) and long-term effectiveness (outcomes
at re-test) on a scale of one to five with five being
the highest effectiveness All 24 interventions rated
included a post-test and 12 included a re-test yield-
ing a total of 36 assigned ratings (Table II) Note
that the two studies recommended for inclusion
during peer review could not be assessed using
the same procedures and raters due to the fact that
the research team was no longer intact These stud-
ies are included in Table II but are marked NR for
not rated
Several measures were taken to insure the reli-
ability of effectiveness assessment First each in-
tervention was labeled with a number and
interventions were sorted randomly so that each re-
viewer got a differently ordered data set Second
the two authors independently made their ratings at
different times with little conversation about how to
Effectiveness of health-promoting media literacy
531
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
education and media analysis For indexes not
health related the keyword lsquohealthrsquo was also used
Studies were also located by reviewing the refer-
ences of primary studies considered in addition to
some articles and reports published about media
literacy in recent years [7 10 11] To further locate
appropriate studies a preliminary list of sources
being considered for inclusion was circulated to
several experts in the field resulting in some addi-
tional recommendations
The literature search was conducted initially by
the first author All publications found were given
to the second author who did her own search and
found several additional studies for a total pool of
65 publications that were reviewed and considered
for inclusion Only 26 met the study selection cri-
teria detailed below
Study selection
The criteria we used to determine study inclusion or
exclusion were as follows
1 Studies were published in English from 1990
through July 2006
2 Studies were peer reviewed (including journal
articles conference proceedings and disserta-
tions) and were publicly available through
a searchable index
3 The sample study design intervention evalua-
tion measures and analysis were described
4 Studies included an experimental media literacy
intervention of 25 or more minutes in length
designed to improve the health of youth
5 The study focused primarily on teaching critical
media literacy skills and did not use media liter-
acy as a small part of a larger curriculum
The time frame for study inclusion was selected
because there have been no comprehensive reviews
of media literacy interventions since 1990 [12]
although the research has continued to advance
Studies that were available to us but were in
press as of July 2006 were excluded from the
analysis [13]
The sample was limited to peer-reviewed publi-
cations due to concerns that non-peer-reviewed
sources would describe less rigorous experimental
procedures and data We debated this decision
however because the youth of the field of media
literacy education research and the small number of
studies that met our criteria made it seem useful to
include sources that had not undergone peer review
but were readily available publicly and seemed to
contribute to the field [14ndash16] including a study
[17] that was reported in a chapter of an edited
book
Some studies were excluded because they did not
describe a rigorous experimental evaluation of
a specific intervention [18 19] One article was
excluded because while it provided information
about the qualitative evaluation conducted none
of the quantitative evaluation measures were de-
scribed and results were not reported [20] Another
study was excluded because the media literacy in-
tervention it used was only 3ndash5 minutes in length
[21] Interventions less than 25 minutes in length
were deemed to be too short to teach critical media
literacy skills and allow students to practice them
using an inquiry approach which is an essential
pedagogical tenet of media literacy education
[22] While most of the selected studies focused
the intervention directly on youth one study was
selected in which the intervention was given to
adult caregivers with a focus on improving the
home nutrition environment to promote the health
of preschoolers [23]
As stated in the fifth criterion above interven-
tions needed to focus on teaching lsquocritical media
literacy skillsrsquo which were operationalized as the
following abilities based on the Center for Media
Literacyrsquos definition [5 p21]
1 To access useful information in the media
2 To analyze media messages using critical think-
ing skills (being able to identify bias and credi-
bility of a source differentiate fact from opinion
determine if a message is unrealistic understand
a messagersquos purpose)
3 To evaluate a message (to determine its truth
applicability)
L J Bergsma and M E Carney
524
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
4 To create media messages which achieve spe-
cific goals
To be included in this review the intervention
needed to teach at least one of the four skills listed
above Although critical media literacy skills may
have been taught in three studies reviewed they
were excluded because it was not clear that this
was the case [24ndash26] Several studies that focused
on teaching youth the skills to resist advertising
pressure to smoke cigarettes were excluded for
the same reason Criterion 5 above also stated that
the intervention must focus lsquoprimarilyrsquo on teaching
critical media literacy skills This distinction was
made because some studies incorporated a small
media literacy component in their intervention as
part of a larger curriculum that taught other knowl-
edge or skills which may have influenced effective-
ness findings [27 28]
Some studies were excluded because they fo-
cused mainly on reducing television use instead
of teaching critical media literacy skills [29ndash31]
Although one of the sources selected did include
the concept of reduced television use in its interven-
tion it was included because it allowed participants
to make informed decisions about media use rather
than simply teaching them that media use should be
avoided [32] Most of the study inclusion or exclu-
sion decisions were made independently by the sec-
ond author based upon the criteria established by
both authors and reviewed by the first author
Where there was any question the authors dis-
cussed their conclusions and a final decision was
made by consensus
During peer review of the manuscript five addi-
tional studies were recommended for inclusion
Upon review we found that one study [33] did
not meet selection criterion 3 and two studies did
not meet criteria 4 and 5 [34 35] Two of the rec-
ommended studies were included [36 37]
Twenty-three studies in 26 publications met the
inclusion criteria [23 32 36ndash59] The same data
from two of the 23 studies were reported in two
publications and two articles reported on different
measures and analyses extracted from the same re-
search study Data from these publications were
grouped and considered as one study
Data extraction and analysis
To enable extraction and analysis of a large quantity
of data we developed an extensive database of in-
formation for each of the selected studies The fol-
lowing sections detail first general descriptive data
and second data that were extracted and analyzed
for each measure
General descriptive data
Table I contains descriptive data about the selected
sample Twenty-eight interventions were detailed
in the 23 studies because two studies employed
multiple experimental groups that experienced dif-
ferent media literacy interventions Austin and
Johnson [40] (two interventions) and Byrne [43]
(three interventions) In addition two studies had
multiple interventions but only some of them were
judged to be media literacy interventions Wilsch
Tiggemann and Wade [37] three of five and Irving
and Berel [48] one of three
Health issues included nutrition (two studies)
nutritioneating disorders prevention (nine) body
image (one) substance abuse prevention (five)
and violence prevention (six) Eighteen studies
recruited participants from schools two from both
schools and community groups and three from non-
school locations Participant sample size ranged
from 17 to 723
Studies were categorized according to target age
as follows children (aged 11 and belowGrade
6 and below) adolescents (aged 12ndash19grades
7ndash12) and college students (mean age 22 and below)
When a study targeted youth in different categories
it was categorized according to the larger number
of participants Thirteen studies targeted children
seven adolescents and three college students
Each studyrsquos research design is included in Table
I While in most cases the research design was
stated explicitly in some instances it was inferred
from the methodology description Because all the
studies used a quasi-experimental design this de-
scriptor was omitted from Table I Most of the stud-
ies selected utilized quantitative measures a few
Effectiveness of health-promoting media literacy
525
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table I Descriptive data concerning systematic review sample
Study Health issue Target population (n) Target age group Research design Number of interventions
Austin and Johnson
[38 39]
Substance abuse (alcohol) Third grade students from
three schools in a northern
mid-western town (246)
Children Solomon four-group
design
1
Austin and Johnson [40] Substance abuse (alcohol) Third grade students from
Roseville MN (225)
Children Solomon four-group
design with two
experimental groups
2
Austin et al [41] Substance abuse (tobacco) Adolescents from 76
Washington state schools
and community groups
mean age 15 (723)
Adolescents Post-test-only design 1
Austin et al [42] Substance abuse (tobacco) Adolescents from
community groups in
three western WA
communities average age
16 (119)
Adolescents Pre-testndashpost-test design 1
Byrne [43] Violence Fourth and fifth grade
students from three west
coast schools (199)
Children Pre-testndashpost-testndashre-test
design with three
experimental groups
3
Coughlin and Kalodner
[44]
Nutrition (eating
disorders)
Female undergraduate
students in introductory
womenrsquos studies course at
a rural university average
age 20 (92)
College students Pre-testndashpost-test design 1
Evans et al [45] Nutrition Fourth and fifth grade
students from two South
Carolina schools (39) and
their parents
Children Pre-testndashpost-test design 1
Fuller et al [36] Body image Fourth grade girls from an
urban school (17)
Children Qualitative focus groups
pre-testndashpost-test design
1
Gonzales et al [46] Substance abuse (tobacco) 10th grade students from
four Los Angeles high
schools (609)
Adolescents Pre-testndashpost-test design 1
LJBerg
smaandMECarn
ey
526
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Hindin [23] and Hindin
et al [47]
Nutrition Low-income mothers
(33) a father and
a grandmother of
preschoolers (aged 3ndash6)
from four NYC Head Start
sites aged 22ndash62 (35)
Children Pre-testndashpost-test
comparison conditionndash
intervention condition
design with no control
group
1
Irving and Berel [48] Nutrition (eating
disorders)
Female college students
from introductory
psychology classes at
Washington State
University average age
19 (110)
College students Post-test-only design with
three experimental groups
and a control group
1
Irving et al [49] Nutrition (eating
disorders)
Female 10th grade
students from two English
classes in a WA high
school average age
15 (41)
Adolescents Post-test-only design 1
Kusel [50] Nutrition
(eating disorders)
Female fourth fifth and
sixth grade students from
two suburban NY school
districts aged 9ndash13 (172)
Children Solomon four-group
design
1
Neumark-Sztainer
et al [51]
Nutrition (eating
disorders)
Female fifth and sixth
grade students from 24
mid-western Girl Scout
troops average age
11 (226)
Children Pre-testndashpost-testndashre-test
design
1
Rabak-Wagener
et al [52]
Nutrition (eating
disorders)
Undergraduate students
from healthful living
classes at a medium-sized
metropolitan university in
the mid-west mainly aged
18ndash23 (105)
College students Pre-testndashpost-test design 1
Rosenkoetter et al [32] Violence First second and third
grade students from 13
classrooms in four schools
in or near a community of
population 50 000 (177)
Children Pre-testndashpost-test design 1
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
527
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Scharrer [53] Violence Sixth grade students from
five classrooms in three
northeast towns (93)
Children Pre-testndashpost-test design
with no control group
1
Sprafkin et al [54] Violence Emotionally disturbed and
learning disabled children
from 29 classes in two
Long Island schools aged
6ndash12 (167)
Children Pre-testndashpost-testndashre-test
design with two
experimental and two
control groups
1
Vooijs and van der
Voort [55]
Violence Fourth fifth and sixth
grade students from 18
classes in six Dutch
schools (437)
Children Pre-testndashpost-testndashre-test
design
1
Vooijs and van der
Voort [56]
Violence Fifth and sixth grade
students from 12 classes in
six Dutch schools (300)
Children Pre-testndashpost-test design 1
Wade et al [57 58] Nutrition (eating
disorders)
Eighth grade students
from four private high
school classes in
Australia average age
134 (86)
Adolescents Post-test-only design with
two experimental groups
Pre-testndashpost-testndashre-test
design with two
experimental groups
1
Wilksch et al [37] Eating disorders (media
internalization)
Eighth grade students
from 11 classes at three
metropolitan private
schools in Australia mean
age 139 (237 100 girls
and 137 boys)
Adolescents Pre-testndashpost-test design
with 3 media literacy
education experimental
groups
3
Wolf-Bloom [59] Nutrition (eating
disorders)
Female adolescents from
two Girl Scout Councils
and a private school in
Northeast Ohio aged 10ndash
15 (70)
Adolescents Pre-testndashpost-testndashre-test
design
1
LJBerg
smaandMECarn
ey
528
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
utilized some qualitative measures as well and one
utilized qualitative measures only [36] Nineteen
studies had one control group one study employed
two different control groups [52] and three studies
had no control group [37 39 50]
Intervention setting
Intervention settings were grouped into the follow-
ing categories (i) in-class (intervention took place
in class during regularly scheduled class time) (ii)
in-school (intervention took place at a school out-
side of the regular classroom) (iii) community
groups or (iv) location not specified (Table II)
Analysis revealed that 19 interventions took place
in-class one both in-class and in community
groups three in school outside of a regular class
or after class three in community groups and two
studies did not specify the intervention setting
Intervention length
This measure was defined as the number of minutes
allotted to the intervention excluding time for pre-
tests post-tests and re-tests When the intervention
took place in a specified number of class sessions
based on analysis of K-12 and college class length
we assigned an average value of 45minutes to a class
session When it was clear that the pre-test or post-
test had been given during the time designated for
the intervention but the pre-testpost-test time was
unspecified we assumed that each evaluation mea-
sure took 15 minutes When a range was provided
for the intervention length we computed an average
Analysis revealed that intervention length ranged
from 25 minutes to 24 hours To better compare
these lengths with intervention effectiveness we
grouped them into the following categories short
(60 minutes or less) average (1ndash5 hours) or long
(more than 5 hours) There were 11 short interven-
tions nine average and eight long (Table II)
Conceptsskills taught
Because the Center for Media Literacyrsquos lsquoCore
Concepts of Media Literacyrsquo [5] has been used
widely in the United States where most of the se-
lected studies were published we used this frame-
work to assess concepts and associated skills
taught Determining core concepts and associated
skills taught required reviewing the publication
multiple times and making many interpretations be-
cause most studies did not specifically state whether
they taught any core concepts The following
guidelines were used to categorize interventions
1 All media messages are lsquoconstructedrsquo Interven-tion taught about how the media differs from
reality evaluating what is shown compared with
real life experiences or the producerproduction
of media messages
2 Media messages are created using a creativelanguage with its own rules Intervention taught
about recognizing advertisingproduction tech-
niques or creatingproducing media messages
3 Different people experience the same messagedifferently Intervention explored how media af-
fect people what people can do to avoid nega-
tive effects of media or that people can take
action to change the media
4 Media have embedded values and points of viewIntervention taught how to identify stereotypes
myths biases values lifestyles andor points
of view represented in or omitted from media
messages
5 Most media messages are constructed to gainprofit andor power Intervention taught about
the purpose of advertising or marketing strate-
gies skepticism toward advertising or creating
counter-advertising
Analysis revealed that the core concepts and as-
sociated skills were taught with the following fre-
quency one (in all 28 interventions) two (17) three
(19) four (28) and five (13) (Table II)
Because our decision to investigate the presence
of the core concepts in the selected studies was
based on prior knowledge of media literacy rather
than a selection of themes that would emerge from
analysis of each text we analyzed the publications
for additional types of knowledge taught Table II
documents the two themes that emerged from this
analysis how media affects health (17) and knowl-
edge about the targeted health issue (14)
Effectiveness of health-promoting media literacy
529
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table II Comparison of intervention setting length core concepts taught and who delivered intervention with effectiveness ratings
Intervention Effectiveness ratingIntervention information
Short
term
Long
term
Setting Length Conceptskills
taughtaWho delivered
interventionb
Fuller et al [36] NR NR School outside class Average 1 2 3 4 5 a b Researchers
Wilksch et al [37] NR NR In-class Short 1 2 4 a Researcher with teacher
present
Byrne [43]mdashbasic Condition 1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashcognitive
activity condition
1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashthird person
condition
1 1 In-class Short 1 3 4 a Unspecified
Irving et al [48]mdashexternally
oriented condition
2 NA Location not specified Short 1 3 4 a Researcher
Irving et al [49] 2 NA School outside class Short 1 2 3 4 5 Teen peer
Scharrer [53] 2 NA In-class Average 1 3 4 a Undergraduate
college student
Wade et al [57 58] 2 1 In-class Average 1 2 3 4 5 a b Teacher assisted by
graduate student
Neumark-Sztainer
et al [51]
2 2 Community groups Long 1 2 3 4 a b Girl Scout troop
leaders
Sprafkin et al [54] 2 3 In-class Long 1 2 4 5 Teacher
Austin et al [42] 3 NA Community groups Average 1 2 3 4 5 b Teen peers
Evans et al [45] 3 NA School outside class Long 1 2 3 4 a b Unspecified
Austin and Johnson
[39 40]3 2 In-class Short 1 2 4 5 Researcher
Kusel [50] 3 2 In-class Average 1 2 3 4 5 a b Researcher
Wolf-Bloom [59] 3 4 Location not specified Average 1 2 3 4 5 a Senior Girl Scout
peers
Austin et al [41] 4 NA In-class and community
groups
Average 1 2 3 4 5 b Teen peers (2)
Coughlin and Kalodner [44] 4 NA In-class Average 1 2 3 4 a b Researcher
Rabak-Wagener et al [52] 4 NA In-class Long 1 2 4 5 a b Researcher and
instructor (assisted)
Rosenkoetter et al [32] 4 NA In-class Long 1 2 3 4 5 a b Researcher
Gonzales et al [46] 5 NA In-class Long 1 2 3 4 5 a b Unspecified
Hindin [23] and Hindin
et al [47]
5 NA Community groups Long 1 2 3 4 5 a b Unspecified
Vooijs and van der
Voort [56]
5 NA In-class Average 1 3 4 b School television
broadcast series
and teacher
Vooijs and van der
Voort [55]
5 5 In-class Long 1 3 4 b Researcher
Austin and Johnson
[40]mdashalcohol-specific
condition
5 5 In-class Short 1 4 5 Researcher
Austin and Johnson
[40]mdashgeneral condition
5 5 In-class Short 1 4 5 Researcher
NR = not rated NA = not applicableaNumbers refer to media literacy core concepts taught and letters refer to (a) how media affect health and (b) knowledge about healthissuebAssumed to be researcher (research project staff) if unspecified
L J Bergsma and M E Carney
530
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Who delivered the intervention
Analysis of each intervention revealed that the per-
son who most frequently delivered the intervention
was the researcher (11) In one study the researcher
was assisted by the class instructor In six studies
where the person who delivered the intervention
was not specified we assumed it was a member
of the research staff Other modes of delivery in-
cluded teen peers (four) some by themselves and
some in teams of two Two interventions involved
the use of college students one utilized a trained
undergraduate student and another utilized the
classroom teacher assisted by a graduate student
Classroom teachers were the primary person deliv-
ering the intervention in three cases although one
intervention utilized a series of school television
broadcasts facilitated by teachers using a guide In
one study the intervention was delivered by Girl
Scout troop leaders (Table II)
Intervention effectiveness
lsquoIntervention effectivenessrsquo was difficult to assess
due in part to the many different health issues
addressed and research designs theoretical models
and outcome measures used Effectiveness was de-
fined very differently in each of the studies which
made it impossible to impose a common framework
upon them In an attempt to find elements by which
to compare the studies the following information
was compiled for each intervention health issue
research design theoretical approach intervention
outcomes (Table III) results at pre- and post-test
results at re-test (delayed post-test) and elaborated
results In addition we recorded how much signif-
icant change the intervention achieved on its meas-
ures at post-test and re-test
This method of standardizing each studyrsquos eval-
uation excluded considerable information For ex-
ample in the research by Austin and Johnson
[38ndash40] analyses were conducted to compare each
measurersquos success in predicting outcomes on other
measures as hypothesized in their Message Inter-
pretation Process Model By choosing to focus only
on the media literacy interventionrsquos direct effect on
each of the selected measures a great deal of useful
information was excluded For this reason we in-
cluded an elaborated description of each studyrsquos
results in the database that was developed from
which to determine the effectiveness ratings
As we reviewed the outcome evaluation meas-
ures to identify ways in which they might be
grouped we found that all of them fell into one of
the following categories knowledge attitudes
behaviors risk factors for the health problem inten-
tions skills or process evaluation Then we re-
reviewed the measures and their results and
recorded which measures corresponded with each
category and the amount of statistically significant
change that the intervention achieved for each cat-
egory at post-test and re-test
Using all of the information compiled about the
results of each study as well as background knowl-
edge about each theoretical approach and the
strengthslimitations of each research design we
attempted to make an unbiased rating of each inter-
ventionrsquos effectiveness Given the diversity of the
data available for comparison we determined that
each studyrsquos rating of effectiveness would primarily
be a function of whether the intervention appeared
to achieve a significant change on evaluated mea-
sures that were of importance according to the
selected theoretical approach Interventions were
rated for short-term effectiveness (outcomes at
post-test) and long-term effectiveness (outcomes
at re-test) on a scale of one to five with five being
the highest effectiveness All 24 interventions rated
included a post-test and 12 included a re-test yield-
ing a total of 36 assigned ratings (Table II) Note
that the two studies recommended for inclusion
during peer review could not be assessed using
the same procedures and raters due to the fact that
the research team was no longer intact These stud-
ies are included in Table II but are marked NR for
not rated
Several measures were taken to insure the reli-
ability of effectiveness assessment First each in-
tervention was labeled with a number and
interventions were sorted randomly so that each re-
viewer got a differently ordered data set Second
the two authors independently made their ratings at
different times with little conversation about how to
Effectiveness of health-promoting media literacy
531
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
4 To create media messages which achieve spe-
cific goals
To be included in this review the intervention
needed to teach at least one of the four skills listed
above Although critical media literacy skills may
have been taught in three studies reviewed they
were excluded because it was not clear that this
was the case [24ndash26] Several studies that focused
on teaching youth the skills to resist advertising
pressure to smoke cigarettes were excluded for
the same reason Criterion 5 above also stated that
the intervention must focus lsquoprimarilyrsquo on teaching
critical media literacy skills This distinction was
made because some studies incorporated a small
media literacy component in their intervention as
part of a larger curriculum that taught other knowl-
edge or skills which may have influenced effective-
ness findings [27 28]
Some studies were excluded because they fo-
cused mainly on reducing television use instead
of teaching critical media literacy skills [29ndash31]
Although one of the sources selected did include
the concept of reduced television use in its interven-
tion it was included because it allowed participants
to make informed decisions about media use rather
than simply teaching them that media use should be
avoided [32] Most of the study inclusion or exclu-
sion decisions were made independently by the sec-
ond author based upon the criteria established by
both authors and reviewed by the first author
Where there was any question the authors dis-
cussed their conclusions and a final decision was
made by consensus
During peer review of the manuscript five addi-
tional studies were recommended for inclusion
Upon review we found that one study [33] did
not meet selection criterion 3 and two studies did
not meet criteria 4 and 5 [34 35] Two of the rec-
ommended studies were included [36 37]
Twenty-three studies in 26 publications met the
inclusion criteria [23 32 36ndash59] The same data
from two of the 23 studies were reported in two
publications and two articles reported on different
measures and analyses extracted from the same re-
search study Data from these publications were
grouped and considered as one study
Data extraction and analysis
To enable extraction and analysis of a large quantity
of data we developed an extensive database of in-
formation for each of the selected studies The fol-
lowing sections detail first general descriptive data
and second data that were extracted and analyzed
for each measure
General descriptive data
Table I contains descriptive data about the selected
sample Twenty-eight interventions were detailed
in the 23 studies because two studies employed
multiple experimental groups that experienced dif-
ferent media literacy interventions Austin and
Johnson [40] (two interventions) and Byrne [43]
(three interventions) In addition two studies had
multiple interventions but only some of them were
judged to be media literacy interventions Wilsch
Tiggemann and Wade [37] three of five and Irving
and Berel [48] one of three
Health issues included nutrition (two studies)
nutritioneating disorders prevention (nine) body
image (one) substance abuse prevention (five)
and violence prevention (six) Eighteen studies
recruited participants from schools two from both
schools and community groups and three from non-
school locations Participant sample size ranged
from 17 to 723
Studies were categorized according to target age
as follows children (aged 11 and belowGrade
6 and below) adolescents (aged 12ndash19grades
7ndash12) and college students (mean age 22 and below)
When a study targeted youth in different categories
it was categorized according to the larger number
of participants Thirteen studies targeted children
seven adolescents and three college students
Each studyrsquos research design is included in Table
I While in most cases the research design was
stated explicitly in some instances it was inferred
from the methodology description Because all the
studies used a quasi-experimental design this de-
scriptor was omitted from Table I Most of the stud-
ies selected utilized quantitative measures a few
Effectiveness of health-promoting media literacy
525
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table I Descriptive data concerning systematic review sample
Study Health issue Target population (n) Target age group Research design Number of interventions
Austin and Johnson
[38 39]
Substance abuse (alcohol) Third grade students from
three schools in a northern
mid-western town (246)
Children Solomon four-group
design
1
Austin and Johnson [40] Substance abuse (alcohol) Third grade students from
Roseville MN (225)
Children Solomon four-group
design with two
experimental groups
2
Austin et al [41] Substance abuse (tobacco) Adolescents from 76
Washington state schools
and community groups
mean age 15 (723)
Adolescents Post-test-only design 1
Austin et al [42] Substance abuse (tobacco) Adolescents from
community groups in
three western WA
communities average age
16 (119)
Adolescents Pre-testndashpost-test design 1
Byrne [43] Violence Fourth and fifth grade
students from three west
coast schools (199)
Children Pre-testndashpost-testndashre-test
design with three
experimental groups
3
Coughlin and Kalodner
[44]
Nutrition (eating
disorders)
Female undergraduate
students in introductory
womenrsquos studies course at
a rural university average
age 20 (92)
College students Pre-testndashpost-test design 1
Evans et al [45] Nutrition Fourth and fifth grade
students from two South
Carolina schools (39) and
their parents
Children Pre-testndashpost-test design 1
Fuller et al [36] Body image Fourth grade girls from an
urban school (17)
Children Qualitative focus groups
pre-testndashpost-test design
1
Gonzales et al [46] Substance abuse (tobacco) 10th grade students from
four Los Angeles high
schools (609)
Adolescents Pre-testndashpost-test design 1
LJBerg
smaandMECarn
ey
526
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Hindin [23] and Hindin
et al [47]
Nutrition Low-income mothers
(33) a father and
a grandmother of
preschoolers (aged 3ndash6)
from four NYC Head Start
sites aged 22ndash62 (35)
Children Pre-testndashpost-test
comparison conditionndash
intervention condition
design with no control
group
1
Irving and Berel [48] Nutrition (eating
disorders)
Female college students
from introductory
psychology classes at
Washington State
University average age
19 (110)
College students Post-test-only design with
three experimental groups
and a control group
1
Irving et al [49] Nutrition (eating
disorders)
Female 10th grade
students from two English
classes in a WA high
school average age
15 (41)
Adolescents Post-test-only design 1
Kusel [50] Nutrition
(eating disorders)
Female fourth fifth and
sixth grade students from
two suburban NY school
districts aged 9ndash13 (172)
Children Solomon four-group
design
1
Neumark-Sztainer
et al [51]
Nutrition (eating
disorders)
Female fifth and sixth
grade students from 24
mid-western Girl Scout
troops average age
11 (226)
Children Pre-testndashpost-testndashre-test
design
1
Rabak-Wagener
et al [52]
Nutrition (eating
disorders)
Undergraduate students
from healthful living
classes at a medium-sized
metropolitan university in
the mid-west mainly aged
18ndash23 (105)
College students Pre-testndashpost-test design 1
Rosenkoetter et al [32] Violence First second and third
grade students from 13
classrooms in four schools
in or near a community of
population 50 000 (177)
Children Pre-testndashpost-test design 1
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
527
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Scharrer [53] Violence Sixth grade students from
five classrooms in three
northeast towns (93)
Children Pre-testndashpost-test design
with no control group
1
Sprafkin et al [54] Violence Emotionally disturbed and
learning disabled children
from 29 classes in two
Long Island schools aged
6ndash12 (167)
Children Pre-testndashpost-testndashre-test
design with two
experimental and two
control groups
1
Vooijs and van der
Voort [55]
Violence Fourth fifth and sixth
grade students from 18
classes in six Dutch
schools (437)
Children Pre-testndashpost-testndashre-test
design
1
Vooijs and van der
Voort [56]
Violence Fifth and sixth grade
students from 12 classes in
six Dutch schools (300)
Children Pre-testndashpost-test design 1
Wade et al [57 58] Nutrition (eating
disorders)
Eighth grade students
from four private high
school classes in
Australia average age
134 (86)
Adolescents Post-test-only design with
two experimental groups
Pre-testndashpost-testndashre-test
design with two
experimental groups
1
Wilksch et al [37] Eating disorders (media
internalization)
Eighth grade students
from 11 classes at three
metropolitan private
schools in Australia mean
age 139 (237 100 girls
and 137 boys)
Adolescents Pre-testndashpost-test design
with 3 media literacy
education experimental
groups
3
Wolf-Bloom [59] Nutrition (eating
disorders)
Female adolescents from
two Girl Scout Councils
and a private school in
Northeast Ohio aged 10ndash
15 (70)
Adolescents Pre-testndashpost-testndashre-test
design
1
LJBerg
smaandMECarn
ey
528
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
utilized some qualitative measures as well and one
utilized qualitative measures only [36] Nineteen
studies had one control group one study employed
two different control groups [52] and three studies
had no control group [37 39 50]
Intervention setting
Intervention settings were grouped into the follow-
ing categories (i) in-class (intervention took place
in class during regularly scheduled class time) (ii)
in-school (intervention took place at a school out-
side of the regular classroom) (iii) community
groups or (iv) location not specified (Table II)
Analysis revealed that 19 interventions took place
in-class one both in-class and in community
groups three in school outside of a regular class
or after class three in community groups and two
studies did not specify the intervention setting
Intervention length
This measure was defined as the number of minutes
allotted to the intervention excluding time for pre-
tests post-tests and re-tests When the intervention
took place in a specified number of class sessions
based on analysis of K-12 and college class length
we assigned an average value of 45minutes to a class
session When it was clear that the pre-test or post-
test had been given during the time designated for
the intervention but the pre-testpost-test time was
unspecified we assumed that each evaluation mea-
sure took 15 minutes When a range was provided
for the intervention length we computed an average
Analysis revealed that intervention length ranged
from 25 minutes to 24 hours To better compare
these lengths with intervention effectiveness we
grouped them into the following categories short
(60 minutes or less) average (1ndash5 hours) or long
(more than 5 hours) There were 11 short interven-
tions nine average and eight long (Table II)
Conceptsskills taught
Because the Center for Media Literacyrsquos lsquoCore
Concepts of Media Literacyrsquo [5] has been used
widely in the United States where most of the se-
lected studies were published we used this frame-
work to assess concepts and associated skills
taught Determining core concepts and associated
skills taught required reviewing the publication
multiple times and making many interpretations be-
cause most studies did not specifically state whether
they taught any core concepts The following
guidelines were used to categorize interventions
1 All media messages are lsquoconstructedrsquo Interven-tion taught about how the media differs from
reality evaluating what is shown compared with
real life experiences or the producerproduction
of media messages
2 Media messages are created using a creativelanguage with its own rules Intervention taught
about recognizing advertisingproduction tech-
niques or creatingproducing media messages
3 Different people experience the same messagedifferently Intervention explored how media af-
fect people what people can do to avoid nega-
tive effects of media or that people can take
action to change the media
4 Media have embedded values and points of viewIntervention taught how to identify stereotypes
myths biases values lifestyles andor points
of view represented in or omitted from media
messages
5 Most media messages are constructed to gainprofit andor power Intervention taught about
the purpose of advertising or marketing strate-
gies skepticism toward advertising or creating
counter-advertising
Analysis revealed that the core concepts and as-
sociated skills were taught with the following fre-
quency one (in all 28 interventions) two (17) three
(19) four (28) and five (13) (Table II)
Because our decision to investigate the presence
of the core concepts in the selected studies was
based on prior knowledge of media literacy rather
than a selection of themes that would emerge from
analysis of each text we analyzed the publications
for additional types of knowledge taught Table II
documents the two themes that emerged from this
analysis how media affects health (17) and knowl-
edge about the targeted health issue (14)
Effectiveness of health-promoting media literacy
529
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table II Comparison of intervention setting length core concepts taught and who delivered intervention with effectiveness ratings
Intervention Effectiveness ratingIntervention information
Short
term
Long
term
Setting Length Conceptskills
taughtaWho delivered
interventionb
Fuller et al [36] NR NR School outside class Average 1 2 3 4 5 a b Researchers
Wilksch et al [37] NR NR In-class Short 1 2 4 a Researcher with teacher
present
Byrne [43]mdashbasic Condition 1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashcognitive
activity condition
1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashthird person
condition
1 1 In-class Short 1 3 4 a Unspecified
Irving et al [48]mdashexternally
oriented condition
2 NA Location not specified Short 1 3 4 a Researcher
Irving et al [49] 2 NA School outside class Short 1 2 3 4 5 Teen peer
Scharrer [53] 2 NA In-class Average 1 3 4 a Undergraduate
college student
Wade et al [57 58] 2 1 In-class Average 1 2 3 4 5 a b Teacher assisted by
graduate student
Neumark-Sztainer
et al [51]
2 2 Community groups Long 1 2 3 4 a b Girl Scout troop
leaders
Sprafkin et al [54] 2 3 In-class Long 1 2 4 5 Teacher
Austin et al [42] 3 NA Community groups Average 1 2 3 4 5 b Teen peers
Evans et al [45] 3 NA School outside class Long 1 2 3 4 a b Unspecified
Austin and Johnson
[39 40]3 2 In-class Short 1 2 4 5 Researcher
Kusel [50] 3 2 In-class Average 1 2 3 4 5 a b Researcher
Wolf-Bloom [59] 3 4 Location not specified Average 1 2 3 4 5 a Senior Girl Scout
peers
Austin et al [41] 4 NA In-class and community
groups
Average 1 2 3 4 5 b Teen peers (2)
Coughlin and Kalodner [44] 4 NA In-class Average 1 2 3 4 a b Researcher
Rabak-Wagener et al [52] 4 NA In-class Long 1 2 4 5 a b Researcher and
instructor (assisted)
Rosenkoetter et al [32] 4 NA In-class Long 1 2 3 4 5 a b Researcher
Gonzales et al [46] 5 NA In-class Long 1 2 3 4 5 a b Unspecified
Hindin [23] and Hindin
et al [47]
5 NA Community groups Long 1 2 3 4 5 a b Unspecified
Vooijs and van der
Voort [56]
5 NA In-class Average 1 3 4 b School television
broadcast series
and teacher
Vooijs and van der
Voort [55]
5 5 In-class Long 1 3 4 b Researcher
Austin and Johnson
[40]mdashalcohol-specific
condition
5 5 In-class Short 1 4 5 Researcher
Austin and Johnson
[40]mdashgeneral condition
5 5 In-class Short 1 4 5 Researcher
NR = not rated NA = not applicableaNumbers refer to media literacy core concepts taught and letters refer to (a) how media affect health and (b) knowledge about healthissuebAssumed to be researcher (research project staff) if unspecified
L J Bergsma and M E Carney
530
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Who delivered the intervention
Analysis of each intervention revealed that the per-
son who most frequently delivered the intervention
was the researcher (11) In one study the researcher
was assisted by the class instructor In six studies
where the person who delivered the intervention
was not specified we assumed it was a member
of the research staff Other modes of delivery in-
cluded teen peers (four) some by themselves and
some in teams of two Two interventions involved
the use of college students one utilized a trained
undergraduate student and another utilized the
classroom teacher assisted by a graduate student
Classroom teachers were the primary person deliv-
ering the intervention in three cases although one
intervention utilized a series of school television
broadcasts facilitated by teachers using a guide In
one study the intervention was delivered by Girl
Scout troop leaders (Table II)
Intervention effectiveness
lsquoIntervention effectivenessrsquo was difficult to assess
due in part to the many different health issues
addressed and research designs theoretical models
and outcome measures used Effectiveness was de-
fined very differently in each of the studies which
made it impossible to impose a common framework
upon them In an attempt to find elements by which
to compare the studies the following information
was compiled for each intervention health issue
research design theoretical approach intervention
outcomes (Table III) results at pre- and post-test
results at re-test (delayed post-test) and elaborated
results In addition we recorded how much signif-
icant change the intervention achieved on its meas-
ures at post-test and re-test
This method of standardizing each studyrsquos eval-
uation excluded considerable information For ex-
ample in the research by Austin and Johnson
[38ndash40] analyses were conducted to compare each
measurersquos success in predicting outcomes on other
measures as hypothesized in their Message Inter-
pretation Process Model By choosing to focus only
on the media literacy interventionrsquos direct effect on
each of the selected measures a great deal of useful
information was excluded For this reason we in-
cluded an elaborated description of each studyrsquos
results in the database that was developed from
which to determine the effectiveness ratings
As we reviewed the outcome evaluation meas-
ures to identify ways in which they might be
grouped we found that all of them fell into one of
the following categories knowledge attitudes
behaviors risk factors for the health problem inten-
tions skills or process evaluation Then we re-
reviewed the measures and their results and
recorded which measures corresponded with each
category and the amount of statistically significant
change that the intervention achieved for each cat-
egory at post-test and re-test
Using all of the information compiled about the
results of each study as well as background knowl-
edge about each theoretical approach and the
strengthslimitations of each research design we
attempted to make an unbiased rating of each inter-
ventionrsquos effectiveness Given the diversity of the
data available for comparison we determined that
each studyrsquos rating of effectiveness would primarily
be a function of whether the intervention appeared
to achieve a significant change on evaluated mea-
sures that were of importance according to the
selected theoretical approach Interventions were
rated for short-term effectiveness (outcomes at
post-test) and long-term effectiveness (outcomes
at re-test) on a scale of one to five with five being
the highest effectiveness All 24 interventions rated
included a post-test and 12 included a re-test yield-
ing a total of 36 assigned ratings (Table II) Note
that the two studies recommended for inclusion
during peer review could not be assessed using
the same procedures and raters due to the fact that
the research team was no longer intact These stud-
ies are included in Table II but are marked NR for
not rated
Several measures were taken to insure the reli-
ability of effectiveness assessment First each in-
tervention was labeled with a number and
interventions were sorted randomly so that each re-
viewer got a differently ordered data set Second
the two authors independently made their ratings at
different times with little conversation about how to
Effectiveness of health-promoting media literacy
531
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table I Descriptive data concerning systematic review sample
Study Health issue Target population (n) Target age group Research design Number of interventions
Austin and Johnson
[38 39]
Substance abuse (alcohol) Third grade students from
three schools in a northern
mid-western town (246)
Children Solomon four-group
design
1
Austin and Johnson [40] Substance abuse (alcohol) Third grade students from
Roseville MN (225)
Children Solomon four-group
design with two
experimental groups
2
Austin et al [41] Substance abuse (tobacco) Adolescents from 76
Washington state schools
and community groups
mean age 15 (723)
Adolescents Post-test-only design 1
Austin et al [42] Substance abuse (tobacco) Adolescents from
community groups in
three western WA
communities average age
16 (119)
Adolescents Pre-testndashpost-test design 1
Byrne [43] Violence Fourth and fifth grade
students from three west
coast schools (199)
Children Pre-testndashpost-testndashre-test
design with three
experimental groups
3
Coughlin and Kalodner
[44]
Nutrition (eating
disorders)
Female undergraduate
students in introductory
womenrsquos studies course at
a rural university average
age 20 (92)
College students Pre-testndashpost-test design 1
Evans et al [45] Nutrition Fourth and fifth grade
students from two South
Carolina schools (39) and
their parents
Children Pre-testndashpost-test design 1
Fuller et al [36] Body image Fourth grade girls from an
urban school (17)
Children Qualitative focus groups
pre-testndashpost-test design
1
Gonzales et al [46] Substance abuse (tobacco) 10th grade students from
four Los Angeles high
schools (609)
Adolescents Pre-testndashpost-test design 1
LJBerg
smaandMECarn
ey
526
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Hindin [23] and Hindin
et al [47]
Nutrition Low-income mothers
(33) a father and
a grandmother of
preschoolers (aged 3ndash6)
from four NYC Head Start
sites aged 22ndash62 (35)
Children Pre-testndashpost-test
comparison conditionndash
intervention condition
design with no control
group
1
Irving and Berel [48] Nutrition (eating
disorders)
Female college students
from introductory
psychology classes at
Washington State
University average age
19 (110)
College students Post-test-only design with
three experimental groups
and a control group
1
Irving et al [49] Nutrition (eating
disorders)
Female 10th grade
students from two English
classes in a WA high
school average age
15 (41)
Adolescents Post-test-only design 1
Kusel [50] Nutrition
(eating disorders)
Female fourth fifth and
sixth grade students from
two suburban NY school
districts aged 9ndash13 (172)
Children Solomon four-group
design
1
Neumark-Sztainer
et al [51]
Nutrition (eating
disorders)
Female fifth and sixth
grade students from 24
mid-western Girl Scout
troops average age
11 (226)
Children Pre-testndashpost-testndashre-test
design
1
Rabak-Wagener
et al [52]
Nutrition (eating
disorders)
Undergraduate students
from healthful living
classes at a medium-sized
metropolitan university in
the mid-west mainly aged
18ndash23 (105)
College students Pre-testndashpost-test design 1
Rosenkoetter et al [32] Violence First second and third
grade students from 13
classrooms in four schools
in or near a community of
population 50 000 (177)
Children Pre-testndashpost-test design 1
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
527
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Scharrer [53] Violence Sixth grade students from
five classrooms in three
northeast towns (93)
Children Pre-testndashpost-test design
with no control group
1
Sprafkin et al [54] Violence Emotionally disturbed and
learning disabled children
from 29 classes in two
Long Island schools aged
6ndash12 (167)
Children Pre-testndashpost-testndashre-test
design with two
experimental and two
control groups
1
Vooijs and van der
Voort [55]
Violence Fourth fifth and sixth
grade students from 18
classes in six Dutch
schools (437)
Children Pre-testndashpost-testndashre-test
design
1
Vooijs and van der
Voort [56]
Violence Fifth and sixth grade
students from 12 classes in
six Dutch schools (300)
Children Pre-testndashpost-test design 1
Wade et al [57 58] Nutrition (eating
disorders)
Eighth grade students
from four private high
school classes in
Australia average age
134 (86)
Adolescents Post-test-only design with
two experimental groups
Pre-testndashpost-testndashre-test
design with two
experimental groups
1
Wilksch et al [37] Eating disorders (media
internalization)
Eighth grade students
from 11 classes at three
metropolitan private
schools in Australia mean
age 139 (237 100 girls
and 137 boys)
Adolescents Pre-testndashpost-test design
with 3 media literacy
education experimental
groups
3
Wolf-Bloom [59] Nutrition (eating
disorders)
Female adolescents from
two Girl Scout Councils
and a private school in
Northeast Ohio aged 10ndash
15 (70)
Adolescents Pre-testndashpost-testndashre-test
design
1
LJBerg
smaandMECarn
ey
528
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
utilized some qualitative measures as well and one
utilized qualitative measures only [36] Nineteen
studies had one control group one study employed
two different control groups [52] and three studies
had no control group [37 39 50]
Intervention setting
Intervention settings were grouped into the follow-
ing categories (i) in-class (intervention took place
in class during regularly scheduled class time) (ii)
in-school (intervention took place at a school out-
side of the regular classroom) (iii) community
groups or (iv) location not specified (Table II)
Analysis revealed that 19 interventions took place
in-class one both in-class and in community
groups three in school outside of a regular class
or after class three in community groups and two
studies did not specify the intervention setting
Intervention length
This measure was defined as the number of minutes
allotted to the intervention excluding time for pre-
tests post-tests and re-tests When the intervention
took place in a specified number of class sessions
based on analysis of K-12 and college class length
we assigned an average value of 45minutes to a class
session When it was clear that the pre-test or post-
test had been given during the time designated for
the intervention but the pre-testpost-test time was
unspecified we assumed that each evaluation mea-
sure took 15 minutes When a range was provided
for the intervention length we computed an average
Analysis revealed that intervention length ranged
from 25 minutes to 24 hours To better compare
these lengths with intervention effectiveness we
grouped them into the following categories short
(60 minutes or less) average (1ndash5 hours) or long
(more than 5 hours) There were 11 short interven-
tions nine average and eight long (Table II)
Conceptsskills taught
Because the Center for Media Literacyrsquos lsquoCore
Concepts of Media Literacyrsquo [5] has been used
widely in the United States where most of the se-
lected studies were published we used this frame-
work to assess concepts and associated skills
taught Determining core concepts and associated
skills taught required reviewing the publication
multiple times and making many interpretations be-
cause most studies did not specifically state whether
they taught any core concepts The following
guidelines were used to categorize interventions
1 All media messages are lsquoconstructedrsquo Interven-tion taught about how the media differs from
reality evaluating what is shown compared with
real life experiences or the producerproduction
of media messages
2 Media messages are created using a creativelanguage with its own rules Intervention taught
about recognizing advertisingproduction tech-
niques or creatingproducing media messages
3 Different people experience the same messagedifferently Intervention explored how media af-
fect people what people can do to avoid nega-
tive effects of media or that people can take
action to change the media
4 Media have embedded values and points of viewIntervention taught how to identify stereotypes
myths biases values lifestyles andor points
of view represented in or omitted from media
messages
5 Most media messages are constructed to gainprofit andor power Intervention taught about
the purpose of advertising or marketing strate-
gies skepticism toward advertising or creating
counter-advertising
Analysis revealed that the core concepts and as-
sociated skills were taught with the following fre-
quency one (in all 28 interventions) two (17) three
(19) four (28) and five (13) (Table II)
Because our decision to investigate the presence
of the core concepts in the selected studies was
based on prior knowledge of media literacy rather
than a selection of themes that would emerge from
analysis of each text we analyzed the publications
for additional types of knowledge taught Table II
documents the two themes that emerged from this
analysis how media affects health (17) and knowl-
edge about the targeted health issue (14)
Effectiveness of health-promoting media literacy
529
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table II Comparison of intervention setting length core concepts taught and who delivered intervention with effectiveness ratings
Intervention Effectiveness ratingIntervention information
Short
term
Long
term
Setting Length Conceptskills
taughtaWho delivered
interventionb
Fuller et al [36] NR NR School outside class Average 1 2 3 4 5 a b Researchers
Wilksch et al [37] NR NR In-class Short 1 2 4 a Researcher with teacher
present
Byrne [43]mdashbasic Condition 1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashcognitive
activity condition
1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashthird person
condition
1 1 In-class Short 1 3 4 a Unspecified
Irving et al [48]mdashexternally
oriented condition
2 NA Location not specified Short 1 3 4 a Researcher
Irving et al [49] 2 NA School outside class Short 1 2 3 4 5 Teen peer
Scharrer [53] 2 NA In-class Average 1 3 4 a Undergraduate
college student
Wade et al [57 58] 2 1 In-class Average 1 2 3 4 5 a b Teacher assisted by
graduate student
Neumark-Sztainer
et al [51]
2 2 Community groups Long 1 2 3 4 a b Girl Scout troop
leaders
Sprafkin et al [54] 2 3 In-class Long 1 2 4 5 Teacher
Austin et al [42] 3 NA Community groups Average 1 2 3 4 5 b Teen peers
Evans et al [45] 3 NA School outside class Long 1 2 3 4 a b Unspecified
Austin and Johnson
[39 40]3 2 In-class Short 1 2 4 5 Researcher
Kusel [50] 3 2 In-class Average 1 2 3 4 5 a b Researcher
Wolf-Bloom [59] 3 4 Location not specified Average 1 2 3 4 5 a Senior Girl Scout
peers
Austin et al [41] 4 NA In-class and community
groups
Average 1 2 3 4 5 b Teen peers (2)
Coughlin and Kalodner [44] 4 NA In-class Average 1 2 3 4 a b Researcher
Rabak-Wagener et al [52] 4 NA In-class Long 1 2 4 5 a b Researcher and
instructor (assisted)
Rosenkoetter et al [32] 4 NA In-class Long 1 2 3 4 5 a b Researcher
Gonzales et al [46] 5 NA In-class Long 1 2 3 4 5 a b Unspecified
Hindin [23] and Hindin
et al [47]
5 NA Community groups Long 1 2 3 4 5 a b Unspecified
Vooijs and van der
Voort [56]
5 NA In-class Average 1 3 4 b School television
broadcast series
and teacher
Vooijs and van der
Voort [55]
5 5 In-class Long 1 3 4 b Researcher
Austin and Johnson
[40]mdashalcohol-specific
condition
5 5 In-class Short 1 4 5 Researcher
Austin and Johnson
[40]mdashgeneral condition
5 5 In-class Short 1 4 5 Researcher
NR = not rated NA = not applicableaNumbers refer to media literacy core concepts taught and letters refer to (a) how media affect health and (b) knowledge about healthissuebAssumed to be researcher (research project staff) if unspecified
L J Bergsma and M E Carney
530
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Who delivered the intervention
Analysis of each intervention revealed that the per-
son who most frequently delivered the intervention
was the researcher (11) In one study the researcher
was assisted by the class instructor In six studies
where the person who delivered the intervention
was not specified we assumed it was a member
of the research staff Other modes of delivery in-
cluded teen peers (four) some by themselves and
some in teams of two Two interventions involved
the use of college students one utilized a trained
undergraduate student and another utilized the
classroom teacher assisted by a graduate student
Classroom teachers were the primary person deliv-
ering the intervention in three cases although one
intervention utilized a series of school television
broadcasts facilitated by teachers using a guide In
one study the intervention was delivered by Girl
Scout troop leaders (Table II)
Intervention effectiveness
lsquoIntervention effectivenessrsquo was difficult to assess
due in part to the many different health issues
addressed and research designs theoretical models
and outcome measures used Effectiveness was de-
fined very differently in each of the studies which
made it impossible to impose a common framework
upon them In an attempt to find elements by which
to compare the studies the following information
was compiled for each intervention health issue
research design theoretical approach intervention
outcomes (Table III) results at pre- and post-test
results at re-test (delayed post-test) and elaborated
results In addition we recorded how much signif-
icant change the intervention achieved on its meas-
ures at post-test and re-test
This method of standardizing each studyrsquos eval-
uation excluded considerable information For ex-
ample in the research by Austin and Johnson
[38ndash40] analyses were conducted to compare each
measurersquos success in predicting outcomes on other
measures as hypothesized in their Message Inter-
pretation Process Model By choosing to focus only
on the media literacy interventionrsquos direct effect on
each of the selected measures a great deal of useful
information was excluded For this reason we in-
cluded an elaborated description of each studyrsquos
results in the database that was developed from
which to determine the effectiveness ratings
As we reviewed the outcome evaluation meas-
ures to identify ways in which they might be
grouped we found that all of them fell into one of
the following categories knowledge attitudes
behaviors risk factors for the health problem inten-
tions skills or process evaluation Then we re-
reviewed the measures and their results and
recorded which measures corresponded with each
category and the amount of statistically significant
change that the intervention achieved for each cat-
egory at post-test and re-test
Using all of the information compiled about the
results of each study as well as background knowl-
edge about each theoretical approach and the
strengthslimitations of each research design we
attempted to make an unbiased rating of each inter-
ventionrsquos effectiveness Given the diversity of the
data available for comparison we determined that
each studyrsquos rating of effectiveness would primarily
be a function of whether the intervention appeared
to achieve a significant change on evaluated mea-
sures that were of importance according to the
selected theoretical approach Interventions were
rated for short-term effectiveness (outcomes at
post-test) and long-term effectiveness (outcomes
at re-test) on a scale of one to five with five being
the highest effectiveness All 24 interventions rated
included a post-test and 12 included a re-test yield-
ing a total of 36 assigned ratings (Table II) Note
that the two studies recommended for inclusion
during peer review could not be assessed using
the same procedures and raters due to the fact that
the research team was no longer intact These stud-
ies are included in Table II but are marked NR for
not rated
Several measures were taken to insure the reli-
ability of effectiveness assessment First each in-
tervention was labeled with a number and
interventions were sorted randomly so that each re-
viewer got a differently ordered data set Second
the two authors independently made their ratings at
different times with little conversation about how to
Effectiveness of health-promoting media literacy
531
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Hindin [23] and Hindin
et al [47]
Nutrition Low-income mothers
(33) a father and
a grandmother of
preschoolers (aged 3ndash6)
from four NYC Head Start
sites aged 22ndash62 (35)
Children Pre-testndashpost-test
comparison conditionndash
intervention condition
design with no control
group
1
Irving and Berel [48] Nutrition (eating
disorders)
Female college students
from introductory
psychology classes at
Washington State
University average age
19 (110)
College students Post-test-only design with
three experimental groups
and a control group
1
Irving et al [49] Nutrition (eating
disorders)
Female 10th grade
students from two English
classes in a WA high
school average age
15 (41)
Adolescents Post-test-only design 1
Kusel [50] Nutrition
(eating disorders)
Female fourth fifth and
sixth grade students from
two suburban NY school
districts aged 9ndash13 (172)
Children Solomon four-group
design
1
Neumark-Sztainer
et al [51]
Nutrition (eating
disorders)
Female fifth and sixth
grade students from 24
mid-western Girl Scout
troops average age
11 (226)
Children Pre-testndashpost-testndashre-test
design
1
Rabak-Wagener
et al [52]
Nutrition (eating
disorders)
Undergraduate students
from healthful living
classes at a medium-sized
metropolitan university in
the mid-west mainly aged
18ndash23 (105)
College students Pre-testndashpost-test design 1
Rosenkoetter et al [32] Violence First second and third
grade students from 13
classrooms in four schools
in or near a community of
population 50 000 (177)
Children Pre-testndashpost-test design 1
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
527
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Scharrer [53] Violence Sixth grade students from
five classrooms in three
northeast towns (93)
Children Pre-testndashpost-test design
with no control group
1
Sprafkin et al [54] Violence Emotionally disturbed and
learning disabled children
from 29 classes in two
Long Island schools aged
6ndash12 (167)
Children Pre-testndashpost-testndashre-test
design with two
experimental and two
control groups
1
Vooijs and van der
Voort [55]
Violence Fourth fifth and sixth
grade students from 18
classes in six Dutch
schools (437)
Children Pre-testndashpost-testndashre-test
design
1
Vooijs and van der
Voort [56]
Violence Fifth and sixth grade
students from 12 classes in
six Dutch schools (300)
Children Pre-testndashpost-test design 1
Wade et al [57 58] Nutrition (eating
disorders)
Eighth grade students
from four private high
school classes in
Australia average age
134 (86)
Adolescents Post-test-only design with
two experimental groups
Pre-testndashpost-testndashre-test
design with two
experimental groups
1
Wilksch et al [37] Eating disorders (media
internalization)
Eighth grade students
from 11 classes at three
metropolitan private
schools in Australia mean
age 139 (237 100 girls
and 137 boys)
Adolescents Pre-testndashpost-test design
with 3 media literacy
education experimental
groups
3
Wolf-Bloom [59] Nutrition (eating
disorders)
Female adolescents from
two Girl Scout Councils
and a private school in
Northeast Ohio aged 10ndash
15 (70)
Adolescents Pre-testndashpost-testndashre-test
design
1
LJBerg
smaandMECarn
ey
528
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
utilized some qualitative measures as well and one
utilized qualitative measures only [36] Nineteen
studies had one control group one study employed
two different control groups [52] and three studies
had no control group [37 39 50]
Intervention setting
Intervention settings were grouped into the follow-
ing categories (i) in-class (intervention took place
in class during regularly scheduled class time) (ii)
in-school (intervention took place at a school out-
side of the regular classroom) (iii) community
groups or (iv) location not specified (Table II)
Analysis revealed that 19 interventions took place
in-class one both in-class and in community
groups three in school outside of a regular class
or after class three in community groups and two
studies did not specify the intervention setting
Intervention length
This measure was defined as the number of minutes
allotted to the intervention excluding time for pre-
tests post-tests and re-tests When the intervention
took place in a specified number of class sessions
based on analysis of K-12 and college class length
we assigned an average value of 45minutes to a class
session When it was clear that the pre-test or post-
test had been given during the time designated for
the intervention but the pre-testpost-test time was
unspecified we assumed that each evaluation mea-
sure took 15 minutes When a range was provided
for the intervention length we computed an average
Analysis revealed that intervention length ranged
from 25 minutes to 24 hours To better compare
these lengths with intervention effectiveness we
grouped them into the following categories short
(60 minutes or less) average (1ndash5 hours) or long
(more than 5 hours) There were 11 short interven-
tions nine average and eight long (Table II)
Conceptsskills taught
Because the Center for Media Literacyrsquos lsquoCore
Concepts of Media Literacyrsquo [5] has been used
widely in the United States where most of the se-
lected studies were published we used this frame-
work to assess concepts and associated skills
taught Determining core concepts and associated
skills taught required reviewing the publication
multiple times and making many interpretations be-
cause most studies did not specifically state whether
they taught any core concepts The following
guidelines were used to categorize interventions
1 All media messages are lsquoconstructedrsquo Interven-tion taught about how the media differs from
reality evaluating what is shown compared with
real life experiences or the producerproduction
of media messages
2 Media messages are created using a creativelanguage with its own rules Intervention taught
about recognizing advertisingproduction tech-
niques or creatingproducing media messages
3 Different people experience the same messagedifferently Intervention explored how media af-
fect people what people can do to avoid nega-
tive effects of media or that people can take
action to change the media
4 Media have embedded values and points of viewIntervention taught how to identify stereotypes
myths biases values lifestyles andor points
of view represented in or omitted from media
messages
5 Most media messages are constructed to gainprofit andor power Intervention taught about
the purpose of advertising or marketing strate-
gies skepticism toward advertising or creating
counter-advertising
Analysis revealed that the core concepts and as-
sociated skills were taught with the following fre-
quency one (in all 28 interventions) two (17) three
(19) four (28) and five (13) (Table II)
Because our decision to investigate the presence
of the core concepts in the selected studies was
based on prior knowledge of media literacy rather
than a selection of themes that would emerge from
analysis of each text we analyzed the publications
for additional types of knowledge taught Table II
documents the two themes that emerged from this
analysis how media affects health (17) and knowl-
edge about the targeted health issue (14)
Effectiveness of health-promoting media literacy
529
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table II Comparison of intervention setting length core concepts taught and who delivered intervention with effectiveness ratings
Intervention Effectiveness ratingIntervention information
Short
term
Long
term
Setting Length Conceptskills
taughtaWho delivered
interventionb
Fuller et al [36] NR NR School outside class Average 1 2 3 4 5 a b Researchers
Wilksch et al [37] NR NR In-class Short 1 2 4 a Researcher with teacher
present
Byrne [43]mdashbasic Condition 1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashcognitive
activity condition
1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashthird person
condition
1 1 In-class Short 1 3 4 a Unspecified
Irving et al [48]mdashexternally
oriented condition
2 NA Location not specified Short 1 3 4 a Researcher
Irving et al [49] 2 NA School outside class Short 1 2 3 4 5 Teen peer
Scharrer [53] 2 NA In-class Average 1 3 4 a Undergraduate
college student
Wade et al [57 58] 2 1 In-class Average 1 2 3 4 5 a b Teacher assisted by
graduate student
Neumark-Sztainer
et al [51]
2 2 Community groups Long 1 2 3 4 a b Girl Scout troop
leaders
Sprafkin et al [54] 2 3 In-class Long 1 2 4 5 Teacher
Austin et al [42] 3 NA Community groups Average 1 2 3 4 5 b Teen peers
Evans et al [45] 3 NA School outside class Long 1 2 3 4 a b Unspecified
Austin and Johnson
[39 40]3 2 In-class Short 1 2 4 5 Researcher
Kusel [50] 3 2 In-class Average 1 2 3 4 5 a b Researcher
Wolf-Bloom [59] 3 4 Location not specified Average 1 2 3 4 5 a Senior Girl Scout
peers
Austin et al [41] 4 NA In-class and community
groups
Average 1 2 3 4 5 b Teen peers (2)
Coughlin and Kalodner [44] 4 NA In-class Average 1 2 3 4 a b Researcher
Rabak-Wagener et al [52] 4 NA In-class Long 1 2 4 5 a b Researcher and
instructor (assisted)
Rosenkoetter et al [32] 4 NA In-class Long 1 2 3 4 5 a b Researcher
Gonzales et al [46] 5 NA In-class Long 1 2 3 4 5 a b Unspecified
Hindin [23] and Hindin
et al [47]
5 NA Community groups Long 1 2 3 4 5 a b Unspecified
Vooijs and van der
Voort [56]
5 NA In-class Average 1 3 4 b School television
broadcast series
and teacher
Vooijs and van der
Voort [55]
5 5 In-class Long 1 3 4 b Researcher
Austin and Johnson
[40]mdashalcohol-specific
condition
5 5 In-class Short 1 4 5 Researcher
Austin and Johnson
[40]mdashgeneral condition
5 5 In-class Short 1 4 5 Researcher
NR = not rated NA = not applicableaNumbers refer to media literacy core concepts taught and letters refer to (a) how media affect health and (b) knowledge about healthissuebAssumed to be researcher (research project staff) if unspecified
L J Bergsma and M E Carney
530
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Who delivered the intervention
Analysis of each intervention revealed that the per-
son who most frequently delivered the intervention
was the researcher (11) In one study the researcher
was assisted by the class instructor In six studies
where the person who delivered the intervention
was not specified we assumed it was a member
of the research staff Other modes of delivery in-
cluded teen peers (four) some by themselves and
some in teams of two Two interventions involved
the use of college students one utilized a trained
undergraduate student and another utilized the
classroom teacher assisted by a graduate student
Classroom teachers were the primary person deliv-
ering the intervention in three cases although one
intervention utilized a series of school television
broadcasts facilitated by teachers using a guide In
one study the intervention was delivered by Girl
Scout troop leaders (Table II)
Intervention effectiveness
lsquoIntervention effectivenessrsquo was difficult to assess
due in part to the many different health issues
addressed and research designs theoretical models
and outcome measures used Effectiveness was de-
fined very differently in each of the studies which
made it impossible to impose a common framework
upon them In an attempt to find elements by which
to compare the studies the following information
was compiled for each intervention health issue
research design theoretical approach intervention
outcomes (Table III) results at pre- and post-test
results at re-test (delayed post-test) and elaborated
results In addition we recorded how much signif-
icant change the intervention achieved on its meas-
ures at post-test and re-test
This method of standardizing each studyrsquos eval-
uation excluded considerable information For ex-
ample in the research by Austin and Johnson
[38ndash40] analyses were conducted to compare each
measurersquos success in predicting outcomes on other
measures as hypothesized in their Message Inter-
pretation Process Model By choosing to focus only
on the media literacy interventionrsquos direct effect on
each of the selected measures a great deal of useful
information was excluded For this reason we in-
cluded an elaborated description of each studyrsquos
results in the database that was developed from
which to determine the effectiveness ratings
As we reviewed the outcome evaluation meas-
ures to identify ways in which they might be
grouped we found that all of them fell into one of
the following categories knowledge attitudes
behaviors risk factors for the health problem inten-
tions skills or process evaluation Then we re-
reviewed the measures and their results and
recorded which measures corresponded with each
category and the amount of statistically significant
change that the intervention achieved for each cat-
egory at post-test and re-test
Using all of the information compiled about the
results of each study as well as background knowl-
edge about each theoretical approach and the
strengthslimitations of each research design we
attempted to make an unbiased rating of each inter-
ventionrsquos effectiveness Given the diversity of the
data available for comparison we determined that
each studyrsquos rating of effectiveness would primarily
be a function of whether the intervention appeared
to achieve a significant change on evaluated mea-
sures that were of importance according to the
selected theoretical approach Interventions were
rated for short-term effectiveness (outcomes at
post-test) and long-term effectiveness (outcomes
at re-test) on a scale of one to five with five being
the highest effectiveness All 24 interventions rated
included a post-test and 12 included a re-test yield-
ing a total of 36 assigned ratings (Table II) Note
that the two studies recommended for inclusion
during peer review could not be assessed using
the same procedures and raters due to the fact that
the research team was no longer intact These stud-
ies are included in Table II but are marked NR for
not rated
Several measures were taken to insure the reli-
ability of effectiveness assessment First each in-
tervention was labeled with a number and
interventions were sorted randomly so that each re-
viewer got a differently ordered data set Second
the two authors independently made their ratings at
different times with little conversation about how to
Effectiveness of health-promoting media literacy
531
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table I Continued
Study Health issue Target population (n) Target age group Research design Number of interventions
Scharrer [53] Violence Sixth grade students from
five classrooms in three
northeast towns (93)
Children Pre-testndashpost-test design
with no control group
1
Sprafkin et al [54] Violence Emotionally disturbed and
learning disabled children
from 29 classes in two
Long Island schools aged
6ndash12 (167)
Children Pre-testndashpost-testndashre-test
design with two
experimental and two
control groups
1
Vooijs and van der
Voort [55]
Violence Fourth fifth and sixth
grade students from 18
classes in six Dutch
schools (437)
Children Pre-testndashpost-testndashre-test
design
1
Vooijs and van der
Voort [56]
Violence Fifth and sixth grade
students from 12 classes in
six Dutch schools (300)
Children Pre-testndashpost-test design 1
Wade et al [57 58] Nutrition (eating
disorders)
Eighth grade students
from four private high
school classes in
Australia average age
134 (86)
Adolescents Post-test-only design with
two experimental groups
Pre-testndashpost-testndashre-test
design with two
experimental groups
1
Wilksch et al [37] Eating disorders (media
internalization)
Eighth grade students
from 11 classes at three
metropolitan private
schools in Australia mean
age 139 (237 100 girls
and 137 boys)
Adolescents Pre-testndashpost-test design
with 3 media literacy
education experimental
groups
3
Wolf-Bloom [59] Nutrition (eating
disorders)
Female adolescents from
two Girl Scout Councils
and a private school in
Northeast Ohio aged 10ndash
15 (70)
Adolescents Pre-testndashpost-testndashre-test
design
1
LJBerg
smaandMECarn
ey
528
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
utilized some qualitative measures as well and one
utilized qualitative measures only [36] Nineteen
studies had one control group one study employed
two different control groups [52] and three studies
had no control group [37 39 50]
Intervention setting
Intervention settings were grouped into the follow-
ing categories (i) in-class (intervention took place
in class during regularly scheduled class time) (ii)
in-school (intervention took place at a school out-
side of the regular classroom) (iii) community
groups or (iv) location not specified (Table II)
Analysis revealed that 19 interventions took place
in-class one both in-class and in community
groups three in school outside of a regular class
or after class three in community groups and two
studies did not specify the intervention setting
Intervention length
This measure was defined as the number of minutes
allotted to the intervention excluding time for pre-
tests post-tests and re-tests When the intervention
took place in a specified number of class sessions
based on analysis of K-12 and college class length
we assigned an average value of 45minutes to a class
session When it was clear that the pre-test or post-
test had been given during the time designated for
the intervention but the pre-testpost-test time was
unspecified we assumed that each evaluation mea-
sure took 15 minutes When a range was provided
for the intervention length we computed an average
Analysis revealed that intervention length ranged
from 25 minutes to 24 hours To better compare
these lengths with intervention effectiveness we
grouped them into the following categories short
(60 minutes or less) average (1ndash5 hours) or long
(more than 5 hours) There were 11 short interven-
tions nine average and eight long (Table II)
Conceptsskills taught
Because the Center for Media Literacyrsquos lsquoCore
Concepts of Media Literacyrsquo [5] has been used
widely in the United States where most of the se-
lected studies were published we used this frame-
work to assess concepts and associated skills
taught Determining core concepts and associated
skills taught required reviewing the publication
multiple times and making many interpretations be-
cause most studies did not specifically state whether
they taught any core concepts The following
guidelines were used to categorize interventions
1 All media messages are lsquoconstructedrsquo Interven-tion taught about how the media differs from
reality evaluating what is shown compared with
real life experiences or the producerproduction
of media messages
2 Media messages are created using a creativelanguage with its own rules Intervention taught
about recognizing advertisingproduction tech-
niques or creatingproducing media messages
3 Different people experience the same messagedifferently Intervention explored how media af-
fect people what people can do to avoid nega-
tive effects of media or that people can take
action to change the media
4 Media have embedded values and points of viewIntervention taught how to identify stereotypes
myths biases values lifestyles andor points
of view represented in or omitted from media
messages
5 Most media messages are constructed to gainprofit andor power Intervention taught about
the purpose of advertising or marketing strate-
gies skepticism toward advertising or creating
counter-advertising
Analysis revealed that the core concepts and as-
sociated skills were taught with the following fre-
quency one (in all 28 interventions) two (17) three
(19) four (28) and five (13) (Table II)
Because our decision to investigate the presence
of the core concepts in the selected studies was
based on prior knowledge of media literacy rather
than a selection of themes that would emerge from
analysis of each text we analyzed the publications
for additional types of knowledge taught Table II
documents the two themes that emerged from this
analysis how media affects health (17) and knowl-
edge about the targeted health issue (14)
Effectiveness of health-promoting media literacy
529
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table II Comparison of intervention setting length core concepts taught and who delivered intervention with effectiveness ratings
Intervention Effectiveness ratingIntervention information
Short
term
Long
term
Setting Length Conceptskills
taughtaWho delivered
interventionb
Fuller et al [36] NR NR School outside class Average 1 2 3 4 5 a b Researchers
Wilksch et al [37] NR NR In-class Short 1 2 4 a Researcher with teacher
present
Byrne [43]mdashbasic Condition 1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashcognitive
activity condition
1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashthird person
condition
1 1 In-class Short 1 3 4 a Unspecified
Irving et al [48]mdashexternally
oriented condition
2 NA Location not specified Short 1 3 4 a Researcher
Irving et al [49] 2 NA School outside class Short 1 2 3 4 5 Teen peer
Scharrer [53] 2 NA In-class Average 1 3 4 a Undergraduate
college student
Wade et al [57 58] 2 1 In-class Average 1 2 3 4 5 a b Teacher assisted by
graduate student
Neumark-Sztainer
et al [51]
2 2 Community groups Long 1 2 3 4 a b Girl Scout troop
leaders
Sprafkin et al [54] 2 3 In-class Long 1 2 4 5 Teacher
Austin et al [42] 3 NA Community groups Average 1 2 3 4 5 b Teen peers
Evans et al [45] 3 NA School outside class Long 1 2 3 4 a b Unspecified
Austin and Johnson
[39 40]3 2 In-class Short 1 2 4 5 Researcher
Kusel [50] 3 2 In-class Average 1 2 3 4 5 a b Researcher
Wolf-Bloom [59] 3 4 Location not specified Average 1 2 3 4 5 a Senior Girl Scout
peers
Austin et al [41] 4 NA In-class and community
groups
Average 1 2 3 4 5 b Teen peers (2)
Coughlin and Kalodner [44] 4 NA In-class Average 1 2 3 4 a b Researcher
Rabak-Wagener et al [52] 4 NA In-class Long 1 2 4 5 a b Researcher and
instructor (assisted)
Rosenkoetter et al [32] 4 NA In-class Long 1 2 3 4 5 a b Researcher
Gonzales et al [46] 5 NA In-class Long 1 2 3 4 5 a b Unspecified
Hindin [23] and Hindin
et al [47]
5 NA Community groups Long 1 2 3 4 5 a b Unspecified
Vooijs and van der
Voort [56]
5 NA In-class Average 1 3 4 b School television
broadcast series
and teacher
Vooijs and van der
Voort [55]
5 5 In-class Long 1 3 4 b Researcher
Austin and Johnson
[40]mdashalcohol-specific
condition
5 5 In-class Short 1 4 5 Researcher
Austin and Johnson
[40]mdashgeneral condition
5 5 In-class Short 1 4 5 Researcher
NR = not rated NA = not applicableaNumbers refer to media literacy core concepts taught and letters refer to (a) how media affect health and (b) knowledge about healthissuebAssumed to be researcher (research project staff) if unspecified
L J Bergsma and M E Carney
530
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Who delivered the intervention
Analysis of each intervention revealed that the per-
son who most frequently delivered the intervention
was the researcher (11) In one study the researcher
was assisted by the class instructor In six studies
where the person who delivered the intervention
was not specified we assumed it was a member
of the research staff Other modes of delivery in-
cluded teen peers (four) some by themselves and
some in teams of two Two interventions involved
the use of college students one utilized a trained
undergraduate student and another utilized the
classroom teacher assisted by a graduate student
Classroom teachers were the primary person deliv-
ering the intervention in three cases although one
intervention utilized a series of school television
broadcasts facilitated by teachers using a guide In
one study the intervention was delivered by Girl
Scout troop leaders (Table II)
Intervention effectiveness
lsquoIntervention effectivenessrsquo was difficult to assess
due in part to the many different health issues
addressed and research designs theoretical models
and outcome measures used Effectiveness was de-
fined very differently in each of the studies which
made it impossible to impose a common framework
upon them In an attempt to find elements by which
to compare the studies the following information
was compiled for each intervention health issue
research design theoretical approach intervention
outcomes (Table III) results at pre- and post-test
results at re-test (delayed post-test) and elaborated
results In addition we recorded how much signif-
icant change the intervention achieved on its meas-
ures at post-test and re-test
This method of standardizing each studyrsquos eval-
uation excluded considerable information For ex-
ample in the research by Austin and Johnson
[38ndash40] analyses were conducted to compare each
measurersquos success in predicting outcomes on other
measures as hypothesized in their Message Inter-
pretation Process Model By choosing to focus only
on the media literacy interventionrsquos direct effect on
each of the selected measures a great deal of useful
information was excluded For this reason we in-
cluded an elaborated description of each studyrsquos
results in the database that was developed from
which to determine the effectiveness ratings
As we reviewed the outcome evaluation meas-
ures to identify ways in which they might be
grouped we found that all of them fell into one of
the following categories knowledge attitudes
behaviors risk factors for the health problem inten-
tions skills or process evaluation Then we re-
reviewed the measures and their results and
recorded which measures corresponded with each
category and the amount of statistically significant
change that the intervention achieved for each cat-
egory at post-test and re-test
Using all of the information compiled about the
results of each study as well as background knowl-
edge about each theoretical approach and the
strengthslimitations of each research design we
attempted to make an unbiased rating of each inter-
ventionrsquos effectiveness Given the diversity of the
data available for comparison we determined that
each studyrsquos rating of effectiveness would primarily
be a function of whether the intervention appeared
to achieve a significant change on evaluated mea-
sures that were of importance according to the
selected theoretical approach Interventions were
rated for short-term effectiveness (outcomes at
post-test) and long-term effectiveness (outcomes
at re-test) on a scale of one to five with five being
the highest effectiveness All 24 interventions rated
included a post-test and 12 included a re-test yield-
ing a total of 36 assigned ratings (Table II) Note
that the two studies recommended for inclusion
during peer review could not be assessed using
the same procedures and raters due to the fact that
the research team was no longer intact These stud-
ies are included in Table II but are marked NR for
not rated
Several measures were taken to insure the reli-
ability of effectiveness assessment First each in-
tervention was labeled with a number and
interventions were sorted randomly so that each re-
viewer got a differently ordered data set Second
the two authors independently made their ratings at
different times with little conversation about how to
Effectiveness of health-promoting media literacy
531
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
utilized some qualitative measures as well and one
utilized qualitative measures only [36] Nineteen
studies had one control group one study employed
two different control groups [52] and three studies
had no control group [37 39 50]
Intervention setting
Intervention settings were grouped into the follow-
ing categories (i) in-class (intervention took place
in class during regularly scheduled class time) (ii)
in-school (intervention took place at a school out-
side of the regular classroom) (iii) community
groups or (iv) location not specified (Table II)
Analysis revealed that 19 interventions took place
in-class one both in-class and in community
groups three in school outside of a regular class
or after class three in community groups and two
studies did not specify the intervention setting
Intervention length
This measure was defined as the number of minutes
allotted to the intervention excluding time for pre-
tests post-tests and re-tests When the intervention
took place in a specified number of class sessions
based on analysis of K-12 and college class length
we assigned an average value of 45minutes to a class
session When it was clear that the pre-test or post-
test had been given during the time designated for
the intervention but the pre-testpost-test time was
unspecified we assumed that each evaluation mea-
sure took 15 minutes When a range was provided
for the intervention length we computed an average
Analysis revealed that intervention length ranged
from 25 minutes to 24 hours To better compare
these lengths with intervention effectiveness we
grouped them into the following categories short
(60 minutes or less) average (1ndash5 hours) or long
(more than 5 hours) There were 11 short interven-
tions nine average and eight long (Table II)
Conceptsskills taught
Because the Center for Media Literacyrsquos lsquoCore
Concepts of Media Literacyrsquo [5] has been used
widely in the United States where most of the se-
lected studies were published we used this frame-
work to assess concepts and associated skills
taught Determining core concepts and associated
skills taught required reviewing the publication
multiple times and making many interpretations be-
cause most studies did not specifically state whether
they taught any core concepts The following
guidelines were used to categorize interventions
1 All media messages are lsquoconstructedrsquo Interven-tion taught about how the media differs from
reality evaluating what is shown compared with
real life experiences or the producerproduction
of media messages
2 Media messages are created using a creativelanguage with its own rules Intervention taught
about recognizing advertisingproduction tech-
niques or creatingproducing media messages
3 Different people experience the same messagedifferently Intervention explored how media af-
fect people what people can do to avoid nega-
tive effects of media or that people can take
action to change the media
4 Media have embedded values and points of viewIntervention taught how to identify stereotypes
myths biases values lifestyles andor points
of view represented in or omitted from media
messages
5 Most media messages are constructed to gainprofit andor power Intervention taught about
the purpose of advertising or marketing strate-
gies skepticism toward advertising or creating
counter-advertising
Analysis revealed that the core concepts and as-
sociated skills were taught with the following fre-
quency one (in all 28 interventions) two (17) three
(19) four (28) and five (13) (Table II)
Because our decision to investigate the presence
of the core concepts in the selected studies was
based on prior knowledge of media literacy rather
than a selection of themes that would emerge from
analysis of each text we analyzed the publications
for additional types of knowledge taught Table II
documents the two themes that emerged from this
analysis how media affects health (17) and knowl-
edge about the targeted health issue (14)
Effectiveness of health-promoting media literacy
529
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table II Comparison of intervention setting length core concepts taught and who delivered intervention with effectiveness ratings
Intervention Effectiveness ratingIntervention information
Short
term
Long
term
Setting Length Conceptskills
taughtaWho delivered
interventionb
Fuller et al [36] NR NR School outside class Average 1 2 3 4 5 a b Researchers
Wilksch et al [37] NR NR In-class Short 1 2 4 a Researcher with teacher
present
Byrne [43]mdashbasic Condition 1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashcognitive
activity condition
1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashthird person
condition
1 1 In-class Short 1 3 4 a Unspecified
Irving et al [48]mdashexternally
oriented condition
2 NA Location not specified Short 1 3 4 a Researcher
Irving et al [49] 2 NA School outside class Short 1 2 3 4 5 Teen peer
Scharrer [53] 2 NA In-class Average 1 3 4 a Undergraduate
college student
Wade et al [57 58] 2 1 In-class Average 1 2 3 4 5 a b Teacher assisted by
graduate student
Neumark-Sztainer
et al [51]
2 2 Community groups Long 1 2 3 4 a b Girl Scout troop
leaders
Sprafkin et al [54] 2 3 In-class Long 1 2 4 5 Teacher
Austin et al [42] 3 NA Community groups Average 1 2 3 4 5 b Teen peers
Evans et al [45] 3 NA School outside class Long 1 2 3 4 a b Unspecified
Austin and Johnson
[39 40]3 2 In-class Short 1 2 4 5 Researcher
Kusel [50] 3 2 In-class Average 1 2 3 4 5 a b Researcher
Wolf-Bloom [59] 3 4 Location not specified Average 1 2 3 4 5 a Senior Girl Scout
peers
Austin et al [41] 4 NA In-class and community
groups
Average 1 2 3 4 5 b Teen peers (2)
Coughlin and Kalodner [44] 4 NA In-class Average 1 2 3 4 a b Researcher
Rabak-Wagener et al [52] 4 NA In-class Long 1 2 4 5 a b Researcher and
instructor (assisted)
Rosenkoetter et al [32] 4 NA In-class Long 1 2 3 4 5 a b Researcher
Gonzales et al [46] 5 NA In-class Long 1 2 3 4 5 a b Unspecified
Hindin [23] and Hindin
et al [47]
5 NA Community groups Long 1 2 3 4 5 a b Unspecified
Vooijs and van der
Voort [56]
5 NA In-class Average 1 3 4 b School television
broadcast series
and teacher
Vooijs and van der
Voort [55]
5 5 In-class Long 1 3 4 b Researcher
Austin and Johnson
[40]mdashalcohol-specific
condition
5 5 In-class Short 1 4 5 Researcher
Austin and Johnson
[40]mdashgeneral condition
5 5 In-class Short 1 4 5 Researcher
NR = not rated NA = not applicableaNumbers refer to media literacy core concepts taught and letters refer to (a) how media affect health and (b) knowledge about healthissuebAssumed to be researcher (research project staff) if unspecified
L J Bergsma and M E Carney
530
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Who delivered the intervention
Analysis of each intervention revealed that the per-
son who most frequently delivered the intervention
was the researcher (11) In one study the researcher
was assisted by the class instructor In six studies
where the person who delivered the intervention
was not specified we assumed it was a member
of the research staff Other modes of delivery in-
cluded teen peers (four) some by themselves and
some in teams of two Two interventions involved
the use of college students one utilized a trained
undergraduate student and another utilized the
classroom teacher assisted by a graduate student
Classroom teachers were the primary person deliv-
ering the intervention in three cases although one
intervention utilized a series of school television
broadcasts facilitated by teachers using a guide In
one study the intervention was delivered by Girl
Scout troop leaders (Table II)
Intervention effectiveness
lsquoIntervention effectivenessrsquo was difficult to assess
due in part to the many different health issues
addressed and research designs theoretical models
and outcome measures used Effectiveness was de-
fined very differently in each of the studies which
made it impossible to impose a common framework
upon them In an attempt to find elements by which
to compare the studies the following information
was compiled for each intervention health issue
research design theoretical approach intervention
outcomes (Table III) results at pre- and post-test
results at re-test (delayed post-test) and elaborated
results In addition we recorded how much signif-
icant change the intervention achieved on its meas-
ures at post-test and re-test
This method of standardizing each studyrsquos eval-
uation excluded considerable information For ex-
ample in the research by Austin and Johnson
[38ndash40] analyses were conducted to compare each
measurersquos success in predicting outcomes on other
measures as hypothesized in their Message Inter-
pretation Process Model By choosing to focus only
on the media literacy interventionrsquos direct effect on
each of the selected measures a great deal of useful
information was excluded For this reason we in-
cluded an elaborated description of each studyrsquos
results in the database that was developed from
which to determine the effectiveness ratings
As we reviewed the outcome evaluation meas-
ures to identify ways in which they might be
grouped we found that all of them fell into one of
the following categories knowledge attitudes
behaviors risk factors for the health problem inten-
tions skills or process evaluation Then we re-
reviewed the measures and their results and
recorded which measures corresponded with each
category and the amount of statistically significant
change that the intervention achieved for each cat-
egory at post-test and re-test
Using all of the information compiled about the
results of each study as well as background knowl-
edge about each theoretical approach and the
strengthslimitations of each research design we
attempted to make an unbiased rating of each inter-
ventionrsquos effectiveness Given the diversity of the
data available for comparison we determined that
each studyrsquos rating of effectiveness would primarily
be a function of whether the intervention appeared
to achieve a significant change on evaluated mea-
sures that were of importance according to the
selected theoretical approach Interventions were
rated for short-term effectiveness (outcomes at
post-test) and long-term effectiveness (outcomes
at re-test) on a scale of one to five with five being
the highest effectiveness All 24 interventions rated
included a post-test and 12 included a re-test yield-
ing a total of 36 assigned ratings (Table II) Note
that the two studies recommended for inclusion
during peer review could not be assessed using
the same procedures and raters due to the fact that
the research team was no longer intact These stud-
ies are included in Table II but are marked NR for
not rated
Several measures were taken to insure the reli-
ability of effectiveness assessment First each in-
tervention was labeled with a number and
interventions were sorted randomly so that each re-
viewer got a differently ordered data set Second
the two authors independently made their ratings at
different times with little conversation about how to
Effectiveness of health-promoting media literacy
531
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table II Comparison of intervention setting length core concepts taught and who delivered intervention with effectiveness ratings
Intervention Effectiveness ratingIntervention information
Short
term
Long
term
Setting Length Conceptskills
taughtaWho delivered
interventionb
Fuller et al [36] NR NR School outside class Average 1 2 3 4 5 a b Researchers
Wilksch et al [37] NR NR In-class Short 1 2 4 a Researcher with teacher
present
Byrne [43]mdashbasic Condition 1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashcognitive
activity condition
1 1 In-class Short 1 3 4 a Unspecified
Byrne [43]mdashthird person
condition
1 1 In-class Short 1 3 4 a Unspecified
Irving et al [48]mdashexternally
oriented condition
2 NA Location not specified Short 1 3 4 a Researcher
Irving et al [49] 2 NA School outside class Short 1 2 3 4 5 Teen peer
Scharrer [53] 2 NA In-class Average 1 3 4 a Undergraduate
college student
Wade et al [57 58] 2 1 In-class Average 1 2 3 4 5 a b Teacher assisted by
graduate student
Neumark-Sztainer
et al [51]
2 2 Community groups Long 1 2 3 4 a b Girl Scout troop
leaders
Sprafkin et al [54] 2 3 In-class Long 1 2 4 5 Teacher
Austin et al [42] 3 NA Community groups Average 1 2 3 4 5 b Teen peers
Evans et al [45] 3 NA School outside class Long 1 2 3 4 a b Unspecified
Austin and Johnson
[39 40]3 2 In-class Short 1 2 4 5 Researcher
Kusel [50] 3 2 In-class Average 1 2 3 4 5 a b Researcher
Wolf-Bloom [59] 3 4 Location not specified Average 1 2 3 4 5 a Senior Girl Scout
peers
Austin et al [41] 4 NA In-class and community
groups
Average 1 2 3 4 5 b Teen peers (2)
Coughlin and Kalodner [44] 4 NA In-class Average 1 2 3 4 a b Researcher
Rabak-Wagener et al [52] 4 NA In-class Long 1 2 4 5 a b Researcher and
instructor (assisted)
Rosenkoetter et al [32] 4 NA In-class Long 1 2 3 4 5 a b Researcher
Gonzales et al [46] 5 NA In-class Long 1 2 3 4 5 a b Unspecified
Hindin [23] and Hindin
et al [47]
5 NA Community groups Long 1 2 3 4 5 a b Unspecified
Vooijs and van der
Voort [56]
5 NA In-class Average 1 3 4 b School television
broadcast series
and teacher
Vooijs and van der
Voort [55]
5 5 In-class Long 1 3 4 b Researcher
Austin and Johnson
[40]mdashalcohol-specific
condition
5 5 In-class Short 1 4 5 Researcher
Austin and Johnson
[40]mdashgeneral condition
5 5 In-class Short 1 4 5 Researcher
NR = not rated NA = not applicableaNumbers refer to media literacy core concepts taught and letters refer to (a) how media affect health and (b) knowledge about healthissuebAssumed to be researcher (research project staff) if unspecified
L J Bergsma and M E Carney
530
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Who delivered the intervention
Analysis of each intervention revealed that the per-
son who most frequently delivered the intervention
was the researcher (11) In one study the researcher
was assisted by the class instructor In six studies
where the person who delivered the intervention
was not specified we assumed it was a member
of the research staff Other modes of delivery in-
cluded teen peers (four) some by themselves and
some in teams of two Two interventions involved
the use of college students one utilized a trained
undergraduate student and another utilized the
classroom teacher assisted by a graduate student
Classroom teachers were the primary person deliv-
ering the intervention in three cases although one
intervention utilized a series of school television
broadcasts facilitated by teachers using a guide In
one study the intervention was delivered by Girl
Scout troop leaders (Table II)
Intervention effectiveness
lsquoIntervention effectivenessrsquo was difficult to assess
due in part to the many different health issues
addressed and research designs theoretical models
and outcome measures used Effectiveness was de-
fined very differently in each of the studies which
made it impossible to impose a common framework
upon them In an attempt to find elements by which
to compare the studies the following information
was compiled for each intervention health issue
research design theoretical approach intervention
outcomes (Table III) results at pre- and post-test
results at re-test (delayed post-test) and elaborated
results In addition we recorded how much signif-
icant change the intervention achieved on its meas-
ures at post-test and re-test
This method of standardizing each studyrsquos eval-
uation excluded considerable information For ex-
ample in the research by Austin and Johnson
[38ndash40] analyses were conducted to compare each
measurersquos success in predicting outcomes on other
measures as hypothesized in their Message Inter-
pretation Process Model By choosing to focus only
on the media literacy interventionrsquos direct effect on
each of the selected measures a great deal of useful
information was excluded For this reason we in-
cluded an elaborated description of each studyrsquos
results in the database that was developed from
which to determine the effectiveness ratings
As we reviewed the outcome evaluation meas-
ures to identify ways in which they might be
grouped we found that all of them fell into one of
the following categories knowledge attitudes
behaviors risk factors for the health problem inten-
tions skills or process evaluation Then we re-
reviewed the measures and their results and
recorded which measures corresponded with each
category and the amount of statistically significant
change that the intervention achieved for each cat-
egory at post-test and re-test
Using all of the information compiled about the
results of each study as well as background knowl-
edge about each theoretical approach and the
strengthslimitations of each research design we
attempted to make an unbiased rating of each inter-
ventionrsquos effectiveness Given the diversity of the
data available for comparison we determined that
each studyrsquos rating of effectiveness would primarily
be a function of whether the intervention appeared
to achieve a significant change on evaluated mea-
sures that were of importance according to the
selected theoretical approach Interventions were
rated for short-term effectiveness (outcomes at
post-test) and long-term effectiveness (outcomes
at re-test) on a scale of one to five with five being
the highest effectiveness All 24 interventions rated
included a post-test and 12 included a re-test yield-
ing a total of 36 assigned ratings (Table II) Note
that the two studies recommended for inclusion
during peer review could not be assessed using
the same procedures and raters due to the fact that
the research team was no longer intact These stud-
ies are included in Table II but are marked NR for
not rated
Several measures were taken to insure the reli-
ability of effectiveness assessment First each in-
tervention was labeled with a number and
interventions were sorted randomly so that each re-
viewer got a differently ordered data set Second
the two authors independently made their ratings at
different times with little conversation about how to
Effectiveness of health-promoting media literacy
531
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Who delivered the intervention
Analysis of each intervention revealed that the per-
son who most frequently delivered the intervention
was the researcher (11) In one study the researcher
was assisted by the class instructor In six studies
where the person who delivered the intervention
was not specified we assumed it was a member
of the research staff Other modes of delivery in-
cluded teen peers (four) some by themselves and
some in teams of two Two interventions involved
the use of college students one utilized a trained
undergraduate student and another utilized the
classroom teacher assisted by a graduate student
Classroom teachers were the primary person deliv-
ering the intervention in three cases although one
intervention utilized a series of school television
broadcasts facilitated by teachers using a guide In
one study the intervention was delivered by Girl
Scout troop leaders (Table II)
Intervention effectiveness
lsquoIntervention effectivenessrsquo was difficult to assess
due in part to the many different health issues
addressed and research designs theoretical models
and outcome measures used Effectiveness was de-
fined very differently in each of the studies which
made it impossible to impose a common framework
upon them In an attempt to find elements by which
to compare the studies the following information
was compiled for each intervention health issue
research design theoretical approach intervention
outcomes (Table III) results at pre- and post-test
results at re-test (delayed post-test) and elaborated
results In addition we recorded how much signif-
icant change the intervention achieved on its meas-
ures at post-test and re-test
This method of standardizing each studyrsquos eval-
uation excluded considerable information For ex-
ample in the research by Austin and Johnson
[38ndash40] analyses were conducted to compare each
measurersquos success in predicting outcomes on other
measures as hypothesized in their Message Inter-
pretation Process Model By choosing to focus only
on the media literacy interventionrsquos direct effect on
each of the selected measures a great deal of useful
information was excluded For this reason we in-
cluded an elaborated description of each studyrsquos
results in the database that was developed from
which to determine the effectiveness ratings
As we reviewed the outcome evaluation meas-
ures to identify ways in which they might be
grouped we found that all of them fell into one of
the following categories knowledge attitudes
behaviors risk factors for the health problem inten-
tions skills or process evaluation Then we re-
reviewed the measures and their results and
recorded which measures corresponded with each
category and the amount of statistically significant
change that the intervention achieved for each cat-
egory at post-test and re-test
Using all of the information compiled about the
results of each study as well as background knowl-
edge about each theoretical approach and the
strengthslimitations of each research design we
attempted to make an unbiased rating of each inter-
ventionrsquos effectiveness Given the diversity of the
data available for comparison we determined that
each studyrsquos rating of effectiveness would primarily
be a function of whether the intervention appeared
to achieve a significant change on evaluated mea-
sures that were of importance according to the
selected theoretical approach Interventions were
rated for short-term effectiveness (outcomes at
post-test) and long-term effectiveness (outcomes
at re-test) on a scale of one to five with five being
the highest effectiveness All 24 interventions rated
included a post-test and 12 included a re-test yield-
ing a total of 36 assigned ratings (Table II) Note
that the two studies recommended for inclusion
during peer review could not be assessed using
the same procedures and raters due to the fact that
the research team was no longer intact These stud-
ies are included in Table II but are marked NR for
not rated
Several measures were taken to insure the reli-
ability of effectiveness assessment First each in-
tervention was labeled with a number and
interventions were sorted randomly so that each re-
viewer got a differently ordered data set Second
the two authors independently made their ratings at
different times with little conversation about how to
Effectiveness of health-promoting media literacy
531
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Intervention outcomes
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Byrne [43]mdashbasic condition 1 1 [ willingness to use aggression but no
significant improvement in ability to critically
evaluate violent characters
Significant Y willingness to use aggression
(still above control) and no effect in ability to
critically evaluate violent characters
Byrne [43]mdashcognitive
activity condition
1 1 Y willingness to use aggression (still well
above control) and no effect on ability to
critically evaluate violent characters in the
media
No significant effects
Byrne [43]mdashthird
person condition
1 1 [ willingness to use aggression and no effect
on ability to critically evaluate violent
characters
No significant effects
Irving and Berel [48]
mdashexternally oriented
condition
2 NA Y perceived realism similarity and
desirability of media portraying thin ideal of
beauty Somewhat Y intention to diet No
effect on body dissatisfaction or
internalization of sociocultural physical
appearance standards
Irving et al [49] 2 NA Y internalization of thin beauty standards and
perceived realism of media portrayals No
effect on body dissatisfaction anxiety about
weight-related appearance awareness of
beauty standards in the media desirability of
looking like a media image positive
expectations associated with looking like
a media image or affect
Scharrer [53] 2 NA Quantitative measures showed some [comprehension of key concepts and critical
thinking about the topic of media violence
Qualitative measures showed enhanced
sophistication in analyzing media violence
Wade et al [57 58] 2 1 Y levels of weight concern No significant
effects for eating disorder risk factors body
dissatisfaction or self-esteem
Significant [ dietary restraint and some +differences in self-esteem
LJBerg
smaandMECarn
ey
532
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Neumark-Sztainer
et al [51]
2 2 Notable + influence on internalization of
sociocultural ideals self-efficacy to impact
weight-related social norms and print media
habits Little effect on dieting behaviors
Dieting behaviors and body-related
knowledge and attitudes were not
significantly different at 3 months follow-up
Sprafkin et al [54] 2 3 Significant [ knowledge of television
realism Emotionally disturbed children
significantly Y their identification with
aggressive television characters but learning
disability children did not No significant
changes in attitudes about television or
viewing habits
Significant changes in perceptions of media
portrayals of reality versus fantasy
knowledge of special effects and veracity of
commercial messages
Austin et al [41] 3 NA Significantly [ knowledge of tobacco
marketing skepticism regarding persuasive
messages efficacy to counter tobacco
advertising perceived peer norms and peer
influence [ desirability of smoking
portrayals Some effect on perceived realism
of media messages and no effect on tobacco
use behavior peer discussion perceived
similarity with people in the media
identification with media portrayals and
expectancies to smoke
Evans et al [45] 3 NA Significant [ motivation to eat fruits and
vegetables (FV) and home environment
changes No significant change in FV eating
Austin and Johnson [38 39] 3 2 Significant change in understanding of
persuasive intent perceptions of realism
perceptions of social norms and pre-drinking
behavior No significant change in perceived
similarity to television identification with
television portrayals and expectancies for
drinking alcohol
Some change was detectable 3 months but
only significant for perceived realism
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
533
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Kusel [50] 3 2 Significantly Y dieting body dissatisfaction
internalization of body stereotypes perceived
realism desirability and expectancies and [self-esteem No effect on changing attitudes
and behaviors about restricting food or
purging body distortion awareness of ideal
body stereotype understanding of persuasive
intent perceived social norms similarity or
identification
At 3-month follow-up significant Y body
dissatisfaction perceived realism and
perceived similarity and [ awareness of body
stereotypes No effect on other measures
Wolf-Bloom [59] 3 4 Significantly Y tendency to compare
physical appearance with models and rating
of television importanceinfluence
Significantly [ self-esteem and body image
Some + effect on ideal-body internalization
No effect on self-reported eating disordered
symptomatology mastery and coping or
perceptions of current ideal figure
discrepancy
At 2-month follow-up there was an effect on
all measures except ideal-body internalization
and eating disordered symptomatology
Austinet al [41] 4 NA Significantly Y perceived realism perceived
similarity expectancies about smoking and
susceptibility to peer influence Significantly
changed perceived peer norms and almost
significantly Y identification with people in
tobacco advertisements
Coughlin and Kalodner [44] 4 NA Women at high risk for an eating disorder
had significantly Y body dissatisfaction
drive for thinness feelings of ineffectiveness
and internalization of societal standards of
beauty No significantly Y perfectionism
physical appearance comparisons or
awareness of societal standards for beauty
No changes in eating disorder risk factors
found among low-risk participants
LJBerg
smaandMECarn
ey
534
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Table III Continued
Intervention Effectiveness rating Outcomes
Short term Long term Post-test Re-test
Rabak-Wagener et al [52] 4 NA Significant improvement in beliefs about
body image but no significant changes in
behavior Women not men had
significantly [ beliefs that models have
ideal body sizeshape and that dieting
exercising decisions should be based more
on looks than health status
Rosenkoetter et al [32] 4 NA Among girls significantly Y viewing of
violent television and identification with
violent characters Significantly Y + attitudes
toward television violence Boys had no
significant change in television violence
viewing identification with violent characters
or attitudes about television violence
Classmates reported Y behavioral aggression
among boys
Gonzales et al [46] 5 NA Significantly [ knowledge about tobacco use
and Y pro-tobacco attitudes and tobacco use
behavior
Hindin [23] and Hindin
et al [47]
5 NA Significant effects on Head Start parentsrsquo
understanding television advertising
attitudes about television ads outcomes
expectations of talking with children about
television ads while co-viewing self-efficacy
television mediation behavior and ability to
read food labels
Vooijs et al [56] 5 NA Significant + effect on measures of perceived
realism and factual knowledge
Vooijs and Van Der Voort [55] 5 5 Significantly Y readiness to see violence
approval of violent actions and perceived
realism Effects were found among fifth and
sixth graders not among fourth graders
Effects were weaker but still significant at 2-
year follow-up
Effectiv
eness
ofhealth
-promotin
gmedia
literacy
535
by guest on June 15 2015 httpheroxfordjournalsorg Downloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
do it and provided only with the data described
above for each study and some basic information
about theoretical models used Finally two public
health graduate students independently rated the
interventions based on the same information In
two cases where the reviewers were evenly split
the rating of the senior researcher was used
Using Cohenrsquos kappa inter-rater statistics were
calculated to measure agreement on assigning the
interventions to effectiveness categories A kappa
statistic of gt061 indicates a substantial strength of
agreement [60] When the rating was 1 the kappa
statistic was 091 when the rating was 5 the kappa
statistic was 080 and when the rating was 2ndash4
there was more variability in the kappa statistic
Overall all four raters had a kappa statistic of
063 for all ratings The effectiveness ratings were
entered into a database (Table II) and used as an
independent variable against which to analyze each
of the four dependent measures
Results
Results of our systematic review were as follows
Intervention setting
Table II illustrates the relationship of intervention
setting with short-term and long-term effectiveness
Because the majority of the interventions occurred
in classrooms in-class interventions were well rep-
resented at every level of effectiveness Insufficient
numbers of the other intervention settings pre-
vented detection of a clear relationship
Intervention length
Table II shows that short interventions were some-
what more likely to be rated as ineffective (rated
1ndash2) whereas longer interventions were more
likely to be rated as effective (rated 3ndash5) although
several interventions did not follow this pattern For
example two interventions of Austin and Johnson
[40] were only 45 minutes but they were given
the highest effectiveness rating Conversely theTable
III
Con
tinu
ed
Intervention
Effectivenessrating
Outcomes
Shortterm
Longterm
Post-test
Re-test
Austin
andJohnson[39]mdash
alcohol-specific
condition
55
Effectsincluded
[understandingof
persuasiveintentviewingofcharacters
as
less
similar
torealpeoplethey
knew
andless
desirable
andYdesireto
belikethe
charactersexpectationofpositive
consequencesfrom
drinkingalcoholand
likelihoodto
choose
analcohol-related
productIndirecteffectsfoundonperceptions
oftelevisionrealism
andviewsofsocial
norm
srelatedto
alcoholTreatmentmore
effectiveam
onggirlsthan
boys
Delayed
effectsat
3monthswereconfirm
ed
onexpectancies
andbehavior
Austin
andJohnson[39]mdash
general
condition
55
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
Treatmentwas
effectivebutsomew
hat
less
than
thealcohol-specifictreatm
ent
[increasedYdecreased+positiveNA
=notapplicable
L J Bergsma and M E Carney
536
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
Neumark-Sztainer et al [51] intervention and the
Sprafkin et al [54] intervention were both long
(540 and 420 minutes respectively) but were given
a rating of 2
Conceptsskills taught
When examining the media literacy core concepts
skills taught effective interventions seemed some-
what more likely than ineffective interventions to
have taught all the core concepts Table IV gives
percentages for the amount that each core concept
was taught in short-term and long-term effective
and ineffective interventions
Because core concepts one and three were taught
in 100 of all cases it is impossible to determine
how they contributed to the effectiveness or inef-
fectiveness of the interventions Differences be-
tween effective and ineffective interventions were
evident for core concepts two and five when these
variables were compared with the measure of short-
term effectiveness For core concept two 73 of
effective interventions taught this concept as com-
pared with 44 of ineffective interventions For
core concept five 73 of effective interventions
taught this concept compared with 33 of ineffec-
tive interventions Differences in the same direction
were found for core concept five when compared
with long-term effectiveness ratings but the same
did not hold for core concept two For core concept
three the relationship was in the opposite direction
with more short-term ineffective interventions
(88 ineffective versus 73 effective) and more
long-term ineffective interventions (86 ineffec-
tive versus 40 effective) teaching this concept
Effective interventions appeared to be somewhat
more likely to have taught knowledge about the
targeted health issue 73 effective compared with
22 ineffective at short term Interestingly ineffec-
tive interventions appeared to be more likely to
teach participants about how media affect their
health 77 of ineffective compared with 53 of
effective interventions at short term and 86 of
ineffective compared with 20 of effective inter-
ventions at long term
Who delivered the intervention
There seemed to be some association between who
delivered the intervention and effectiveness at short
term with 10 (73) of the 15 effective interven-
tions and four (44) of the nine ineffective inter-
ventions being delivered by research staff (Table II)
Discussion
Media literacy education has the potential to be
a useful health-promoting strategy for ameliorating
a number of harmful health behaviors To date
however evidence for its potential is based more
on theory than on rigorous demonstrations of effi-
cacy or effectiveness In this review we have docu-
mented a selection of peer-reviewed studies in
which media literacy education has been used to
promote health among youth and have taken a first
step toward developing a framework for research
that will ultimately determine the effectiveness of
this health promotion strategy The research ques-
tion that guided this effort was lsquoWhat are the context
and process elements of an effective health-
promoting media literacy education interventionrsquo
Because this field of research is relatively new
there is a limited pool of studies that have been
reported and those that are available vary greatly
Table IV Percentages of ineffective and effective
interventions teaching media literacy core concepts andhealth-related information
Core concept or other
knowledgeaShort term Long term
EffectiveIneffectiveEffectiveIneffective
One 100 100 100 100
Two 73 44 40 57
Three 73 88 40 86
Four 100 100 100 100
Five 73 33 80 43
How media affects
health
53 77 20 86
Knowledge about
selected health issue
73 22 20 43
aSee Conceptsskills taught section above for the core conceptrelated to each number
Effectiveness of health-promoting media literacy
537
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
with respect to intervention methodological preci-
sion and outcome measurements As a result we
conducted a systematic review of a small sample
of studies with no statistical analysis of results that
can lead to correlations of context and process var-
iables with effectiveness ratings The most defini-
tive trends in this review emerged around the
variables concerning the intervention setting (con-
text) and who taught the intervention (process)
Our review shows a clear trend that most health-
promoting media literacy programs are conducted
in classrooms Although it is interesting to specu-
late on why this is so the data provide no reason to
believe that health-promoting media literacy educa-
tion programs conducted in community or other
non-school settings would not be effective
More of the short-term effective interventions
were delivered by research staff than by others It
seems reasonable to assume that research staff
would be more familiar with the concepts and more
facile with the pedagogical techniques of the inter-
vention than someone who had received a limited
amount of training (eg others who were trained to
deliver the educational program in some of the stud-
ies received anywhere from an hour to a day of
training) This finding is also corroborated by many
observations of the first author in her own work and
discussion with others who train people to incorpo-
rate media literacy education into their curricular
areas It often takes teachers who are trained in
the concepts and skills of media literacy education
at least a year of consistent practice to become
skilled enough that they can readily identify and
facilitate media literacy teachable moments in
which they feel confident with allowing students
to discover concepts for themselves [5 61]
Although we would like to draw more global
conclusions from the data regarding intervention
length and conceptsskills taught our review does
not provide definitive trends with regard to these
variables What the data do tell us is that there
has been a number of both effective and ineffec-
tive health-promoting media literacy education
programs that have varied widely with regard to
length and conceptsskills taught Both effective
and ineffective interventions may be short or long
Although it makes intuitive sense that education
provided for a longer period of time should result
in greater change because the participants have
more opportunity to learn media literacy concepts
and practice associated inquiry skills we cannot
draw this conclusion from our review Although
the most effective interventions seemed somewhat
more likely to have taught all of the core media
literacy concepts as well as knowledge about the
targeted health issue because of the small sample
size the data on conceptsskills taught in each in-
tervention is inconclusive as to whether certain
core concepts may have more or less impact on
effectiveness
Our review results outline characteristics of the
majority of the important health-promoting media
literacy studies that were conducted from 1990
through July 2006 including health issues
addressed (nutrition body image and eating disor-
ders violence and alcohol and tobacco abuse) We
were unable to locate studies about media literacy
education interventions focused on prevention of
unsafe sexual behaviors even though this is a key
health concern There is some evidence that this
type of education is taking place [62] but we could
find no peer-reviewed research on media literacy
curricula that address this issue This may be due
to the human subjects protection concerns involved
with children under 18 as well as the fact that many
educational settings shy away from dealing with
this controversial issue
Limitations
Despite our methodological efforts to minimize bi-
as and errors it is important to note that this review
relied on many subjective judgments For example
the literature search was a collaborative effort of
both authors and could have been done more sys-
tematically to improve yield Even though we uti-
lized numerous methods to identify studies that met
our selection criteria we inadvertently omitted
studies that would have contributed to this research
as evidenced by study inclusion recommendations
emanating from peer review
Systematic reviews are also biased by the amount
of textual table and figure information allowed by
L J Bergsma and M E Carney
538
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
the journals in which the studies are published For
example studies describing lengthy media literacy
curricula are often forced to omit important details
about methodology used such as specific skills
taught or pedagogical approach used Contacting
the primary investigator of each study would prob-
ably clarify some questions about methods used
but was beyond the scope of this study An example
of this bias can be found in our own reporting of
this review Spatial limitations for tables prevented
us from including several additional columns of
data that we extracted and textual limitations pre-
vented us from citing every study considered and
the reasons for excluding each one
Our assignment of intervention effectiveness rat-
ings could also be biased by several factors For
one the ratings were influenced by the number
and type of outcome measures in each study This
made results difficult to compare because some
studies demonstrated significant change on all of
the variables they measured but did not investigate
many factors Conversely other studies that mea-
sured more variables found significant change on
several important factors but were portrayed as
having a low percentage of change overall because
a difference was not detectable on many variables
Another limitation of the present review is that
the methodology used cannot determine if the four
selected measures are the main factors responsible
for effectiveness of health-promoting media literacy
education In fact media literacy education propo-
nents maintain that successful media literacy edu-
cation results not so much from lsquowhatrsquo is taught
as lsquohowrsquo it is taught [58 63] Although interven-
tion setting length and who taught the intervention
provide some information about how these health-
promoting media literacy interventions were taught
investigating the pedagogical approach used would
more significantly inform the lsquohowrsquo question Me-
dia literacy education is grounded in inquiry-based
process-oriented pedagogy Unfortunately whether
the pedagogical approach used in the sample stud-
ies was one of inquiry or indoctrination was un-
clear Only curriculum review and observation of
its teaching can provide reliable data on pedagogi-
cal approach
Implications and recommendations
Systematic literature reviews that identify both what
we know and do not know can help in planning new
research [8] The main benefit of this review is that it
provides a detailed picture of the health-promoting
media literacy education research that has been con-
ducted and published in the years from 1990 through
July 2006 This information can help future re-
searchers interested in investigating both what con-
text and process elements make a health-promoting
media literacy education intervention effective and
what are the outcome measures that best demon-
strate effectiveness It should also provide support
to health educators wanting to implement media lit-
eracy education as a health promotion strategy but
facing questions from others about its effectiveness
Likewise the effectiveness analysis of interven-
tion outcomes provides some insights that may help
to improve the health and media literacy outcomes
of such efforts The majority of outcomes involved
knowledge and attitudes and revealed less about
actually preventing or changing risky health behav-
ior which would demonstrate an effective interven-
tion This has implications for the fact that we need
to design studies which measure more behavioral
outcomes such as amount of smoking or clinical
outcomes such as BMI change Our review did suc-
ceed in quantifying effectiveness to allow for anal-
ysis of four variables across many different types of
studies We hope the current review will inform the
work of other researchers to clarify the outcome
measures that best demonstrate health-promoting
media literacy education effectiveness
Future research is needed to examine other aspects
of media literacy education that could be responsible
for effectiveness For example researchers could an-
alyze the pedagogical style used the amounttype of
training provided to the instructor the key concepts
and skills taught the health behavior theory used to
guide the intervention etc Studies could also be an-
alyzed specifically for variations among participants
of different ages genders racesethnic groups socio-
economic status or other demographic variables
This review provides a framework for organizing
research about media literacy education Our analysis
Effectiveness of health-promoting media literacy
539
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
and discussion of the dependent measures suggest
that researchers should be more explicit about the
media literacy core conceptsskills they include in
their interventions and should more carefully ad-
dress who delivered the intervention with what fidel-
ity in what setting for how long and utilizing what
pedagogical approach Although it was not published
at the time this review was conducted the Alliance
for a Media Literate Americarsquos (AMLA) Core Prin-ciples of Media Literacy Education [64] served as an
implicit foundation for developing our inclusion cri-
teria and determining what variables to explore since
the lead author of this review was one of 10 authors
of the Core Principles published in April 2007 TheAMLArsquos Core Principles document now provides
a significant foundation for developing media liter-
acy education research frameworks in the future
Overall more research evaluating the effective-
ness of media literacy education to address a variety
of relevant public health concerns is greatly needed
The small size of the sample that met the criteria for
inclusion in this review is indicative of the lack of
peer-reviewed research on health-promoting media
literacy education Additional rigorous research
carefully reported using the measures established
in the framework for this review and those addi-
tional measures recommended in the discussion
especially pedagogical approach and more defini-
tive outcome measures will help to improve media
literacy education and advance it as a useful health
promotion strategy
Acknowledgements
Many thanks to Sue Forster-Cox Assistant Profes-
sor New Mexico State University for her help in
preparing the thesis that led to this article Thanks
also to Jennifer Peters Mel amp Enid Zuckerman
College of Public Health University of Arizona
for her help in preparing the tables
References
References marked with an asterisk indicate studies included
in the systematic review
1 Committee on Public Education American Academy of Pe-diatrics Children adolescents and television Pediatrics2001 107 423ndash6
2 Strasburger VC Wilson BJ Children Adolescents and theMedia Thousand Oaks CA Sage Publications 2002
3 Kaiser Family Foundation Generation M Media in theLives of 8ndash18 Year-OldsAvailable at httpwwwkfforgentmediauploadGeneration-M-Media-in-the-Lives-of-8-18-Year-olds-Reportpdf Accessed 06 January 2006
4 Heins M Cho C Media Literacy An Alternative to Censor-ship 2nd edn New York NY Free Expression PolicyProject 2002
5 Thoman E Jolls T MediaLit KitmdashLiteracy for the 21st Cen-tury An Overview and Orientation Guide to Media LiteracyEducation Available at httpmedialitorgmedialitkithtmlAccessed 06 February 2006
6 Dennis E Out of sight and out of mind the media literacyneeds of grown-ups Am Behav Sci 2004 48 202ndash11
7 Center for Media Studies Conference Report Setting Re-search Directions for Media Literacy and Health EducationAvailable at httpwwwmediastudiesrutgersedumh_con-ferenceconf7012pdf Accessed 06 January 2006
8 Centre for Reviews and Dissemination Undertaking System-atic Reviews of Research on Effectiveness CDRrsquos Guidancefor Those Carrying Out or Commissioning Reviews CDRReport 4 2nd edn Available at httpwwwyorkacukinstcrdpdfcrd4_ph0pdf Accessed 15 February 2007
9 Lipsey MW Wilson DB Practical Meta-Analysis Thou-sand Oaks CA Sage Publications 2001
10 Kaiser Family Foundation Key Facts Media LiteracyAvailable at httpwwwkfforgentmediauploadKey-Facts-Medai-Literacypdf Accessed 06 January 2006
11 Yates BL Media literacy a health education perspectiveJ Health Educ 1999 30 180ndash4
12 Brown JA Television lsquolsquoCritical Viewing Skillsrsquorsquo EducationMajor Media Literacy Projects in the United States andSelected Countries Hillsdale NJ Lawrence Erlbaum Asso-ciates 1991
13 Pinkleton BP Austin EW Cohen M et al State-wide evalu-ation of the effectiveness of media literacy Training to preventtobacco use among adolescents Health Commun 2007 2123ndash34
14 Bergsma LJ Ingram M Blowing Smoke Project EvaluationFinal Report Available at httpwwwblowingsmokearizonaedufinalresults2pdf Accessed 06 February 2006
15 Graham JW Hernandez R A Pilot Test of the AdSmartsCurriculum A Report to the Scott Newman Center LosAngeles CA Scott Newman Center 1993
16 Hobbs R Girls and Young Womenrsquos Understanding of Di-etary Supplement Advertising Assessing Critical AnalysisSkills Report prepared for the Office on Womenrsquos healthDepartment of Health and Human Services Available athttpwwwreneehobbsorgreneersquos20web20sitePubli-cationssample_of_recent_publicationshtm Accessed 06February 2006
17 Levine MP Piran N Stoddard C Mission more probablemedia literacy activism and advocacy as primary preven-tion In Piran N Levine M Steiner-Adair C (eds) Prevent-ing Eating Disorders A Handbook of Interventions andSpecial Challenges Philadelphia PA Brunner-Routledge1999 3ndash25
L J Bergsma and M E Carney
540
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
18 Hill SC Lindsay GB Using health infomercials to developmedia literacy skills J Sch Health 2003 73 239ndash41
19 Rich M Health literacy via media literacy Am Behav Sci2004 48 165ndash88
20 Moore J DeChillo N Nicholson B et al Flashpoint aninnovative media literacy intervention for high-risk adoles-cents Juv Fam Court J 2000 23ndash34
21 Nathanson AI Yang M The effects of mediation contentand form on childrenrsquos responses to violent television HumCommun Res 2003 29 111ndash34
22 Bergsma LJ Empowerment education the link betweenmedia literacy and health promotion Am Behav Sci 200448 152ndash64
23 Hindin TJ Development and evaluation of a nutrition ed-ucation intervention on Head Start parentsrsquo ability to medi-ate the impact of TV food advertising to their children DissAbstr Int 2001 62 2264
24 Kline S Countering childrenrsquos sedentary lifestyles an eval-uative study of a media-risk education approach Childhood2005 12 239ndash58
25 Posavac HD Posavac SS Weigel RG Reducing the impactof media images on women at risk for body image distur-bance three targeted interventions J Soc Clin Psychol 200120 324ndash40
26 Slater MD Rouner D Murphy K et al Adolescent counter-arguing of TV beer advertisements evidence for effective-ness of alcohol education and critical viewing discussionsJ Drug Educ 1996 26 143ndash58
27 Levine MP Smolak L Schermer F Media analysis and re-sistance by elementary school children in the primary pre-vention of eating problems Eat Disord 1996 4 310ndash22
28 McVey G Davis R A program to promote positive bodyimage a 1-year follow-up evaluation J Early Adolesc 200222 96ndash108
29 Clocksin BD Integrated health and physical education pro-gram to reduce media use and increase physical activity inyouth Diss Abstr Int 2005 66 126
30 Nathanson AI Mediation of childrenrsquos television viewingworking towards conceptual clarity and common under-standing In Gudykunst WB (ed) Communication Year-book 25 Mahwah NJ Lawrence Erlbaum Associates2001 115ndash51
31 Robinson TN Wilde ML Navracruz LC et al Effects ofreducing childrenrsquos television and video game use on aggres-sive behavior Arch Pediatr Adolesc Med 2001 155 17ndash23
32 Rosenkoetter LI Rosenkoetter SE Ozretich RA et al Miti-gating the harmful effects of violent television J Appl DevPsychol 2004 25 25ndash47
33 Piran N LevineMP Irving LM GOGIRLSMedia literacyactivism and advocacy project Healthy Weight J 2000 1489
34 Goldberg ME Niedermeier KE Bechtel LJ et al Heighten-ing adolescent vigilance toward alcohol advertising to fore-stall alcohol use J Public Policy Mark 2006 25 147ndash59
35 Yamamiya Y Cash TF Melnyk SE et al Womenrsquos expo-sure to thin-and-beautiful media images body image effectsof media-ideal internalization and impact-reduction inter-ventions Body Image 2005 2 74ndash80
36 Fuller HA Damico AM Rodgers S Impact of a health andmedia literacy curriculum on 4th-grade girls a qualitativestudy J Res Child Educ 2004 19 66ndash78
37 Wilksch SM Tiggemann M Wade TD Impact of interac-tive school-based media literacy lessons for reducing inter-nalization of media ideals in young adolescent girls andboys Int J Eat Disord 2006 39 385ndash93
38 Austin EW Johnson KK Direct and indirect effects ofmedia literacy training of third gradersrsquo decision-makingfor alcohol Proc Int Commun Assoc 1995 1ndash40
39 Austin EW Johnson KK Immediate and delayed effects ofmedia literacy training on third gradersrsquo decision making foralcohol Health Commun 1997 9 323ndash49
40 Austin EW Johnson KK Effects of general and alcohol-specific media literacy training on childrenrsquos decision mak-ing about alcohol J Health Commun 1997 2 17ndash42
41 Austin EW Pinkleton BE Funabiki R The desirabilityparadox in the effects of media literacy training Proc IntCommun Assoc 2005 1ndash46
42 Austin EW Pinkleton BE Hust SJT et al Evaluation of anAmerican legacy foundationWashington state departmentof health media literacy pilot study Health Commun 200518 75ndash95
43 Byrne S Effective and lasting media literacy interventionsProc Int Commun Assoc 2005 1ndash35
44 Coughlin JW Kalodner C Media literacy as a preventionintervention for college women at low- or high-risk for eat-ing disorders Body Image 2006 3 35ndash43
45 Evans AE Dave J Tanner A et al Changing the homenutrition environment effects of a nutrition and media liter-acy pilot intervention Fam Community Health 2006 2943ndash54
46 Gonzales R Glik D Davoudi M et al Media literacy andpublic health integrating theory research and practice fortobacco control Am Behav Sci 2004 48 189ndash201
47 Hindin TJ Contento IR Gussow JD A media literacynutrition education curriculum for Head Start parents aboutthe effects of television advertising on their childrenrsquos foodrequests J Am Diet Assoc 2004 104 192ndash8
48 Irving LM Berel SR Comparison of media-literacy pro-grams to strengthen college womenrsquos resistance to mediaimages Psychol Women Q 2001 25 103ndash11
49 Irving LM DuPen J Berel S A media literacy program forhigh school females Eat Disord 1998 6 119ndash31
50 Kusel AB Primary prevention of eating disorders throughmedia literacy training of girls Diss Abst Int 1999 60 1859
51 Neumark-Sztainer D Sherwood NE Coller T et al Pri-mary prevention of disordered eating among preadolescentgirls feasibility and short-term effect of a community-basedintervention J Am Diet Assoc 2000 100 1466ndash73
52 Rabak-Wagener J Eickhoff-Shemek J Kelly-Vance LThe effect of media analysis on attitudes and behaviors re-garding body image among college students J Am CollHealth 1998 47 29ndash35
53 Scharrer E lsquolsquoI noticed more violencersquorsquo The effects of a me-dia literacy program on critical attitudes toward media vio-lence J Mass Media Ethics 2006 21 69ndash86
54 Sprafkin J Watkins LT Gadow KD Efficacy of a televisionliteracy curriculum for emotionally disturbed and learningdisabled children J Appl Psychol 1990 11 225ndash44
55 Vooijs MW Van Der Voort THA Learning about televi-sion violence the impact of a critical viewing curriculum onchildrenrsquos attitudinal judgments of crime series J Res DevEduc 1993 26 133ndash42
Effectiveness of health-promoting media literacy
541
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from
56 Vooijs MW Van Der Voort THA Teaching children toevaluate television violence critically the impact of a Dutchschools television project J Educ Telev 1993 19 139ndash52
57 Wade TD Davidson S OrsquoDea JA Enjoyment and per-ceived value of two school-based interventions designed toreduce risk factors for eating disorders in adolescents Aust EJ Adv Mental Health 2002 1 Available at httpwwwau-seinetflinderseduaujournalvol1iss2wadepdf Accessed1 February 2006
58 Wade TD Davidson S OrsquoDea JA A preliminary con-trolled evaluation of a school-based media literacy programand self-esteem program for reducing eating disorder riskfactors Int J Eat Disord 2003 33 371ndash83
59 Wolf-Bloom M Using media literacy training to preventbody dissatisfaction and subsequent eating problems in earlyadolescent girls Diss Abstr Int 1998 59 4515
60 Landis RJ Koch GG The measurement of observer agree-ment for categorical data Biometrics 1977 33 159ndash74
61 Thoman E Jolls T Media literacy education lessons fromthe Center for Media Literacy In Schwarz G Brown PU(ed) Media Literacy Transforming Curriculum andTeachingmdash104th Yearbook of the National Society forthe Study of Education Malden MA Blackwell Publish-ing 2005 180ndash205
62 Teen Futures Media Network Teen Aware Sex Media andYouAvailable at httpdeptswashingtonedutawareview-cgisection Accessed 06 April 2006
63 Worsnop C Screening Images Mississauga OntarioWright Communications 1998
64 Alliance for a Media Literate America Core Principlesof Media Literacy Education Available at httpwwwamlainfoorguploadscE7AcE7AyUoKFH3zm-aNaVy7P-QAMLA-Core-MLE-Princ-6-07-Final-Revpdf Accessed07 May 2007
Received on October 23 2006 accepted on October 16 2007
L J Bergsma and M E Carney
542
by guest on June 15 2015httpheroxfordjournalsorg
Dow
nloaded from