Health Education & Behavior38(3) 251 –260© 2011 by SOPHEReprints and permission: sagepub.com/journalsPermissions.navDOI: 10.1177/1090198110372333http://heb.sagepub.com

Salud America! Developinga National Latino Childhood Obesity Research Agenda

Amelie G. Ramirez, DrPH, MPH1, Patricia Chalela, DrPH, MPH1, Kipling J. Gallion, MA1, Lawrence W. Green, DrPH2, and Judith Ottoson, EdD, MPH3

Abstract

U.S. childhood obesity has reached epidemic proportions, with one third of children overweight or obese. Latino children have some of the highest obesity rates, a concern because they are part of the youngest and fastest-growing U.S. minority group. Unfortunately, scarce research data on Latinos hinders the development and implementation of evidence-based, culturally appropriate childhood obesity interventions. In response, the Salud America! network conducted a national Delphi survey among researchers and stakeholders to identify research priorities to address Latino childhood obesity and compare differences by occupation and race or ethnicity. The resulting first-ever National Latino Childhood Obesity Research Agenda provides a framework to stimulate research and collaboration among investigators, providers, and communities, and inform policy makers about the epidemic’s seriousness and specific needs for priority funding. The agenda ranks family as the main ecological level to prevent Latino childhood obesity—followed by community, school, society, and individual—and ranks top research priorities in each level.

Keywords

childhood obesity, Latinos, research agenda, Delphi survey, national, obesity prevention

Background

Obesity rates have increased for all age groups over the past 30 years (Anderson & Butcher, 2006; Institute of Medicine [IOM], 2005, 2007). However, although obesity rates have doubled for adults, they have nearly tripled for children (Anderson, Butcher, & Levine, 2003; IOM, 2005, 2007; Ogden et al., 2006; Ogden, Yanovski, Carroll, & Flegal, 2007) and have increased almost fivefold among those aged 6 to 11 (Centers for Disease Control and Prevention [CDC], 2005; Ogden et al., 2006; Ogden et al., 2007; Robert Wood Johnson Foundation [RWJF], 2009). It is estimated that almost one third of U.S. children are either overweight or obese (IOM, 2005, 2007; Ludwig, 2007; Ogden et al., 2007). By 2010, estimates predict that almost 50% of U.S. children will be overweight (Wang & Lobstein, 2006).

This trend is alarming for medicine and public health given the documented link between being an overweight child and becoming an obese adult (Anderson et al., 2003; Must et al., 2005) and the chronic diseases associated with obesity. The probabilities of a child’s obesity persisting into adulthood increase from an estimated 20% at age 4 to 80% in adolescence (Guo & Chumlea, 1999).

Latino children, who belong to the largest, youngest, and fastest-growing U.S. minority group, have one of the highest rates of obesity. Data from the National Health and Nutrition Examination Survey (NHANES) show that 38% of Mexican American children are obese or overweight, compared with 30.7% of non-Hispanic Whites and 34.9% of African American children (Ogden, Carrol, & Flegal, 2008). Latino children as early as age 4 have obesity rates that have exceeded those of their White and African American counterparts (Mei et al., 1998; Ogden et al., 2006), and Mexican American children ages 6 to 11 are at the highest risk for obesity compared to other racial/ethnic groups (Ogden et al., 2006; Ogden et al., 2008).

1University of Texas Health Science Center at San Antonio, TX, USA2University of California at San Francisco, CA, USA3Independent Evaluation Consultant, San Francisco, CA, USA

Corresponding Author:Amelie G. Ramirez, Institute for Health Promotion Research, The University of Texas Health Science Center at San Antonio, 8207 Callaghan Road, Suite 353, San Antonio, TX 78230; phone: (210) 348-0255E-mail: ramirezag@uthscsa.edu

at UTHSC AT SAN ANTONIO on May 24, 2011heb.sagepub.comDownloaded from

252 Health Education & Behavior 38(3)

These figures are of great importance, given that 34% of the Latino population is composed of children younger than 18, compared with only 25% of the total population and 21% of the non-Hispanic White population (U.S. Census Bureau, 2008). The limited data on Latino children led the 2002 Latino Consortium of the American Academy of Pediatrics’ Center for Child Health Research to call for more research on Latino child health issues, greater inclusion of Latino children in medical research, and analysis of study data by pertinent Latino subgroups (Flores et al., 2002).

The problem of childhood obesity is complex and multifac-eted, with the interplay of genetic, physiologic, behavioral, and environmental factors. Given the rapid rise in childhood obesity during the past three decades within a genetically stable popula-tion, there is strong evidence that genetic factors are not the pri-mary reason for this epidemic. Thus, there is growing emphasis placed on environmental factors (Lobstein, Baur, & Uauy, 2004) and the critical role of the environment as a predisposing factor for many of the more proximal behavioral determinants (Flynn et al., 2006). Social ecological models, originated in Bronfen-brenner’s work (1986, 1989, 2005), have been used to study the complex interaction affecting individual lifestyle behaviors like obesity (Booth et al., 2001; Davison & Birch, 2001; Sallis & Owen, 1997; Story, Kaphingst, Robinson-O’Brien, & Glanz, 2008). According to ecological systems theory, development or changes in individual characteristics cannot be effectively explained without considering the immediate context (or ecologi-cal niche, in which the person is embedded) and broader interact-ing contexts. In the case of childhood obesity, for example, the immediate ecological niche includes family (parents, siblings, etc.) and school (teachers, peers, etc.), which are in turn embed-ded in larger social contexts, including the community and society in general (Davison & Birch, 2001).

Two Institute of Medicine (IOM) committees on preventing childhood obesity issued recent reports that recognize the pre-vention of childhood obesity as a national priority and under-score the need to understand the complex interactions across social, environmental, and policy contexts that have created an adverse environment for maintaining a healthy weight. The committees also highlight the importance of a population-based approach involving multiple sectors and stakeholders to provide a nurturing environment where children can grow up physically and emotionally healthy (IOM, 2005, 2007).

Unfortunately, the scarce data available on Latinos makes it difficult to develop and implement evidence-based, culturally appropriate interventions that are effective in preventing and addressing childhood obesity among this increasingly important population. What is known, however, is that interventions that address multiple levels of an ecologic framework are most effec-tive because they address contributing factors across those many levels, and there are reciprocal causal relationships among them. Hence, intervention efforts targeted at reducing rates of obesity among Latino children must necessarily address children’s pre-vailing environments, including the society, community, school,

family, and the individual, and must, by definition, transcend traditional boundaries of intervention if definitive progress is to be made in reversing, or even just slowing, the current trends. Interventions also must be responsive and culturally sensitive to the communities at risk, and ideally tailored down to the family and individual levels of intervention.

In response to this urgent need, Salud America! The RWJF Research Network to Prevent Obesity among Latino Children, was created. Directed by the Institute for Health Promotion Research (IHPR) at the University of Texas Health Science Center at San Antonio, and with support from the RWJF, Salud America! aims to reduce and prevent Latino childhood obesity by uniting and increasing the number of Latino researchers engaged in intervention-related research on Latino childhood obesity and seeking environmental and policy-relevant solu-tions to the epidemic. In addition, the 5-year program aims to advance the RWJF goal to reverse the childhood obesity epi-demic by 2015 by building the evidence base for childhood obesity prevention strategies among Latinos.

Building from a social ecological model approach, Salud America! conducted a national Delphi survey among research-ers, policy makers, community leaders, and other Latino child-hood obesity stakeholders to identify and assess the top priorities for research aimed at preventing obesity among Latino children. This article reports on the results of the Delphi survey and the research priorities that guided the establishment of the first-ever National Latino Childhood Obesity Research Agenda in the United States. It also presents a general com-parison of the researchers’ perception of priorities with those of other stakeholders, who are considered to be the potential end users of the research, and by race or ethnicity.

MethodA modified three-round Delphi method was selected for this study as a practical approach to collect opinions and achieve consensus on the top research priorities to prevent or reverse childhood obesity among Latinos. The Delphi method, origi-nally developed by the RAND Corporation in the 1950s, is a widely used technique for consensus building that uses specific rounds of questionnaires to collect data from a panel of selected participants (Dalkey, 1969; Linstone & Turoff, 1975; Turoff & Hiltz, 1996; Young & Jamieson, 2001). The technique has several advantages: It allows the inclusion of a large number of participants with diverse backgrounds and geographic loca-tions and at the same time provides convenience, anonymity, a controlled feedback process, and a variety of statistical analy-sis techniques to interpret and summarize the collected data. It also removes the shortcomings of face-to-face group discus-sions, such as the influence of dominant individuals, manipula-tion, or group pressure for conformity (Dalkey, 1972; Delbecq, Van de Ven, & Gustafson, 1975; Hsu & Sandford, 2007).

The research team used a Web-based Delphi to facilitate not only the participation of a panel of experts from a variety

at UTHSC AT SAN ANTONIO on May 24, 2011heb.sagepub.comDownloaded from

Ramirez et al. 253

of locations across the nation but also the storage and process-ing of data, maintenance of participants’ anonymity, and provi-sion of rapid feedback (Bowles, Holmes, Naylor, Liberatore, & Nydick, 2003; Witkin & Altschuld, 1995).

ParticipantsPotential survey participants were identified using convenience and snowball sampling methods that included: members of the Salud America! network; recommendations from the Salud America! National Advisory Committee (NAC); and individuals who already had expressed interest in participating. In addition, various program communication materials, such as a website, e-newsletters, and e-alerts also were used to invite potential participants.

Participation was open to existing and new Salud America! network members. Even if they joined the network after the survey had begun and they had not participated in any prior rounds, they were allowed to participate in an open round because the network continued to recruit members with per-tinent expertise. This allowed the replacement of participant dropouts and kept the number of participants in each round above 100. It also facilitated the participation of individuals representing diverse viewpoints and increased the decision quality (or reduction in group error) and reliability of results (collective group opinion) with a large number of participants (Ali, 2005; Murphy et al., 1998; Powell, 2003; Skulmoski, Hartman, & Krahn, 2007).

ProcessThe three-round Delphi survey was conducted over 3 months between May 1 and July 30, 2008. Participants were given 2 weeks to respond to each round. Personalized reminders were e-mailed by the beginning of the second week to encour-age participation. A third week was used to analyze data and draft the next round’s questionnaire. On the completion of the third and final round, four additional weeks were used for final analysis and reporting of results.

All potential participants were sent an e-mail invitation letter encouraging their participation and giving details of the purpose and process of the Delphi survey, use of information, and assurance of anonymity. The invitation also included a specific URL link to the website and survey, where further instructions were provided.

Building from a social ecologic model approach, the Delphi questionnaire was designed by the Salud America! research team and reviewed by the program’s National Advisory Com-mittee, which features nearly two dozen national researchers and leaders in the field of health and obesity. Next, the ques-tionnaire was transferred onto a website designed with support from the IT group of the Department of Epidemiology and Biostatistics at the University of Texas Health Science Center at San Antonio. The questionnaire and website were then

pilot-tested with 10 people, resulting in minor revisions in wording to clarify ambiguities and suggestions to improve site navigation.

The website featured a log-in function, password protection, and step-by-step instructions for completing each survey round. Electronic questionnaires for the Delphi website were designed in a user-friendly manner, with simple check options to rate issues and avoid participant fatigue. To facilitate navigation, each of five main areas of research was contained on a single page that ended with an open comment box so participants could add any comments after each main area. A general com-ment box at the end of the survey allowed participants to address Web-site functionality and general survey opinions.

An extensive review of the literature by Salud America! identified five main research areas associated with childhood obesity intervention research: society, community, school, fam-ily, and individual. These also generally correspond to the eco-logical levels of potential intervention. The Salud America! National Advisory Committee reviewed these five main research areas and suggested 45 initial specific research priorities to be assessed during the Delphi’s first round, distributed among the five levels. After the initial questionnaire, the second- and third-round questionnaire provided additional information—feedback on the group and individual ratings’ central tendency and dis-persion scores (means and standard deviations) from previous rounds. Ultimately, the aim of Round 3 was to rank the top five specific research priorities identified by participants for each of the five main research areas to establish the National Latino Childhood Obesity Research Agenda and to anticipate issues in translating the research into policy and practice.

Data AnalysisData refinement was conducted in preparation for analysis, and variables were recorded as needed. All statistical analyses were performed using SPSS for Windows 16.0 (SPSS Inc., 2008).

Descriptive statistical analysis included frequencies, cross-tabulations, and measures of central tendency and dispersion. For each round, means and standard deviations for all specific research priorities were reviewed to rank them in order of impor-tance. The standard deviation size was used to break ties in mean ranks and to identify areas in which consensus within and across stakeholder groups appears potentially problematic. To determine the most agreed-on specific research priorities that would advance to the next survey round to be rated again, the study used researcher-defined criteria that included (a) a group mean score >3.5 and (b) a standard deviation ≤1, indicating strong consensus for inclusion among respondents.

In addition to basic descriptive analysis, independent sam-ples t-test and one-way analysis of variance (ANOVA) were conducted to assess mean differences by occupation (research-ers and potential end-user stakeholders) and race or ethnicity. To determine the relevance of a statistically significant result, the effect size or the magnitude of the difference in means scores

at UTHSC AT SAN ANTONIO on May 24, 2011heb.sagepub.comDownloaded from

254 Health Education & Behavior 38(3)

between groups was assessed by using eta squared (η2; Agresti & Finlay, 1997; Cohen, 1988; Rosner, 1995).

Also, given that Round 1 respondents were encouraged to use the comment section of the questionnaire to suggest items to the list of specific priorities to be considered by respondents in Round 2, content analysis was used to analyze suggested research priorities. After reading through all the suggestions, the research team, through discussion and agreement, identified common statements and developed related specific potential research priorities to add to the Round 2 questionnaire.

ResultsAs shown on Table 1, a total of 313 people participated in the Delphi survey. Participants were primarily females (80.2%) and predominantly Hispanic/Latino (49.2%), followed by Whites (41.5%), African Americans (2.9%), Asians/Pacific Islanders (2.2%), and other ethnicities (3.8%). Participants had diverse areas of expertise, institutional backgrounds, and geographical locations. Most participants were academicians or researchers (about 33%), followed by health educators or educators (15.3%), administrators or managers and clinicians

(11.8%), and public health workers (11.2%). Delphi respon-dents were located in 31 U.S. states—mostly Texas, California, Illinois, Maryland, and New York.

Round 1Of the 579 invitations sent for the first round, 177 individuals responded (30.6%), of whom 165 had complete data for inclu-sion in the analysis. Round 1 used a structured questionnaire with 45 specific research priorities categorized under five main research areas (individual, family, school, community, and society). Using a 5-point Likert-type scale (ranging from 1 = not important at all to 5 = extremely important), partici-pants were asked to rate the importance of each research prior-ity in reversing the Latino childhood obesity epidemic. All specific research priorities for each research area met the inclu-sion criteria and all but one received rating scores between 4 (very important) and 5 (extremely important). Dispersion was very small, with standard deviations equal or less than 1, suggesting high participant agreement. In addition, participants were given an open comment box to add any specific research priority that they considered was missing in each of the five main research areas, which would then be assessed by Round 2 participants. After content analysis, 22 additional research priorities were included in the Round 2 questionnaire.

Round 2For Round 2, a total of 103 people completed the survey, of whom 57 had completed Round 1 (55.3%) and 46 were new participants (44.7%) who had recently joined the Salud America! network. The Round 2 questionnaire included overall group ratings from the first round, plus the participant’s own initial response. Using the same 5-point Likert-type scale, participants were asked to reconsider their first response and re-rate the issues based on the overall first-round results.

In addition to the 45 original specific research priorities, par-ticipants were asked to rate the 22 additional priorities suggested by respondents in the first round, for a total of 67 specific research priorities, which could have affected the response rate. An open box for comments at the end of the survey allowed participants to include any clarification or comments related to the survey. After analyzing the Round 2 results, the consistent rating and low standard deviations of research priorities in each research area showed that consensus had been approached by participants as to the top research priorities. Given that all the research issues included for assessment met the researcher-established criteria for selection (X ≥ 3.5 and SD ≤ 1), the top five research priori-ties receiving the highest mean scores and the smallest stan-dard deviations were included for a final ranking in Round 3.

Round 3

For the third and final round, 194 people completed surveys, of which 93 completed a survey in Round 1 and 2 (47.9%)

Table 1. Delphi Participants’ Demographic Characteristics

Total

Demographics n %

Gender Female 251 80.2Male 62 19.8Total 313 100.0

Ethnicity Hispanic/Latino 154 49.2White 130 41.5African American 9 2.9Asian or Pacific Islander 7 2.2Other 12 3.8No response 1 0.3Total 313 100.0

Occupation Academician or researcher 105 33.5Health educator or educator 48 15.3Administrator or manager 37 11.8Clinician 37 11.8Public health 35 11.2Student 16 5.1Social services 9 2.9Health policy 6 1.9Research associate, assistant, 5 1.6 or coordinator

Lobbyist or advocate 4 1.3Administration assistant or secretarial 3 1.0Community 2 0.6Consultant 2 0.6Government employee 2 0.6Legal 2 0.6Total 313 100.0

at UTHSC AT SAN ANTONIO on May 24, 2011heb.sagepub.comDownloaded from

Ramirez et al. 255

and 101 were new recruits (52.1%) from the Salud America! network.

Participants in this final round were asked to review each research priority and rank it using a 1 to 5 scale (with 1 rep-resenting the first most important priority and 5 representing the fifth most important priority, a reversal of importance from the other rounds). Participants also were asked to rank the five main research areas—society, community, school, family, and individual—based on their perception of their importance in reversing the obesity epidemic among Latino children. Although simple rankings usually produce ordinal scales that are not suitable for statistical treatments that assume interval scales, the wording of the ranking instructions in this case produced a Likert-type ordinal scale.

The first column of Table 2 shows the final rankings with the specific research priorities selected by our national group of Delphi participants, which constitutes the first National Latino Childhood Obesity Research Agenda. The top society-related research priority selected by Delphi participants was “policies that subsidize accessibility of healthy foods to improve diet among Latino families”; the top community-related research priority was “built-environment policies involving collaborations with multiple stakeholders to promote physical activity”; the top school-related research priority was “health, nutrition, and active physical education classes as part of the school curriculum”; the top family-related research priority was “engaging Latino families as advocates of child-hood obesity prevention initiatives at the community and school levels”; and the top individual-related research priority was “programs making physical activity more attractive than watching TV or playing video games.”

The authors also were interested in comparing the rankings between researchers and the potential end-user stakeholders to determine any possible difference in which consensus may appear potentially problematic. Table 2 also shows that signifi-cant differences were found in only 3 of the 25 research priori-ties within the society, community, or school research areas.

In the society research area, researchers ranked “Policies that provide health care access for screening and treatment of childhood obesity” fifth place (X = 3.62 SD = 1.33), whereas poten-tial end user stakeholders ranked it in fourth place (X = 3.18, SD = 1.50, t = 2.00, p = .05). The magnitude of the differences in the means was small (η2 = .02). In the community research area, researchers considered “Policies that limit the sale of unhealthy foods and drinks in public institutions” a top research priority, ranking it in first place, compared to a fourth-place ranking by potential end user stakeholders (X = 2.70, SD = 1.48, vs. X = 3.25, SD = 1.55, t = 2.34, p = .02). How-ever, the effect size was small (η2 = .03). Finally, at the school level, researchers ranked “Health, nutrition, and active physical education classes as part of the school curriculum” in third place whereas potential end user stakeholders considered it the top research priority (X = 3.11, SD = 1.40, vs. X = 2.64, SD = 1.40, t = 2.19, p = .03). The magnitude of the difference between groups also was small (η2 = .02).

Table 3 shows the comparison in ranking scores by race or ethnicity. A one-way ANOVA found statistically significant differences at the p ≤.05 level by race or ethnicity for one society-related research priority: “Policies to improve nutrition and physical activity education in the media and in community settings” (F = 6.71, p < .01). Post hoc comparison using the Bonferroni correction test indicated that Hispanics (X = 2.95, SD = 1.37) differ significantly from Whites (X = 3.56, SD = 1.15, p < .01) and participants in the Other category (X = 3.81, SD = 1.11, p < .04). Hispanic ranked this priority in second place, whereas Whites and others ranked it in fifth place. The actual difference in mean rankings between the groups was moderate (η2 = .07).

Even though a significant difference also was found for the school-related research priority “Policies that determine stan-dards for nutrition and physical activity,” a multiple comparison analysis using the Bonferroni correction test did not show any significant difference between racial and ethnic groups.

In the final round, participants also were asked to rank the five main research areas. Table 4 shows that “family” was ranked as the most important research area to prevent and/or reverse the obesity epidemic among Latino children, followed by community, school, society, and individual. There were no significant differences in mean ranking scores between researchers and potential end user stakeholders or between racial and ethnic groups.

DiscussionThe presented Delphi survey was part of a strategy by Salud America! The RWJF Research Network to Prevent Obesity among Latino Children to identify key specific research pri-orities regarding childhood obesity and establish the first-ever National Latino Childhood Obesity Research Agenda. The 25 top specific research priorities identified by the national panel of participants represent areas in critical need of knowledge, and constitute specific foci for research aimed at preventing or reversing the obesity epidemic and its consequences among Latino children. Results from studies that address these priori-ties will represent an important step in the advancement of scientific knowledge on Latino childhood obesity and its con-tributing factors and identify the most effective interventions to address this national public health problem that is in urgent need of solutions. For example, this national research agenda was the basis for a recent Salud America! call for proposals, guiding the submission and selection among 89 applications of 20 pilot research projects aimed at preventing or reversing obesity and its consequences among Latino children.

Few significant differences were found between researchers and potential end-user stakeholders or by race or ethnicity, suggesting an overall agreement on ranking of top research priorities. The small differences found in 4 of the 25 research priorities may help investigators recognize diverse perceptions among specific research issues that may warrant further study and may be important when fostering collaboration among

at UTHSC AT SAN ANTONIO on May 24, 2011heb.sagepub.comDownloaded from

256

Tab

le 2

. Top

Res

earc

h Pr

iori

ties

Iden

tifie

d by

Del

phi P

artic

ipan

ts b

y R

anki

ng O

rder

and

by

Occ

upat

ion

Res

earc

h Pr

iori

ty

Gen

eral

Gro

up

Ran

king

Aca

dem

icia

ns

and

Res

earc

hers

Pote

ntia

l En

d U

ser

Stak

ehol

ders

RO

MSD

RO

MSD

RO

MSD

pη2

Soci

ety

Polic

ies

that

sub

sidi

ze a

cces

sibi

lity

of h

ealth

y fo

ods

to im

prov

e di

et a

mon

g La

tino

fam

ilies

1

2.53

1.39

12.

561.

391

2.51

1.40

.813

Prog

ram

s to

influ

ence

sta

te a

nd lo

cal l

egis

latio

n at

diff

eren

t le

vels

reg

ardi

ng p

hysi

cal a

ctiv

ity a

nd h

ealth

y f

oods

ava

ilabl

e to

chi

ldre

n 2

2.81

1.47

22.

671.

522

2.88

1.45

.353

Polic

ies

that

mak

e pl

aygr

ound

s, s

choo

ls, p

arks

, and

rec

reat

iona

l fac

ilitie

s av

aila

ble

for

phys

ical

act

ivity

for

Lat

ino

child

ren

and

fam

ilies

dur

ing

non–

scho

ol d

ays

33.

051.

293

2.86

1.22

33.

141.

33.1

55

Polic

ies

to im

prov

e nu

triti

on a

nd p

hysi

cal a

ctiv

ity e

duca

tion

in t

he m

edia

and

in c

omm

unity

set

tings

4

3.28

1.30

43.

281.

385

3.28

1.28

.991

Polic

ies

that

pro

vide

hea

lth c

are

acce

ss fo

r sc

reen

ing

and

trea

tmen

t of

chi

ldho

od o

besi

ty5

3.33

1.46

53.

621.

334

3.18

1.50

.047

.02

Com

mun

ityBu

ilt-e

nvir

onm

ent

polic

ies

invo

lvin

g co

llabo

ratio

ns w

ith m

ultip

le s

take

hold

ers

to p

rom

ote

phys

ical

act

ivity

12.

731.

332

2.75

1.33

12.

721.

34.8

79C

olla

bora

tions

am

ong

com

mun

ity, s

choo

ls, a

nd fa

mili

es t

o ge

nera

te a

fter

-sch

ool o

ppor

tuni

ties

for

child

ren

to

be m

ore

phys

ical

ly a

ctiv

e2

2.83

1.33

32.

921.

332

2.78

1.33

.491

Com

mun

ity p

artic

ipat

ory

proc

esse

s th

at e

ncou

rage

Lat

ino

fam

ilies

to d

iscus

s th

eir

need

s an

d de

fine

actio

n st

rate

gies

33.

041.

534

3.20

1.58

32.

961.

51.3

03Po

licie

s th

at li

mit

the

sale

of u

nhea

lthy

food

s an

d dr

inks

in p

ublic

inst

itutio

ns4

3.07

1.54

12.

701.

484

3.25

1.55

.020

.03

Nei

ghbo

rhoo

d sa

fety

influ

ence

on

outd

oor

recr

eatio

n an

d ph

ysic

al a

ctiv

ity5

3.33

1.25

53.

421.

265

3.29

1.25

.489

Scho

olH

ealth

, nut

ritio

n, a

nd a

ctiv

e ph

ysic

al e

duca

tion

clas

ses

as p

art

of t

he s

choo

l cur

ricu

lum

12.

801.

413

3.11

1.40

12.

641.

40.0

30.0

2Po

licie

s m

anda

ting

daily

, act

ive

phys

ical

edu

catio

n cl

asse

s fo

r st

uden

ts b

ased

on

age

and

heal

th s

tatu

s2

2.92

1.57

12.

651.

603

3.05

1.54

.094

Brea

kfas

t an

d lu

nch

food

cho

ices

32.

971.

402

2.87

1.40

23.

021.

40.4

86Po

licie

s th

at d

eter

min

e st

anda

rds

for

nutr

ition

and

phy

sica

l act

ivity

43.

111.

294

3.13

1.24

43.

101.

32.8

98C

olla

bora

tions

bet

wee

n pa

rent

s, t

each

ers,

sch

ool a

dmin

istr

ator

s, a

nd c

omm

unity

lead

ers

to in

fluen

ce

leg

isla

tion

rela

ted

to h

ealth

y nu

triti

on a

nd p

hysi

cal a

ctiv

ity5

3.20

1.38

53.

241.

395

3.18

1.38

.784

Fam

ilyEn

gagi

ng L

atin

o fa

mili

es a

s ad

voca

tes

of c

hild

hood

obe

sity

prev

entio

n in

itiat

ives

at t

he c

omm

unity

and

sch

ool l

evel

s1

2.81

1.43

22.

921.

411

2.76

1.45

.468

Com

preh

ensi

ve in

terv

entio

ns t

hat

trea

t th

e fa

mily

as

the

unit

of a

naly

sis

22.

851.

571

2.81

1.66

22.

871.

53.8

09Pa

rent

al k

now

ledg

e, a

ttitu

des,

and

beh

avio

rs (

mod

elin

g) r

elat

ed t

o fo

od c

onsu

mpt

ion

and

food

pre

fere

nces

32.

951.

273

3.02

1.35

32.

921.

24.6

34Fa

mily

acc

ess

to p

hysi

cal a

ctiv

ity o

ppor

tuni

ties

at p

arks

, pla

ygro

unds

, sch

ools

, and

rec

reat

iona

l fac

ilitie

s4

3.17

1.39

43.

101.

364

3.21

1.41

.598

Food

lite

racy

edu

catio

n on

hea

lthy

eatin

g on

a li

mite

d bu

dget

53.

221.

355

3.16

1.30

53.

241.

39.7

00In

divi

dual

Prog

ram

s m

akin

g ph

ysic

al a

ctiv

ity m

ore

attr

activ

e th

an w

atch

ing

TV

or

play

ing

vide

o ga

mes

12.

621.

431

2.52

1.40

22.

671.

45.4

96K

now

ledg

e, a

ttitu

des,

and

ski

lls o

f Lat

ino

child

ren

and

adol

esce

nts

rega

rdin

g nu

triti

on2

2.75

1.36

32.

941.

341

2.65

1.36

.173

Kno

wle

dge,

att

itude

s, a

nd s

kills

of L

atin

o ch

ildre

n an

d ad

oles

cent

s re

gard

ing

phys

ical

act

ivity

32.

841.

232

2.87

1.28

32.

821.

21.7

70T

ime

that

Lat

ino

child

ren

and

adol

esce

nts

spen

d on

sed

enta

ry a

ctiv

ities

suc

h as

TV

, vid

eo g

ames

, and

the

Inte

rnet

43.

321.

394

3.14

1.42

43.

411.

36.2

08Pe

rcep

tions

abo

ut w

alki

ng, b

ikin

g, a

nd u

sing

pub

lic t

rans

it as

pre

ferr

ed t

rans

port

atio

n m

odal

ities

53.

471.

485

3.52

1.48

53.

441.

48.7

28

Not

e: R

O =

ran

king

ord

er. R

esea

rch

prio

ritie

s ar

e ra

nked

with

in e

ach

of fi

ve m

ain

rese

arch

are

as. M

ean

scor

es fo

r im

port

ance

are

from

low

est

to h

ighe

st (

1 be

ing

the

mos

t im

port

ant

prio

rity

and

5

bein

g th

e fif

th m

ost

impo

rtan

t pr

iori

ty).

at UTHSC AT SAN ANTONIO on May 24, 2011heb.sagepub.comDownloaded from

257

Tab

le 3

. Top

Res

earc

h Pr

iori

ties

Iden

tifie

d by

Del

phi P

artic

ipan

ts b

y R

ace

or E

thni

city

and

Ran

king

Ord

er

Res

earc

h Pr

iori

ty

His

pani

csW

hite

sO

ther

RO

MSD

RO

MSD

RO

MSD

pη2

Soci

ety

Polic

ies

that

sub

sidi

ze a

cces

sibi

lity

of h

ealth

y fo

ods

to im

prov

e di

et a

mon

g La

tino

fam

ilies

1

2.68

1.47

12.

271.

283

2.81

1.33

.107

Polic

ies

to im

prov

e nu

triti

on a

nd p

hysi

cal a

ctiv

ity e

duca

tion

in t

he m

edia

and

in c

omm

unity

set

tings

2

2.95

1.37

53.

561.

155

3.81

1.11

.002

.07

Prog

ram

s to

influ

ence

sta

te a

nd lo

cal l

egis

latio

n at

diff

eren

t le

vels

reg

ardi

ng p

hysi

cal a

ctiv

ity a

nd h

ealth

y fo

ods

avai

labl

e to

chi

ldre

n 3

3.02

1.44

22.

711.

501

2.19

1.33

.072

Polic

ies

that

mak

e pl

aygr

ound

s, s

choo

ls, p

arks

, and

rec

reat

iona

l fac

ilitie

s av

aila

ble

for

phys

ical

act

ivity

fo

r La

tino

child

ren

and

fam

ilies

dur

ing

non–

scho

ol d

ays

43.

061.

313

2.95

1.30

43.

441.

21.3

87

Polic

ies

that

pro

vide

hea

lth c

are

acce

ss fo

r sc

reen

ing

and

trea

tmen

t of

chi

ldho

od o

besi

ty

53.

291.

444

3.51

1.43

22.

751.

65.1

49C

omm

unity

Col

labo

ratio

ns a

mon

g co

mm

unity

, sch

ools

, and

fam

ilies

to

gene

rate

aft

er-s

choo

l opp

ortu

nitie

s fo

r ch

ildre

n to

be

mor

e ph

ysic

ally

act

ive

12.

711.

343

3.01

1.34

12.

671.

18.2

97

Built

-env

iron

men

t po

licie

s in

volv

ing

colla

bora

tions

with

mul

tiple

sta

keho

lder

s to

pro

mot

e ph

ysic

al a

ctiv

ity2

2.74

1.39

12.

591.

283

3.20

1.08

.254

Com

mun

ity p

artic

ipat

ory

proc

esse

s th

at e

ncou

rage

Lat

ino

fam

ilies

to

disc

uss

thei

r ne

eds

and

defin

e ac

tion

stra

tegi

es3

2.92

1.45

43.

171.

604

3.20

1.78

.522

Polic

ies

that

lim

it th

e sa

le o

f unh

ealth

y fo

ods

and

drin

ks in

pub

lic in

stitu

tions

43.

201.

532

3.00

1.51

22.

671.

72.3

83N

eigh

borh

ood

safe

ty in

fluen

ce o

n ou

tdoo

r re

crea

tion

and

phys

ical

act

ivity

53.

421.

255

3.23

1.26

53.

271.

28.6

04Sc

hool

Hea

lth, n

utri

tion,

and

act

ive

phys

ical

edu

catio

n cl

asse

s as

par

t of

the

sch

ool c

urri

culu

m1

2.66

1.38

32.

921.

393

2.87

1.64

.471

Polic

ies

man

datin

g da

ily, a

ctiv

e ph

ysic

al e

duca

tion

clas

ses

for

stud

ents

bas

ed o

n ag

e an

d he

alth

sta

tus

22.

871.

602

2.88

1.50

53.

601.

60.2

26Br

eakf

ast

and

lunc

h fo

od c

hoic

es3

2.98

1.42

43.

041.

421

2.60

1.18

.541

Col

labo

ratio

ns b

etw

een

pare

nts,

tea

cher

s, s

choo

l adm

inis

trat

ors,

and

com

mun

ity le

ader

s to

influ

ence

le

gisl

atio

n re

late

d to

hea

lthy

nutr

ition

and

phy

sica

l act

ivity

43.

151.

305

3.27

1.51

43.

201.

27.8

51

Polic

ies

that

det

erm

ine

stan

dard

s fo

r nu

triti

on a

nd p

hysi

cal a

ctiv

ity5

3.34

1.29

12.

881.

242

2.73

1.34

.034

Fam

ily Enga

ging

Lat

ino

fam

ilies

as

advo

cate

s of

chi

ldho

od o

besi

ty p

reve

ntio

n in

itiat

ives

at

the

com

mun

ity a

nd

scho

ol le

vels

12.

811.

472

2.88

1.39

12.

401.

50.4

97

Com

preh

ensi

ve in

terv

entio

ns t

hat

trea

t th

e fa

mily

as

the

unit

of a

naly

sis

22.

841.

533

2.88

1.61

22.

731.

75.9

43Pa

rent

al k

now

ledg

e, a

ttitu

des,

and

beha

vior

s (m

odel

ing)

rel

ated

to

food

con

sum

ptio

n an

d fo

od p

refe

renc

es3

3.07

1.25

12.

831.

323

2.93

1.1

.458

Food

lite

racy

edu

catio

n on

hea

lthy

eatin

g on

a li

mite

d bu

dget

43.

131.

355

3.24

1.34

53.

731.

44.2

77Fa

mily

acc

ess

to p

hysi

cal a

ctiv

ity o

ppor

tuni

ties

at p

arks

, pla

ygro

unds

, sch

ools

, and

rec

reat

iona

l fac

ilitie

s5

3.15

1.45

43.

171.

404

3.20

1.01

.991

Indi

vidu

alPr

ogra

ms

mak

ing

phys

ical

act

ivity

mor

e at

trac

tive

than

wat

chin

g T

V o

r pl

ayin

g vi

deo

gam

es1

2.63

1.38

12.

641.

511

2.40

1.40

.828

Kno

wle

dge,

att

itude

s, a

nd s

kills

of L

atin

o ch

ildre

n an

d ad

oles

cent

s re

gard

ing

nutr

ition

22.

641.

382

2.80

1.33

43.

271.

28.2

30K

now

ledg

e, a

ttitu

des,

and

ski

lls o

f Lat

ino

child

ren

and

adol

esce

nts

rega

rdin

g ph

ysic

al a

ctiv

ity3

2.79

1.20

32.

871.

293

3.00

1.20

.808

Tim

e th

at L

atin

o ch

ildre

n an

d ad

oles

cent

s sp

end

on s

eden

tary

act

iviti

es s

uch

as T

V, v

ideo

gam

es, a

nd

the

Inte

rnet

43.

311.

425

3.39

1.34

22.

931.

44.5

01

Perc

eptio

ns a

bout

wal

king

, bik

ing,

and

usi

ng p

ublic

tra

nsit

as p

refe

rred

tra

nspo

rtat

ion

mod

aliti

es5

3.63

1.43

43.

291.

495

3.40

1.72

.321

Not

e: R

O =

ran

king

ord

er. R

esea

rch

prio

ritie

s ar

e ra

nked

with

in e

ach

of fi

ve m

ain

rese

arch

are

as. M

ean

scor

es fo

r im

port

ance

are

from

low

est

to h

ighe

st (

1 be

ing

the

mos

t im

port

ant

prio

rity

and

5 b

eing

th

e fif

th m

ost

impo

rtan

t pr

iori

ty).

at UTHSC AT SAN ANTONIO on May 24, 2011heb.sagepub.comDownloaded from

258 Health Education & Behavior 38(3)

Table 4. Participants’ Ranking of the Five Main Research Areas

Research Priority RO M SD

Family 1 2.18 1.13Community 2 2.61 1.15School (day care, preschool, 3 3.00 1.17 grade school, high school)

Society 4 3.53 1.57Individual 5 3.68 1.44

Note: RO = ranking order. 1 = first most important; 2 = second most important; 3 = third most important; 4 = fourth most important; and 5 = fifth most important.

different groups to have a unified front to influence policy and practice to reverse the Latino childhood obesity epidemic, or producing research perceived as the most relevant or impor-tant needs by stakeholders who would be the end users of that research.

Some limitations to this study should be noted. First, par-ticipants were not randomly selected, which could affect the generalization of results and make selection bias possible. Second, results are dependent on the composition of the panel. However, rather than being a shortcoming, the researchers believe that the process used was an effective way to obtain broad participation from individuals with different back-grounds and geographic locations across the United States, reflecting a broad range of points of views, interests, and expe-riences. Third, there was significant attrition of participants, especially in Round 2. This may be the result of the length of the survey, the time commitment required, and the busy sched-ules of some participants. However, by allowing participation of new members of the Salud America! network, researchers were able not only to replace participant dropouts but to involve a heterogeneous group of national panelists, and to increase the reliability of results (collective group opinion) by increasing the number of participants (Ali, 2005; Murphy et al., 1998; Powell, 2003; Skulmoski et al., 2007).

The scarce data available on Latinos highlights the impor-tance of this national research agenda and the urgent need for further research to identify the most effective strategies in preventing and addressing childhood obesity among this very important group. To be effective, interventions should address multiple levels of an ecologic framework to address contribut-ing factors across those levels. Hence, intervention efforts targeted at reducing rates of obesity among Latino children must necessarily address children’s prevailing environments, including the society, community, school, family, and the indi-vidual, and must, by definition, transcend traditional bound-aries of intervention if definitive progress is to be made in reversing the current epidemic trends.

This National Latino Childhood Obesity Research Agenda provides a framework for Salud America! and other research entities to present and pursue research and collaborations to

help address this need. Not only is a research agenda essential in empowering investigators, educators, health care providers, and communities to collaborate on childhood obesity preven-tion and control, but it also is crucial in informing and moti-vating policy makers to recognize the seriousness of obesity among Latino children and the need for allocation of funding and resources. Policy makers at the federal, state, and local levels need to take into account the specific health needs of the growing Latino population, particularly Latino children, and ways in which those needs can be met.

Implication for PractitionersThe Salud America! National Latino Childhood Obesity Research Agenda represents a foundation for new and innova-tive research to identify the most effective policies and pro-grams for reversing the obesity epidemic among Latino children. The agenda will guide investigators to build the evidence base for childhood obesity prevention and will help identify the most promising obesity-prevention strategies specifically tailored for Latino communities. Specifically, results from this study will guide the development of a call for proposals to support 20 pilot projects aimed at identifying effective prevention and control strategies and encouraging partnerships and collaborations between researchers and the intended end users at different contexts (i.e., school, family, community, and policy makers) in the planning, implementa-tion, evaluation, and dissemination of results. It also will guide other researchers in developing new and innovative ecological interventions focusing on the identified research areas and priorities to fight Latino childhood obesity. Results from these studies will inform public health professionals, school administrators, health practitioners, community orga-nizations, and policy makers about the most effective strate-gies and interventions to support program design and implementation, allocation of specific resources, policy devel-opment, and further research.

It will generate interventions and best practices so that practitioners at all levels can effectively address the problem of Latino childhood obesity and prevent its devastating health ramifications both in the immediate and long term to ensure that all children have the opportunity to grow up physically and emotionally healthy.

Acknowledgments

The authors wish to thank Cliff Despres for his assistance in editing this manuscript, Bill Sanns and Andrea Whitlock for helping develop the Web-based questionnaire, Emma Mancha for her assistance with literature review, and Edgar Muñoz for his support with data analysis. Special appreciation is extended to the Salud America! National Advisory Committee members for their contributions to the initial survey design, and to all Delphi participants, whose contributions were vital to this study and the establishment of the first-ever National Latino Childhood Obesity Research Agenda.

at UTHSC AT SAN ANTONIO on May 24, 2011heb.sagepub.comDownloaded from

Ramirez et al. 259

Declaration of Conflicting InterestsThe authors declared no potential conflicts of interests with respect to the authorship and/or publication of this article.

FundingThe authors disclosed receipt of the following financial support forthe research and/or authorship of this article:

This research project is funded by a Robert Wood Johnson Foun-dation award, number ID 64756.

References

Agresti, A., & Finlay, B. (1997). Statistical methods for the social sciences (3rd ed.). Upper Saddle River, NJ: Prentice Hall.

Ali, A. K. (2005). Using the Delphi technique to search for empirical measures of local planning agency power. Qualitative Report, 10, 718-744.

Anderson, P., & Butcher, K. (2006). Childhood obesity: Trends and potential causes. Future of Children, 16, 19-45.

Anderson, P. M., Butcher, K. F., & Levine, P. B. (2003). Economic perspectives on childhood obesity. Federal Reserve Bank of Chicago, Economic Perspectives, 3Q, 30-48. Retrieved from http://www.chicagofed.org/publications/economicperspectives/2003/3qeppart3.pdf

Booth, S. L., Mayer, J., Sallis, J. F., Ritenbaugh, C., Hill, J. O., & Birch, L. L. (2001). Environmental and societal factors affect food choice and physical activity: Rationale, influences and leverage points. Nutrition Reviews, 59, 21-39.

Bowles, K. H., Holmes, J. H., Naylor, M. D., Liberatore, M., & Nydick, R. (2003). Expert consensus for discharge referral deci-sions using online Delphi. AMIA Annual Symposium Proceedings, 2003, 106-109.

Bronfenbrenner, U. (1986). Ecology of the family as a context for human development: Research perspectives. American Psycho-logical Association, 22, 723-742.

Bronfenbrenner, U. (1989). Ecological systems theory. Annals of Child Development, 6, 187-249.

Bronfenbrenner, U. (2005). The bioecological theory of human development. In U. Bronfenbrenner (Ed.), Making human beings human: Bioecological perspectives on human development (pp. 3-15). Thousand Oaks, CA: Sage Publications.

Centers for Disease Control and Prevention. (2005). QuickStats: Prevalence of overweight among children and teenagers, by age group and selected period—United States, 1963-2002. Morbidity and Mortality Weekly Report, 54(8), 203.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillside, NJ: Lawrence Erlbaum.

Dalkey, N. C. (1969). An experimental study of group opinion. Futures, 1, 408-426.

Dalkey, N. C. (1972). The Delphi method: An experimental study of group opinion. In N. C. Dalkey, D. L. Rourke, R. Lewis, & D. Snyder (Eds.), Studies in the quality of life: Delphi and decision-making (pp. 13-54). Lexington, MA: Lexington Books.

Davison, K. K., & Birch, L. L. (2001). Childhood overweight: a con-textual model and recommendations for future research. Obesity Reviews, 2(3), 159-171.

Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning. Glenview, IL: Scott, Foresman.

Flores, G., Fuentes-Afflick, E., Barbot, O., Carter-Pokras, O., Claudio, L., Lara, M., . . . Weitzman, M. (2002). The health of Latino children: Urgent priorities, unanswered questions, and a research agenda. Journal of the American Medical Association, 288, 82-90.

Flynn, M. A., McNeil, D. A., Maloff, B., Mutasingwa, D., Wu, M., Ford, C., & Tough, S. C. (2006). Reducing obesity and related chronic disease risk in children and youth: A synthesis of evi-dence with “best practice” recommendations. Obesity Reviews, 7(Suppl. 1), 7-66.

Guo, S. S., & Chumlea, W. C. (1999). Tracking of body mass index in children in relation to overweight in adulthood. American Journal of Clinical Nutrition, 70, 145S-148S.

Hsu, C., & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment Research & Evaluation, 12(10), 1-8.

Institute of Medicine. (2005). Preventing childhood obesity: Health in the balance (J. P. Kaplan, C. T. Liverman, & V. I. Kraak, Eds.; Committee on Prevention of Obesity in Children and Youth). Washington, DC: National Academies Press.

Institute of Medicine. (2007). Progress in preventing childhood obesity: How do we measure up? (J. P. Kaplan, C. T. Liverman, V. I. Kraak, & S. L. Wisham, Eds.; Committee on Progress in Preventing Child-hood Obesity). Washington, DC: National Academies Press.

Linstone, H. A., & Turoff, M. (1975). The Delphi method: Techniques and applications (pp. 3-12). Reading, MA: Addison-Wesley.

Lobstein, T., Baur, L., & Uauy, R. (2004). Obesity in children and young people: A crisis in public health. Obesity Reviews, 5(Suppl. 1), 4-85.

Ludwig, D. S. (2007). Childhood obesity: The shape of things to come. New England Journal of Medicine, 357, 2325-2327.

Mei, Z., Scanlon, K. S., Grummer-Strawn, L. M., Freedman, D. S., Yip, R., & Trowbridge, F. L. (1998). Increasing prevalence of overweight among U.S. low-income preschool children: The Centers for Disease Control and Prevention Pediatric Nutrition Surveillance, 1983 to 1995. Pediatrics, 101(1), E12.

Murphy, M. K., Black, N. A., Lamping, D. L., McKee, C. M., Sanderson, C. F., Askham, J., & Marteau, T. (1998). Consensus development methods, and their use in clinical guideline devel-opment. Health Technology Assessment, 2(3), i-iv, 1-88.

Must, A., Naumova, E. N., Phillips, S. M., Blum, M., Dawson-Hughes, B., & Rand, W. M. (2005). Childhood overweight and maturational timing in the development of adult overweight and fatness: The Newton Girls Study and its follow-up. Pediatrics, 116, 620-627.

Ogden, C. L., Carroll, M. D., Curtin, L. R., McDowell, M. A., Tabak, C. J., & Flegal, K. M. (2006). Prevalence of overweight and obesity in the United States, 1999-2004. Journal of the American Medical Association, 295, 1549-1555.

Ogden, C. L., Carroll, M. D., & Flegal, K. M. (2008). High body mass index for age among U.S. children and adolescents, 2003-2006. Journal of the American Medical Association, 299, 2401-2405.

at UTHSC AT SAN ANTONIO on May 24, 2011heb.sagepub.comDownloaded from

260 Health Education & Behavior 38(3)

Ogden, C. L., Yanovski, S. Z., Carroll, M. D., & Flegal, K. M. (2007). The epidemiology of obesity. Gastroenterology, 132, 2087-2102.

Powell, C. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing, 41, 376-382.

Robert Wood Johnson Foundation. (2009). Active living research: Fast facts. Retrieved from http://www.activelivingresearch.org/resourcesearch/resourcesforpolicymakers/fastfacts

Rosner, B. (1995). Fundamentals of biostatistics (4th ed.). Belmont, CA: Dexbury Press.

Sallis, J. F., & Owen, N. (1997). Ecological models. In K. Glanz, F. M. Lewis, & B. K. Rimer (Eds.), Health behavior and health education (pp. 403-424). San Francisco, CA: Jossey-Bass.

Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi method for graduate research. Journal of Information Technology Education, 6, 1-21.

SPSS Inc. (2008). SPSS for Windows. Release 16.0 ed. Chicago, IL: Author.

Story, M., Kaphingst, K. M., Robinson-O’Brien, R., & Glanz, K. (2008). Creating health food and eating environments: Policy and environ-mental approaches. Annual Review of Public Health, 29, 253-272.

Turoff, M., & Hiltz, S. R. (1996). Computer-based Delphi processes. In M. Adler & E. Ziglio (Eds.), Gazing into the oracle: The Delphi method and its application to social policy and public health (pp. 56-88). London, UK: Kingsley.

U.S. Census Bureau. (2008). US Hispanic population surpasses 45 mil-lion: Now 15 percent of total. Retrieved from http://www.census .gov/Press-Release/www/releases/archives/population/011910.html

Wang, Y., & Lobstein, T. (2006). Worldwide trends in childhood over-weight and obesity. International Journal of Pediatric Obesity, 1, 11-25.

Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs assessment: A practical guide. Thousand Oaks, CA: Sage.

Young, S. J., & Jamieson, L. M. (2001). Delivery methodology of the Delphi: A comparison of two approaches. Journal of Park and Recreation Administration, 19, 42-58.

at UTHSC AT SAN ANTONIO on May 24, 2011heb.sagepub.comDownloaded from