http://jis.sagepub.com

Journal of Information Science

DOI: 10.1177/0165551508092258 2009; 35; 82 originally published online Oct 23, 2008; Journal of Information Science

Michael Mackert, Brad Love and Pamela Whitten Patient education on mobile devices: an e-health intervention for low health literate audiences

http://jis.sagepub.com/cgi/content/abstract/35/1/82 The online version of this article can be found at:

Published by:

http://www.sagepublications.com

On behalf of:

Chartered Institute of Library and Information Professionals

can be found at:Journal of Information Science Additional services and information for

http://jis.sagepub.com/cgi/alerts Email Alerts:

http://jis.sagepub.com/subscriptions Subscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.co.uk/journalsPermissions.navPermissions:

http://jis.sagepub.com/cgi/content/refs/35/1/82 Citations

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 82

Patient education on mobiledevices: an e-healthintervention for low healthliterate audiences

Michael Mackert and Brad Love

Department of Advertising, The University of Texas at Austin, USA

Pamela Whitten

College of Communication Arts & Sciences, Michigan State University, USA

Abstract.

Providing health information to low health literate audiences remains a challenge. Beyond message design,realistic delivery models are needed for delivering information to a traditionally hard-to-reach audience.This study investigated two e-health interventions to provide health information on mobile devices – one pro-viding diabetes information and one offering childcare information. Both were well-received, and most of thesubjects’ usability issues related to the translation of these interventions to the mobile device’s smaller screen.The diabetes website was effective in providing information to study participants (as measured by pre- andpost-tests of knowledge), while the childcare website was not. Continued work in this area could exploreimproved design strategies for mobile devices – a delivery model that could be used in doctors’ offices, forexample. Effective delivery of health information to low health literate audiences is an important issue, andthis research highlights a critical element by targeting another potential delivery model.

Keywords: health literacy; mobile devices; usability; website

1. Introduction

Information systems (IS) technology and research has vast potential to help improve health out-comes. IS developments have been used, for example, to offer educational content to patients [1],provide decision-making models to healthcare practitioners [2, 3], and examine the environment’sinfluence on health [3]. In addition, digital technologies also have demonstrated an ability to helpconfront daunting health issues [1] and connect with hard-to-reach patients [4–6].

Correspondence to: Michael Mackert, Department of Advertising, The University of Texas at Austin, 1University Station A1200, Austin, TX 78712, USA. Email: mackert@mail.utexas.edu

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

Michael Mackert et al.

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 83

The potential to customize digital media content offers unique options for working with popula-tions not readily reached by traditional communication avenues. One such population includesindividuals with reduced health literacy. Lacking the skills to obtain, process, and understand basicinformation needed for health decisions, these individuals are less able to mange their own care [7,8] and suffer worse health outcomes as a result [9–14]. Reduced health literacy is also associatedwith frequent hospitalizations [15], incomplete understanding of health issues [16, 17], andincreased healthcare costs [18, 19].These negative outcomes become even more urgent when considering health literacy’s status as

a resolvable issue that impacts individual quality of life, public health, and national economies [20].In addition, recent projects have pointed to other pressing reasons for addressing and improvinghealth literacy. Senior citizens with low health literacy have increased mortality rates, adding to theconcern surrounding the world’s aging population [21]. Rapidly shifting immigration trends alsopose a real challenge as significant numbers of people are forced to seek health information in a non-native language and navigate significant cultural barriers [22].As a result of these factors, health officials throughout the world are confronted by the challenge

of reduced literacy [23], with half or more of the world’s adult population possessing limited abil-ity to understand and act on information [8, 24]. In the USA, the seriousness of the situation has ledboth the federal government and the National Academies’ Institute of Medicine to label the problemof health literacy a top priority for public health officials, researchers, and practitioners [25].Globally, scholars have suggested that an international index of health literacy could be useful inevaluating countries’ health promotion and prevention efforts and capacity to advance health issueswithin a given population [26].The scale of the problem has brought a number of individuals and institutions to work around

individuals’ low health literacy using internet technologies [27], traditional-media campaigns [28],and interpersonal channels [29]. In this context, digital media are particularly recognized for theirhealth-education potential because of the technology’s adaptability, relative cost effectiveness, andaccessibility [30].This research aims to move health literacy research in a new direction by taking full advantage

of the benefits of information technologies, specifically mobile digital media. The study focuses onthe development of mobile-device-based health interventions designed for lower-literate audiences.Using mobile devices such as personal digital assistants (PDAs) allows individuals to benefit fromthe customization common to digital media while having maximum convenience when accessinginformation. With ultra-portable technology like PDAs, individuals can be reached where and whenit is most convenient for them, whether in a waiting room, at home, or on a job site.A focus on handheld technology looks to the future and the inevitable growth of portable dig-

ital equipment [31]. Using PDAs also produces the benefits of involving the patient in the learn-ing process while meeting growing consumer demand for some control over health-informationgathering [32].To discuss these issues, this paper starts with a discussion of health literacy and related issues,

explains the research method used to evaluate the interventions, provides the results of the evalua-tion, and then discusses the findings along with suggestions for future research.

2. Literature review

Reduced health literacy is a challenge that reaches across society as education level, age, geographiclocation, income and ethnicity are all associated with health literacy [33], increasing the need forhealth literacy researchers to develop interventions to improve communication with low literatepatients. Low general literacy prevents millions from obtaining essential health information, lead-ing governments and international institutions to declare the development of literacy skills an inter-national priority [34, 35]. The emergence of digital communication adds a new dimension to thisissue with its many possibilities [36], specifically its ability to tailor information and connect withhard-to-reach audiences [37, 38].

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

Michael Mackert et al.

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 84

According to Harris Interactive, 110 million Americans (66% of the adult population) are alreadyusing online sources for health information, up significantly from 54 million people in 1998 [39]. Aspatients increasingly search for health information through digital sources, providers are adapting aswell. Given the significant percentage of Americans with both general and health literacy problems,making health information accessible and comprehensible to low literate populations is vital.Evidence suggests many low literate people in the industrialized world already use digital media

such as the web. The National Assessment of Educational Progress found that 41% of eighth-gradersin free and reduced lunch programs have home internet access [40]. This group is certain to growas low literate individuals increasingly employ digital technologies like handheld computers athome and at work. Future adoption will be driven by factors such as decreasing costs and the fre-quency of government business being conducted online.Of course, access by itself would solve little. Information available through handheld machines

or public computer terminals must be accessible to less literate audiences in order to create somechange. These audiences appear to be mostly excluded at the moment. One study looking at read-ability levels of 50 websites intended for public viewing found that on average the sampled medicalinformation was at a 10th-grade reading level according to the Flesch reading score and the Flesch-Kincaid reading level assessment [41]. Similarly, the Children’s Partnership evaluated 1000 web-sites and only found 10 of them to be accessible to low literate adults [42].Another study looking at the readability and accessibility of the English- and Spanish-language

information presented by 14 search engines found that almost all sites required high-school-level orgreater reading ability. In addition, a team of 34 physicians determined that not more than 20% oflinks provided on the first results page led to relevant content, and less than half of the clinical com-ponents of each subject were covered accurately and substantively [43]. Common, low quality infor-mation on the web opens the door for less valuable information to gain distribution [44], particularlysince not all users may bother checking the credibility of websites [45] or have the backgroundknowledge to judge the quality of presented information.Technology offers a number of opportunities to counter this trend of inaccurate, less accessible

online information. Several projects have developed customized websites intended to provide accu-rate information in a way that is approachable for the intended audience. In such cases, users tendto appreciate the tailored nature of the material and find it more personally relevant [46].Even in economically powerful countries such as Germany, Canada, the USA, Finland, and

Norway, up to a quarter of the adults in the International Adult Literacy Survey failed to reach theminimum functional competency level with everyday texts [47] – never mind medical texts orphysician instructions. The potential of digital technologies lies in the ability to customize contentand adapt materials for these reduced literacy groups without dispensing significant resources(when compared to making changes in print materials).Handheld technologies take this potential into a new area by opening up opportunities for

more convenient access to information. Smaller size, increased portability, and extended batterylife, for example, all offer benefits not available from currently employed kiosks or computers, allin a presentation format capable of being equally effective as large-screen monitors [48]. Existingresearch has shown benefits from the usage of PDAs, including better documentation, improvedprocedure selection [49], and adherence to guidelines [50]; other research has pointed to diffi-culties encountered when patients are asked to adopt PDAs for use in managing their health [51].However, the number of studies examining the usefulness of these devices is actually quite smallin number, and those in existence tend to focus on usefulness for medical professionals, not thegeneral public [52, 53].

3. Method

This study was designed to test the efficacy of mobile devices as a delivery model for e-health inter-ventions created specifically for low health literate audiences. Three primary research questionsguided this investigation:

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

Michael Mackert et al.

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 85

RQ1: How do participants respond to learning health information from specially-designed websiteson a mobile device?

RQ2: How do users perceive the websites and the mobile device in regard to design and usability?

RQ3: Can these websites, offered on mobile devices, improve subjects’ knowledge of health infor-mation?

Successfully answering these questions would lay the foundation for larger-scale evaluations,such as randomized trials comparing such mobile e-health interventions to video or printed educa-tional materials or experiments to test specific usability elements.

3.1. The websites and mobile devices

The research presented here evaluates reactions to two different websites. Participants looked at thedifferent websites which were designed to provide health information to low literate and lowhealth-literate audiences. Both websites were designed using Macromedia Flash, Adobe GoLive,Adobe Photoshop, Adobe Image ready, BBEdit, Apple Final Cut Pro, Apple QuickTime Player Pro,Discreet Cleaner 6, and Soernson Squeeze, and are formatted so that any current web browser canview the material. Researchers and designers from the Michigan State University CommunicationTechnology Laboratory created the two websites. A team of healthcare providers and educatorsdeveloped and approved the content provided in the websites. Since the websites are designed fora low literate audience, users were presented with information visually by animated healthproviders. Most information is presented through audio means, and text is used sparingly.The first website, Diabetes and You,1 was designed to provide information about diabetes to non-

diabetics. Some of the features of the website are basic information about diabetes, signs and symp-toms of the disease, tips for eating and exercise that could help prevent diabetes, and a tool that letsusers measure their own potential for getting diabetes. The website features an African Americanmale doctor and a white female doctor. Simple, VCR-style controls allow the user to rewind, pause,and advance through content sections. Users can surf through the website in any order, but follow-ing the doctor’s instructions takes the user through in a simple, linear manner.The second website, Child Care Center,2 offers information about promoting healthy eating habits

in kids, dealing with newborns, and what to do when children become sick. Four characters are fea-tured in this website:

(1) Nancy, a female nurse who welcomes users to the website;

(2) Dr Garcia, a Hispanic female doctor;

(3) Dr Smith, a while female doctor; and

(4) Dr Lee, an Asian male doctor.

The design of the Child Care website is more sophisticated and allows users to browse throughat their own will. It also features more advance production values (i.e. the impression of a movingcamera as the user enters a doctor’s office).Finally, it is worth noting that these websites were designed to be used primarily as educational

interventions (given to people on CD-ROM, shown in healthcare clinics, etc.) rather than to be puton the internet for general use. The goal of these websites was to provide important health infor-mation to low health literate audiences, to prevent unnecessary calls to doctors’ offices and visits toemergency rooms. To meet the needs of these individuals, the websites were designed less to pro-vide information regarding an immediate need for a specific piece of health information; instead,low health literate users’ education through this type of intervention could reduce unnecessarystress on healthcare systems in the long term.The mobile device used for this project was a Hewlett-Packard iPAQ RX1950 Pocket PC, running

Microsoft Windows Mobile™ Version 5.0. This personal digital assistant (PDA) was selected givenits reasonable price and relative simplicity. Macromedia Flash Player for Pocket PC was installed on

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

the PDA so that study participants could view the websites on the device. This style of PDA wasselected for the study over other potential mobile devices due to the specific design of these web-sites – they were intended to be given to people as an educational intervention as opposed to beingreleased as a general health website that someone might access via a mobile phone.

3.2. Study procedures and sample

Subjects for this study were recruited from the waiting room of a clinic in an acute care hospital.This clinic serves primarily lower-income patients, approximately 20,000 per year. In addition toprimary care, the clinic also operates a diabetes clinic, a procedures clinic, and an anticoagulationclinic. Participants were invited to take part in the study while waiting for scheduled appointmentsor after their appointments were complete. All appropriate Institutional Review Board approvalswere acquired, and subjects were compensated for their participation in the study.Upon agreeing to participate in the study, subjects first completed the Rapid Estimate of Adult

Literacy in Medicine (REALM), a 66-item assessment of health literacy that involves correctly pro-nouncing words from a range of health topics [54, 55]. Scoring levels of the REALM include 0–18(3rd grade or lower), 19–44 (4th–6th grade), 45–60 (7th–8th grade), and 61–66 (9th grade).Participants then completed an orally-administered pre-test of knowledge on the topic covered

by the website. Next, subjects with no experience using a PDA or stylus were given basic instruc-tions on usage, as well as a few moments to practice with the stylus and feel more comfortable withthe technology. A research assistant then observed subjects as they explored a copy of the websiteloaded on the PDA, followed by a post-test of knowledge to assess any gain in knowledge thatresulted from viewing the intervention. The research assistant then administered an open-endedinterview to gage their reaction to the content and design of the website, as well as their feelingsabout use of the PDA.Finally, participants completed the Short Test of Functional Health Literacy in Adults

(STOFHLA) as another assessment of their health literacy [56, 57]. Scores on the STOFHLA rangefrom 1 to 36; scores of 0–16 represent inadequate health literacy, 17–22 is a marginal level of healthliteracy, and scores in the 23–36 range demonstrate adequate health literacy. The STOFHLA, as asecond measure of health literacy, was placed at the end of data collection so that it could be omit-ted if necessary (e.g. if subjects needed to leave the clinic to get the bus).There were 50 participants (N = 50) in the Diabetes and You segment of the study. The sample

was 71.4% female and 28.6% male, with an average age of 39.5 (SD = 13.9). This was a diverse sam-ple, including African Americans (39.5%), whites (35.4%), Hispanics (22.9%), and Asians (2.1%).The majority of this group had used a computer before (88%), but only 25% of respondents reportedhaving used a PDA. Most subjects (78%) stated they had access to the internet.This group’s average score on the STOFHLA was 25.1 (SD = 13.6). This broke down as 23.8% of

the sample with inadequate health literacy, 4.8% with marginal health literacy, and 71.4% with ade-quate health literacy. The average score on the REALM was 58.08 (SD = 17.3). This included 8% ofthe sample with health literacy at the 3rd grade or lower, 4.0% in the 4th–6th grade level, 12.0% at7th–8th grade capacity, and 76.0% at the level of 9th grade or higher. The REALM and STOFHLAwere highly correlated (r = 0.87, p <.001).The Child Care Center study included 29 subjects (N = 29). That sample included a larger pro-

portion of females (72.7%) to males (27.3%), with an average age of 46.5 (SD = 15.3). This was alsoa diverse group, consisting of African Americans (50%), whites (27.3%), and Hispanics (22.7%).The majority of these subjects had used a computer before (87%), but only 17% of respondents hadused a PDA. Slightly more than half of the subjects (52%) stated they had access to the internet.This group’s average score on the STOFHLA was 13.8 (SD = 15.9). This included 62.5% of the

sample with inadequate health literacy and 37.5% with adequate health literacy. The average scoreon the REALM was 46.9 (SD = 13.4). This included 17.2% of the sample with health literacy at the3rd grade or lower, 13.8% in the 4th–6th grade level, 24.1% at 7th–8th grade capacity, and 44.8%at the level of 9th grade or higher. Again, the two measures of health literacy were highly correlated(r = 0.93, p < 0.001).

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 86

Michael Mackert et al.

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

4. Results

The results of this study are presented below, organized by the primary research questions thatguided the investigation.

4.1. RQ1: How do participants respond to learning health information from specially designedwebsites on a mobile device?

Respondents were asked to rate the information in these websites in terms of their understanding ofthe information, the accuracy of the information, and the amount of information presented. All sub-jects understood the information in the Diabetes and You website (100%) and most (97.8%) con-sidered it to be accurate. Most (89.8%) felt that the website had the right amount of information,with the remainder feeling that it contained too little information. The Child Care Center receivedsimilar ratings, with most users understanding the information (92.9%) and considering it accurate(96.4%). The majority of respondents (88.9%) felt that it contained the right amount of information,with the remainder feeling that it included too much information. This difference is not surprising,given the relative depth of content areas on the Child Care Center website.Considering the appearance of the website, subjects were asked to consider the doctor narrators

and graphics on the site. For the Diabetes and You participants, all (100%) felt the narrators werehelpful, and most liked the way they looked (95%) and the general graphics within the site (95.9%).Again, the Child Care Center received similar ratings by users, as most (96.3%) of participants feltthat the narrators were helpful. All also approved of the narrators’ appearance (100%) and the gen-eral graphical appearance of the website (100%).

Table 1Diabetes and You user involvement

Section Mean Standard deviation

Website Introduction 2.6 0.5Why Should You Care? 2.5 0.6Signs and Symptoms 2.6 0.6How to Keep Your Risk Low 2.6 0.5Are You at Risk? 2.7 0.5

Table 2Child Care Center user involvement

Section and subsection names Visited (N) Mean Standard deviation

Website Introduction 29 2.7 0.5Sickness Introduction 23 2.8 0.4Fever 13 1.9 0.6Vomiting and Diarrhea 9 2.1 0.8Constipation 6 2.0 0.9Common Cold 10 1.8 0.8Newborn Introduction 23 2.7 0.5Feeding Newborns 13 2.2 0.7Appearance Issues 6 2.5 0.5Skin Rashes 5 2.6 0.5Spitting Up 4 2.3 0.1Health Eating Introduction 21 2.9 0.4How to Feed/Avoiding Obesity 11 2.7 0.6Feeding Newborns 4 2.3 0.1Feeding Toddlers 2 1.5 0.7Feeding Pre-school Children 12 2.3 0.7Feeding School-Age Children and Adolescents 3 2.0 0.0

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 87

Michael Mackert et al.

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

In addition to self-report measures, as participants worked their way through the linear Diabetesand You website, a graduate research assistant observed their use of the website. The research assis-tant rated subjects’ involvement in the website on a scale of 0 (bored and distracted) to 3 (highlyengaged) as they moved through the different content sections. Table 1 provides the average userinvolvement for each of the Diabetes and You sections. Overall, respondents were interested in thesite and its contents, with the interactive risk-assessment tool (‘Are You At Risk?’) as the most engag-ing section of content; this is consistent with initial pilot testing of the Diabetes and You website [58].Subjects were given the opportunity to explore the Child Care Center in whatever order they

wished, and they were not required to visit all sections. As with the Diabetes and You website, sub-jects were observed by a research assistant and rated from 0 (bored and distracted) to 3 (highlyengaged) as they navigated the Child Care Center. Table 2 provides information about the number ofpeople who visited each of the website’s content areas and their level of engagement with each sec-tion. Again, overall the entire website was received relatively well. The pages introducing eachmain content area were most engaging, which is not surprising – these particular areas provided thegreatest opportunity for user interaction, as users were given a menu of content options from whichto choose. The actual content areas did not include this type of user interaction.

4.2. RQ2: How do users perceive the websites and the mobile device in regard todesign and usability?

Subjects were asked to rate both the website itself and the PDA in terms of usability on a scale of 1(easy) to 10 (hard). Respondents rated the Diabetes and You website (M = 1.7; SD = 1.4) and the PDA(M = 2.5; SD = 2.5) as relatively simple to use. Not surprisingly, these two ratings were correlated sig-nificantly with one another (r = 0.39, p < 0.01). The Child Care Center website (M = 3.3; SD = 3.5), aswell as the use of the PDA to explore it (M = 3.5; SD = 3.4), were rated slightly less positively. In thiscase the relationship between usability of the site and the mobile device were even more related(r = 0.94, p < 0.001). The interrelationship between site and device usability is hardly surprising, par-ticularly when one considers that these websites were not designed for usage on the smaller screensof mobile devices. Navigation buttons and other website design elements were at times quite small onthe PDA screen. The Child Care Center website requires more navigation than the Diabetes and Youwebsite, creating more opportunities for frustration hitting the appropriate navigational devices withthe PDA stylus.In addition to asking users for their opinions of usability, the graduate researchers made note of

navigation and usability issues as they observed subjects exploring these websites. For the Diabetesand You website, users experienced some sort of issue on 7.7% of the pages. Such problems arosemost frequently while subjects used the risk-assessment tool (22.0% of users experienced trouble onthis page, representing 47.8% of all usage issues), which required users to drag sliders across thescreen to input their age and height. This is directly the result of the Diabetes and You website notbeing designed for use on mobile devices, as the tool’s controls were quite small on the screen anddifficult to manage with a stylus.As with the Diabetes and You website, navigational/usage issues were noted by the research assis-

tant. For the Child Care Center, users encountered some sort of navigational/usage issue on 14.0%of pages. The most common source of such problems was actually the initial lobby/introductionpage – 38.1% of users experienced problems on that page (33.3% of all problems). These problemswere typically that users did not immediately know to click on the doors to move deeper into thecontent areas, as the highlighting of the doors was not as noticeable on the smaller PDA screens.

4.3. RQ3: Can these websites, offered on mobile devices, improve subjects’ knowledge ofhealth information?

In an effort to test the educational value of these websites, respondents were tested for knowledge beforeand after the interventions using 18-item pre-tests and post-tests. For Diabetes and You, the average scoreon the diabetes pre-test was 12.7 (SD = 2.6), while the post-test was 14.8 (SD = 2.4). A paired samples

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 88

Michael Mackert et al.

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

t-test (t(47) = 6.07, p < 0.001) demonstrated the statistical significance of that improvement. For the ChildCare Center, the average score on the child care pre-test was 7.6 (SD = 2.2), which improved to an aver-age of 7.8 on the post-test (SD = 3.2). A paired samples t-test (t(28) = 0.74, p = 0.47) indicated that this dif-ference was not statistically significant; part of the reason for the lack of improvement could be thatsubjects were asked to explore only those sections which they found interesting, while the pre- and post-tests consisted of questions from each main content area.To look for a possible relationship between health literacy and knowledge gain, the correlation between

health literacy (asmeasured by the STOFHLA and the REALM) and increase in knowledgewas calculated.There was no significant relationship between increase in knowledge and health literacy as measured byeither the REALMor the STOFHLA for either website. To further confirm this, amultiple regression analy-sis was run with REALM and STOFHLA scores as independent variables and improvement in knowledgeas the dependent variable. Neither the REALM nor STOFHLA were significant predictors of knowledgeimprovement in the model for either intervention. Both sets of analyses seem to indicate that health liter-acy did not determine participants’ ability to acquire information from the website.

5. Discussion

Several interesting conclusions can be drawn from this research, not the least of which is the fact thatmobile devices could become a reasonable delivery model for providing health information to lowhealth literate audiences. These individuals overwhelmingly approved of the design of these interven-tions, which is particularly impressive when one considers that these websites were not designed foruse on the smaller screen of a mobile device. While certain design elements provided some usabilityissues, proper consideration of mobile device usage from the design stage of intervention developmentcould help eliminate such problems. It is reasonable to suggest that websites designed from the start tooperate on mobile devices are even more likely to provide positive results in such interventions.The improvement of diabetes knowledge after subjects viewed the Diabetes and You website is

consistent with previous research, as is the lack of a relationship between gain in knowledge and par-ticipants’ health literacy [58]. This lack of relationship between knowledge gain and health literacyfurther demonstrates that digital media have the capability to help work around patient-educationbarriers presented by health literacy. What is different in this study, though, is the lack of improve-ment among subjects exploring the Child Care Center website. Other work, with a larger sample (N =547), showed that participants did indeed increase knowledge as a result of viewing the Child CareCenter [59]. The difference could simply be one of numbers, but it could also be a result of these sub-jects exploring only some of the website – while the pre- and post-tests covered knowledge from theentire website. The other research, which used the same pre- and post-tests, involved an online sur-vey with participants exploring the website from home, where subjects could plausibly have hadmore time to spend viewing the site than this study’s participants taking part in a clinic waiting room.It is also possible that the Child Care Center website does not translate as an educational interven-tion to the mobile device as well as the Diabetes and You website does. Future work must continueto explore these issues and website design elements that could help or hinder the translation tosmaller mobile devices. Another useful direction for future research would be testing different deliv-ery models (mobile device, other digital media, paper brochures), though adequately matching thelevel and range of information for each type of content would be a challenge to such efforts.The previous point speaks to one of the limitations of this work, namely the smaller sample sizes

– particularly with the Child Care Center website. Continued exploration of the use of these typesof interventions, delivered on mobile devices, is important if researchers and professionals want tofully understand the potential for mobile devices as a health education delivery model. Of course,all research involves trade-offs, and in this particular study subjects were given the option toexplore whatever sections of the Child Care Center interested them most. An alternative strategy –limiting participants to one section, and then only testing them on the information in that section –would be a stronger test of the educational potential of the Child Care Center. As these interventionswere available only in English, a more in-depth examination of issues specific to English as a

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 89

Michael Mackert et al.

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

second language (ESL) audiences and their particular needs would also be useful. Issues to considerwould include how English interventions can better serve ESL audiences, as well as challenges thatmight arise in translating such applications to other languages and cultures. On a related note, theDiabetes and You website was initially designed to provide information to non-diabetics, but dia-betics were not excluded from the study of that intervention. Poor management of chronic diseaseslike diabetes is one of the greatest challenges facing health professionals, so it was considered ben-eficial to include them in them in that study; future interventions could productively test knowl-edge gain regarding health issues of which participants might have less pre-existing knowledge.Researchers must also continue to advance their understanding of health literacy, as well as meas-

ures of health literacy. The REALM, for example, does not assess whether or not subjects know whata particular health term means – only that they are able to pronounce it correctly. While the REALMand STOFHLA are useful in some contexts, more varied and accurate measures of health literacywould be helpful in assessing individuals’ health literacy and generalizing across studies. This alsohelps explain the high percentage of subjects in this study that are deemed to have ‘adequate’ or ‘9thgrade or higher’ health literacy. The problem is not that these subjects do not have lower levels ofhealth literacy; it is more that the existing measures do not completely tap into all the varied dimen-sions of health literacy that these individuals may indeed struggle with.Health researchers and professionals must continue to explore new and innovative strategies for

providing health information to low health literate audiences. Handheld mobile devices, perhapsprovided to patients while they wait in a doctor’s office, are one promising delivery channel. Thisresearch suggests that specially tailored interventions, designed to reach low health literate audi-ences, can be effective when provided on smaller mobile devices. If this is true for websites not opti-mized for mobile delivery, how much greater might the potential be for interventions designed fromthe start for this channel? Health communication researchers and professionals must continue toadvance their knowledge in this area, to improve our ability to provide effective health informationto a traditionally hard-to-reach audience.Moving beyond the USA, and more developed nations in general, this work also points to the

potential for the use of similar interventions in developing countries without extensive internetaccess. Delivering cheaper mobile devices loaded with interventions intended for lower health lit-erate audiences could provide crucial health education to developing countries. IS researchers canmake a significant contribution to these efforts by providing technology and usability expertise.Joint projects involving both health communication and IS researchers have the promise to morefully exploit the use of technology in communicating with low health literate audiences.

Notes

1 https://webspace.utexas.edu/msm2326/www/diabetes/2 https://webspace.utexas.edu/msm2326/www/childcarecenter/

Acknowledgements

This project was supported by funding from The University of Texas at Austin Office of the VicePresident for Research.

References

[1] D. Nicholas, P. Huntington, P. Williams and P. Blackburn, Digital health information provision and healthoutcomes, Journal of Information Science 27(4) (2001) 265–76.

[2] D. Masys, Effects of current and future information technologies on the health care workforce, HealthAffairs 21(5) (2002) 33–41.

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 90

Michael Mackert et al.

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

[3] M. Ulierua, M. Hadzicb and E. Chang, Soft computing agents for e-Healthnext term in application to theresearch and control of unknown diseases, Information Sciences, 176(9) (2006) 1190–1214.

[4] A.N. Uhegbu, Information communication networking in rural communities: the case of women in ImoState, Nigeria, Journal of Information Science 26(1) (2000) 51–9.

[5] S. Arunachalam, Reaching the unreached: how can we use information and communication technologiesto empower the rural poor in the developing world through enhanced access to relevant information?Journal of Information Science 28(6) (2002) 513–22.

[6] E. Hunter, H. Travers and B. McCulloch, Bridging the information gap: IT and health in Indigenouspopulations, Australasian Psychiatry 11(S1) (2003) S51-S56.

[7] Ad Hoc Committee on Health Literacy, Health Literacy: Report of the Council on Scientific Affairs,Journal of the American Medical Association 281 (1999) 552–557.

[8] L. Nielsen-Bohlman, A. Panzer and D. Kindig (eds), Health Literacy: A Prescription to End Confusion.(National Academy of Sciences, Washington, DC, 2004).

[9] D.A. DeWalt, N.D. Berkman, S. Sheridan, K.N. Lohr and M.P. Pignone, Literacy and health outcomes: Asystematic review of the literature, Journal of General Internal Medicine 19(12) (2004) 1228–39.

[10] S.C. Kalichman, E. Benotsch, T. Suarez, S. Catz, J. Miller and D. Rompa, Health literacy and health-relatedknowledge among persons living with HIV/AIDS, American Journal of Preventive Medicine 18(4) (2000)325–31.

[11] S.T. Lindau, C. Tomori, T. Lyons, L. Langseth, C.L. Bennett and P. Garcia, The association of health liter-acy with cervical cancer prevention knowledge and health behaviors in a multiethnic cohort of women,American Journal of Obstetrics and Gynecology 186(5) (2002) 938–43.

[12] M.A. Chisholm, J. Fair and C.A. Spivey, Health literacy and transplant patients and practitioners. PublicHealth 121(10) (2007) 800–803.

[13] H.S. Yin, S.G. Forbis and B.P. Dreyer, Health literacy and pediatric health, Current Problems in Pediatricand Adolescent Health Care 37(7) (2007) 258–86.

[14] P.C. Smith, J.H. Brice and J. Lee, The relationship between functional health literacy and adherence toemergency department discharge instructions among Spanish-speaking patients, Annals of EmergencyMedicine 50(3,S1) (2007) S55–6.

[15] D. Baker, R. Parker, M. Williams and W. Clark, Health literacy and the risk of hospital admission, Journalof General Internal Medicine 13(12) (1998) 850–51.

[16] M. Williams, D. Baker, R. Parker and J. Nurss, relationship of functional health literacy to patients' knowl-edge of their chronic disease, Archives of Internal Medicine 158(2) (1998) 166–72.

[17] J. Gazmararian, M. Williams, J. Peel and D. Baker, Health literacy and knowledge of chronic disease,Patient Education and Counseling 51(3) (2003) 267–75.

[18] D. Howard, J. Gazmararian and R. Parker, The impact of low health literacy on the medical costs ofMedicare managed care enrollees, The American Journal of Medicine 118(4) (2005) 371–37.

[19] B. Weiss and R. Palmer, Relationship between health care costs and very low literacy skills in a medicallyneedy and indigent medicaid population, The Journal of the American Board of Family Practice 17(1)(2004) 44–7.

[20] J. Gazmararian, J. Curran, R. Parker, J. Bernhardt and B. DeBuono, Public health literacy in America: anethical imperative, American Journal of Preventive Medicine 28(3) (2005) 317–22.

[21] R. Sudore, K. Yaffe, S. Satterfield, T.B. Harris, K.M. Mehta, E.M. Simonsick, A.B. Newman, C. Rosano,R. Rooks, S.M. Rubin, H.N. Ayonayon and D. Schillinger, Limited literacy and mortality in the elderly:the Health, Aging, and Body Composition Study. Journal of General Internal Medicine 21(8) (2006)806–12.

[22] L. Cooper, M. Hill and N. Powe, Designing and evaluating interventions to eliminate racial and ethnicdisparities in health care, Journal of General Internal Medicine 17(6) ( 2002) 477–86.

[23] N.J. Gray, J.D. Klein, P.R. Noyce, T.S. Sesselberg and J.A. Cantrill, The Internet: a window on adolescenthealth literacy, Journal of Adolescent Health 37(3) ( 2005) 243.e1–e7.

[24] D.L. Manning and C. Dickens, Health literacy: more choice, but do cancer patients have the skills todecide? European Journal of Cancer Care, 15(5) (2006) 448–52.

[25] R.H. Carmona, Health literacy: a national priority, Journal of General Internal Medicine 21(8) (2006) 803.[26] I. S. Kickbusch, Health literacy: addressing the health and education divide, Health Promotion

International 16(3) (2001) 289–97.[27] J. Homewood, Consumer health information e-mails: content, metrics and issues, Aslib Proceedings: New

Information Perspectives 56(3) (2004) 166–79.

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 91

Michael Mackert et al.

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

[28] A. Jorm, H. Christensen and K. Griffiths, The impact of beyondblue: the national depression initiative onthe Australian public's recognition of depression and beliefs about treatments, Australian and NewZealand Journal of Psychiatry 39(4) (2005) 248–54.

[29] H. Wilkin and S. Ball-Rokeach, Reaching at-risk groups: the importance of health storytelling in LosAngeles Latino media, Journalism 7(3) (2006) 299–320.

[30] G. Eysenbach, Credibility of health information and digital media: new perspectives and implications foryouth. In: M.J. Metzger and A.J. Flanagin (eds), Digital Media, Youth, and Credibility (MIT Press,Cambridge, MA, 2007).

[31] P. Trocchia and J. Swinder, A phenomenological investigation of internet usage among older individuals,Journal of Consumer Marketing 17(7) (2000) 605–16.

[32] D. Lewis, Computer-based approaches to patient education, Journal of the American Medical InformaticsAssociation 6(4) (1999) 272–82.

[33] M. Paasche-Orlow, R. Parker, J. Gazmararian, L. Nielsen-Bohlman and R. Rudd, The prevalence of lim-ited health literacy, Journal of General Internal Medicine 20(2) (2005) 175–84.

[34] B. Birru and R. Steinman, Online health information and low-literacy African Americans, Journal ofMedical Internet Research 6(3) (2004) 25.

[35] D. Nutbeam and I. Kickbush, Advancing health literacy: a global challenge for the 21st century, HealthPromotional International 15(3) (2000) 183–4.

[36] J. Powell, M. Darvell and J. Gray, The doctor, the patient and the world-wide web: how the internet ischanging healthcare, Journal of the Royal Society of Medicine 96(2) (2003) 74–6.

[37] V.J. Strecher, T. Greenwood, C. Wang and D. Dumont, Interactive multimedia and risk communication.Journal of the National Cancer Institute Monographs 25 (1999) 134–9.

[38] D. Hanauer, E. Dibble, J. Fortin and N. Col, Internet Use Among Community College Students: Implicationsin Designing Healthcare Interventions, Journal of American College Health 52(5) (2004) 197–202.

[39] H. Taylor, Cyberchrondriacs Update. [Web site press release] (2002). [Harris Interactive Web site]Available at: http://www.harrisinteractive.com/harris_poll/index.asp?PID = 299 (accessed 16 July 2005).

[40] A. Poftak, Net-wise teens: safety, ethics, and innovation, Technology and Learning 23(1) (2002) 36–49.[41] M. Graber, C. Roller and B. Kaeble, Readability levels of patient education material on the World Wide

Web, Journal of Family Practice 48(1) (1999) 58–61.[42] W. Lazarus and F. Mora, Online Content for Low-income and Underserved Americans: The Digital

Divide's New Frontier, a Strategic Audit of Activities and Opportunities (2000). Available at:http://www.childrenspartnership.org/pub/low_income/index.html (accessed 16 July 2005).

[43] G.K. Berland, M.N. Elliott, L.S. Morales, J.I. Algazy, R.L. Kravitz, M.S. Broder, D.E. Kanouse, J.A. Muñoz,J.A. Puyol, M. Lara, K.E. Watkins, H. Yang and E.A. McGlynn, Health information on the internet: acces-sibility, quality, and readability in English and Spanish, Journal of the American Medical Association285(20) (2001) 2612–21.

[44] G. Eysenbach, J. Powell, O. Kuss and E. Sa, Empirical studies assessing the quality of health informationfor consumers on the World Wide Web, Jounal of the American Medical Association 287(20) (2002)2691–2700.

[45] G. Eysenbach and C. Kohler, How do consumers search for and appraise health information on the worldwide web? Qualitative study using focus groups, usability tests, and indepth interviews, British MedicalJournal 324 (2002) 573–7.

[46] A. Oenema, J. Brug and L. Lechner, Web-based tailored nutrition education: results of a randomized con-trolled trial, Health Education Research 16(6) (2001) 647–60.

[47] Organization for Economic Cooperation and Development, Literacy Skills for the Knowledge Society:Further Results from the International Adult Literacy Survey (OECD, Paris, 1997).

[48] M. Jones, G. Marsden, N. Mohd-Nasir, K. Boone, and G. Buchanan, Improving web interaction on smalldisplays, Computer Network 31 (1999) 1129–11.

[49] S. Bird, R. Zarum and F. Renzi, Emergency medicine resident patient care documentation using a hand-held computerized device, Academic Emergency Medicine 8(12) (2001) 1200–1203.

[50] R. Shiffman, K. Freudigman, C. Brandt, Y. Liaw and D. Navedo, A guideline implementation system usinghandheld computers for office management of asthma: effects on adherence and patient outcomes,Pediatrics 105(4) (2000) 767–73.

[51] K. Keshavjee, M.L. Lawson, M. Malloy, S. Hubbard and M. Grass, Technology failure analysis: under-standing why a diabetes management tool developed for a personal digital assistant (pda) didn’t work ina randomized control trial. In: AMIA Annual Symposium Proceedings (AMIA, Bethesda, 2003).

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 92

Michael Mackert et al.

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from

[52] S. Fischer, T. Stewart, S. Mehta, R. Wax and S. Lapinsky, Handheld computing in medicine, Journal ofthe American Medical Informatics Associaton 10(2) (2003) 139–49.

[53] Y.-C. Lua, Y. Xiaoa, A. Sears and J.A. Jacko, A review and a framework of handheld computer adoptionin healthcare, International Journal of Medical Informatics 74(5) (2005) 409–22.

[54] C.A. Coyne, R. Xu, P. Raich, K. Plomer, M. Dignan, L.B. Wenzel, D. Fairclough, T. Habermann, L. Schnell,S. Quella and D. Cella , Randomized, controlled trial of an easy-to-read informed consent statement forclinical trial participation: a study of the eastern cooperative oncology group, Journal of ClinicalOncology 21(5) (2003) 836–42.

[55] T.C. Davis, S.W. Long, R.H. Jackson, E.J. Mayeaux, R.B. George, P.W. Murphy and M.A. Crouch, Rapidestimate of adult literacy in medicine: a shortened screening instrument, Family Medicine 25(6) (1993)391–5.

[56] D. Baker, M. Williams, R. Parker, J. Gazmarian and J. Nurss, Development of a brief test to measure func-tional health literacy, Patient Education and Counseling 38(1) (1999) 33–42.

[57] R.M. Parker, D.W. Baker, M.V. Williams, and J.R. Nurss, The Test of Functional Health Literacy in Adults:a new instrument for measuring patients' literacy skills, Journal of General Internal Medicine 10(10)(1995) 537–41.

[58] P. Whitten, B. Love, L. Buis and M. Mackert, Health education online for individuals with low health lit-eracy: evaluation of the Diabetes and You website, Journal of Technology in Human Services 26(1) (2008)77–88.

[59] M. Mackert, P. Whitten and A. Garcia, Interventions for low health literate audiences, Journal ofComputer-Mediated Communication 13(2) 504–15.

Journal of Information Science, 35 (1) 2009, pp. 82–93 © CILIP, DOI: 10.1177/0165551508092258 93

Michael Mackert et al.

at UNIV OF TEXAS AUSTIN on January 19, 2010 http://jis.sagepub.comDownloaded from