The Completeness and Readability of Multiple Sclerosis Information Available on UK Websites

2010

Mohammad Khasawneh Swansea University

9/26/2010

Submitted to the School of Human and Health Science

Swansea University Fulfillment of the Requirement of the

Degree of MSc of Health Informatics

1

DECLARATION

This work has not previously been accepted in substance for any degree and is not being concurrently submitted in candidature for any degree. Signed ...................................................................... (candidate) Date ........................................................................ STATEMENT 1 This work is the result of my own investigations, except where otherwise stated. Other sources are acknowledged by footnotes giving explicit references. A bibliography is appended. Signed ..................................................................... (candidate) Date ........................................................................

2

DECLARATION

This work has not previously been accepted in substance for any degree and is not being concurrently submitted in candidature for any degree. Signed ...................................................................... (candidate) Date ........................................................................ STATEMENT 1 This work is the result of my own investigations, except where otherwise stated. Other sources are acknowledged by footnotes giving explicit references. A bibliography is appended. Signed ..................................................................... (candidate) Date ........................................................................

3

Abstract

Background: Information needs of MS patients have been well established and

documented. There is a gap between the available information about MS and the

information needs of MS patients. The internet is one of the sources used by MS patients

for seeking information about their disease.

Aim: This research well evaluate the readability and completeness of MS information

available on UK websites. It will identify whether the MS information available on UK

websites is complete, easy to understand and read by patients; and whether it acts as

useful educational material. It will try to focus on addressing patients‟ needs when

investigating two variables (completeness and readability).

Method: This study was a “quantitative descriptive evaluation”. A valid data collection

instrument developed by Harland and Bath (2007) was used to evaluate the completeness

of MS information, and the SMOG readability formula was used to evaluate the

readability of MS information. The website sample was identified using a Google search

engine. A total of 27 UK websites that contain information about MS and written in

English form the sample used in this research.

Results: None of the websites archived full score in term of overall completeness. The

majority of the websites (74.1%) archived scores below 50% in overall completeness.

The UK Multiple Sclerosis Society was identified as the best website in terms of overall

completeness. Biomedical/Charity and Government websites achieved the highest score

in term of overall completeness (53.5%: 47.0% respectively), but Personal websites

achieved the lowest score amongst all categories (12.5%). 96.3% of websites partially

addressed the information needs of newly diagnosed patients. Moreover websites

achieved completeness score (32.8%) in post diagnosis section. (85.8%) of Family and

carer information was found to be the most incomplete information on the websites.

However the website score was the best in the overall content presentation section

4

(68.7%). 66.6% of information on the websites goes beyond the average reading ability

of the UK population (5 – 8).

Conclusion: The completeness of MS information on UK websites was found to vary.

Most websites contained incomplete information about MS. Patients need to be careful

about making decisions based on online information. There is a gap between the

information being provided to patients on the websites and their actual needs. This may

be a result of the divide between the authors of websites and patients. The mean reading

level of MS information on UK websites is 10.06 which is beyond the average reading

abilities of the UK population (Nicholls et al., 2009).

5

Contents

Index of figures 7

Index of tables 7

List of abbreviations 7

Introduction and Background 8

Multiple Sclerosis (MS) 8

Changes in Health Information Consumption 10

Literature Review 11

Importance of health information and patient‟s education 13

Health Communication 14

Internet as a Health Information Source 15

i- Seeking Health Information from the Internet 15

ii- Reasons to use the internet as a health information source 17

Health Information Quality 19

i- Definitions and dimensions of health information quality 19

ii- Addressing Patients‟ needs in assessing the quality of online health information

20

iii- Information Completeness 22

iv- Online health information completeness 23

v- Readability of Health Information 25

vi- The readability of Online Health Information 28

Information Needs of Patients with MS 29

Summary and Gaps in Research 32

Research Question 34

Method 34

Study Design 34

Website Sample 39

i- Inclusion Criteria 43

6

Data Collection 45

i- Information Completeness Data Collection tool 45

ii- Information Readability Data collection tool 48

Results 50

Websites Source 50

Overall Completeness 52

Newly Diagnosed Information Completeness 56

Completeness of Post Diagnosis Phase Information 57

Completeness of Family and Carers Information 60

Overall Presentation of the Content on the Website 60

Readability 62

Discussion 64

Overall Completeness 64

Different areas of MS Information Completeness 66

Readability 68

Limitation 70

Conclusion 70

References 73

Appendix A: PageRank Technology 105

Appendix B: Consent for Using MS Information Completeness Tool 106

Appendix C: Multiple Sclerosis Evaluation Tool 107

7

Index of figures

Figure 1: The Equation of SMOG readability formula

Figure 2: Sources of the Websites Used in This Research

Figure 3: The Frequency of the Scores Achieved by the Websites

Figure 4: The relation between the percentage of overall completeness score and the

source of the website

Figure 5: The percentage of overall completeness score achieved by the websites in

Different Sections

Figure 6: The Differences between the mean score achieved by MS Specific Websites

and Non-MS Specific Website in Post Diagnosis Phase Information section

Figure 7: The Differences between the mean score achieved by MS Specific Websites

and Non-MS Specific Website in Family and Carers Information section

Figure 8: SMOG readability score achieved by the websites, and it`s relation to UK

population reading abilities

Index of tables

Table 1: Framework of National Standards for Literacy and Numeracy (National Literacy

Trust 2010)

Table 2: Website sample used in Identifying Keyword

Table 3: Keywords Identified using “Google Keywords Tool”

Table 4: Inclusion Criteria

Table 5: the similarities of the SMOG score with the UK National Adult Literacy

Standards

Table 6: The rank of 27 websites based of overall completeness score achieved

List of abbreviations

BBC British Broadcasting Corporation

CASP Critical Appraisal Skills Programme

CNS Central Nervous System

EDSS Expanded Disability Status Scale

MS Multiple Sclerosis

NHS National Health Service

SMOG Simple Measure of Gobbledegook

UK United Kingdom

8

Introduction and Background

Multiple Sclerosis (MS)

As this research will investigate the readability and completeness of MS information on

UK websites, this section will provide an overview of the disease, its symptoms and its

disease prognosis.

In the human body the Central Nervous System (CNS) is composed of two main

components: an elongated spinal cord, and a large and complex brain. Neuron is the basic

functional unit in the CNS. A fatty materials called Myelin Sheaths coats the neurons and

the nerve cells, protects the nerve cells and plays a key role in the process of transmitting

neural impulses (Iezzoni, 2010; Brodal, 2010; Zivadinov and Bakshi, 2004).

Multiple Sclerosis (MS) was first described by Charcot and Vulpian in 1866 (Compston

et al., 2005b) as a chronic autoimmune disease in which a demyelination and destruction

to the component of the CNS occurs (Compston et al., 2005a; Reipert, 2004; Bartlett and

Kilpatrick, 1991; Courtney et al., 2009; Porth, 2007; Harland and Bath, 2007;

Noseworthy et al., 2000; Frohman, Racke and Raine, 2006). Damage to the myelin

sheaths results in an interruption or even complete stop of the neural impulse flow

(Reipert, 2004; Courtney et al., 2009; Mei, 2004). MS usually occurs in young adults (20-

40 years old) - it is estimated that 75% of patients have the disease before the age of 50

(Mei, 2004).There are also notable gender differences in the prevalence of MS (Male:

Female ratio 1:3) (Sadovnick, 2009; Rosati, 2001).

MS is a disease which results in a deficit in the function of the central nervous system

and leads to a wide range of physical and psychological symptoms. Despite that these

symptoms vary from person to person. This generally depends on the demyelinated part

of the CNS (Mei, 2004; Richards et al., 2002). The signs and symptom of MS include

weakness, sensory symptoms, ataxia, bladder symptoms, fatigue, cramps, poor memory,

headache, mental problems, Dysphasia and a few more (Frohman, Racke and Raine,

2006; Glowacki, 2009; Noseworthy et al., 2000; Porth, 2007; Mei, 2004).

9

Despite the fact that the causes of MS are unclear, it has been accepted that it is triggered

by an interaction between environmental and genetic factors (Compston et al., 2005a;

Lublin and Reingold, 1996; Reipert, 2004; Rejdak, Jackson and Giovannoni, 2010). The

Expanded Disability Status Scale (EDSS) is used to quantify disability in MS based on a

number of functional systems; the scale ranges between 0-10 (Kurtzke, 1983; Richards et

al., 2002).

In terms of the disease course and prognosis, MS can be classified into four subtypes.

„Relapsing remitting‟, is the most common subtype of the disease and involves cyclical

unpredictable relapses followed by partial or complete recovery. „Primary prognosis‟ is

characterized by the slow but continuous progress of the disease and symptoms over

time. The secondary progressive subtype which is more common than the primary one is

characterized by initial periods of relapsing and remitting followed by the steady and

continued progress of the disease and it`s symptoms. The progressive relapsing subtype

which is the least common is characterized by steady and continued neurological damage

with superimposed relapse and remission. Although there is significant recovery after

relapse, the symptoms gradually get worse between relapses (Compston and Coles, 2008;

Miller and Leary, 2007; Boiko et al., 2002; Daumer et al., 2007; National Multiple

Sclerosis Society, 2010; Rejdak, Jackson and Giovannoni, 2010; A.Achiron, 2006;

Richards et al., 2002). The signs and symptoms and clinical course are highly variable

from person to person, and during different disease course (Marrie and Goldman, 2007).

In the UK MS is a common disease (Markby, 1987; McDonnell and Hawkins, 1999;

Richards et al., 2002; Thomas et al., 2009), and it is estimated that there are more than

100,000 people who suffer from it (Hepworth, Harrison and James, 2003; Richards et al.,

2002), Its prevalence in England and Wales is between 104 to 156 per 100,000 (Richards

et al., 2002) and in Scotland is estimated that 1 in 500 have it - one of the highest

percentages in the world (McDonnell and Hawkins, 1999). It is estimated that the average

cost for patients who have EDSS between 1 to 3.5 is £3350 and £9560 for those with

EDSS between 6.5 -8 (Richards et al., 2002).

10

The information needs of MS patients has been well established (Matson and Brooks,

1977; Daumer et al., 2007; Forbes, While and Taylor, 2007; Harland and Bath, 2007;

Hatzakis et al., 2006; Hay et al., 2008; Lejbkowicz et al., 2010; Matti et al., 2010;

Sadovnick, 2009; Somerset, Sharp and Campbell, 2002; Chiovetti, 2006b; Slade, Tennant

and Ford, 2003), therefore patients used different sources to seek information about their

disease (Bishop et al., 2009; Hay et al., 2008; Matti et al., 2010). According to Bishop et

al (2009) understanding the source which provides information for MS patients is

important to ensure that this source provides high quality information.

Changes in Health Information Consumption

Patients used to seek health information from a number of “traditional” sources or

channels (Dutta-Bergman, 2004; Vivian, 2002; DeFleur and Cronin, 1991). There are two

main channels available for seeking health information (Brashers, Goldsmith and Hsieh,

2002). These include face to face encounters such as personal conversation and health

care professionals, friends …etc (Cotten and Gupta, 2004; Dolan et al., 2004; Park,

Chung and Yoo, 2009) and mediated communication channels such as television,

newspapers, books, radio, Internet...etc (Birchley, Pullan and DeFriend, 2003; Cotten and

Gupta, 2004; Brodie et al., 1999; Dolan et al., 2004; Brashers, Goldsmith and Hsieh,

2002; Park, Chung and Yoo, 2009). Although the doctor has been the primary

information source for patients for a long time (Akerkar and Bichile, 2004a; Ellis and

Thomson, 2003; McMullan, 2006; Campbell and Nolfi, 2005), people frequently turn to

other information sources to get as much information as they can about their condition,

and to have a full understanding of the alternative treatment options (Narhi, 2007; Taylor

and Leitman, 2002; Fox and Rainie, 2006; Coulter, 2002). A number of research studies

and surveys show that patients have problems in obtaining complete and reliable health

information (Broom, 2005; Chen and Siu, 2001; Eysenbach and Kohler, 2002a; Hart,

Henwood and Wyatt, 2004; Kiley, 2000; Ende J et al., 1989; Coulter, Entwistle and

Gilbert, 1999), and the information patients get from their doctor doesn‟t meet their

desires (Broom, 2005; Chen and Siu, 2001; Eysenbach and Kohler, 2002a; Hart,

Henwood and Wyatt, 2004; Kiley, 2000; Ende J et al., 1989). Torne and his colleagues

(2004) demonstrated that the ability of the doctor to exchange information effectively

11

with his patients depends largely on the communication skills he has, and the patient visit

to the doctor is too short for an effective information exchange (Bliemel and Hassanein,

2007). Furthermore, there are a number of limitations associated with seeking health

information from traditional sources. It can be time consuming, out of date and space

requiring (Anderson, Rainey and Eysenbach, 2003), lack of interactivity (Doian, 2006),

along with it having a rigid one-way nature (SPICH, 1999).

In the last few years with increased use of information technology there has been a

dramatic change in the environment in which patients seek and consume health

information (Willard and Ginsburg, 2008; Kummervold et al., 2008; Fox and Fallows,

2003a; Hesse et al., 2005). A number of authors have demonstrated that different factors

such as changed attitudes, knowledge and approaches and technology used in healthcare

have made people more capable of managing and finding health information from

different sources (Williams et al., 2002; Civan et al., 2006; Pratt et al., 2006; Anderson,

Rainey and Eysenbach, 2003; Cline and Haynes, 2001b). For example, the unlimited

access of patients to information on the internet leads to an increase in health information

consumption (Dutta-Bergman, 2004; Brashers, Goldsmith and Hsieh, 2002). It also leads

to an exponential increase in the number of patients turning to the internet as one of the

most important and valuable health information sources (Health;, 1999; Fox, 2006b; Fox

and Fallows, 2003a; Nammacher and Schmitt, 1998; Baker et al., 2003; Morahan-Martin,

2004; Eysenbach and Kohler, 2003; Bates et al., 2006; Lemire et al., 2008). Internet was

perceived to be well accepted by MS patients (Lejbkowicz et al., 2010), and to be an

important source for MS patients to seek information about their disease (Bishop et al.,

2009; Hay et al., 2008), more than that Lejbkowicz et al (2010) concluded that internet

use by MS patients contribute to the wellbeing of those patients.

Literature Review

Critical appraisal has referred to the process of systematically and cautiously assessing

research to magistrate its value and relevance in certain contexts (Hill and Spittlehouse,

2003). The Critical Appraisal Skills Programme (CASP) is a framework which helps

people understand scientific evidence such as research articles by producing simple

appraisal checklists which cover the main areas in critical appraisal (i.e. validity results

12

and relevance) (Burls, 2009). CASP was used in this research as a framework for

reviewing literature.

The databases „CINAHL‟ and „Pubmed‟ were searched for articles about the topic of

interest in this research. The following keywords were used: “Internet”, “health” and

“website”, “health” and “online”, “world wide web”, “health communication”, “health

information” and “online”, “information sources”, “health information” and “sources”,

“information quality”, “health information quality”, “information completeness”,

“information readability”, “health information completeness”, “health information

readability”, “online information completeness”, “online information readability”,

“information comprehensiveness”, “multiple sclerosis”, “information seeking” and

“multiple sclerosis”, “information needs” and “multiple sclerosis”, “online information”

and “multiple sclerosis”, “patients education”, “patients education” and “chronic

condition”, “patients education” and “multiple sclerosis”. Limitations were applied to

include studies from between 2002 -2010. The abstract of each article was read by the

researcher and only articles which were relevant to the topic of research were chosen.

Also articles in the reference list of the selected articles were retrieved if they were found

to be relevant to the topic of the study.

Nivivo 7 software is an analytical tool which is frequently used in literature reviews,

because it helps the researcher to organize the article codifying them, identifying

common themes , and grouping ideas in different articles (Beekhuyzen, 2007). Nivivo 7

was used to review the literature in this research.

The first section provides an overview of the importance of patients‟ education and the

role of communication in health care. The second section will talk about the process of

seeking health information from the internet by patients and advantages which attract

patients to the internet. The third section will explain the health information quality

concept and will provide an overview of the dimensions involved in this concept. It will

provide detail about the completeness and readability dimension of health information

13

and the detail of the status of these dimensions in the online health information

environment.

The fourth section will provide detail about the information needs of MS patients with

specific focus on UK patients. The final section will summarize the literature review and

the identified research gaps.

Importance of health information and patient’s education

Based on the sense making theory; patients seek health information to make sense of their

own condition (Dervin, 1992). For instance in a UK study, it is estimated that 80% of UK

patients were very likely to seek information about their medical condition, and how to

cope with this medical condition (Ellins and Coulter, 2005). It has been accepted that

providing patients with information will lead to better understanding of their own health

condition and have greater impact on the success of health care treatment. On the other

hand the shortage in health information will lead patients to misleading actions and

misconceptions (Pellise, Sell and EuroSpine Patient Line Task, 2009; Gross et al., 2006;

Department of Health, 2004; The Joint Commission, 2007b), or even can lead to fatal

consequences (The Joint Commission, 2007a). In the last few years there has been more

emphasis on the importance of patient‟s education (Al-Sowielem and Elzubier, 1998),

especially for patients with chronic conditions (Cooper et al., 2001; Godibile et al., 2005;

Chiovetti, 2006a), and it has become a key element of good medical and pharmaceutical

practice (Howard et al., 1999; Huang et al., 2002).

Patient‟s education gains this importance because of its impact on enhancing patients‟

level of knowledge about their condition as well as its therapeutic values (Saounatsou et

al., 2001; Huang et al., 2002), It can enhance their adaptation to a new lifestyle, along

with improving therapy adherence and overall healthcare outcomes (Saounatsou et al.,

2001; Gruesser et al., 1997; Melnyk, Shevchuk and Remillard, 2000; Goudswaard et al.,

2004). Furthermore, patient‟s education can makes them more active in the healthcare

process by making more informed decisions about their health with their doctor (Gross et

al., 2006; Molenaar et al., 2000). In the UK the white paper “Better information, better

choices, better health” (2004) shaped out the strategy for health information provision in

14

the UK, with emphasis on people‟s right to have access to high quality, accurate and

comprehensive health information.

Since then the empowerment of patients with chronic diseases (such as MS) has been

identified as key for managing their diseases (Matti et al., 2010). Providing MS patients

with information about different stages of their disease, the symptoms associated with

each stage and the available options for managing their disease has been identified as a

key factor for MS patients‟ empowerment (Somerset, Sharp and Campbell, 2002;

Somerset et al., 2003). The advantage and benefits of this empowerment has been

manifested by a number of positive psychological and clinical outcomes such as an

increase in patients‟ compliance, and satisfaction, enhanced coping and adjustment after

diagnosis, and improving the overall quality of life for patients with MS (Somerset, Sharp

and Campbell, 2002; Bishop et al., 2009).

Information exchange or communication is central to patients‟ education (Pellise, Sell

and EuroSpine Patient Line Task, 2009; Thorne et al., 2004), and health communication

which is based on complete and reliable information is essential for patients care and

education (Donohue et al., 2009; Seematter-Bagnoud and Santos-Eggimann, 2007).

Health Communication

Heath communication can be defined as:

“The art and technique of informing, influencing, and motivating individual, institutional,

and public audiences about important health issues”

(U.S. Department of Health and Human Services, 2000)

Health communication is increasingly recognized as a key element for improving public

and individual heath and is considered the cornerstone for effective patient education

(Rimal and Lapinski, 2009; Parrott, 2004), not only because it can help increase the

levels of awareness of individuals about different health-related issues, but also because it

15

can contribute to all aspect of health promotion, disease prevention and overall

improvement of health outcomes in different contexts (Parrott, 2004; U.S. Department of

Health and Human Services, 2002). Information and information exchange is at the heart

of health communication (Lee and Garvin, 2003). According to the U.S. department of

health report “Healthy People 2010: Understanding and Improving Health”; in order to

motivate people for use and understand the health message content, and to achieve the

effective communication of health information, a number of attributes need to be

included in the health message. Content optimization and comprehensibility is one of

these key attributes (U.S. Department of Health and Human Services, 2002), Arkin

(2008) demonstrated that this can be done by identifying the perception and needs of the

health message audience.

Internet as a Health Information Source

Seeking Health Information from the Internet

The internet is has been identified as a type of interactive health communication, which

can be defined as:

“The interaction of an individual (consumer, patient, caregiver, or professional) with or

through an electronic device or communication technology to access or transmit health

information, or to receive or provide guidance and support on a health-related issue”

(SPICH, 1999)

In the last few years the internet has extensively spread round the world, and internet

users exponentially increased (NetRatings, 2005; Fox and Fallows, 2003b; Lenhart, 2003;

Manasian, 2003) It is estimated that 26% of the world‟s population use the internet

(Miniwatts Marketing Group, 2010). Chair (2004) describes the internet as a free tool to

distribute information, which changes the way in which people seek and consume health

information. A number of surveys and research shows that the number of people who

access the internet to search for health information has rapidly increased (Interactive,

2008; McMullan, 2006), and looking for health information is one of the main reasons for

16

using the internet (Health;, 1999; Fox, 2006b; Fox and Fallows, 2003a; Nammacher and

Schmitt, 1998; Baker et al., 2003; Morahan-Martin, 2004; Eysenbach and Kohler, 2003).

People search for health information online to educate themselves and have a complete

understanding of their health issues, along with making more informed decisions about

treatment choices, medications and lifestyle choices (Lenhart, 2003; Fox, 2006b;

Sciamanna et al., 2002). Moreover it has been accepted that health information on the

internet leads to more informed and “empowered” patients (Akerkar and Bichile, 2004a;

Fox; and Fallows;, 2003; Murray et al., 2003a; Henwood et al., 2003; Fox, 2006b; Laing,

Hogg and Winkelman, 2004; KJ;, 1999), and it has an influence over their decision-

making and health behaviour (Dutta-Bergman, 2003a; Rains, 2007; Willard and

Ginsburg, 2008; Murray et al., 2003b).

The rapid growth in consuming health information from the internet has been

accompanied by an exponential increase in the number of websites that provide health

information. It is estimated that the number of health websites on the internet is more

than 100,000(Cline and Haynes, 2001a; Dearness and Tomlin, 2001; Fox, 2005).

Evidence shows that women use the internet to seek health information more than men

(72%:53% respectively), and there is a positive correlation between levels of education

and seeking health information from the internet (Eng et al., 1998; Commerce, 1998;

Rice, 2006; Hesse et al., 2005; Voelker, 1998; Skinner, Biscope and Poland, 2003;

Lenhart, 2000; Tu and Hargraves, 2003; Fox and Fallows, 2003c).

In the UK there has been rapid and wide spread use of the internet. The number of

households with internet access increased from 14.26 million in 2006 to 18.31 million

households with internet access in 2009, which represent 70% of the UK population

(Office for National Statistics, 2009). Like other countries, levels of education is one of

the important determinates for using and seeking information from the internet in the UK

(Office for National Statistics, 2009; Bucy, 2000). According to the office for National

Statistics (2009), 42% of people in UK use internet to search for health information

(Office for National Statistics, 2009), and according to a study conducted in the UK by

17

Brown and Williams (2003), 90% (n=1,100) of respondents reported that they use the

internet to seek health information.

Reasons to use the internet as a health information source

One of the most important reasons behind the rapid increase in internet use as a health

information source is the advantage that the internet has over other traditional health

information sources (Bush, McAuley and Pecaitis, 2000; Khechine, Pascot and Premont,

2008). These advantages make people increasingly turn to the internet to seek health

information and rate the internet as their favourite health information source (Willard and

Ginsburg, 2008; Kummervold et al., 2008; Peterson and Fretz, 2003; Eysenbach, 2003;

Fox and Fallows, 2003a). They also rate the information they get from the internet as

parallel to the information they acquire from their doctor (Diaz, Friedmann and Moulton,

2000). The doctor is no longer their primary health information source (Ellis and

Thomson, 2003)

The internet has been referred as the largest health information source in the world

(Akerkar and Bichile, 2004b; Hussain, Agyeman and Das Carlo, 2004). Because of the

ease of internet access, (Fox, 2006b; NetRatings, 2005; Dumitru et al., 2007; Rideout;,

2001; Neelapala et al., 2008), people have continued and rapid access to the information

source (Hussain, Agyeman and Das Carlo, 2004). All of these make the process of

seeking health information easier and more convenient than ever before (Park, Chung and

Yoo, 2009; Cline and Haynes, 2001a; Fox and Fallows, 2003a; Morahan-Martin, 2004).

Furthermore, the internet is an interactive learning environment, and its content can be

presented in a number of ways (e.g. text, sound, video ...etc). Its interactive nature

attracts patients because it allows them to have more control and search for information

that meets their needs (Pereira and Bruera, 1998; McMillan, 1999; SPICH, 1999).

Information seeking occurs in an environment in which a variety of sources are available

(Brashers, Goldsmith and Hsieh, 2002). The internet contains a variety of information

sources and is the place in which traditional information can meet (Park, Chung and Yoo,

18

2009), hence allowing patients to seek information from more than one source and from

more than one perspective (McMillan, 1999; Health;, 1999).

Embarrassment has been referred to as a problem when seeking health information from

the doctor, especially when referring to sensitive topics (Broom, 2005; Gray et al., 2005;

Berger, Wagner and Baker, 2005; Ziebland et al., 2004; Fox and Raine, 2001).

Anonymity is one of the key internet features which allows patients to browse without

being identified (Napoli, 2000), and avoiding the consequences of social interaction

(Christopherson, 2007; Homsi and Kaplan-Myrth, 2005). As such anonymity is another

advantage of the internet which makes patients turn to it for seeking health information

(Joinson, 2001; Broom, 2005; MORI, 2001; Plantin and Daneback, 2009; Fox and Raine,

2001).

Despite the popularity of the internet as a health information source (Health;, 1999; Fox,

2006b; Fox and Fallows, 2003a; Nammacher and Schmitt, 1998; Baker et al., 2003;

Morahan-Martin, 2004; Eysenbach and Kohler, 2003), and the advantages that the

internet has over other health information sources (Bush, McAuley and Pecaitis, 2000;

Khechine, Pascot and Premont, 2008), the quality of online information has long been a

debate issue (Dutta, Pfister and Kosmoski, 2010), and a number of authors demonstrated

that quality of online information is utmost important issue (Liu and Huang, 2005;

Parker, 2004; Rieh, 2002). This concern has been linked to the popularity of internet as

health information source (Dutta, Pfister and Kosmoski, 2010), as well as with the

process of publishing information on the internet (Hawkins, 1999; Liu and Huang, 2005).

However, the quality of online health information was always a questionable and tension

issue (Berland et al., 2001; Morahan-Martin, 2004; Briggs et al., 2002); while some sites

provide high quality health information, others provide information that is factual but

incomplete (Gustafson et al., 1999; Widman and Tong, 1997; Jones, 1999)

19

Health Information Quality

Definitions and dimensions of health information quality

Information quality simply means “fitness for use” (Wang and Strong, 1996; Tayi and

Ballou, 1998; Stvilia, Mon and Yi, 2009). A number of authors demonstrated that

information quality is a multi-dimensional concept (Klein, 2001; Knight and Burn, 2005;

Stvilia, Mon and Yi, 2009; Lee et al., 2002), and depending on different philosophical

points of view, a variety of characteristics and attributes are involved in this concept

(Knight and Burn, 2005; Parker et al., 2006).

The purpose of the framework is to identify different questions and relevant variables

which need to be answered in order to aid users (Porter, 1991). Despite that there are a

number of models and frameworks for dimensions and attributes for information quality

(Klein, 2002; Eppler and Muenzenmayer, 2002; Kahn, Strong and Wang, 2002; Knight

and Burn, 2005; Klein, 2001). Defining the quality of information using frameworks is

not enough because information quality is also dependent on the context in which it will

be used (Shankar and Watts, 2003). The quality of information needs to be measured and

assessed within its generation context (Shanks and Corbitt, 1999), and its intended use

(Katerattanakul and Siau, 1999; Parker et al., 2006). Completeness and the capacity to

understand is one of the common attributes of information quality between different

quality frameworks (Knight and Burn, 2005).

Regarding the quality of health information, there have been a number of efforts for

identifying user needs and improving health information quality. This includes

developing a number of frameworks and checklists for health information quality

(Eysenbach et al., 2002; Eysenbach and Kohler, 2002b; Fox and Rainie, 2006). Moreover

a number of instruments and quality criteria have been developed especially for assessing

the quality of online health information (COULTER et al., 2006; Anderson, McKemmish

and Manaszewicz, 2003); such as the Health on the Net Foundation‟s principles (The

Health On the Net Foundation, 2010), the European Commission‟s quality criteria for

health related websites (European Commission, 2002; Wilson, 2002), and criteria

developed by the International Patient Decision Aid Standards (IPDAS) (Elwyn et al.,

20

2006). The variation in quality criteria reflects the complexity of health information, and

the wide variation in consumer needs (Civan and Pratt, 2006). Despite all of these

frameworks and quality criteria for assessing health information quality, Eysenbach and

his colleague (2002) who reviewed 79 empirical studies assessing online consumer health

information found that Accuracy, Completeness, Readability, Design and Disclosures

were the most used and cited quality criteria.

Accuracy of information refer to the extent to which the information provided is valid

and correct (Hirsh, 1999; Anderson, McKemmish and Manaszewicz, 2003). The accuracy

of information also refers to research methodologies, reliability, validity of measures,

along with the actual evaluation of evidence used to make an argument (Dutta, 2001).

Completeness refer to the extent to which a document contains the important information

pieces regarding a certain topic, and its main focus being on the content of the

information (Bruce, 1994). Readability means to what extent the information is easy to

read, and understandable by people. Design refers to the overall layout of the website,

including the text, photos, videos.. etc (Ambre et al., 1998; Hirsh, 1999). And finally

Disclosure refers to a statement of the sources of the information provided and date of

publication, along with information about the individual or organization which provides

this information on the site, and contact details (Anderson, McKemmish and

Manaszewicz, 2003; European Commission, 2002).

Addressing Patients’ needs in assessing the quality of online health information

According to Strong and Wang (1997), quality of information can‟t be assessed

independently from its intended audience and needs to be assessed based on their needs

and from their point of view (Knight and Burn, 2005). In spite of large number of studies

which assess the quality of online health information (Eysenbach et al., 2002; Gagliardi

and Jadad, 2002), most of these studies assess the quality of health information from the

expert‟s perspective and neglect consumer‟s needs (Potts and Wyatt, 2002; Bliemel and

Hassanein, 2007). Bliemel and Hassanein (2007) demonstrated that there is a need for

more research which assesses the quality of online information from the consumers‟ point

of view (Bliemel and Hassanein, 2007).

21

People‟s evaluation of information quality is a dynamic process. And information may

have different meanings for the same user at different points of time or even at different

stages of the information search process (Tang and Solomon, 1998). Hence different

situation trigger criteria to be used by people to evaluate information, therefore

identifying the relevance of criteria for evaluating information needs to start by the user

of that information (Tang and Solomon, 1998; Barry, 1994). Salaun and Flores (2001)

define high information quality as:

“Information which satisfies criteria of appreciation specified by the user, together with

a certain standard of requirement”

(Salaun and Flores, 2001)

Eysenbach et al (2002) found that accuracy, completeness and readability are the most

cited content quality criteria by studies which assess the quality of online health

information (Eysenbach et al., 2002; Stvilia, Mon and Yi, 2009). Completeness of

information is at the heart of communicating science to the public (Pellechia, 1997), and

is one of the critical elements for information quality (Dutta-Bergman, 2003b; Dutta,

2001). Eysenbach et al (2002) demonstrated that despite there being a number of studies

which suggest there is a problem with completeness and the scope of online health

information, accuracy is the most commonly assessed dimension of online health

information quality. Gustafson et al (1999) demonstrate that the evaluator needs to

consider more than the accuracy of information when assessing its quality. According to

the web watcher study “How Do People Evaluate a Web Site‟s Credibility?”(n=2440),

users rate completeness of information as the most important when evaluating online

health information (Fogg et al., 2002), and in another study conducted by (Stvilia, Mon

and Yi, 2009) patients rank information completeness as the second most important

dimension for online health information quality after accuracy.

On the other hand information needs to be readable and easy to understand in order to

meet the minimum quality requirements (Kušec, 2004). Ideally all health information

22

which is directed to a certain audience, needs to consider the reading ability of that

audience (Gemoets et al., 2004). Readability is a common quality indicator and should be

included in health information regardless of its type and content (Forsythe, 1992).If

patients can`t read and understand the information they find online it will be useless

(Eysenbach et al., 2002; Risk and Petersen, 2002), and must be readable in order to avoid

any misinterpretation that may occur in the future (Pothier and Pothier, 2009). Because of

the high number of people who use the internet to get health related information, it is

important for online patient orientated material to be easy to understand (Greywoode et

al., 2009).

Information Completeness

Dutta (2001) demonstrated that there are two attributes involved in message

completeness; the message itself and the audience of that message. Taking the message

part, complete scientific information is information that contains all basic categories

which form the building blocks for that piece of information, and the absence of any of

these categories will make the information incomplete (Dutta, 2001; Korpac et al., 1994;

Toulmin, 1958). In the same way, complete health information is the information that

contains all necessary elements. The form of these elements is varied; it may be a logical

link of ideas that make the claim, or scientific research evidence. In other words the

necessary elements for health information completeness is anything which is a central

and important part of the message or claim being made in that piece of information

(Dutta, 2001).

On the other hand, users are key and central to information behaviour (Dutta, 2001), and

they can determine what is relevant and central to their information needs (Dutta, 2001;

Li and Bukovac, 1999). Furthermore strong arguments are arguments which contain the

central relevant element as defined by the user of these arguments (Petty and Cacioppo,

1990). In the relevance approach Cuadra and Katter (1993) demonstrated that the

situational status of users have an impact over their information needs (CUADRA and

KATTER, 1993). In more simple terms the information needs of people (this includes the

amount, content and the quality of the information) is a result of a specific situation

23

which drives information search (Dutta, 2001). For example the information needs of

patient who is trying to diagnose himself based on available information is different from

the information needs of a patient who is newly diagnosed with a specific disease

(Dervin, 1983).

A number of authors demonstrated that health information completeness is the most

important criteria in decision making (Dutta, 2001; Dutta-Bergman, 2003b; Eysenbach et

al., 2002; Dutta-Bergman, 2003a). This is not only because incomplete information can

leave out the important and relevant key information needed by the consumer for

effective decision making (Pellechia, 1997; Dutta, 2001; Dutta-Bergman, 2003b;

Eysenbach et al., 2002), but also because certain information parts serve as a decision

making tool (Dutta, 2001). Complete health information can be defined as:

“The extent to which the dissection of a health topic is comprehensive, balanced and

adequate in its portrayal”

(Dutta, 2001; Ambre et al., 1998)

Completeness of health information is important for a number of reasons; firstly,

incomplete health information will make it difficult to understand the entirety of the

health problem and its solution. Therefore complete information which provides all

relevant and important detail is essential to adequately understand the topic (Dutta, 2001;

Dutta-Bergman, 2003b). Secondly, the processing of the health message in different

situations will be affected by the degree of information completeness; hence complete

information is important for effective decision making. Finally, complete information

will increase user confidence (Dutta, 2001).

Online health information completeness

The internet is a huge source of health information, and there are hundreds of thousands

of sites which provide this (Cline and Haynes, 2001a; Dearness and Tomlin, 2001; Fox,

2005). Some authors describe the internet as the biggest medical library in the world

(Akerkar and Bichile, 2004b; Hussain, Agyeman and Das Carlo, 2004). However,

24

evidence shows that the quality of online health information varies (Berland et al., 2001;

Morahan-Martin, 2004; Briggs et al., 2002); while some sites provide high quality health

information, others provide information that is factual but incomplete (Gustafson et al.,

1999; Widman and Tong, 1997; Jones, 1999). Gustfson et al (1999) demonstrate that

some health websites show only one side of the picture, either because it is written by a

health care professional who assumes that the user has background knowledge, or

because the site owner has a financial interest in the information they provide. Either

way customers are left with limited and incomplete information.

Interestingly some of authors have demonstrated that complete health information is not

an important issue (Risk and Petersen, 2002; Dutta, Pfister and Kosmoski, 2010). They

state that because there are a large number of health websites, and people are more likely

to visit more than one when doing their research. Hence a given website need not provide

all the necessary information, because people will get the information they need by

visiting multiple websites (Risk and Petersen, 2002). Yet (Crespo, 2004) demonstrated

that a growing body of research shows that few users seek information from multiple

sources or compare sources when seeking health information from the internet, which

again suggests users are victims of incomplete information (Dutta, 2001).

A growing body of literature shows that online health information is incomplete (Berland

et al., 2001; Impicciatore et al., 1997; Silberg, Lundberg and Musacchio, 1997; Mathur et

al., 2005; Morr et al., 2010; Labovitch, Bozic and Hansen, 2006).For instance Eysenbach

et al. (2002) found that 90% of studies which assess the completeness of online health

information found that this information is incomplete, moreover a study conducted by

Berland et al (2001) shows there is a significant shortage in completeness of online health

information, especially in covering topics which are important to patients; for example he

found that 45% of English websites and 22% of Spanish websites insufficiently covered

the necessary health information for patients, and according to the same study patients

who seek health information from the internet find it difficult to get complete health

information.

25

Regarding Multiple Sclerosis websites, a number of studies have found that there is a

problem with the quality of online MS patients‟ information (Harland and Bath, 2007;

Hatzakis et al., 2006; Lejbkowicz et al., 2010), especially in the completeness of this

information (Harland and Bath, 2007). Few websites go beyond disease definition and

self-management information (Hatzakis et al., 2006).

Readability of Health Information

Literacy can be defined as:

“An individual‟s ability to read, writes, and speaks in English and computes and solves

problems at levels of proficiency necessary to function on the job and in society, to

achieve one‟s goals, and to develop one‟s knowledge and potential”

(The National Literacy Act, 2007)

Literacy and numeracy are the basic and fundamental skills every adult should have in

order to effectively function and progress in his work and his community (Williams et al.,

2003; John, Stephen and Andrew, 2001). As part of the “Skills for Life” strategy in the

UK, a framework of national standards for literacy and numeracy was developed

(Williams et al., 2003). This framework (Table 1) represents the skills that an individual

may have at different levels (Pothier and Pothier, 2009).

26

Low literacy is an international problem (Graber, Roller and Kaeble, 1999b). Evidence

shows that low literacy is a common problem in the UK (DfEE, 1999; Williams et al.,

2003). A national survey revealed that 16% of the UK population (around 5.2 million)

have low literacy levels, (Williams et al., 2003). According to a study by (Lee, 1999) the

individual levels of literacy are 3 to5 grades below the highest school completed. The

term “Health literacy” appeared thirty years ago (Ratzan, 2001; Coulter and Ellins,

2006), It can be defined as:

Level Literacy (reading) An adult classified at this level...

Equivalent to...

Entry level

1

- Understands short texts with repeated language patterns on

familiar topics

- Can obtain information from commons signs and symbols

National

curriculum level 1

Entry level

2

- Understands short straightforward texts on familiar topics

- Can obtain information from short documents, familiar

sources and signs and symbols

Level expected of

a seven-year-old

(national

curriculum level

2)

Entry level

3

- Understands short straightforward texts on familiar topics

accurately and independently

- Can obtain information from everyday sources

Level expected of

an 11-year-old

(national

curriculum levels

3-4)

Level 1 - Understands short straightforward texts of varying length on

a variety of topics accurately and independently

- Can obtain information from different sources

GCSE grades D-G

(national

curriculum level

5)

Level 2 - Understands a range of texts of varying complexity

accurately and independently

- Can obtain information of varying length and detail from

different sources

GCSE grades A-C

(national

curriculum levels

6-8)

Table 1: Framework of National Standards for Literacy and Numeracy (National Literacy

Trust 2010)

27

“The degree to which individuals have the capacity to obtain, process, and understand

basic health information and services needed to make appropriate health decisions”

(The Joint Commission, 2007b)

Evidence shows that patients with limited health literacy will have difficulty

understanding information about their medical condition, are less likely to adhere to their

prescribed treatment and are at high risk of misinterpreting medical instructions

(Williams et al., 1998b; Williams et al., 1998a; Nielsen-Bohlman, Panzer and Kindig,

2004; Dreger and Tremback, 2002). Moreover health literacy has an impact over heath

information seeking behaviour; and over the process of patients‟ access to key

information about their disease and health care treatment (Ta-Min, 2006b; Rudd,

Moeykens and Colton, 1999); for instance more literate patients are more likely to seek

health information from more than one sources (Ishikawa, Takeuchi and Yano, 2008),

while people with limited health literacy are less likely to seek health information from

written sources (John et al., 1995; von Wagner et al., 2009; Rudd et al., 2007).

Limited health literacy is not only a barrier to seeking health information but can also be

a problem once the information is accessed (von Wagner et al., 2009; Wolf et al., 2007).

The close link between literacy levels and individual health related knowledge and

attitude (von Wagner et al., 2009), as well as the link between patients‟ understanding of

the health information and its impact over the decision making process (Brown et al.,

2010; Zafar et al., 2009; Jansen, Otten and Stiggelbout, 2004), all mean health literacy

has a key role and impact over treatment compliance and overall health care outcomes

(Nielsen-Bohlman, Panzer and Kindig, 2004; Williams et al., 1995; Berkman et al., 2004;

U.S. Department of Health and Human Services, 2002; French and Larrabee, 1999).

The individual ability to read and understand written instruction is not the only

importance and impact of literacy level (Berkman et al., 2004). With increased reliance of

health care systems on written information for improving health and guiding medical

care, along with increases in the level of complexity of this information, low literacy has

been recognized as barrier for individuals to effectively function in the health care

environment (Berkman et al., 2004) . Moreover low literacy has been perceived as a

28

barrier to promoting clinical effectiveness (Colledge et al., 2008). This ultimately makes

readability of health information a central factor which needs to be investigated to

educate patients(Kohlmaier, 2008).

Readability is one of the important components of health literacy (Walsh and Volsko,

2008). It is importance to match written health education material with patients‟ reading

abilities (Ta-Min, 2006b; Coey, 1996). It has been estimated that 52% of the UK

population have a reading level between 9 and 11 years of age (Wilson et al., 1998),

which, according to Nicholls et al. (2009) corresponds the with 5- 8 score based on

SMOG (Nicholls et al., 2009). A number of authors demonstrate that that there is a

mismatch between patients‟ levels of literacy and the readability of health information

material (McCray, 2005; French and Larrabee, 1999; Estrada et al., 2000; Forbis and

Aligne, 2002; Paasche-Orlow, Taylor and Brancati, 2003). This creates a barrier for

people to understand health information (Gilmour, 2007). Although researchers suggest

that any written health information material needs to be written below 6th

grade level in

order to be understood by patients (Anon, 1999; Gottlieb and Rogers, 2004; Walsh and

Volsko, 2008), the majority is written at a reading level above this (Graber, Roller and

Kaeble, 1999a; Graber, D'Alessandro and Johnson-West, 2002; Berland et al., 2001;

Wilson et al., 1998), This makes it difficult for patients to understand (Anon, 1999;

Gottlieb and Rogers, 2004). Unfortunately there is no baseline data about the reading

level within which health information need be written in UK (Ibrahim et al., 2008).

The readability of Online Health Information

There are no age or gender limits for people seeking health information from the internet

(Fox and Fallows, 2003b). People vary in their level of literacy, and reading abilities

(Graber, Roller and Kaeble, 1999a). For instance in the UK while 45% of the 20-24 age

group achieved level 1 literacy, the age group of 35-34 and 45-54 achieved 40% and 36%

respectively (Williams et al., 2003).

29

Readability of health information on certain websites is important because information

which is difficult to read will create a barrier for consuming online health information

(Eysenbach et al., 2002; Gilmour, 2007).

A number of studies which assess the readability of online health information found that

information written at high levels beyond the reading abilities of patients (Graber, Roller

and Kaeble, 1999a; Graber, D'Alessandro and Johnson-West, 2002; Berland et al., 2001).

Patients reported that the information was complicated and difficult to read (Berland et

al., 2001; Murero, D'Ancona and Karamanoukian, 2001). For instance one study which

assesses the quality of NHS direct online found that the readability of information on this

site is written at a level higher than recommended by researchers (Webster and Williams,

2005). Moreover Eysenbach et al (2002) demonstrated that despite of different tools and

formulas used by studies in assessing the readability of online health information these

studies find out the reading level of online information is too high.

Information Needs of Patients with MS

A number of studies which assess the needs of MS patients identified certain needs for

those patients (Kersten et al., 2000; Forbes et al., 2003). Fabros et al (2003) identified

four main categories for MS patients. These include psychological needs, social needs,

physical needs and information needs. Information has been identified as a priority for

MS patients (Slade, Tennant and Ford, 2003). Matson and Brook (1977) developed a

framework for explaining the adjustment process people go through after they are

diagnosed with MS. They concluded that the adjustment process takes place in four

stages; denial, resistance, affirmation and integration. They found that information plays

a key role in the adjustment process, and most adjustment occurs in the denial stage in

which patients seek as much information as they can from a variety of channels trying. In

the resistance stage patients try to seek information about the prevalence of MS and the

available cure options. In the affirmation stage patients start telling other people about the

disease and reach the integration stage when they fully accept the condition (Matson and

Brooks, 1977).

30

MS patients‟ need for information has been well documented (Matson and Brooks, 1977;

Daumer et al., 2007; Forbes, While and Taylor, 2007; Harland and Bath, 2007; Hatzakis

et al., 2006; Hay et al., 2008; Lejbkowicz et al., 2010; Matti et al., 2010; Sadovnick,

2009; Somerset, Sharp and Campbell, 2002; Chiovetti, 2006b; Slade, Tennant and Ford,

2003). This information need has been identified as a universal phenomena (Kersten et

al., 2000). MS is characterized by a relapsing and remitting nature of MS, hence the

clinical course, sign and symptom and degree of disability associated with MS is highly

variable (Heesen et al., 2007; Vickrey et al., 2000). Evidences shows that the information

needs of patients with MS vary according to the different stages of the disease; for

instance newly diagnosed patients have more and different information needs (Forbes,

While and Taylor, 2007; Slade, Tennant and Ford, 2003; Hepworth, Harrison and James,

2003). Patients start to seek information before the diagnosis and continue after the

diagnosis (Box, Hepworth and Harrison, 2003; Hepworth, Harrison and James, 2003).

MS patients seek health information from sources such as doctors, books, leaflets and

internet (Matti et al., 2010; Bishop et al., 2009). Research has identified a gap between

the information needed by MS patients and the information provided and available to

them (Matti et al., 2010; Harland and Bath, 2007; Hay et al., 2008), Moreover MS

patients have reported that this information insufficient and does not satisfy their desires;

(Matti et al., 2010)

The internet has been identified as the most important health information source for MS

patients after doctors (Bishop et al., 2009; Hay et al., 2008). But according to Hepworth

et al (2003) only 5% of UK patients use the internet to get information about their

disease. Harland and Bath (2007) state this may indicate a defect in the quality of online

MS information or inadequacy in the relevance of online information. According to

Bishop et al (2009) understanding the source which provides information for MS patients

is important to ensure that this source provides high quality information. Moreover they

concluded that further research is required to assess the quality of sources which provide

information for MS patients. Matti et al (2010) found a deficit between the information

being provided by different MS information sources and the amount of information

needed by MS patients. A limited number of studies assessed the quality of online MS

31

information and concluded on similar grounds (Lejbkowicz et al., 2010; Hay et al., 2008;

Harland and Bath, 2007; Hatzakis et al., 2006; Hoffmann et al., 2009).

In the UK a study conducted by Robinson (1996) to identify the needs of MS patients,

discovered that that patients have a problem in finding and accessing relevant and high

quality information regarding their disease. He concludes that the information needs of

UK patients vary during different stages of the disease (Robinson, Hunter and Neilson,

1996). According to a more recent study which specifically investigated the information

needs of MS patients in the UK, Hepworth, Harrison and James (2003) demonstrated that

despite of the importance of information for MS patients, the information needs of those

patients was not adequately met regarding coping strategies. 43% of patients with MS in

the reported that they didn‟t receive any information after the diagnosis, and 50% refer to

lack of information as a negative experience of the diagnostic process.. Hepworth also

emphasised the importance of information which is sensitive to the needs of MS patients.

Hepworth and his colleague identified 2 main categories of information needs for MS

patients in UK. These include; “Information about MS” and “information that helps

people with MS interact with the world around them”. The information about MS

category includes information about MS in general (such as a general explanation of the

disease, symptoms, drugs and treatments ..etc) and specific information about MS (such

as the disease type prognosis, the characteristics of the symptoms ...etc).

The other category “Information that helps people with MS interact with the world

around them” includes; different information about the service provider, information

about MS patients‟ employment and the implication of retirement, information about aids

and application. This category also involves information for families of patients with MS,

along with information about leisure activities. Hepwoth emphasises that in order to

satisfy the information needs of MS patients, it is important that information on all topics

identified in his study are made available to patients (Hepworth, Harrison and James,

2003).

32

Family members and carers are important part of MS patients live, because they plays an

important role in their life; this role might be at the physical level by providing any

required physical assistance and supervision required for those patients, the role of family

members may exceed this role to the emotional and social role by providing emotional

and social support for MS patients from one side and by taking the responsibility of

family care from the other side (The National Collaborating Centre for Chronic

Conditions, 2004). In Hepworth et al (2003) survey 46.4% of respondents rated

information for family/friends as important, and (15.4%) find it difficult to get this

information

Summary and Gaps in Research

According to the literature review, 80% of UK patients seek information about their

condition. The importance of providing patients with information has been well

established, and the impact of this information over disease treatment, coping with the

disease, and overall health outcomes has been well identified, especially for patients with

chronic conditions ( such as MS). Providing MS patients with information has been

identified as a way of dramatically changing the way in which those patients experience

their disease.

Health Communication is the key to providing patients with health information and it is

increasingly recognized as a way of improving public health. Information is the basic

building block of health communication and identifying the needs of the audience is

important for optimizing content.

The internet has increasingly become a favourable source for patients to seek health

information and the numbers of patients who access the internet for this purpose

increases every year. This increase has been accompanied by an increase the number of

health information websites. Internet features such as availability, anonymity, privacy

and interactivity have been referred as the main reasons behind its appeal. However,

limited education has been identified as the main determinant for seeking health

information from the internet.

33

Information quality is a concept which involves a variety of dimensions, and there are a

number of frameworks to explain these. A number of authors have demonstrated that the

quality of information needs to be measured within its generation context, and its

intended use. There are a number of frameworks and quality criteria which have been

developed to assess the quality of health information and, each one of has different

attributes. According to the literature review there are common criteria for assessing the

quality of health information. These include Accuracy, Completeness, Readability,

Design and Disclosures.

According to the literature review most studies which assess the quality of online health

information assesses it from the expert perspective and neglect patient‟s needs. The

quality of health information need to be assessed based on patients‟ needs and from their

point of view. Eysenbach et al (2002) found that accuracy, completeness and readability

are the most important quality criteria for health information, and users rate completeness

as most important to them.

Complete health information has been recognized as key for effective decision making

because it will lead to better and more comprehensive understanding. Complete health

information is not only the information which contains all the required elements, but also

the information which addresses the patients‟ needs. The literature review shows that

there is a problem in the completeness of available online health information, and there is

a shortage in the research which assesses the completeness of this information. The

literature review also shows that MS information available online is incomplete and does

not meet the information needs of MS patients.

Patient levels of literacy and reading abilities have been recognized as important factors

for patients to effectively function in their health environment. It has also been

recognized that literacy and reading ability levels have an impact on patients‟

understanding of their own condition, following the prescribed treatment and the

treatment outcome.

34

In the UK it is estimated that 52% of the population have the reading ability of entry level

2. Hence health information which is directed to patients (including the internet) needs to

be written below this reading level. Yet there is a gap between information being

provided for patients and their reading abilities, and most available online health

information is written at reading levels beyond their reading abilities.

Multiple Sclerosis patients have different needs and information needs is one of the most

important. The research shows that the information needs of MS patients vary during

different stages of the disease; for instance newly diagnosed patients need significant

amounts of information about the disease its diagnosis and treatment. Although patients

seek information from a variety of sources, patients reported that this information does

not meet their desire and there is a gap between.

Research Question

What is the completeness and readability of multiple sclerosis patients‟ information

available on UK websites?

Method

The purpose of this research is to identify whether the MS information available on UK

websites is complete, easy to understand and read by patients, and whether it forms a

useful educational material. The research will try to focus on addressing patients‟ needs

when investigating these two variables (the completeness and readability). There are four

main sections in this method chapter; study design, websites sample, data collection, and

data analysis.

Study Design

This research will use “Quantitative Descriptive evaluation” as its study design. The

research will evaluate two variables from online MS information; completeness and

readability.

The research process aims to improve and contribute to the scientific body of knowledge

by examining and developing explanatory concepts, and explaining and describing

35

certain phenomena (Bowling, 2009). Health system research is a multidisciplinary subset

of research (Pope, 1992), which aims to improve public health by improving and

enhancing the efficiency and effectiveness of health care systems (Varkevisser,

Pathmanathan and Brownlee, 1991; Varkevisser, Mwaluko and Le Grand, 2001). The

research design is a central part of any research (Morse and Richards, 2002), and requires

the researcher to have the insight to look at his research at different levels (Vaus, 2003).

The research design is the overall logical structure of the research which serves as and

produces a work plan (Vaus, 2003). Moreover, it provides a framework for data

collection and data analysis (Bryman, 2008). The design of any research needs to be

shaped to answer the research question (Morse and Richards, 2002) and must also ensure

that the evidence obtained enables the researcher to answer the initial question as

unambiguously as possible (Vaus, 2003).

Research can be classified in terms of qualitative and quantitative paradigms (Bowling,

2009; Bryman, 2008; Miller and Salkind, 2002). The qualitative approach uses a system

of inquiries to inform the researcher‟s understanding of a certain social or cultural

phenomena, and provides in-depth understanding of human behaviour, and the factors

that influence this behaviour (Merriam, 1998; Wilson, 2007; Johnson, Christensen and

Christensen, 2010). There are general and specific factors that make a qualitative

approach unsuitable for this research. From the general standpoint the qualitative

approach tends to depart from the main objectives of the research (Cassell and Symon,

1994). The qualitative approach is highly subjective and depending on the characteristics

of the researcher different conclusions can be drawn based on the same information

(Johnson, Christensen and Christensen, 2010). This makes it difficult to establish

reliability and validity (Key, 2002). Moreover unlike quantitative research, in qualitative

research it is difficult to explain and understand the differences between the quality and

quantity of the information obtained, and it requires a high level of experience (Matveev,

2002).

The specific reason for not using qualitative approach is directly related to the purpose of

this research and research question. The purpose of this research is to assess the

36

completeness and readability of MS patient‟s information available on UK websites;

These are quantitative variables (Johnson, Christensen and Christensen, 2010). The

qualitative approach assesses and identifies intangible factors and variables of an issue

(such as social norms, socioeconomic status, gender roles …etc), and tends to examine

the perception, experience, meaning and emotions of a problem or a phenomena (Mack et

al., 2004; Bowling, 2009; Frankel and Devers, 2000; Johnson, Christensen and

Christensen, 2010).This makes the qualitative approach unsuitable for the types of

variables involved in this research. According to (Kohlmaier, 2008) the qualitative

approach will not give quantitative information about the health websites accessed by

health consumers.

The quantitative approach simply allows one to observe and measure scientifically

(Creswell, 1994). It is a systematic empirical examination of numerical properties of a

phenomena or problem, and uses statistical methods to understand and investigate

(Frangos, 2009; Sukamolson, 2005). The quantitative approach is a more suitable method

for this research for general and specific reasons. From a general standpoint, unlike the

qualitative approach, for the quantitative approach the researcher needs no experience or

specific training (Mack et al., 2004). Furthermore, quantitative research is very specific in

stating and examining the research problem (Frankfort-Nachmias and Nachmias, 1994),

and it can draw more valid and objective conclusions by following the research objective.

Moreover, quantitative research can collect more valid and reliable data and can draw

more accurate conclusions based on this data because of the strict control procedure the

researcher need to follow when collecting the data (Matveev, 2002).

The specific reasons for using this approach were directly inspired by the research

question and the objectives of this research. The research question in this project is

“What is the completeness and readability of multiple sclerosis patients‟ information

available on UK websites?” A number of authors demonstrated that when the research

question is “what is the reality?” and the reality in this question is objective, the research

clearly points towards the quantitative paradigm (Sukamolson, 2005; Remenyi, 2007;

Polit and Beck, 2004). Moreover quantitative research aims to understand the extent to

37

which a given set of quality properties and attributes fulfil a set of requirements regarding

specific audiences (Olsina and Rossi, 2000), which directly relate and fit with the

research question and purpose of this research. According to Bowling (2009) the

quantitative approach is appropriate when assessing the quality of online health

information. Eysenbach et al (2002) demonstrated that most studies which assess the

quality of health information use the quantitative approach.

Research can be further classified as exploratory research, explanatory research,

correlation research and descriptive research (Sukamolson, 2005). Descriptive research

simply means describing the data and characteristics of the problem or phenomena under

study (Jonassen, 2004; Berg and Latin, 2007). Descriptive design is one of the non-

experimental research strategies which use scientific processes that help in understanding

and defining the existence and characteristics of certain phenomena or a problem

(Heppner, Wampold and Kivlighan, 2007). The number of variables which can be

employed in the descriptive approach is one of its unique features. Although the

descriptive approach is like other approaches in that it can use multiple variables in the

analysis process, it is unique because it requires one variable only (Jonassen, 2004).

According to Jonasse (2004), the appropriate approach for carrying out an accurate

assessment of the topic at hand, is ultimately determined by the type of question being

asked by the researcher. The descriptive approach involves the research question, design

and the data analysis that will be employed in studying certain phenomena (Jonassen,

2004; Sim and Wright, 2000). This approach is mainly concerned with answering the

question “what is” (Berg and Latin, 2007; Jonassen, 2004), which is well-suited to the

research question of this study. Moreover, descriptive research can be used to collect

information about phenomena in its current state and to describe what exists regarding to

variables in certain situation (Key, 2002), which make descriptive approach appropriate

to use because this research aim to collect information about the readability and

completeness of the MS patients information which currently available on UK websites .

As well as that, Eysenbach et al (2002) found after a review of 71 pieces of assessing the

38

quality of online health information that all these studies use quantitative descriptive

designs based on scoring instruments.

Evaluation research can be defined as:

“Systematic determination of merit, worth, and significance of something or someone

using criteria against a set of standards”

(Miller, Vandome and McBrewster, 2009)

Evaluation is a research approach (Clarke and Dawson, 1999), which is different from

others because of its intended objectives rather than the nature of its design or the

execution method (Rossi and Freeman, 1993; Berk and Rossi, 1990; Powell, 2006). This

approach has been chosen in this research because unlike other approaches, it can

generate more usable, accurate and valid results through its systematic scientific method

(Miller and Salkind, 2002).

Evaluation is appropriate in this research because it fits with its purpose, which is to

gather information about the content (completeness) and readability of MS related

information on UK websites. For instance the evaluation approach is seen as an

information gathering process, that aims to provide objective knowledge about the

performance of a certain service when measured against pre-identified criteria (Leger and

Walsworth-Bell, 1999) . Evaluation research applies scientific methods in gathering and

analyzing data about the content, structure or the outcomes of a programme project or

system (Clarke and Dawson, 1999).

The evaluation approach also fits with the nature of websites, because websites are

considered a type of information system (Berendt and Spiliopoulou, 2000), and system

evaluation involves collecting information about different aspects of certain systems used

by specific people with the aim of reducing uncertainties, and improving effectiveness

and decision making (Clarke and Dawson, 1999).

39

Website Sample

A non-random purposive sampling design was used in this study. The Google Search

engine was used to locate MS websites and to formulate the sample, by entering different

keywords. Only websites from the first page were considered and assessed against the

inclusion criteria. A total number of 27 websites form the last sample were used in this

research.

Targeted population refers to the entire set of elements on which the researcher will aim

to base a conclusion (Burns and Grove, 2005; Kazerooni, 2001). Elements are the unit of

analysis or subjects on which the measurement will be carried out. For instance this could

be people, behaviours, events … etc (Cooper and Schindler, 2003; Burns and Grove,

2005). Census research is research which collects data from the entire population. Census

research is ideal because it can collect complete information about the population and

lead to more accurate results (Bryman and Bell, 2007; Brase and Brase, 2007). On the

other hand, the accessible population is part of the target population which the researcher

has access to (Burns and Grove, 2005). A number of limitations and barriers such as

money, time, the huge size of the population will make the option of collecting the data

from the entire population impractical (Cooper and Schindler, 2003; Brase and Brase,

2007). Hence a representative sample will be drawn. The characteristics of this sample

are similar to those of the accessible population (Cooper and Schindler, 2003; Polit and

Beck, 2004; Jonassen, 2004; Sukamolson, 2005). The researcher carried out the research

using the drawn representative sample and then generalizing his result from which the

sample has been drawn (Gravetter and Forzano, 2008). The extents to which the

researcher can apply his results to the population depend heavily on how much the

sample is representative (Gravetter and Forzano, 2008). A biased sample is a sample

where characteristics are significantly different from the characteristics of a population

(Gravetter and Forzano, 2008; Sukamolson, 2005). Selection bias may occur when the

sample is selected in a way which increases the probability of bias in the sample

(Gravetter and Forzano, 2008).

40

Bowling (2009) demonstrated that the selected website sample has a great impact on the

results of the study. There are two main sample designs in research; random and non-

random (Polit and Beck, 2004; Bowling, 2009; Bryman, 2008). Although a true random

sampling design can be used to draw the website sample by choosing random website

addresses, this is not a practical option, because seeking information form a certain

addresses rarely used by people (Rizzo et al., 2008), the majority of users, use search

engines when search for information (Eysenbach and Kohler, 2002b).

Power calculation is a mathematical method which is used in random sampling to

estimate the size of the sample (Bowling, 2009; Bryman, 2008; Polit and Beck, 2004;

Dattalo, 2008). The purpose of this research is to evaluate the readability and

completeness of MS information websites in UK, and hence the population of websites

accessed by UK patients. The internet is vast and is a rapidly changing information

source (Rice and Katz, 2001). Moreover, there is a large number of health websites on

the internet (it is estimated that the number of health websites is more than 100,000)

(Cline and Haynes, 2001a; Dearness and Tomlin, 2001; Fox, 2005). Therefore it is

impossible to identify the actual size of the population (Schafer, 2002), and applying a

power calculation to determine sample size is not a feasible option (Kohlmaier, 2008).

This ultimately makes a random sampling design unsuitable for this research.

In order to draw a representative samples of websites, a non-random purposive sampling

design has been chosen and used (Bowling, 2009; Neuendorf, 2002). Moreover, all

studies which assess the quality of online health information use this sampling design

(Eysenbach et al., 2002). Non-random sampling is the process of selecting the population

element in a non-random way (Bryman, 2008). It is useful when the researcher is unable

to identify the elements of the population (Henry, 1990). Purposive sampling is “a sample

with a purpose in mind” (Trochim, 2006). It is a type of non-random sampling in which

the researcher selects the sample subjectively in a way that appears best for getting a

representative sample and achieving research goals (Tongco, 2007).

The internet is a huge source of health information, and there are hundreds of thousands

of health websites available (Cline and Haynes, 2001a; Dearness and Tomlin, 2001; Fox,

41

2005). Seeking information from specific addresses is a infrequently used method by

internet users (Rizzo et al., 2008). The search engine is a tool for searching on the

internet which has an index of billions of websites (Heath, 2008). Search engines help

users find resources and websites about certain topics by entering a keyword about their

subject of interest (Deacon et al., 2007). They were used in this research to locate

websites which provide information about MS, because 89% people use non-specialized

search engines (such as Google, Yahoo, Bing etc.) when they search for health

information (Fox and Rainie, 2002). Moreover, most studies which assess the quality of

online health information use search engines to locate health websites (Eysenbach et al.,

2002).

Google is one of the most popular search engines on the internet, and is used in this

research to locate MS websites because it is the most used search engine by internet

users (Nielsen, 2009), and the most used search engine by UK users (64% of internet

users use Google to search on the internet) (Nielsen, 2006). Furthermore, the search

results generated by Google highly overlap with the search results of other search engines

(Spink et al., 2006; Rather, Lone and Shah, 2008). In order to prioritise the search results,

Google use technology called “Page Rank” which employs complex algorithms to

develop a rank for every web page depending on a huge number of variables (appendix A

provide detail about page rank technology used in Google search engine) (Google,

2010a). The rank of any webpage is constantly changing depending on different

keywords used in the search, and different keywords will results in different websites hits

(Deacon et al., 2007).

In the literature review we didn‟t find any study which identified the keywords mostly

used by MS patients when searching for information online. With the aim of identifying

these keywords the “Google Keywords Tool” was used. The Google Keywords Tool was

used because it associates a website with previously entered enquiries by Google users,

and it identifies the keywords used mostly by users based on the content of that website.

Moreover keywords can be specified by language or country (Google, 2010b). Google

Keywords Tool is a free and available on this link (https://adwords.google.co.uk).

42

With the purpose of identifying the most used search keyword by UK users when they

search for information about MS, a sample of MS information websites (Table 2) was

identified in August 2010 by entering the term “Multiple Sclerosis” in the Google search

engine. Because 94% of internet users click on one of the first five results (Online1,

2006), the first five usable MS patients information websites was chosen (Table 2), and

the Google Keywords Tool was applied to the website sample. The language was limited

to English keywords, and the region was set to the United Kingdom. A total of 21

keywords were identified using the Google Keyword tool. Table 3 shows these keywords

ranked by the total number of monthly searches entered by UK users and the total

number globally. Although the actual words used by MS patents when they search for

health information is still unknown this seems to be the best way to address the problem.

This method corresponds with the way in which people search for health information

online, and allows us to evaluate the information which appears and is commonly

accessed (Fox, 2006a; Littleton, 1998).

Website Name Addresses

UK Multiple Sclerosis

Society

http://www.mssociety.org.uk/

Multiple Sclerosis Trust http://www.mstrust.org.uk/

Patients UK http://www.patient.co.uk/health/Multiple-

Sclerosis.htm

Multiple Sclerosis Resource

Centre

http://www.msrc.co.uk/

BBC - Health http://www.bbc.co.uk/health/physical_health/conditio

ns/ms1.shtml

Table 2: Website sample used in Identifying Keyword

43

Inclusion Criteria

The website sample was gathered in August 2010 by entering different key words which

were identified using the Google Keywords Tool into Google search engine. Only results

from the first page were considered, because 97% of people who seek health information

from internet click on a link on the first result page (iProspect, 2006; Online1, 2006), and

95% of people only look at the results on the first page (Online1, 2006). An inclusion

criterion was developed to evaluate the returned websites (Table 4).

Number Conditions

1- The website needs to contain information about MS

and this information needs to be aimed at patients

2- The language of the websites is English

3- The website needs to be located on UK

Table 4: Inclusion Criteria

Keyword UK Monthly Search’s

Number

Globally Monthly

Search’s Number

multiple sclerosis 135,000 1,220,000

ms society 22,200 90,500

ms symptoms 18,100 110,000

multiple sclerosis

symptoms

9,900 90,500

what is ms 6,600 110,000

symptoms of ms 5,400 40,500

multiple sclerosis society 4,400 27,100

ms support 2,900 40,500

ms treatment 2,900 33,100

ms early symptoms 2,400 12,100

ms research 1,600 27,100

multiple sclerosis

treatment

1,600 33,100

what is multiple sclerosis 1,600 9,900

symptoms of multiple

sclerosis

1,600 18,100

ms care 1,000 33,100

ms diagnosis 1,000 8,100

ms multiple sclerosis 880 8,100

fatigue ms 720 5,400

multiple sclerosis care 590 2,900

ms resources 480 14,800

multiple sclerosis

exercise

320 3,600

Table 3: Keywords Identified using “Google Keywords Tool”

44

As the purpose of this research is to evaluate the completeness and readability of MS

information websites accessed by MS patients, the first condition in inclusion criteria is

that the website contains information about MS and that this information is aimed at

patients. Based on the supposition that websites state their directed audience (Petch,

2004), the websites were evaluated to identify who it was aimed at . The language of the

website was the second condition of the inclusion criteria. Because the majority of the

UK population speaks English (there are more than 55 million people who speak English

in the UK) (NationMaster, 2010), which makes English the dominant language in UK

(BBC, 2010), only websites written in this language were included in the study. Because

the aim of the research was to evaluate the completeness and readability of MS

information on UK websites only websites from the UK were considered. To do this the

search criteria in Google search engine was specified to include pages from the UK only,

and double checked for their origin. Forums and encyclopaedia, MS websites which aim

to collect money, and MS centre websites were not included in the sample.

This sampling method is a mirror of how people search for relevant health information

(HERRMAN-ROSE, 2008). A total number of 28 MS websites which met the above

criteria were identified. One site was excluded because it not provide information itself,

but just listed links from different MS websites and put them together in the home page .

This made the final total number of websites sample 27.

Because the internet is continuously changing (Petch, 2004), an application called

HTTrack was used to download a “frozen” copy of each website, to ensure that the

information collected from the website sample leads to accurate results. HTTTrack is an

application which allows user to capture a copy of any website at a certain point in time,

and to download the entire website content (including the images, videos, flash content of

the website) from the server of the website to the user‟s computer (Roche, 2010),

HTTrack is freely available online at this link (http://www.httrack.com/).

According to Bowling (2009) results can be generalized if the external validity is high

and if each website has a similar chance of being included in the sample. A limitation in

this sample is the potential to attribute results to the entire population.

45

Data Collection

Data collection is a very important part of quantitative research (Ngau and Kumssa,

2004). Data collection instruments refer to the tool which is used to collect information

(Keith F. Punch, 2006). Reliability and the validity of measurements are key concepts in

the measurement of quantitative research. These two concepts (reliability and validity)

are important quality indicators of the measurement tool, and it is important the

researcher considers them (Jackie Green, 2006). Reliability refers to the stability and

consistency of the data collection tools (Vaus, 2001), and the ability of the data collection

to consistently generate the same finding every time it is used(Jackie Green, 2006;

Jackson, 2007). Reliability of data collection is a major concern of the researcher when

collecting data (Brink and Walt, 2007). On the other hand validity refers to the extent a

test score is supported by evidence and theory (APA, 1999). Eysenbach (2002)

emphasised the importance of the validation of tools used to measure the quality of online

health information in order to ensure their suitably.

This research uses a scoring tool developed by Harland and Bath (2007) to assess the

completeness of online MS information. Harland and Bath give their consent to use their

tool in this research (appendix C).

Information Completeness Data Collection tool

The MS Information Completeness tool, developed by Harland and batch (2007) was

used to evaluate the completeness of Online MS information.

According to the literature review, with the increase in the amount of health information

available online, in the last few years (Cline and Haynes, 2001a; Dearness and Tomlin,

2001; Fox, 2005), and increased concerns about the quality of online health information

(Berland et al., 2001), a number of evaluation tools have appeared with the aim of

assessing and evaluating the quality of online health information (Harland and Bath,

2007; Anderson, McKemmish and Manaszewicz, 2003; Kim et al., 1999; Gagliardi and

Jadad, 2002; Purcell, Wilson and Delamothe, 2002; Hanif et al., 2009). None of these

tools are considered “ideal” for assessing the quality of online health information

(Eysenbach et al., 2002; Childs, 2005). Evaluation tools can be classified into two main

46

types; generic evaluation tools and disease specific evaluation tool (Harland and Bath,

2007).

Generic evaluation tools are tools which are not directed for certain diseases or

conditions, and can be applied to websites which provides a variety of health information

for different conditions (Harland and Bath, 2007; Purcell, Wilson and Delamothe, 2002).

Most of the evaluation tools available for assessing the quality of online health

information are generic evaluation tools (Eysenbach et al., 2002; Gagliardi and Jadad,

2002; Harland and Bath, 2007). Using non-validated tools for assessing the quality of

online health information may lead to misplaced trust in websites and sometimes harmful

consequences (Harland and Bath, 2007). A number of authors raise concerns about

potential harm to the public and patients which may result in using generic evaluation

tools for assessing the quality of online health information (Purcell, Wilson and

Delamothe, 2002; Harland and Bath, 2007; Hanif et al., 2009). Despite widely available

choices of generic tools, generic evaluation tools were not used in this research because

their applicability varies, and there is doubt over their reliability and validity (Hanif et al.,

2009; Childs, 2005). Moreover, these tools involve too many subjective measures

(Ademiluyi, Rees and Sheard, 2003; Seidman, Steinwachs and Rubin, 2003), and none of

the available generic evolution tools are designed specifically to assess the completeness

of online health information (Eysenbach et al., 2002; Harland and Bath, 2007).

On the other hand disease specific evaluation tools are tools designed specifically to

assess the quality of information regarding specific diseases or conditions (Harland and

Bath, 2007). The disease specific evaluation tool was used in this research to collect data

about the completeness of online MS information because Harland and Bath (2007)

concluded and recommend (after comparing different tools to assess MS information

quality) to use specific tools to assess the quality of MS information available online.

The literature review identifies a gap in research which assesses the quality of online

information based on patient‟s needs, and identifies that MS patients in the UK require a

considerable amount of information about their disease during its different stages. The

different available information sources fail to meet information needs, and there is a gap

in completeness of online MS information (see the literature review). Therefore this

47

research aims to evaluate the completeness and readability of MS information available

on UK, and will address MS patient‟s needs when assessing the information. The data

collection tool needs to fit with the research design and the research question (Polit and

Beck, 2004).

Hepworth and Harrison (2002) use qualitative and quantitative methods (n= 2030) to

identify different aspects of information needs for MS patients in the UK (this includes

identifying the information which is perceived to be important to MS patients, identifying

information which is difficult to find, and the information patients wish to find). Based on

the results of Hepworth and Harrison, Harland and Bath (2007) have developed and

validated a tool to assess the completeness of MS information online. They designed the

tool to reflect the needs of MS patients in the UK by assessing how well the content and

style of MS websites meet and match their information needs. Harland and Bath validated

their instrument using a sample of 40 MS websites. The MS Information completeness

tool achieved a very high score in a reliability test (Cronbach‟s alpha 0.930), and meets

the acceptable standard for internal consistency. Harland and Bath‟s (2007) MS

information completeness tool (Appendix C) was used in this investigation because it fits

perfectly with its purpose and population. According to the literature review this was the

only available tool for evaluating the comprehensiveness of MS information.

The tool is composed of four main sections and involves 48 closed ended (Yes/No)

questions. The first section of the tool involves eleven questions about information for

patients who are newly diagnosed with MS. The second section has twenty three

questions about people who are in the post diagnosis phase, the third section involve

questions about information for the family and career of MS patients. And the final

section includes questions about the overall content and style of the website. Harland and

Bath state that the information needs of MS patients varies during different stages of the

disease, and they include the family and career because the needs of family and career

may vary between patients. After opening the website the researcher took 15-25 minutes

to browse the websites and identify its structure and different sections. Then the tool was

applied to each website, and the data was numerically coded (Yes=1/ No=0). Each

website was given a score between 0 and 48.

48

Information Readability Data collection Tool

Readability of online MS information is another variable that was assessed in this

research. An automated version of the SMOG (Simple Measure of Gobbledegook)

readability formula was used to assess the readability of MS information on the selected

website sample by copying a randomly selected sample text from the each website and

applying the tool accordingly.

According to the literature review online health information is written at a very high

level (Graber, Roller and Kaeble, 1999a; Graber, D'Alessandro and Johnson-West, 2002;

Berland et al., 2001), and patients who seek it found that it was difficult to read and

complicated (Berland et al., 2001; Murero, D'Ancona and Karamanoukian, 2001).

According to our knowledge and the literature review there are no studies which evaluate

the readability of online MS information.

Readability formulas is a mathematical tool, which is used to identify the reading level of

certain texts (Feathers and Rivers, 2004; VICKI S. FREIMUTH, 1979). In the last few

years there have more than 40 readability formulas developed (Ta-Min, 2006a). These

formulas estimate the reading age score which reflects the education level required to

read and understand certain text (Bauman, 1997). Although these formulas differ from

each other in weight and types of variables (Ta-Min, 2006a), the study conducted by

(Meade and Smith, 1991b) shows that different formulas are accurate and reasonably

correlate each other. In order to choose the best methods for assessing the readability of

certain text some readability formulas are more suitable for certain audiences than others

(Freda, 2002). Eysenbach et al. (2002) found that there are four main formulas used to

assess the readability of online health information. These include; the SMOG formula,

the FOG formula, the Flesch-Kincaid formula and the Fry Readability Graph.

The SMOG readability formula is a tool developed by McLaughlin (1969) to give

estimates of the grade the person should have in order to understand the assessed text.

This tool works by counting three samples of 10 consecutive sentences near to the start

middle and end of the text, the words are counted and the square root of the number of

the polysyllabic words is calculated and added to 3 (Figure 1 show the equation used in

49

SMOG formula) (Paz et al., 2009; Dougal, 2006; LAUGHLIN, 1969; McLaughlin, 2008;

VICKI S. FREIMUTH, 1979; Meade and Smith, 1991b).

For the purpose of this research the SMOG readability formula was chosen and used

because unlike with other tools it reflects the actual level of education required for

reading certain texts (Elliott and Shneker, 2009), and it is widely used to revise and

assess the readability of patients directed health documents (Hedman, 2008; Ley and

Florio, 1996). SMOG readability score highly correlates with the score of other

readability formulas (Meade and Smith, 1991a; Meade, Diekmann and Thornhill, 1992).

Moreover it can estimate the readability of text for British people more accurately,

because it is validated on British material and readers (Paz et al., 2009; McLaughlin,

2008).

With aim to identify the readability of MS information on UK websites for British

people, an automated online version of SMOG readability formula was used on this

research (http://www.addedbytes.com/code/readability-score/). A randomly selected

sample from each websites was copied and pasted in to the online application which

calculated the readability based on the previously described formula. The selected text

was checked to make sure that it was complete and had no mistakes in punctuation to

ensure accuracy (McLaughlin, 2008).

According Nicholls et al (2009) patients‟ information needs to be written between levels

5-8 (SMOG Score), because 52% of adults in the UK have reading abilities in this range.

The following Table from the National Institute of Adults Continuing Education

(NIACE) shows the similarities of the SMOG score with the UK National Adult Literacy

Standards:

Figure 1: The Equation of SMOG readability formula

50

A common limitation of all readability formulas is that it may not be able to reflect other

factors that disturb or effect text understanding and comprehension such as use of

medical jargon and writing style or use of culturally specific terms (Eysenbach et al.,

2002). But this seems to be the most feasible way of assessing the readability of MS

information online.

Results

Websites Source

A total of 27 MS websites were evaluated in this research. All of the evaluated websites

are websites that contain information about MS. As the source of the website has an

effect on the credibility and relevance of the health information (Davies and Macdowall,

2006), the websites have been classified into six main categories; governmental websites,

which are part of a governmental health organization or institution; biomedical/charity

websites which are websites related to organizations that are not allied or part of an

institution; personal websites which are MS websites produced by non-physicians and do

not follow or represent an organization or institution; commercials ,websites are those

with profit purposes either by marketing certain health care products/services or by

hosting advertisements on the website; physicians‟ websites are websites produced by

individuals or groups of physicians, and the website is not affiliated to an organization,

institution or company; news websites are television, newspaper, radio and other media

websites be they part of a governmental organization or not, unidentified websites are

websites where one cannot establish the author. The previously described type of

SMOG Score National Adult Literacy Standard

5-8 Entry Level 2

9-10 Entry Level 3

11-12 Level 1

14-15 Level 2

Table 5: the similarities of the SMOG score with the UK National Adult

Literacy Standards

51

classification was used because it is used in the majority of research which assesses the

quality of online health information (Roon and Soot, 1999; Molassiotis and Xu, 2004;

Hellawell et al., 2000).

The different sources of websites used in this research are shown in the pie chart in

Figure 2. The majority of websites (37.04%) were biomedical or charity websites such as

UK Multiple Sclerosis Society website and Multiple Sclerosis trust website. Commercial

websites form 25.93% of the total website sample most of which belong to market

research companies or to companies that provide services/products for patients or host

online advertisement. News and governmental websites each form about 11.11% of the

website sample. News websites included some of the main media websites in the UK

such as the BBC and The Guardian. These websites provided information about MS for

patients. Government websites mainly included the NHS website and others which are

part of the NHS or Department of Health. The other 11.11% of websites were personal

websites written by patients affected by MS with the aim of providing other MS patients

with information about the disease. The Final 3.70% were websites produced by health

care professionals.

Figure 2: Sources of the Websites Used in This Research

52

The website sample was also classified according to their content source; MS Specific

websites and Non-MS Specific Websites. MS specific websites are websites which

contain information only about the MS disease. Non-MS specific websites are those

which provide information about different diseases and conditions MS being one of them.

MS Specific websites form 51.85% of the sample while non-MS Specific websites form

about 48.15 of the total sample.

Overall Completeness

The MS information completeness tool used in this research has a total of 48 Yes/No

questions for measuring the completeness of MS information. The questions are divided

into four main parts for measuring different areas of information completeness. The

overall completeness score was calculated out of 48.

Figure 3 summarizes the completeness scores of all 27 websites. None of the website in

the sample achieved a 100% completeness score, the score ranging from 5-45. As can be

seen from the figure only 14.8% of the websites achieved a high score in overall

Figure 3: The frequency of the scores Achieved by the websites

53

completeness (above 75%). Only a small number of websites (25.9%) achieved a score

above 50% (between 24 - 48) in overall completeness. However, the majority of websites

(74.1%) achieved a score below 50% (between 0-23), and 37.0% of websites achieved

very low score (below 25%). The mean level of information completeness on websites

was (19.41).

Table 6 shows all 27 websites ranked by their overall achieved completeness score. The

UK Multiple Sclerosis Society was ranked first, and it achieved a high overall

completeness score of 93.75%. Interestingly the MS Society Scotland website achieved a

low score in overall completeness (41.6%). In addition to the UK, Multiple Sclerosis

Society website, the Multiple Sclerosis Trust, NHS Choice and MS Active Source

websites contained very complete information about MS and they achieved an overall

score above 75%. On the other hand, the BBC health website achieved very low

completeness score (12.2 %), and Shift MS website attained the lowest score in overall

completeness (10.4%).

54

Rank Website Score

1 UK Multiple Sclerosis Society 45

2 Multiple Sclerosis Trust 38

3(=) NHS Choice 36(=)

3(=) MS Active Source 36(=)

4 The Multiple Sclerosis Resource Centre 32

5 The Multiple Sclerosis International Federation

(MSIF)

31

6 Action MS 26

7 Proventus 23

8 NHS Clinical Knowledge 22

9 Guardian Health & Wellbeing 21

10(=) MS Society Scotland 20(=)

10(=)

11

Young MS

DLA and Attendance Allowance

20(=)

19

12(=) Independent 17(=)

12(=) Brain and Spine 17(=)

13 MS Diagnosis 15

14 Bupa's Health Information 13

15(=) The Patients Voice 12(=)

15(=) Klearview to MS 12(=)

16(=) Net Doctor 11

16(=) Patient UK 11

17(=) Home Health 10

17(=) MS Decision 10

18 Multiple sclerosis a - Personnel Account 9

19 Dr Romi Saha 7

20 BBC Health 6

21 Shift MS 5

Table 6: The rank of 27 websites based of overall completeness score achieved

55

Figure 4 summarizes the relation between the mean overall completeness score and the

type of website. The research found that compared to other website sources,

Biomedical/Charity and Government websites achieved the highest score in term of

overall completeness (53.5%: 47.0% respectively). On the other hand the score of the

commercial and news websites were close to each other (35.0%: 30.0% respectively). But

Physician websites achieved a low score (about 18.75%), and personal websites achieved

the lowest score among all categories (12.5%).

The MS information completeness tool is composed of four main sections; the first

section assesses the completeness of information for newly diagnosed patients with MS,

the second section assesses the completeness of information for patients in the post-

diagnosis phase. The third section of this tool considers the completeness of information

Figure 4: The relation between the percentage of overall completeness score and the

source of the website

56

for families and carers of MS patients and the final section assesses the overall content

presentation on the website.

Figure 5 summarizes the overall score achieved by the websites in the different areas of

the tool. The overall content presentation score was the highest (68.7%) of all sections

and the family and carer section was the lowest (14.2%). Websites achieved a score of

54.8% in the newly diagnosed information section, and of 32.8% in the post diagnosis

section. In the next part of this chapter detailed results about each section will be

discussed.

Newly Diagnosed Information Completeness

The first section of the MS Information Completeness Tool assesses the completeness of

information for MS patients who are newly diagnosed with the disease. This section is

composed of 11 questions assessing the provision of complete information about

Figure5: The percentage of overall completeness score achieved by the websites in

different sections

57

different areas such as physiology, the course of the disease, symptoms, management,

exercise and diet. The score of this section was calculated out of 11.

The average score achieved by the websites in this section was 54.8%. The highest

achieved in this section was 11 and the lowest was 2. The mean score was 6.05. In this

section only the Multiple Sclerosis Trust website achieved a full score and contained full

information for newly diagnosed MS patients. The majority of websites 96.3% only

partially address the information needs of newly diagnosed patients. Moreover 59.3%

achieved an overall score of below 6 in this section. 3 websites (11.1%) achieved an

overall score of 2. These websites were the BBC Health website, Dr Romi Saha website

and Shift MS website.

A number of areas in this section were found to be complete. The majority of the

websites (81.5%) give information about the physiology of MS, and about 74.1% of the

websites explained how MS was diagnosed. An explanation of the disease‟s course was

addressed by 81.5% of the websites, and 85.2% of the websites included information

about the physical symptoms of MS. 74.1% of the websites contained information about

the available drug treatments.

There are some areas in the newly diagnosed information section which are poorly

addressed by the websites. For instance only 22.2% contained a separate information

section for newly diagnosed MS patients, and only 29.6% contained information about

symptom management. Moreover, the majority (66.7%) did not discuss the role of the

MS specialist nurse, and only 37% discussed how patients can tell family and friends

about the disease.

Completeness of Post Diagnosis Phase Information

The second section of the MS completeness tool assesses the completeness of MS

information for patients who are in the post diagnosis phase. This section is composed of

23 questions which address different information for patients such as balance problems,

bowel and bladder problems, emotional changes, fatigue, memory and visual problems

58

and spasticity. This section also assesses whether the website is providing a balance of

information about nutrition, physiotherapy, sex, pregnancy, different drug treatments and

comparing these drugs. This section assesses whether the website provides practical

information about issues such as work and MS, home modification for MS patients and

leisure activities. The score for this section was calculated out of 23.

The average overall score achieved by the websites in this section was low (32.8%). The

highest score achieved by the websites was 23 and the lowest score was 0. The mean

score was quite low (7.56). The highest score was achieved by the UK Multiple Sclerosis

Society website and the lowest by the BBC health website because it did contain any of

the information in this section. Interestingly, unlike the UK Multiple Sclerosis Society

website, the MS Society Scotland website achieved a low score (7) in this section. The

majority of websites (92.6%) only partially address the information required by patients

in the post diagnosis phase, 51.9% achieved a score of 7 or below in this section.

The research found that only 33.3% of the websites contained a separate section about

with practical information about how to live with MS. It was also discovered that only

37% of the websites contained a contact facility that allows patients to converse with

fellow MS suffers.

As stated above, websites only partially address the information needs of MS patients in

this section. For instance the websites achieved an average score of between(48.1% and

59.3% in the questions about balance and waking information, bowel and bladder

information, emotional changes, memory, fatigue, visual information and spasticity; this

score was not significantly high or low. On the other hand a number of areas were found

to be significantly incomplete and were not address by the websites, for instance; only

37% of the websites provide information about different drug treatments available for MS

and how they compared with each other, and only 25.9% provided information about

pregnancy and MS. 66.7% of websites did not any contain information about the

complementary medicines for MS disease and 70.4% of the websites did not provide

59

information about physiotherapy for MS patients. Providing patient with information

about MS research was only done by 33.3% of the websites.

With regards to important everyday practical issues (Hepworth, Harrison and James,

2003) such as access and transport, home modification, work etc., the websites contained

incomplete information. Information about access and transport and aid and appliances

information was only provided by 14.8% of the websites. The majority of websites

(88.2%) did not contain information about Benefits and entitlements for MS patients in

the UK, and only 18.2% provided patients with information about leisure activities and

holiday information. Information about home changes and occupational therapy was

addressed by only 14.8% of the websites. Only seven websites addressed the issue of

work for MS patients.

Interestingly it was found that there a variation in the score achieved by MS specific

websites and Non-MS Specific website in this section. As can be seen from Figure 6, MS

specific websites contained more complete information in this section and achieved an

average overall score of 40.6%, while the Non-MS Specific websites achieved only

24.4%.

Figure 6: The Differences between the mean score achieved by MS Specific Websites

and Non-MS Specific Website in Post Diagnosis Phase Information section

60

Completeness of Family and Carers Information

This section evaluated the websites regarding the completeness of information for family

and carers of patients with MS. It was composed of 7 questions; the questions covering

areas about the practical and emotional challenges for MS patient‟s families, experiences

from other carers and MS family members, support groups and contact available for

carers and family members. The score for this section was calculated out of 7.

The average overall score attained by the websites in this section was 14.2% which was

the lowest score among all the categories. The highest score was 6 and the lowest score

was 0. The highest score was only attained by the UK Multiple Sclerosis Society

Website, and remarkably the lowest score (0) was attained by 18 (66.7%) of the websites.

The mean score for this section was (1.0) which is significantly low.

In general the research found that the websites contain highly incomplete or absent

information for carers and family members of patients with MS. Only 25.9% of the

websites contained a separate information section with this information. Moreover, only

29.6% of the websites provided general caring advice for carers and family members.

The information about the practical and emotional challenges that carer family members

may face was provided by less than 12% of the websites. Only 7.4% of the websites

reported the personal experiences of other carers and family members of MS patients and

most 92.6% did not provide carers and family members with a contact facility to contact

each other‟s. The majority of websites (88.9%) did not provide information about support

groups and contact for carers family members.

61

The figure above shows the relation between the score achieved by MS specific websites

and by Non-MS Specific websites. Interestingly it was found that there wide variation in

completeness of family and carer information between these two categories; while the

MS specific websites achieved a 23.4% overall score in this section, Non-MS Specific

websites achieved a low overall score (4.3%).

Overall Presentation of the Content on the Website

This section evaluated the overall qualities of the content on the websites, and whether

the content, presentation and tone meets the needs of MS patients in UK. This section

was composed of 7 questions about issues such as the presentation of the content, the

potential of the information to motivate MS patients, the presentation tone of the text on

the websites and contacts for MS patients.

Websites achieved average overall score of 68.7% which was the highest score among all

categories. The mean score of the websites in this section was 4.81 and the highest was 7

which was attained by 18.5% of the websites. The lowest score was 1 which was

attainted by the Multiple sclerosis a - Personnel Account website.

Figure 7: The Differences between the mean score achieved by MS Specific Websites and

Non-MS Specific Website in Family and Carers Information section

62

The websites achieved a high score in different areas of this section. The majority of

websites (85.2%) provided information about other useful resources for MS patients and

provided patients with contact address and telephone number for support. Regarding the

presentation of the content it was found that 85.2% of the websites provided the

information in a concise manner and provided the information in way that helps to inspire

and motivate MS patients. Moreover, the research found that 88.9% of the websites

presented and discussed their information content in a positive manner that helps to

support MS patients.

Some areas on this varied between different websites. For instance only 44.4% of the

websites included sections written by MS patients. 44.4% of the websites did not contain

a text size adjusting facility to make it easier to read for patients. The final question in

this section was a question for the researcher about whether he would recommend the

website for MS patients as an information source based on his extensive review of the

websites. The researcher recommended only 10 websites. All of the recommended

websites ranked within the first 10 websites in table 6.

Readability

The readability of information on the websites was assessed using an automated version

of the SMOG readability formula. The output of the formula was calculated as a scale

variable which represents the years of education required to read a piece of text. Table 5

represent the correspondence between the SMOG readability score and literacy levels in

the UK. The results were analyzed based on this table. According to Nicholas et al.

(2009) 52% of the UK population have a reading ability which ranges from 5-8 and the

recommended range for written materials needs to be below this range.

63

The mean score of readability was 10.06. The highest readability score was 15.50 which

was achieved by the Proventus website and the lowest was 5.30 by the Shift MS website.

Figure 8 represents different categories of SMOG score and the frequency of these scores

achieved by the websites. 66.6% of websites required 11 years of education or below in

order to read the text in the websites. Of these, 33.3% achieved a readability score

ranging from 5-8 (Entry Level 2), and 33.3% achieved a score ranging from 9-10 (Entry

Level 3). On the other hand, 25.9% of the websites achieved a readability score which

ranged between 11-12 (Level 1), and only two websites (7.4%) achieved a score of 14 or

above (Level 2). As can be seen from the figure above 66.6% of the websites achieved a

readability score beyond the reading abilities of the UK population and only 33.3%

achieved score within the readability range of UK population. None of the websites

achieved a readability score below 5.

The median readability score for Government websites was (9.43) and for

biomedical/charity websites 10.36. The majority (60%) of biomedical/charity websites

achieved a readability score which ranged from 10-12. The mean score for commercial

Figure 8: SMOG readability score achieved by the websites, and it`s relation to

UK population reading abilities

64

and personal websites were close to each other (10.10 : 10.03 respectively), but 50% of

commercial websites achieved a score between 6-8. The mean score for news websites

was 9.70 and for physician websites was 10.15.

Discussion

The results of this research find that online MS information on UK websites contained

highly incomplete information about MS, and the information available on these websites

does not meet the different needs of UK patients. Furthermore this study also found that

MS information available on UK websites is written at a high reading level which goes

beyond the average reading abilities of the UK population. This discussion section will

falls into three sections; the first section will discuss the overall completeness of MS

information on UK websites, the section will discuss different specific areas of MS

information completeness on the websites (e.g. Newly diagnosed information, Family

and Carer Information .. etc), and the final parts will discuss the readability of this

information.

Overall Completeness

The literature review revealed that providing MS patients with information has a positive

impact on psychological and clinical outcomes for MS patients, and that it is essential for

the decision making process of these patients (Somerset, Sharp and Campbell, 2002;

Somerset et al., 2003).Nevertheless, a shortage in providing patients with information

may lead to misconception and harmful consequences (Pellise, Sell and EuroSpine

Patient Line Task, 2009; Gross et al., 2006; Department of Health, 2004; The Joint

Commission, 2007b). According to the literature review also completeness of health

information was an area of interest for a lot of research. In general, this research

concluded that online health information is incomplete (Kohlmaier, 2008; Larkin, 2002;

Sandvik, 1999; Croft and Peterson, 2002; Saithna, Ajayi and Davis, 2008; Morr et al.,

2010; Bichakjian et al., 2002; Fant, Clark and Kemper, 2005). This research found

confirms this result because it found online MS information on UK websites is

incomplete, and the completeness of this information is highly variable. Moreover

Eysenbach et al. (2002) after reviewing of 71 studies which assess the quality of online

65

health information, demonstrated that in spite different methods and instruments used in

different studies for assessing the completeness of online health information, 90% of

these studies concluded that online health information is incomplete. The results of this

research match the results of previous studies. None of the websites contained all the

information needed by MS patients, and the majority (74.1%) didn‟t even contain even

50% . A significant number of websites (37%) contained highly incomplete information

about MS. The result of this study confirms that there is a gap between information

needed by MS patients and information available to them (Matti et al., 2010; Harland and

Bath, 2007; Hay et al., 2008).

One of the important features which attracts patients to use the internet as a health

information source is the wide scope of available information which allows them to

search for information that meets their needs (Pereira and Bruera, 1998; McMillan, 1999;

SPICH, 1999). Although the internet was an important information source for MS

patients in many studies (Bishop et al., 2009; Hay et al., 2008), Hepworth et al (2003)

found that in the UK only 5% of MS patients turn to the internet to seek information

about their disease, as the quality of the information source has an impact on the

information seeking behaviour from this source (Park, Chung and Yoo, 2009). The

results of this research may explain Hepworth‟s finding. However Harland and Bath

(2007) demonstrated that the low quality and irrelevance of online MS information might

be the reason behind low internet use by MS patients in UK.

The UK MS Society website contained highly complete information about MS that meets

the needs of MS patients in UK. Moreover this website was ranked as the most complete

MS website. This result corresponds with the results of Harland and Bath (2007) which

rank the UK MS Society website among the top five most complete MS websites, and

found that it contained highly complete information for MS patients. Conversely,

according to (Huntington et al., 2007) the BBC websites is one of the largest content

based websites in the UK and it is one of the most popular sources of different

information for UK people (45.8% of the British use the BBC website to get

information). But in Huntington et al‟s study (2007), users report that health information

66

available on the BBC website is too short and didn‟t meet their information needs. This

study corresponds with Huntington et al‟s as it found that the BBC website contained

highly incomplete information about MS and was ranked as the second worst website in

terms of MS information completeness. Moreover, it achieved a very low score in newly

diagnosed and content presentation sections, and it did not contained any information for

patients in the post-diagnosis phase or for family and carers of MS patients.

In this research it was found that commercial and physicians‟ websites contained highly

incomplete information about MS and they achieved a low score in overall completeness

(35% : 18.75%). On the other hand biomedical/charity and governmental websites were

the best sources in terms of overall information completeness. This result corresponds

with Gustafson et al (1999) finding who demonstrate that website authors have an impact

over information completeness on the website; for instance he states that physicians

might not include all the required information because they assume that patients have

background knowledge about their disease, and commercial websites may contain

information which only achieves the financial goal of the website owner.

Different areas of MS Information Completeness

Because of the fact that information the needs of people is a result of certain situation

which trigger the information searching process (Dutta, 2001), a number of studies found

that the information needs of MS patients vary (Forbes, While and Taylor, 2007; Slade,

Tennant and Ford, 2003; Hepworth, Harrison and James, 2003; Lejbkowicz et al., 2010;

Heesen et al., 2007), and according to (Baker, 1996) the information needs of MS

patients and their family didn‟t receive enough attention and coverage. In this research it

was discovered that not only is there a shortage in coverage of online MS information,

but also the completeness of this information on UK websites is highly variable among

different information needs categories. This result corresponds also with the results of

Harland and Bath (2007) who also found that only a few websites addressed all

information needs.

67

Evidence shows that newly diagnosed patients need different and significant amounts of

information once they are diagnosed with MS (Forbes, While and Taylor, 2007; Slade,

Tennant and Ford, 2003; Hepworth, Harrison and James, 2003). Moreover it has been

accepted that providing newly diagnosed patients with information plays a key role in

their adjustment process, but despite this the majority of websites (96.3%) only partially

addressed the information needs of those patients. Although the MS society website

achieved a very high score in this section, only the Multiple Sclerosis Trust websites

achieved the full score and contained all the information required by MS patients.

The signs and symptoms of MS are highly variable, and can differ from person to person

during the different course of the disease (Mei, 2004; Richards et al., 2002). According

to Box et al (2003) and MacLean, Russell (2005) new symptoms and managing these

symptoms was a source of anxiety for MS patients, especially newly diagnosed patients.

Moreover, they found that there is a shortage in information about the management of

new symptoms (MacLean and Russell, 2005; Box, Hepworth and Harrison, 2003). It was

found in this research that information about managing the symptoms of MS was highly

incomplete on UK websites, and only 29.6% of the websites provided this information.

MS Patients reported that generic MS information is readily offered and accessible

(Gregory, Disler and Firth, 1996). This research confirms this and found that some areas

of MS information were more complete than others and some information more available

than others; this information includes information about the physiology of MS (81.5%),

the diagnosis process of MS (74.1%), physical signs and symptoms of MS (85.2%), and

the available drug treatments for MS (74.1%). On the other hand Gregory et al. (1996)

found that practical information which is more important to MS patients was difficult to

find and obtain by MS patients, and UK patients in Hepworth et al‟s survey refer to

issues such as benefits and entitlements (44.1%), access and transport (21.9%), home

modifications (25.5%), work (20.9%) .. etc. as being more important to them because it is

part of their everyday life. UK patients reported that in general it is difficult to find and

obtain this information (Hepworth, Harrison and James, 2003). In this research it was

discovered that practical information about MS was highly incomplete in UK websites

68

and only a few websites contained and discussed this information; this includes

information about aid and appliances, home changes, occupational therapy (14.8%),

benefits and entitlements for MS patients in the UK(11.8%), leisure activities and holiday

information (18.2%), which might again explain the low usage of internet by UK MS

patients. This result corresponds also with result of Berlamd et al (2001) who found that

online health education materials suffers from extreme shortage on coverage of

information which important for patients.

Carers and family members have an important role in the life and disease progress of MS

patients. This role may involve physical support, supervision, social and emotional

support, and taking the responsibility of caring for all the family (The National

Collaborating Centre for Chronic Conditions, 2004). Providing family members and

carers of MS patients with information was perceived to help them in understanding the

disease and providing the required support and help for MS patients (Hepworth, Harrison

and James, 2003). In Hepworth et al‟s (2003) survey of the information needs of MS

patients and their families in the UK, 46.4% of respondents rated information for

family/friends as important to them, and 15.4% reported that they find it difficult to get

this information. This research has discovered that information for family and carers of

MS patients is highly incomplete on UK MS websites. A very small number of websites

contained this information, and the information was almost always insufficient.

Concise and simple information is an important part of health education because it is

more likely to be remembered (Moreno et al., 1997; Hoffmann and Worrall, 2004). It was

found in this research 85.2% of MS websites provided their information in a concise

manner, and most of this information (88.9%) was discussed in positive way that helps in

supporting and inspiring MS patients.

Readability

Health Literacy plays an important role in the understanding of medical and health

information (Williams et al., 1998b; Williams et al., 1998a; Nielsen-Bohlman, Panzer and

Kindig, 2004; Dreger and Tremback, 2002), and limited health literacy was perceived to

be not only a barrier for consuming health information from the internet (Gilmour, 2007),

but it can be a problem once the information is accessed (von Wagner et al., 2009; Wolf

69

et al., 2007). Furthermore, evidence shows that there is a correlation between education

levels and seeking health information from the internet (Eng et al., 1998; Commerce,

1998; Rice, 2006; Hesse et al., 2005; Voelker, 1998; Skinner, Biscope and Poland, 2003;

Lenhart, 2000; Tu and Hargraves, 2003; Fox and Fallows, 2003c). Despite there being no

up to date study available about the recommended reading level of health information

materials directed to MS patients or UK patients in general, it is estimated that 52% of

the UK population have a reading ability between 9 and 11 years of age (Wilson et al.,

1998), This corresponds with the 5- 8 score based on SMOG (Nicholls et al., 2009).

According to Eysenbach et al‟s (2002) review of reading levels, online health information

was found to be beyond patient‟s reading abilities. Furthermore a number of researchers

found that there is a mismatch between patients‟ levels of literacy and the readability of

health information materials (McCray, 2005; French and Larrabee, 1999; Estrada et al.,

2000; Forbis and Aligne, 2002; Paasche-Orlow, Taylor and Brancati, 2003). The results

of this research correspond with the results of previous studies because it found that the

reading level of MS information available on UK website (66.6%) goes beyond reading

abilities of UK population.

Cognitive deficit is one of the symptoms of MS and it is estimated that 50% of MS

patients have cognitive deficit symptoms. Cognitive deficit may include; loss of memory,

weakened ability for learning, processing, and recalling information (Chiovetti, 2006b).

According to our knowledge there is no up to date study which assesses the literacy level

of MS patients and there is no available recommendation about the reading abilities of

those patients. Moreover there is a knowledge gap in updated recommendations about the

reading level health education materials directed to UK patients needs to be written. A

common limitation of all readability formulas is they may not be able to reflect other

factors that affect text comprehension such as the use of medical jargon and writing style

or using cultural specific terms (Eysenbach et al., 2002).

70

Limitation

This study was limited by the small sample size (27) and the convenience sampling

strategy used for sampling. Therefore it has not been possible to generalize the findings

to all UK multiple sclerosis websites in the UK (Bowling, 2009). Although Google

Keywords was used to identify the keywords used by MS patients when searching for

health information the actual keywords used by MS patients are still unknown. However

this method was a mirror of how people search and find health information online (Fox,

2006a; Littleton, 1998).

Another limitation was the page rank technology used by Google for ranking search

results. This means that the page rank of certain websites is not fixed and may change

from time to time (Google, 2010a). Moreover, the internet is rapidly changing and the

content of websites may be updated which makes the results of this study valid for only a

limited period of time (Rice and Katz, 2001). A common limitation of readability

formulas is that they may not be able to reflect other factors that affect text

comprehension such as the use of medical jargon and writing styles or the use of

culturally specific terms (Eysenbach et al., 2002). Further research is needed to extend

and confirm the validity and reliability of this research results.

Conclusion

The information needs of MS patients have been well established and documented

(Daumer et al., 2007; Harland and Bath, 2007; Hay et al., 2008; Lejbkowicz et al., 2010;

Matti et al., 2010; Sadovnick, 2009; Chiovetti, 2006b). These needs vary according to the

stage of the disease and information was perceived as playing a key role in managing it

(Forbes, While and Taylor, 2007; Slade, Tennant and Ford, 2003; Hepworth, Harrison

and James, 2003). MS patients can use different sources to attain information about their

disease (Bishop et al., 2009; Hay et al., 2008; Matti et al., 2010). The internet has

increasingly become a favourite source for patients to use to achieve health information

(Willard and Ginsburg, 2008; Kummervold et al., 2008; Peterson and Fretz, 2003;

Eysenbach, 2003; Fox and Fallows, 2003a), and this is no different for MS patients

(Bishop et al., 2009; Hay et al., 2008). As limited literacy was identified as a barrier to

71

understanding and following written health education material (Gilmour, 2007), the

reading levels of online information are worth considering as important (Ta-Min, 2006b;

Coey, 1996).

This study is a descriptive evaluation which aimed to evaluate the completeness and

readability of MS information available on 27 identified UK websites. In general all 27

MS websites contained information about different issues of MS, but the majority of sites

failed to meet the information needs of patients in UK. Therefore patients‟ needs to be

careful when seeking information form MS websites and only a small number contain

what they want. However, incompleteness of online MS information might explain the

low use of internet by MS patients in the UK. Patients need to be careful about making

decisions based on online information and doctors need to be consulted.

Furthermore, the incomplete information available on websites as well as their inability

to address all the information needed of MS patients indicates that there is gap between

the websites and the patients. This gap is reflected in the availability of generic

information and the lack of practical information which is more important for patients. As

such, it is recommended that patients become part of the health education material

synthesis process. If the information us synthesized by the author alone they may neglect

the patients‟ needs (Gustafson et al., 1999).

Although the BBC website was a popular source of information (Huntington et al., 2007),

it was discovered that this website contains highly incomplete information for MS

patients. Nevertheless there was evidence of good MS information sources such as the

UK MS Society website the best source for MS patients in terms of information

completeness. As such patients need to be directed to good websites rather than popular

websites. Furthermore, health care professional need to be knowledgeable about the

information sources and the quality of information of these sources based on valid,

reliable and up to date research. This way they can direct patients appropriately.

72

It was further discovered that the mean reading level of MS information on UK websites

is 10.06 which is beyond the average reading ability of the UK population (Nicholls et

al., 2009). This means MS information on websites is too complicated to be understood

by UK patients. Further research is needed to establish the reading levels of MS patients.

Moreover, as cognitive deficit is one of the main symptoms of MS and it is estimated that

50% of MS patients have cognitive deficit symptoms (and this deficit may vary during

different disease stages), information materials need to be based on cognitive abilities.

This study was limited by the small sample size (27) used (Bowling, 2009), and the

convenience sampling strategy used for identifying the website sample. Because the

internet is a rapidly changing environment the results of this research may only be valid

only for a limited period of time (Rice and Katz, 2001). Moreover readability formulas

may not be able to reflect other factors that affect text comprehension (Eysenbach et al.,

2002).

73

References

A.Achiron. (2006) 'Measuring disability progression in multiple sclerosis', Journal of

Neurology 253(16), pp. 31-36.

Ademiluyi, G., Rees, C.E. and Sheard, C.E. (2003) 'Evaluating the reliability and

validity of three tools to assess the quality of health information on the Internet', Patient

Education and Counseling 50(2), pp. 151-155.

Akerkar, S.M. and Bichile, L.S. (2004a) 'Doctor patient relationship: changing

dynamics in the information age', J Postgrad Med 50(2), pp. 120-2.

Akerkar, S.M. and Bichile, L.S. (2004b) 'Health information on the internet: patient

empowerment or patient deceit?', Indian J Med Sci 58(8), pp. 321-6.

Al-Sowielem, L. and Elzubier, A. (1998) 'Compliance and knowledge of hypertensive

patients attending PHC centres in Al-Khobar, Saudi Arabia', Eastern Mediterranean

Health Journal 4(2), pp. 301-307.

Ambre, J., Guard, R., Frances, M., Renner, J. and Rippen, H. (1998) Criteria for

Assessing the Quality of Health Information on the Internet. Health Summit Working

Group. Available at: http://www.mlanet.org/tech_is/meb/criteria.pdf (Accessed

23/07/2010).

Anderson, J., McKemmish, S. and Manaszewicz, R. (2003) QUALITY CRITERIA

MODELS USED TO EVALUATE HEALTH WEBSITES. In: 10th Asia Pacific Special

Health and Law Librarians Conference.

Anderson, J.G., Rainey, M.R. and Eysenbach, G. (2003) 'The impact of

CyberHealthcare on the physician-patient relationship', Journal of Medical Systems

27(1), pp. 67-84.

Anon. (1999) 'Health literacy: report of the Council on Scientific Affairs. Ad Hoc

Committee on Health Literacy for the Council on Scientific Affairs, American Medical

Association', JAMA 281(6), pp. 552-7.

APA. (1999) Standards for Educational and Psychological Testing. Washington, DC:

American Psychological Association.

Baker, L., Wagner, T.H., Singer, S. and Bundorf, M.K. (2003) 'Use of the Internet and

e-mail for health care information - Results from a national survey', Jama-Journal of the

American Medical Association 289(18), pp. 2400-2406.

Baker, L.M. (1996) 'A study of the nature of information needed by women with

multiple sclerosis', Library & Information Science Research 18(1), pp. 67-81.

74

Barry, L. (1994) 'User-Defined Relevance Criteria: An Exploratory Study.', The

Journal of the American society for Information Science pp. 149-159.

Bartlett, P.F. and Kilpatrick, T.J. (1991) 'NEUROIMMUNOLOGY OF

DEMYELINATING DISEASE', Current Opinion in Neurology and Neurosurgery 4(2),

pp. 181-185.

Bates, B.R., Romina, S., Ahmed, R. and Hopson, D. (2006) 'The effect of source

credibility on consumers' perceptions of the quality of health information on the Internet',

Medical Informatics and the Internet in Medicine 31(1), pp. 45-52.

Bauman, A. (1997) 'The comprehensibility of asthma education materials', Patient

Education and Counseling 32 pp. 51-59.

BBC. (2010) United Kingdom country profile BBC. Available at:

http://news.bbc.co.uk/1/hi/world/europe/country_profiles/1038758.stm (Accessed

12/5/2010).

Beekhuyzen, J. (2007) PUTTING THE PIECES OF THE PUZZLE TOGETHER:

USING NVIVO FOR A LITERATURE REVIEW. In: 4th QUALIT Conference Qualitative

Research in IT & IT in Qualitative Research. Wellington, New Zealand.

Berendt, B. and Spiliopoulou, M. (2000) 'Analysis of navigation behaviour in web sites

integrating multiple information systems', Vldb Journal 9(1), pp. 56-75.

Berg, K.E. and Latin, R.W. (2007) Essentials of Research Methods in Health, Physical

Education, Exercise Science, and Recreation. Lippincott Williams & Wilkins.

Berger, M., Wagner, T.H. and Baker, L.C. (2005) 'Internet use and stigmatized illness',

Social Science & Medicine 61(8), pp. 1821-1827.

Berk, A. and Rossi, H. (1990) Thinking About Program Evaluation. CA: SAGE.

Berkman, N.D., DeWalt, D.A., Pignone, M.P., Sheridan, S.L., Lohr, K.N., Lux, L.,

Sutton, S.F., Swinson, T. and Bonito, A.J. (2004) Literacy and Health Outcomes.

Evidence Report/Technology Assessment. AHRQ Publication. Available at:

http://www.ahrq.gov/downloads/pub/evidence/pdf/literacy/literacy.pdf (Accessed

28/07/2010).

Berland, G.K., Elliott, M.N., Morales, L.S., Algazy, J.I., Kravitz, R.L., Broder, M.S.,

Kanouse, D.E., Munoz, J.A., Puyol, J.A., Lara, M., Watkins, K.E., Yang, H. and

McGlynn, E.A. (2001) 'Health information on the Internet - Accessibility, quality, and

readability in English and Spanish', Jama-Journal of the American Medical Association

285(20), pp. 2612-2621.

75

Bichakjian, C.K., Schwartz, J.L., Wang, T.S., Hall, J.M., Johnson, T.M. and Biermann,

J.S. (2002) 'Melanoma information on the Internet: Often incomplete - A public health

opportunity?', Journal of Clinical Oncology 20(1), pp. 134-141.

Birchley, D., Pullan, R. and DeFriend, D. (2003) 'Patient attitudes to the Internet and

analysis of the potential role of a dedicated colorectal website - a prospective study',

Annals of the Royal College of Surgeons of England 85(6), pp. 398-401.

Bishop, M., Frain, M.P., Espinosa, C.T. and Stenhoff, D.M. (2009) 'Sources of

information about multiple sclerosis: Information seeking and personal, demographic,

and MS variables', Journal of Vocational Rehabilitation 31

Bliemel, M. and Hassanein, K. (2007) 'Consumer satisfaction with online health

information retrieval: A model and empirical study', E-Service Journal 5(2), pp. 83-84.

Boiko, A., Vorobeychik, G., Paty, D., Devonshire, V., Sadovnick, D. and Neurologists,

U.M.C. (2002) 'Early onset multiple sclerosis - A longitudinal study', Neurology 59(7),

pp. 1006-1010.

Bowling, A. (2009) Reserch methods in health investigating health and health service.

Buckingham: Open University Press.

Box, V., Hepworth, M. and Harrison, J. (2003) 'Identifying information needs of people

with multiple sclerosis', Nurs Times 99(49), pp. 32-6.

Brase, C.H. and Brase, C.P. (2007) Understandable Statistics: Concepts and Methods.

Cengage Learning.

Brashers, D.E., Goldsmith, D.J. and Hsieh, E. (2002) 'Information seeking and avoiding

in health contexts', Human Communication Research 28(2), pp. 258-271.

Briggs, P., Burford, B., De Angeli, A. and Lynch, P. (2002) 'Trust in online advice',

Social Science Computer Review 20(3), pp. 321-332.

Brink, H. and Walt, C.V.d. (2007) Fundamentals of Research Methodology for Health-

care Professionals. Juta and Company Ltd.

Brodal, P. (2010) The Central Nervous System: Structure and Function. New York:

Oxford University Press.

Brodie, M., Kjellson, N., Hoff, T. and Parker, M. (1999) 'Perceptions of Latinos,

African Americans, and Whites on Media as a Health Information Source', Howard

Journal of Communications 10(3), pp. 147 - 167.

Broom, A. (2005) 'Virtually he@lthy: The impact of Internet use on disease experience

and the doctor-patient relationship', Qualitative Health Research 15(3), pp. 325-345.

76

Brown, S.M., Culver, J.O., Osann, K.E., MacDonald, D.J., Sand, S., Thornton, A.A.,

Grant, M., Bowen, D.J., Metcalfe, K.A., Burke, H.B., Robson, M.E., Friedman, S. and

Weitzel, J.N. (2010) 'Health literacy, numeracy, and interpretation of graphical breast

cancer risk estimates', Patient Education and Counseling In Press, Corrected Proof

Bruce, H.W. (1994) 'A COGNITIVE VIEW OF THE SITUATIONAL DYNAMISM

OF USER-CENTERED RELEVANCE ESTIMATION', Journal of the American

Society for Information Science 45(3), pp. 142-148.

Bryman, A. (2008) Social Research Methods

New York: Oxford University Press.

Bryman, A. and Bell, E. (2007) Business research methods. Oxford: Oxford University

Press.

Bucy, E.P. (2000) 'Social access to the Internet', Harvard International Journal of

Press-Politics 5(1), pp. 50-61.

Burls, A. (2009) What is critical appraisal? Evidence-based medicine. Available at:

http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/What_is_critical_appr

aisal.pdf (Accessed 12/08/2010).

Burns, N. and Grove, S.K. (2005) The practice of nursing research: conduct, critique,

and utilization. Elsevier Health Sciences.

Bush, J., McAuley, A. and Pecaitis, F. (2000) 'Navigating the options', Health Manag

Technol 21(11), pp. 22, 24-5.

Campbell, R.J. and Nolfi, D.A. (2005) 'Teaching elderly adults to use the Internet to

access health care information: Before-after study', Journal of Medical Internet Research

7(2),

Cassell, C. and Symon, G. (1994), 'Qualitative research in work contexts', in (Ed, C.

Cassell, G. S.), Qualitative methods in organizational research Thousand Oaks, CA,

SAGE, pp.

Chen, X. and Siu, L.L. (2001) 'Impact of the media and the internet on oncology: survey

of cancer patients and oncologists in Canada', J Clin Oncol 19(23), pp. 4291-7.

Childs, S. (2005) 'Judging the quality of internet-based health information',

Performance Measurement and Metrics 6(2), pp. 80-96.

Chiovetti, A. (2006a) 'Bridging the gap between health literacy and patient education for

people with multiple sclerosis', J Neurosci Nurs 38(5), pp. 374-8.

Chiovetti, A. (2006b) 'Bridging the gap between health literacy and patient education

for people with multiple sclerosis', Journal of Neuroscience Nursing

77

Christopherson, K.M. (2007) 'The positive and negative implications of anonymity in

Internet social interactions: "On the Internet, Nobody Knows You're a Dog"', Computers

in Human Behavior 23(6), pp. 3038-3056.

Civan, A. and Pratt, W. (2006) Supporting Consumers by Characterizing the Quality of

Online Health Information: A Multidimensional Framework. In: Proceedings of the 39th

Hawaii International Conference on System Sciences.

Civan, A., Skeels, M.M., Stolyar, A. and Pratt, W. (2006) 'Personal health information

management: consumers' perspectives', AMIA Annu Symp Proc pp. 156-60.

Clarke, A. and Dawson, R. (1999) Evaluation research: an introduction to principles,

methods, and practice London: SAGE.

Cline, R.J. and Haynes, K.M. (2001a) 'Consumer health information seeking on the

Internet: the state of the art', Health Educ Res 16(6), pp. 671-92.

Cline, R.J.W. and Haynes, K.M. (2001b) 'Consumer health information seeking on the

Internet: the state of the art', Health Education Research 16(6), pp. 671-692.

Coey, L. (1996) 'Readability of printed educational materials used to inform potential

and actual ostomates', Journal of Clinical Nursing 5(6), pp. 359-366.

Colledge, A., Car, J., Donnelly, A. and Majeed, A. (2008) 'Health information for

patients: time to look beyond patient information leaf lets', Journal of the Royal Society

of Medicine 101(9), pp. 447-453.

Commerce, U.D.o. (1998) Falling Through the Net, II: New Data on the Digital Divide.

National Telecommunications and Information Administration. Available at:

http://www.ntia.doc.gov/ntiahome/net2/falling.html (Accessed 24/12/2009).

Compston, A. and Coles, A. (2008) 'Multiple sclerosis', The Lancet 372(9648), pp.

1502-1517.

Compston, A., Noseworthy, J., Hans Lassmann, Miller, D., Smith, K., Wekerle, H. and

Confavreux, C. (2005a) McAlpine„s Multiple Sclerosis. London: Churchill Livingstone

Elsevier.

Compston, A., Noseworthy, J., Hans Lassmann, Miller, D., Smith, K., Wekerle, H. and

Confavreux, C. (2005b), 'The story of Multiple Sclerosis', in McAlpine„s Multiple

Sclerosis London, Churchill Livingstone Elsevier, pp. 3-62.

Cooper, D.R. and Schindler, P.S. (2003) Business research methods. New York:

Mcgraw-Hill.

Cooper, H., Booth, K., Fear, S. and Gill, G. (2001) 'Chronic disease patient education:

lessons from meta-analyses', Patient Education and Counseling 44(2), pp. 107-117.

78

Cotten, S.R. and Gupta, S.S. (2004) 'Characteristics of online and offline health

information seekers and factors that discriminate between them', Social Science &

Medicine 59(9), pp. 1795-1806.

Coulter, A. (2002) The Autonomous Patient: Ending Paternalism in Medical Care.

London: Stationery Office.

Coulter, A. and Ellins, J. (2006) Patient-focused interventions: A review of the evidence

The Health Foundation, 2010. Available at:

http://www.health.org.uk/publications/research_reports/patientfocused.html (Accessed

28/07/2010).

COULTER, A., ELLINS, J., SWAIN, D., CLARKE, A., HERON, P., RASUL, F.,

MAGEE, H. and SHELDON, H. (2006) Assessing the quality of information to support

people in making decisions about their health and healthcare. Public Health Resource

Unit. Available at:

http://www.pickereurope.org/Filestore/PIE_reports/project_reports/Health-information-

quality-web-version-FINAL.pdf (Accessed 26/06/2010).

Coulter, A., Entwistle, V. and Gilbert, D. (1999) 'Sharing decisions with patients: Is the

information good enough?', British Medical Journal 318(7179), pp. 318-322.

Courtney, A.M., Treadaway, K., Remington, G. and Frohman, E. (2009) 'Multiple

Sclerosis', Medical Clinics of North America 93(2), pp. 451-476.

Crespo, J. (2004) 'Training the health information seeker: Quality issues in health

information web sites', Library Trends 53(2), pp. 360-374.

Creswell, W. (1994) Research Design: Qualitative & Quantitative Approaches.

London: SAGE.

Croft, D.R. and Peterson, M.W. (2002) 'An evaluation of the quality and contents of

asthma education on the World Wide Web', Chest 121(4), pp. 1301-1307.

CUADRA, C.A. and KATTER, R.V. (1993) 'OPENING THE BLACK BOX OF

„RELEVANCE', Journal of Documentation 23(4), pp. 291-303.

Dattalo, P. (2008) Determining Sample Size: Balancing Power, Precision, and

Practicality. Oxford University Press

Daumer, M., Neuhaus, A., Lederer, C., Scholz, M., Wolinsky, J.S., Heiderhoff, M. and

Sylvia Lawry Centre Multiple, S. (2007) 'Prognosis of the individual course of disease -

steps in developing a decision support tool for Multiple Sclerosis', BMC Medical

Informatics and Decision Making 7

79

Davies, M. and Macdowall, W. (2006) Health Promotion Theory. Maidenhead: Open

University Press.

Deacon, D., Pickering, M., Golding, P. and Murdock, G. (2007) Researching

Communications: A Practical Guide to Methods in Media and Cultural Analysis.

London: Hodder Arnold.

Dearness, K.L. and Tomlin, A. (2001) 'Development of the national electronic library

for mental health: providing evidence-based information for all', Health Info Libr J

18(3), pp. 167-74.

DeFleur, M.L. and Cronin, M.M. (1991) 'Completeness and Accuracy of Recall in the

Diffusion of the News From a Newspaper vs. a Television Source', Sociological Inquiry

61(2), pp. 148-166.

Department of Health. (2004) Better information, better choices, better health Putting

information at the centre of health. Department of Health. Available at:

http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/documents/di

gitalasset/dh_4098599.pdf (Accessed 16/07/2010).

Dervin. (1992), 'From the mind's eye of the user: the sense-making qualitative-

quantitative methodology', in (Eds, Glazier, J. D. and Powell, R. R.), Qualitative

Research in Information Management, Libraries Unlimited, pp. 61-84.

Dervin, B. (1983) AN OVERVIEW OF SENSE-MAKING RESEARCH: CONCEPTS,

METHODS, AND RESULTS TO DATE. In: International Communication Association

annual meeting. Dallas.

DfEE. (1999) A fresh start: Improving numeracy and literacy. Department for

Education and Employment (DfEE. Available at:

http://www.nrdc.org.uk/anr_details.asp?ID=135 (Accessed 29/07/2010).

Diaz, J.A., Friedmann, P.D. and Moulton, A.W. (2000) 'Patients' use of the Internet for

medical information', Journal of General Internal Medicine 15 pp. 110-110.

Doian, G.C. 2006 University of Glamorgan, pp. 323.

Dolan, G., Iredale, R., Williams, R. and Ameen, J. (2004) 'Consumer use of the internet

for health information: a survey of primary care patients', International Journal of

Consumer Studies 28(2), pp. 147-153.

Donohue, J.M., Huskamp, H.A., Wilson, I.B. and Weissman, J. (2009) 'Whom Do Older

Adults Trust Most to Provide Information About Prescription Drugs?', American Journal

of Geriatric Pharmacotherapy 7(2), pp. 105-116.

Dougal, L. (2006) Does Level of Educational Attainment and Readability Level of Text

Affect Understanding Of Health Leaflets? A Questionnaire Based Study Using

80

Readability Formulae. Available at: http://www.era.lib.ed.ac.uk/handle/1842/2363

(Accessed 14/5/2010).

Dreger, V. and Tremback, T. (2002) 'Optimize patient health by treating literacy and

language barriers', Aorn J 75(2), pp. 280-5, 287, 289-93; quiz 297-300, 303-4.

Dumitru, R.C., Buerkle, T., Potapov, S., Lausen, B., Wiese, B. and Prokosch, H.U.

(2007) 'Use and perception of Internet for health related purposes in Germany: results of

a national survey', International Journal of Public Health 52 pp. 275-285.

Dutta-Bergman, M. (2003a) 'Trusted online sources of health information: differences in

demographics, health beliefs, and health-information orientation', J Med Internet Res

5(3), pp. e21.

Dutta-Bergman, M.J. (2003b) 'Health communication on the web: The roles of web use

motivation and information completeness', Communication Monographs 70(3), pp. 264 -

274.

Dutta-Bergman, M.J. (2004) 'Primary sources of health information: Comparisons in the

domain of health attitudes, health cognitions, and health behaviors', Health

Communication 16(3), pp. 273-288.

Dutta, M. 2001, 'The impact of Internet information completeness: The moderating role

of Web use motivation', UNIVERSITY OF MINNESOTA,

Dutta, M.J., Pfister, R. and Kosmoski, C. (2010) 'Consumer Evaluation of Genetic

Information Online: The Role of Quality on Attitude and Behavioral Intentions', Journal

of Computer-Mediated Communication 15(4), pp. 592-605.

Ellins, J. and Coulter, A. (2005) How engaged are people in their healthcare. Health

Foundation. Available at:

http://www.health.org.uk/publications/research_reports/how_engaged_are.html

(Accessed 26/07/2010).

Elliott, J.O. and Shneker, B.F. (2009) 'A health literacy assessment of the epilepsy.com

website', Seizure-European Journal of Epilepsy 18(6), pp. 434-439.

Ellis, M.J.H. and Thomson, C.E. (2003) 'Consumer health information on the WWW: an

evaluation of information on verrucae', 13(3), pp. 130-135.

Elwyn, G., O'Connor, A., Stacey, D., Volk, R., Edwards, A., Coulter, A. and

Collaboration, I. (2006) 'Developing a quality criteria framework for patient decision

aids: online international Delphi consensus process', British Medical Journal 333(7565),

pp. 417-419.

81

Ende J, Lewis K, Ash A and M, M. (1989) 'Measuring patients‟ desire for autonomy:

Decision making and information seeking preferences among medical patients', J Gen

Intern Med

Eng, T.R., Maxfield, A., Patrick, K., Deering, M.J., Ratzan, S.C. and Gustafson, D.H.

(1998) 'Access to Health Information and Support: A Public Highway or a Private

Road?', JAMA 280(15), pp. 1371-1375.

Eppler, M. and Muenzenmayer, P. (2002) Measuring information quality in the web

context: A survey of state-of-the-art instruments and an application methodology. In: 7th

International Conference on Information Quality.

Estrada, C.A., Hryniewicz, M.M., Higgs, V.B., Collins, C. and Byrd, J.C. (2000)

'Anticoagulant patient information material is written at high readability levels', Stroke

31(12), pp. 2966-2970.

European Commission. (2002) 'eEurope 2002: Quality Criteria for Health Related

Websites', J Med Internet Res 4(3), pp. E15.

Eysenbach, G. (2003) 'The impact of the Internet on cancer outcomes', Ca-a Cancer

Journal for Clinicians 53(6), pp. 356-371.

Eysenbach, G. and Kohler, C. (2002a) 'How do consumers search for and appraise

health information on the world wide web? Qualitative study using focus groups,

usability tests, and in-depth interviews', BMJ 324(7337), pp. 573-577.

Eysenbach, G. and Kohler, C. (2002b) 'How do consumers search for and appraise

health information on the world wide web? Qualitative study using focus groups,

usability tests, and in-depth interviews', British Medical Journal 324(7337), pp. 573-577.

Eysenbach, G. and Kohler, C. (2003) 'What is the prevalence of health-related searches

on the World Wide Web? Qualitative and quantitative analysis of search engine queries

on the internet', AMIA Annu Symp Proc pp. 225-9.

Eysenbach, G., Powell, J., Kuss, O. and Sa, E.-R. (2002) 'Empirical Studies Assessing

the Quality of Health Information for Consumers on the World Wide Web: A Systematic

Review', JAMA 287(20), pp. 2691-2700.

Fant, K.E., Clark, S.J. and Kemper, A.R. (2005) 'Completeness and complexity of

information available to parents from newborn-screening programs', Pediatrics 115(5),

pp. 1268-1272.

Feathers, K.M. and Rivers, D. (2004) Infotext: reading and learning. Pippin Publishing

Corporation.

Fogg, B.J., Marable, L., Stanford, J. and Tauber, E.R. (2002) How Do People Evaluate

a Web Site‟s Credibility? Results from a Large Study. Stanford Persuasive Technology

82

Lab, Stanford University. Available at:

http://www.consumerwebwatch.org/pdfs/stanfordPTL.pdf (Accessed 25/07/2010).

Forbes, A., While, A., Dyson, L., Grocott, T. and Griffiths, P. (2003) 'Impact of clinical

nurse specialists in multiple sclerosis - synthesis of the evidence', Journal of Advanced

Nursing 42(5), pp. 442-462.

Forbes, A., While, A. and Taylor, M. (2007) 'What people with multiple sclerosis

perceive to be important to meeting their needs', Journal of Advanced Nursing 58(1), pp.

11-22.

Forbis, S.G. and Aligne, C.A. (2002) 'Poor readability of written asthma management

plans found in national guidelines', Pediatrics 109(4),

Forsythe, D.E. (1992) 'Using ethnography to build a working system: rethinking basic

design assumptions', Proc Annu Symp Comput Appl Med Care pp. 505-9.

Fox, S. (2005) Health Information Online. Pew internet & american life project.

Available at:

http://www.pewinternet.org/~/media//Files/Reports/2005/PIP_Healthtopics_May05.pdf.p

df (Accessed 29/12/2009).

Fox, S. (2006a) Online Health Search 2006. PEW Internet and American Life.

Available at:

http://www.pewinternet.org/~/media/Files/Reports/2006/PIP_Online_Health_2006.pdf.pd

f (Accessed 11/5/2010).

Fox, S. (2006b) Online health search 2006. Pew Internet and American Life Project.

Available at: http://www.pewinternet.org/Reports/2006/Online-Health-Search-2006.aspx

(Accessed 18/12/2009).

Fox, S. and Fallows, D. (2003a) Health searches and email have become more

commonplace, but there is room for improvement in searches and overall Internet access.

PEW INTERNET & AMERICAN LIFE PROJECT. Available at:

http://www.pewinternet.org/Reports/2003/Internet-Health-Resources.aspx?r=1 (Accessed

19/12/2009).

Fox, S. and Fallows, D. (2003b) Internet Health Resources. PEW INTERNET &

AMERICAN LIFE PROJECT. Available at:

http://www.pewinternet.org/~/media//Files/Reports/2003/PIP_Health_Report_July_2003.

pdf.pdf (Accessed 28/12/2009).

Fox, S. and Fallows, D. (2003c) Internet health resources. Pew Internet and American

Life Project. Available at: http://www.pewinternet.org (Accessed 18/07/2010).

Fox, S. and Raine, L. (2001) The online health care revolution: How the Web helps

Americans take better care of themselves. Pew Internet & American Life Project.

83

Available at: http://www.pewinternet.org/Reports/2000/The-Online-Health-Care-

Revolution.aspx (Accessed 23/12/2009).

Fox, S. and Rainie, L. (2002) Vital decisions: How Internet users decide what

information to trust when they or their loved ones are sick. Pew Internet and American

Life Project. Available at:

http://www.pewinternet.org/~/media//Files/Reports/2002/PIP_Vital_Decisions_May2002

.pdf.pdf (Accessed 11/5/2010).

Fox, S. and Rainie, L. (2006) Vital decisions: how Internet users decide what

information to trust when they or their loved ones are sick. Internet and American Life

Project. Available at: http://www.pewinternet.org (Accessed 15/07/2010).

Fox;, S. and Fallows;, D. (2003) Health searches and email have become more

commonplace, but there is room for improvement in searches and overall Internet access.

PEW INTERNET & AMERICAN LIFE PROJECT. Available at:

http://www.pewinternet.org/Reports/2003/Internet-Health-Resources.aspx?r=1 (Accessed

19/12/2009).

Frangos, C. (2009) Proceedings of the 2nd International Conference: Quantitative and

Qualitative Methodologies in the Economic and Administrative Sciences. Athens:

Frankel, R. and Devers, K. (2000) 'Qualitative research: a consumer's guide', Educ

Health (Abingdon). 13(1), pp. 113.

Frankfort-Nachmias, C. and Nachmias, D. (1994) Research methods in the social

sciences (4th ed.). New York: St. Martin's Press.

Freda, M.C. (2002) Perinatal Patient Education: A Practical Guide with Education

Handouts for Patients. Lippincott Williams & Wilkins.

French, K.S. and Larrabee, J.H. (1999) 'Relationships among educational material

readability, client literacy, perceived beneficence, and perceived quality', Journal of

Nursing Care Quality 13(6), pp. 68-82.

Frohman, E.M., Racke, M.K. and Raine, C.S. (2006) 'Multiple Sclerosis -- The Plaque

and Its Pathogenesis', N Engl J Med 354(9), pp. 942-955.

Gagliardi, A. and Jadad, A.R. (2002) 'Examination of instruments used to rate quality of

health information on the internet: chronicle of a voyage with an unclear destination',

British Medical Journal 324(7337), pp. 569-573.

Gemoets, D., Rosemblat, G., Tse, T. and Logan, R. (2004) 'Assessing readability of

consumer health information: An exploratory study', Medinfo 2004: Proceedings of the

11th World Congress on Medical Informatics, Pt 1 and 2 107 pp. 869-873.

84

Gilmour, J.A. (2007) 'Reducing disparities in the access and use of Internet health

information. A discussion paper', International Journal of Nursing Studies 44(7), pp.

1270-1278.

Glowacki, K. (2009) Multiple Sclerosis. Available at:

www.acuarts.com/files/Wei_Syndrome_2.pdf (Accessed 7/5/2010).

Godibile, A., d‟Ivernois, J.-F., Deccache, A., Tamara Cirillo a, M.-. and Albano, G.

(2005) Studies on therapeutic patient education in chronic diseases : a survey of

literature frome 1997 to 2004. Center of Health Sciences Education. Available at:

http://hal.archives-

ouvertes.fr/docs/00/15/11/41/PDF/Studies_on_therapeutic_patient_education_in_chronic

_diseases_a_survey_of_literature_from_1997_to_2004.pdf (Accessed 8/5/2010).

Google. (2010a) PageRank Technology. Google. Available at:

http://www.google.co.uk/corporate/tech.html (Accessed 12/5/2010).

Google. (2010b) Search-based Keyword Tool Help. Google. Available at:

http://www.google.com/support/sktool/bin/topic.py?topic=15806 (Accessed 12/5/2010).

Gottlieb, R. and Rogers, J.L. (2004) 'Readability of Health Sites on the Internet',

International Electronic Journal of Health Education 7 pp. 38-42.

Goudswaard, A.N., Stolk, R.P., Zuithoff, N.P.A., de Valk, H.W. and Rutten, G. (2004)

'Long-term effects of self-management education for patients with Type 2 diabetes taking

maximal oral hypoglycaemic therapy: a randomized trial in primary care', Diabetic

Medicine 21(5), pp. 491-496.

Graber, M.A., D'Alessandro, D.M. and Johnson-West, J. (2002) 'Reading level of

privacy policies on Internet health Web sites', Journal of Family Practice 51(7), pp. 642-

645.

Graber, M.A., Roller, C.M. and Kaeble, B. (1999a) 'Readability levels of patient

education material on the World Wide Web', Journal of Family Practice 48(1), pp. 58-

61.

Graber, M.A., Roller, C.M. and Kaeble, B. (1999b) 'Readability Levels of Patient

Education Material on the World Wide Web', Journal of Family Practice 155 pp. 807-

812.

Gravetter, F.J. and Forzano, L.-A.B. (2008) Research Methods for the Behavioral

Sciences Cengage Learning.

Gray, N.J., Klein, J.D., Noyce, P.R., Sesselberg, T.S. and Cantrill, J.A. (2005) 'Health

information-seeking behaviour in adolescence: the place of the internet', Social Science

& Medicine 60(7), pp. 1467-1478.

85

Gregory, R., Disler, P. and Firth, S. (1996) 'Wellbeing and multiple sclerosis findings of

a survey in the Manawatu Wanganui area of New Zealand', Community Mental Health in

New Zealand 9 pp. 32-42.

Greywoode, J., Bluman, E., Spiegel, J. and Boon, M. (2009) 'Readability analysis of

patient information on the American Academy of Otolaryngology-Head and Neck

Surgery website', Otolaryngol Head Neck Surg 141(5), pp. 555-8.

Gross, D.P., Ferrari, R., Russell, A.S., Battie, M.C., Schopflocher, D., Hu, R.W.,

Waddell, G. and Buchbinder, R. (2006) 'A population-based survey of back pain beliefs

in Canada', Spine 31(18), pp. 2142-2145.

Gruesser, M., Hartmann, P., Schlottmann, N., Lohmann, F.W., Sawicki, P.T. and

Joergens, V. (1997) 'Structured patient education for out-patients with hypertension in

general practice: a model project in Germany', Journal of Human Hypertension 11(8),

pp. 501-506.

Gustafson, D.H., Robinson, T.N., Ansley, D., Adler, L., Brennan, P.F. and Sci Panel

Interactive Commun, H. (1999) 'Consumers and evaluation of interactive health

communication applications', American Journal of Preventive Medicine 16(1), pp. 23-

29.

Hanif, F., Read, J.C., Goodacre, J.A., Chaudhry, A. and Gibbs, P. (2009) 'The role of

quality tools in assessing reliability of the Internet for health information', Informatics

for Health & Social Care 34(4), pp. 231-243.

Harland, J. and Bath, P. (2007) 'Assessing the quality of websites providing information

on multiple sclerosis: evaluating tools and comparing sites', Health Informatics J 13(3),

pp. 207-21.

Hart, A., Henwood, F. and Wyatt, S. (2004) 'The role of the Internet in patient-

practitioner relationships: findings from a qualitative research study', J Med Internet Res

6(3), pp. e36.

Hatzakis, M.J., Allen, C., Haselkorn, M., Anderson, S.M., Nichol, P., Lai, C. and

Haselkorn, J.K. (2006) 'Use of medical informatics for management of multiple

sclerosis using a chronic-care model', Journal of Rehabilitation Research and

Development 43(1), pp. 1-15.

Hawkins, D. (1999) 'What Is Credible Information?', Online 23 pp. 86-89.

Hay, M.C., Strathmann, C., Lieber, E., Wick, K. and Giesser, B. (2008) 'Why Patients

Go Online Multiple Sclerosis, the Internet, and Physician-Patient Communication',

Neurologist 14(6), pp. 374-381.

86

Health;, S.P.o.I.C.a. (1999) Wired for health and wellbeing:The emergence of

interactive health communication. Department of Health and Human Services, U.S.

Government Printing Office. Available at: http://classweb.gmu.edu/gkreps/820/023.pdf

(Accessed 22/12/2009).

Heath, T. 2008, 'Information-seeking on the Web with Trusted Social Networks – from

Theory to Systems', The Open University,

Hedman, A.S. (2008) 'Using the SMOG Formula to Revise a Health-Related Document',

American Journal of Health Education 39 pp. 61-64.

Heesen, C., Kopke, S., Richter, T. and Kasper, J. (2007) 'Shared decision making and

self-management in multiple sclerosis a consequence of evidence', Journal of Neurology

254 pp. 116-121.

Hellawell, G.O., Turner, K.J., Le Monnier, K.J. and Brewster, S.F. (2000) 'Urology and

the Internet: an evaluation of Internet use by urology patients and of information

available on urological topics', Bju International 86(3), pp. 191-194.

Henry, G. (1990) Practical Sampling. Newbury Park: SAGE.

Henwood, F., Wyatt, S., Hart, A. and Smith, J. (2003) ''Ignorance is bliss sometimes':

constraints on the emergence of the 'informed patient' in the changing landscapes of

health information', Sociology of Health & Illness 25(6), pp. 589-607.

Heppner, P.P., Wampold, B.E. and Kivlighan, D.M. (2007) Research design in

counseling. Cengage Learning.

Hepworth, M., Harrison, J. and James, N. (2003) 'Information needs of people with

multiple sclerosis and the implications for information provision based on a national UK

survey', Aslib Proceedings 55(5-6), pp. 290-303.

HERRMAN-ROSE, B. 2008, 'SOCIOCULTURAL TAILORING IN BREAST CANCER

WEBSITES: A CONTENT ANALYSIS', the University of Missouri - Columbia,

Columbia.

Hesse, B.W., Nelson, D.E., Kreps, G.L., Croyle, R.T., Arora, N.K., Rimer, B.K. and

Viswanath, K. (2005) 'Trust and sources of health information - The impact of the

Internet and its implications for health care providers: Findings from the first Health

Information National Trends Survey', Archives of Internal Medicine 165(22), pp. 2618-

2624.

Hill, A. and Spittlehouse, C. (2003) 'What is critical appraisal?', Evidence-Based

Medicine 3(2), pp. 1-8.

Hirsh, S.G. (1999) 'Children's relevance criteria and information seeking on electronic

resources', Journal of the American Society for Information Science 50(14), pp. 1265-

1283.

87

Hoffmann, T., Twork, S., Pöhlau, D. and Kugler, J. (2009) 'Patientenorientierung im

Internet – qualitative Bewertung von Internetseiten für Multiple-Sklerose-Betroffene',

Patient-Centered Information in the Internet – A Qualitative Assessment of Websites for

Patients with Multiple Sclerosis 36(04), pp. 151-157.

Hoffmann, T. and Worrall, L. (2004) 'Designing effective written health education

materials: Considerations for health professionals', Disability and Rehabilitation 26(19),

pp. 1166-1173.

Homsi, M. and Kaplan-Myrth, A. (2005) Online Anonymity and John Doe lawsuits.

Available at: http://www.cippic.ca/online-anonymity-and-john-doe-lawsuits/ (Accessed

31/12/2009).

Howard, J., Wildman, K., Blain, J., Wills, S. and Brown, D. (1999) 'The importance of

drug information from a patient perspective', Journal of Social and Administrative

Pharmacy 16 pp. 115-126.

Huang, L.-H., Chen, S.-W., Yu, Y.-P., Chen, P.-R. and Lin, Y.-C. (2002) 'The

effectiveness of health promotion education programs for community elderly', J Nurs

Res 10(4), pp. 261-70.

Huntington, P., Nicholas, D., Jamali, H.R. and Russell, C. (2007) 'Health information

for the consumer: NHS vs the BBC', Aslib Proceedings 59(1), pp. 46-67.

Hussain, N., Agyeman, A. and Das Carlo, M. (2004) 'Access, attitudes, and concerns of

physicians and patients toward e-mail use in health-related communication', Tex Med

100(3), pp. 50-7.

Ibrahim, S.Y., Reid, F., Shaw, A., Rowlands, G., Gomez, G.B., Chesnokov, M. and

Ussher, M. (2008) 'Validation of a health literacy screening tool (REALM) in a UK

Population with coronary heart disease', Journal of Public Health 30(4), pp. 449-455.

Iezzoni, L.I. (2010) Multiple Sclerosis. Greenwood.

Impicciatore, P., Pandolfini, C., Casella, N. and Bonati, M. (1997) 'Reliability of health

information for the public on the world wide web: Systematic survey of advice on

managing fever in children at home', British Medical Journal 314(7098), pp. 1875-1879.

Interactive, H. (2008) Number of Cyberchondriacs Adults going Online for Health

Information. Harris Interactive. Available at:

http://www.harrisinteractive.com/vault/HI_HealthCareNews2008Vol8_Iss08.pdf

(Accessed 18/07/2010).

iProspect. (2006) iProspect Search Engine User Behavior Study. iProspect.com, Inc.

Available at: http://www.iprospect.com/about/whitepaper_seuserbehavior_apr06.htm

(Accessed 12/5/2010).

88

Ishikawa, H., Takeuchi, T. and Yano, E. (2008) 'Measuring functional, communicative,

and critical health literacy among diabetic patients', Diabetes Care 31(5), pp. 874-879.

Jackie Green, J.S. (2006) Evaluation. McGraw-Hill International.

Jackson, S.L. (2007) Research methods: a modular approach. Cengage Learning.

Jansen, S.J.T., Otten, W. and Stiggelbout, A.M. (2004) 'Review of determinants of

patients preferences for adjuvant therapy in cancer', Journal of Clinical Oncology

22(15), pp. 3181-3190.

John, C., Stephen, R. and Andrew, S. (2001) Building a Level Playing Field: The Need

To Expand and Improve the National and State Adult Education and Literacy Systems.

NCSALL Occasional. Mass: National Center for the Study of Adult Learning and

Literacy. Available at: www.ncsall.net/fileadmin/resources/research/op_comings2.pdf

(Accessed 29/07/2010).

John, M.S., David, J.K., Lynn, A.H. and Craig, C. (1995) 'Comprehension of Discharge

Instructions by Patients in an Urban Emergency Department', Annals of emergency

medicine 25(1), pp. 71-74.

Johnson, B., Christensen, L. and Christensen, L.B. (2010) Educational Research:

Quantitative, Qualitative, and Mixed Approaches SAGE.

Joinson, A.N. (2001) 'Self-disclosure in computer-mediated communication: The role of

self-awareness and visual anonymity', European Journal of Social Psychology 31(2), pp.

177-192.

Jonassen, D.H. (2004) HANDBOOK OF RESEARCH ON EDUCATIONAL

COMMUNICATIONS AND TECHNOLOGY A Project of the Association for Educational

Communications and Technology Lawrence Erlbaum Associates.

Jones, J. (1999) 'Development of a self-assessment method for patients to evaluate

health information on the Internet', Journal of the American Medical Informatics

Association pp. 540-544.

Kahn, K., Strong, M. and Wang, Y. (2002) 'Information quality benchmarks: Product

and service performance.', Communications of the ACM 45(4), pp. 84-192.

Katerattanakul, P. and Siau, K. (1999) Measuring information quality of web sites:

Development of an instrument. In: Proceedings of the 20th international conference on

Information Systems. Charlotte, North Carolina, United States.

Kazerooni, E.A. (2001) 'Population and Sample', Am. J. Roentgenol. 177(5), pp. 993-

999.

Keith F. Punch, K.P. (2006) Developing effective research proposals. SAGE.

Kersten, P., McLellan, D.L., Gross-Paju, K., Grigoriadis, N., Bencivenga, R., Beneton,

C., Charlier, M., Ketelaer, P. and Thompson, A.J. (2000) 'A questionnaire assessment of

89

unmet needs for rehabilitation services and resources for people with multiple sclerosis:

results of a pilot survey in five European countries', Clinical Rehabilitation 14(1), pp.

42-49.

Key, J.P. (2002) Research Design in Occupational Education. Available at:

http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/index.htm (Accessed

04/08/2010).

Khechine, H., Pascot, D. and Premont, P. (2008) 'Use of health-related information from

the Internet by English-speaking patients', Health Informatics J 14(1), pp. 17-28.

Kiley, R. (2000) 'Finding health information on the Internet: health consumers', Hosp

Med 61(11), pp. 799-801.

Kim, P., Eng, T.R., Deering, M.J. and Maxfield, A. (1999) 'Published criteria for

evaluating health related web sites: review', British Medical Journal 318(7184), pp. 647-

649.

KJ;, R. (1999) 'Patient Empowerment in the United States: A Critical Commentary',

Health Expectations, 2 pp. 82-92.

Klein, B.D. (2001) 'User perceptions of data quality: Internet and traditional text

sources', Journal of Computer Information Systems 41(4), pp. 9-15.

Klein, D. (2002) When do users detect information quality problems on the World Wide

Web? In: American Conference in Information Systems.

Knight, S.-a. and Burn, J. (2005) 'Developing a Framework for Assessing Information

Quality on the World Wide Web', Informing Science Journal 8

Kohlmaier, B. 2008, 'A quantitative evaluation of the accuracy, transparency, readability

and navigability of online consumer health information about coeliac', Unpublished MSc

thesis, Swansea University,

Korpac, A., Bisanz, G., Dukewich, T., Robinson, K., Bisanz, J., Thibodeau, M., Hubbard,

K. and Leighton, J. (1994) Assessing Scientific Literacy: A Taxonomy for Classifying

Questions and Knowledge About Scientific Research. Technical Report University of

Alberta. Available at:

http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtS

earch_SearchValue_0=ED381341&ERICExtSearch_SearchType_0=no&accno=ED3813

41 (Accessed 1).

Kummervold, P.E., Chronaki, C.E., Lausen, B., Prokosch, H.U., Rasmussen, J., Santana,

S., Staniszewski, A. and Wangberg, S.C. (2008) 'eHealth Trends in Europe 2005-2007:

A Population-Based Survey', Journal of Medical Internet Research 10(4),

90

Kurtzke, J.F. (1983) 'RATING NEUROLOGIC IMPAIRMENT IN MULTIPLE-

SCLEROSIS - AN EXPANDED DISABILITY STATUS SCALE (EDSS)', Neurology

33(11), pp. 1444-1452.

Kušec, S. (2004) 'Diabetes-related websites:are they readable?', Diabetic Voice 29(3),

Labovitch, R.S., Bozic, K.J. and Hansen, E. (2006) 'An evaluation of information

available on the Internet regarding minimally invasive hip arthroplasty', Journal of

Arthroplasty 21(1), pp. 1-5.

Laing, A., Hogg, G. and Winkelman, D. (2004) 'Healthcare and the information

revolution: re-configuring the healthcare service encounter', Health Serv Manage Res

17(3), pp. 188-99.

Larkin, M. (2002) 'Completeness of web-based melanoma information questioned',

Lancet 359(9300), pp. 50-50.

LAUGHLIN, G.H.M. (1969) 'SMOG grading: a new readability formula', JOURNAL

OF READING pp. 939-946.

Lee, P.P. (1999) 'Why literacy matters - Links between reading ability and health',

Archives of Ophthalmology 117(1), pp. 100-103.

Lee, R.G. and Garvin, T. (2003) 'Moving from information transfer to information

exchange in health and health care', Social Science & Medicine 56(3), pp. 449-464.

Lee, Y.W., Strong, D.M., Kahn, B.K. and Wang, R.Y. (2002) 'AIMQ: a methodology

for information quality assessment', Information & Management 40(2), pp. 133-146.

Leger, A.S. and Walsworth-Bell, J. (1999) Change Promoting Research for Health

Services. Buckingham: Open University Press.

Lejbkowicz, I., Paperna, T., Stein, N., Dishon, S. and Miller, A. (2010) 'Internet Usage

by Patients withMultiple Sclerosis: Implications to ParticipatoryMedicine and

Personalized Healthcare', Multiple Sclerosis International 2010 pp. 7.

Lemire, M., Pare, G., Sicotte, C. and Harvey, C. (2008) 'Determinants of Internet use as

a preferred source of information on personal health', International Journal of Medical

Informatics 77(11), pp. 723-734.

Lenhart, A. (2000) Who's Not Online: 57% of those without Internet access say they do

not plan to log on. Pew Internet and American Life Project. Available at:

http://www.pewinternet.org/~/media//Files/Reports/2000/Pew_Those_Not_Online_Repor

t.pdf.pdf (Accessed 25/12/2009).

Lenhart, A. (2003) Ever-Shifting Internet Population. Pew Internet & American Life

Project. Available at:

91

http://www.pewinternet.org/~/media//Files/Reports/2003/PIP_Shifting_Net_Pop_Report.

pdf.pdf (Accessed 30/12/2009).

Ley, P. and Florio, T. (1996) 'The use of readability formulas in health care',

Psychology, Health & Medicine 1(1), pp. 7 - 28.

Li, H.R. and Bukovac, J.L. (1999) 'Cognitive impact of banner ad characteristics: An

experimental study', Journalism & Mass Communication Quarterly 76(2), pp. 341-353.

Littleton, D. (1998) 'A review of strategies for finding health information on the World-

Wide Web', Med Ref Serv Q 17(2), pp. 51-5.

Liu, Z.M. and Huang, X.B. (2005) 'Evaluating the credibility of scholarly information

on the web: A cross cultural study', International Information & Library Review 37(2),

pp. 99-106.

Lublin, F.D. and Reingold, S.C. (1996) 'Defining the clinical course of multiple

sclerosis: Results of an international survey', Neurology 46(4), pp. 907-911.

Mack, N., Woodsong, C., MacQueen, K.M., Guest, G. and Namey, E. (2004)

Qualitative Research Methods: A Data Collector‟s Field Guide. FHI website. Available

at:

http://www.fhi.org/NR/rdonlyres/etl7vogszehu5s4stpzb3tyqlpp7rojv4waq37elpbyei3tgm

c4ty6dunbccfzxtaj2rvbaubzmz4f/overview1.pdf (Accessed 10/5/2010).

MacLean, R. and Russell, A. (2005) 'Innovative ways of responding to the information

needs of people with MS', Br J Nurs 14(14), pp. 754-7.

Manasian, D. (2003) 'Digital dilemmas: a survey of the Internet society', Engineering

Management Review, IEEE 31(3), pp. 84.

Markby, D.P. (1987) 'DISTRIBUTION OF MULTIPLE-SCLEROSIS IN THE

UNITED-KINGDOM', Journal of Neurology Neurosurgery and Psychiatry 50(4), pp.

505-506.

Marrie, R.A. and Goldman, M. (2007) 'Validity of performance scales for disability

assessment in multiple sclerosis', Multiple Sclerosis 13(9), pp. 1176-1182.

Mathur, S., Shanti, N., Brkaric, M., Sood, V., Kubeck, J., Paulino, C. and Merola, A.A.

(2005) 'Surfing for scoliosis: The quality of information available on the Internet', Spine

30(23), pp. 2695-2700.

Matson, R.R. and Brooks, N.A. (1977) 'Adjusting to multiple sclerosis: An exploratory

study', Social Science & Medicine (1967) 11(4), pp. 245-250.

92

Matti, A.I., McCarl, H., Klaer, P., Keane, M.C. and Chen, C.S. (2010) 'Multiple

sclerosis: patients' information sources and needs on disease symptoms and management',

Patient Prefer Adherence 4 pp. 157-61.

Matveev, A.V. (2002) THE ADVANTAGES OF EMPLOYING QUANTITATIVE AND

QUALITATIVE METHODS IN INTERCULTURAL RESEARCH: PRACTICAL

IMPLICATIONS FROM THE STUDY OF THE PERCEPTIONS OF INTERCULTURAL

COMMUNICATION COMPETENCE BY AMERICAN AND RUSSIAN MANAGERS.

Available at: http://www.russcomm.ru/eng/rca_biblio/m/matveev01_eng.shtml (Accessed

4/08/2010).

McCray, A.T. (2005) 'Promoting health literacy', Journal of the American Medical

Informatics Association 12(2), pp. 152-163.

McDonnell, G.V. and Hawkins, S.A. (1999) 'High incidence and prevalence of multiple

sclerosis in south east Scotland: evidence of a genetic predisposition', Journal of

Neurology Neurosurgery and Psychiatry 66(3), pp. 411-411.

McLaughlin, H. (2008) SMOG. Harry McLaughlin. Available at:

http://www.harrymclaughlin.com/SMOG.htm (Accessed 14/5/2010).

McMillan, S.J. (1999) 'Health communication and the internet: Relations between

interactive characteristics of the medium and site creators, content, and purpose', Health

Communication 11(4), pp. 375-390.

McMullan, M. (2006) 'Patients using the Internet to obtain health information: How this

affects the patient-health professional relationship', Patient Education and Counseling

63(1-2), pp. 24-28.

Meade, C.D., Diekmann, J. and Thornhill, D.G. (1992) 'Readability of American Cancer

Society patient education literature', Oncol Nurs Forum 19(1), pp. 51-5.

Meade, C.D. and Smith, C.F. (1991a) 'READABILITY FORMULAS - CAUTIONS

AND CRITERIA', Patient Education and Counseling 17(2), pp. 153-158.

Meade, C.D. and Smith, C.F. (1991b) 'Readability formulas: Cautions and criteria',

Patient Education and Counseling 17(2), pp. 153-158.

Mei, I.A.F.v.d. (2004) The role of past sun exposure in Multiple Sclerosis. University

of Tasmania. Available at: eprints.utas.edu.au/884/1/Front_matter_Ingrid_4_final.pdf

(Accessed 6/7/2010).

Melnyk, P.S., Shevchuk, Y.M. and Remillard, A.J. (2000) 'Impact of the dial access

drug information service on patient outcome', Annals of Pharmacotherapy 34(5), pp.

585-592.

93

Merriam, B. (1998) Qualitative research and case study applications in education. San

Francisco: Jossey-Bass.

Miller, D.C. and Salkind, N.J. (2002), 'Applied and evalucation reserch', in (Eds,

Laughton, D., Novak, V. and Axelsen, D. E.), Handbook of Resrech Design London,

SAGE Publication, pp. 79.

Miller, D.H. and Leary, S.M. (2007) 'Primary-progressive multiple sclerosis', The

Lancet Neurology 6(10), pp. 903-912.

Miller, F.P., Vandome, A.F. and McBrewster, J. (2009) Evaluation. VDM Publishing

House Ltd.

Miniwatts Marketing Group. (2010) INTERNET USAGE STATISTICS The Internet Big

Picture. Miniwatts Marketing Group,. Available at:

http://www.internetworldstats.com/stats.htm (Accessed 18/07/2010).

Molassiotis, A. and Xu, M. (2004) 'Quality and safety issues of web-based information

about herbal medicines in the treatment of cancer', Complementary Therapies in

Medicine 12(4), pp. 217-227.

Molenaar, S., Sprangers, M.A.G., Postma-Schuit, F.C.E., Rutgers, E.J.T., Noorlander, J.,

Hendriks, J. and De Haes, H. (2000) 'Feasibility and effects of decision aids', Medical

Decision Making 20(1), pp. 112-127.

Morahan-Martin, J.M. (2004) 'How Internet users find, evaluate, and use online health

information: A cross-cultural review', Cyberpsychology & Behavior 7(5), pp. 497-510.

Moreno, C., Alvarado, M., Balcazar, H., Lane, C., Newman, E., Ortiz, G. and Forrest, M.

(1997) 'Heart disease education and prevention program targeting immigrant Latinos:

Using focus group responses to develop effective interventions', Journal of Community

Health 22(6), pp. 435-450.

MORI. (2001) Nearly 3 out of 5 internet users would seek help for mental health

problems on the net MORI. Available at: http://www.ipsos-

mori.com/researchpublications/researcharchive/poll.aspx?oItemId=1444 (Accessed

24/12/2009).

Morr, S., Shanti, N., Carrer, A., Kubeck, J. and Gerling, M.C. (2010) 'Quality of

information concerning cervical disc herniation on the Internet', Spine Journal 10(4), pp.

350-354.

Morse, J.M. and Richards, L. (2002) Read me first for a user's guide to qualitative

methods. London: SAGE.

94

Murero, M., D'Ancona, G. and Karamanoukian, H. (2001) 'Use of the Internet by

patients before and after cardiac surgery: telephone survey', J Med Internet Res 3(3), pp.

E27.

Murray, E., Lo, B., Pollack, L., Donelan, K., Catania, J., Lee, K., Zapert, K. and Turner,

R. (2003a) 'The impact of health information on the Internet on health care and the

physician-patient relationship: national U.S. survey among 1.050 U.S. physicians', J Med

Internet Res 5(3), pp. e17.

Murray, E., Lo, B., Pollack, L., Donelan, K., Catania, J., White, M., Zapert, K. and

Turner, R. (2003b) 'The impact of health in formation on the Internet on the physician-

patient relationship - Patient perceptions', Archives of Internal Medicine 163(14), pp.

1727-1734.

Nammacher, M.A. and Schmitt, K. (1998) 'Consumer use of the Internet for health

information: A population survey', Journal of the American Medical Informatics

Association pp. 1044-1044.

Napoli, P.M. (2000), 'Consumer use of medical information from electronic and paper

media: a literature review', in (Eds, Rice, R. E. and Katz, J. E.), The Internet and health

communication: experiences and expectations79-98.

Narhi, U. (2007) 'Sources of medicine information and their reliability evaluated by

medicine users', Pharmacy World & Science 29 pp. 688-694.

National Literacy Trust (2010) how can i assess the readability of my document or write

more clearly. National Literacy Trust Available at:

http://www.literacytrust.org.uk/about/faqs/710_how_can_i_assess_the_readability_of_m

y_document_or_write_more_clearly (Accessed 31/07/2010).

National Multiple Sclerosis Society. (2010) About MS: What is multiple sclerosis?

National Multiple Sclerosis Society. Available at: http://www.nationalmssociety.org

(Accessed 9/7/2010).

NationMaster. (2010) BRITISH LANGUAGE STATS. NationMaster.com. Available at:

http://www.nationmaster.com/red/country/uk-united-kingdom/lan-language&b_cite=1

(Accessed 12/5/2010).

Neelapala, P., Duvvi, S.K., Kumar, G. and Kumar, B.N. (2008) 'Do gynaecology

outpatients use the Internet to seek health information? A questionnaire survey', Journal

of Evaluation in Clinical Practice 14(2), pp. 300-304.

NetRatings, N. (2005) the global standard for Internet audience measurement and

analysis. Nielsen // NetRatings. Available at: http://www.nielsen-

online.com/pr/pr_050318.pdf (Accessed 23/12/2009).

95

Neuendorf, K.A. (2002) The content analysis guidebook London: SAGE.

Ngau, P. and Kumssa, A. (2004) Research design, data collection, and analysis: a

training manual. United Nations Centre for Regional Development, Africa Office,.

Nicholls, S., Hankins, M., Hooley, C. and Smith, H. (2009) 'A survey of the quality and

accuracy of information leaflets about skin cancer and sun-protective behaviour available

from UK general practices and community pharmacies', Journal of the European

Academy of Dermatology and Venereology 23(5), pp. 566-569.

Nielsen-Bohlman, L., Panzer, A. and Kindig, D. (2004) Health Literacy: A Prescription

to End Confusion. Washington, DC: Institute of Medicine/National Academies Press.

Nielsen. (2006) WHO BENEFITS MOST FROM SEARCH? Nielsen//NetRatings.

Available at: http://www.nielsen-online.com/pr/PR_033006_UK.pdf (Accessed

11/5/2010).

Nielsen. (2009) NIELSEN ONLINE ANNOUNCES DECEMBER U.S. SEARCH SHARE

RANKINGS. Nielsen Online enables. Available at: http://www.nielsen-

online.com/pr/pr_090114.pdf (Accessed 11/5/2010).

Noseworthy, J.H., Lucchinetti, C., Rodriguez, M. and Weinshenker, B.G. (2000)

'Multiple Sclerosis', N Engl J Med 343(13), pp. 938-952.

Office for National Statistics. (2009) Internet Access Households and Individuals.

Office for National Statistics. Available at:

http://www.statistics.gov.uk/pdfdir/iahi0809.pdf (Accessed 9/5/2010).

Olsina, L. and Rossi, G. (2000) Web Engineering: A Quantitative Methodology for

Quality Evaluation and Comparison of Web Applications. Available at:

www.emis.de/journals/SADIO/vol.3.1/olsinasantos.pdf (Accessed 10/5/2010).

Online1. (2006) Nutzverhalten – Zahlen & Fakten zum www 2006. Online1. Available

at: http://www.online1.ch/wissen/nutzer-verhalten-suchmaschinen.htm (Accessed

12/5/2010).

Paasche-Orlow, M.K., Taylor, H.A. and Brancati, F.L. (2003) 'Readability standards for

informed-consent forms as compared with actual readability', New England Journal of

Medicine 348(8), pp. 721-726.

Park, J., Chung, H. and Yoo, W.S. (2009) 'Is the Internet a primary source for consumer

information search?: Group comparison for channel choices', Journal of Retailing and

Consumer Services 16(2), pp. 92-99.

Parker, M. (2004) A generic business intelligence data model to analyse data within a

small to medium medical practice (SMMP). In: South African Institute of Computer

Scientists and Information Technologists (SAICSIT),. Stellenbosch.

96

Parker, M., Moleshe, V., Harpe, R.D.l. and Wills, G. (2006) An evaluation of

Information quality frameworks for the World Wide Web. In: 8th Annual Conference on

WWW Applications. Bloemfontein.

Parrott, R. (2004) 'Emphasizing "Communication" in Health Communication', The

Journal of Communication 54(4), pp. 751-787.

Paz, S.H., Liu, H.H., Fongwa, M.N., Morales, L.S. and Hays, R.D. (2009) 'Readability

estimates for commonly used health-related quality of life surveys', Quality of Life

Research 18(7), pp. 889-900.

Pellechia, M.G. (1997) 'Trends in science coverage: A content analysis of three US

newspapers', Public Understanding of Science 6(1), pp. 49-68.

Pellise, F., Sell, P. and EuroSpine Patient Line Task, F. (2009) 'Patient information and

education with modern media: the Spine Society of Europe Patient Line', European

Spine Journal 18 pp. S395-S401.

Pereira, J. and Bruera, E. (1998) 'The Internet as a resource for palliative care and

hospice: A review and proposals', Journal of Pain and Symptom Management 16(1), pp.

59-68.

Petch, T. (2004) Content Analysis of Selected Health Information Websites Final

Report. Action for Health. Available at:

www.sfu.ca/act4hlth/pub/working/Content%20Analysis.pdf (Accessed 12/5/2010).

Peterson, M.W. and Fretz, P.C. (2003) 'Patient use of the Internet for information in a

lung cancer clinic', Chest 123(2), pp. 452-457.

Petty, R.E. and Cacioppo, J.T. (1990) 'INVOLVEMENT AND PERSUASION -

TRADITION VERSUS INTEGRATION', Psychological Bulletin 107(3), pp. 367-374.

Plantin, L. and Daneback, K. (2009) 'Parenthood, information and support on the

internet. A literature review of research on parents and professionals online', Bmc Family

Practice 10

Polit, D.F. and Beck, C.T. (2004) Nursing research: principles and methods Lippincott

Williams

Pope, C. (1992) 'What use is medical sociology to health service research', Medical

Sociology News 18 pp. 17-25.

Porter, M.E. (1991) 'TOWARDS A DYNAMIC THEORY OF STRATEGY', Strategic

Management Journal 12 pp. 95-117.

97

Porth, C. (2007), 'Unit X: Nervous System', in (Eds, Zuccarini, M., Kogut, H. and Shiff,

D.), Essentials of pathophysiology: concepts of altered health states Philadelphia,

Lippincott, pp. 809-811.

Pothier, L. and Pothier, D.D. (2009) 'Patient-orientated web sites on laryngectomy: is

their information readable?', European Journal of Cancer Care 18(6), pp. 594-597.

Potts, H.W.W. and Wyatt, J.C. (2002) 'Survey of doctors' experience of patients using

the Internet', J Med Internet Res 4(1), pp. e5.

Powell, R. (2006) 'Evaluation research: an overview', 55(1),

Pratt, W., Unruh, K., Civan, A. and Skeels, M. (2006) 'Personal health information

management', Communications of the Acm 49(1), pp. 51-55.

Purcell, G.P., Wilson, P. and Delamothe, T. (2002) 'The quality of health information on

the internet', BMJ 324(7337), pp. 557-558.

Rains, S.A. (2007) 'The anonymity effect: The influence of anonymity on perceptions of

sources and information on health websites', Journal of Applied Communication

Research 35(2), pp. 197-214.

Rather, R.A., Lone, F.A. and Shah, G.J. (2008) Overlap in Web Search Results: A Study

of Five Search Engines. Library Philosophy and Practice. Available at:

http://www.webpages.uidaho.edu/~mbolin/rather-lone-shah.pdf (Accessed 11/5/2010).

Ratzan, S.C. (2001) 'Health literacy: communication for the public good', Health

Promotion International 16(2), pp. 207-214.

Reipert, B. (2004) 'Multiple sclerosis: a short review of the disease and its differences

between men and women', The Journal of Men's Health & Gender 1(4), pp. 334-340.

Rejdak, K., Jackson, S. and Giovannoni, G. (2010) 'Multiple sclerosis: a practical

overview for clinicians', Br Med Bull pp. ldq017.

Remenyi, D. (2007) Proceedings of the 6th European Conference on Research

Methodology for Business and Management Studies. Acadimic Conference Limited.

Rice, R. and Katz, J. (2001) The Internet and health communication: experiences and

expectations. Thousands Oaks: SAGE.

Rice, R.E. (2006) 'Influences, usage, and outcomes of Internet health information

searching: Multivariate results from the Pew surveys', International Journal of Medical

Informatics 75(1), pp. 8-28.

98

Richards, R., Sampson, F., Beard, S. and Tappenden, P. (2002) A review of the natural

history and epidemiology of multiple sclerosis: implications for resource allocation and

health economic models. Health Technology Assessment. Available at:

http://www.hta.ac.uk/fullmono/mon610.pdf (Accessed 9/7/2010).

Rideout;, V. (2001) Generation Rx.com: how young people use the Internet for health

information. A Kaiser Family Foundation survey. Available at:

http://www.kff.org/entmedia/loader.cfm?url=/commonspot/security/getfile.cfm&PageID

=13719 (Accessed 23/12/2009).

Rieh, S.Y. (2002) 'Judgment of information quality and cognitive authority in the Web',

Journal of the American Society for Information Science and Technology 53(2), pp. 145-

161.

Rimal, R.N. and Lapinski, M.K. (2009) 'Why health communication is important in

public health', Bulletin of the World Health Organization 87(4), pp. 247-247.

Risk, A. and Petersen, C. (2002) 'Health Information on the Internet: Quality Issues and

International Initiatives', JAMA 287(20), pp. 2713-2715.

Rizzo, L., Moser, R.P., Waldron, W., Wang, Z. and Davis, W.W. (2008) ANALYTIC

METHODS TO EXAMINE CHANGES ACROSS YEARS USING HINTS 2003 & 2005

DATA. National Institute of Health. Available at:

http://hints.cancer.gov/docs/HINTS_Data_Users_Handbook-2008.pdf (Accessed

12/5/2010).

Robinson, I., Hunter, M. and Neilson, S. 1996 the Multiple Sclerosis Society, Uxbridge.

Roche, X. (2010) About HTTrack. Available at: http://www.httrack.com/ (Accessed

02/09/2010).

Roon, A.J. and Soot, L.C. (1999) 'Vascular surgery and the Internet: A poor source of

patient-oriented information - Discussion', Journal of Vascular Surgery 30(1), pp. 90-91.

Rosati, G. (2001) 'The prevalence of multiple sclerosis in the world: an update',

Neurological Sciences 22(2), pp. 117-139.

Rossi, H. and Freeman, E. (1993) Evaluation: A systimatics Approch. CA: SAGE.

Rudd, E., Moeykens, A. and Colton, C. (1999) Health and literacy: a review of medical

and public health literature. In: Annual Review of Adult Learning and Literacy.

Newyork.

Rudd, R.E., Anderson, J.E., Oppenheimer, S. and Nath, C. (2007) Health Literacy An

Update of Medical and Public Health Literature. In: Review of Adult Learning and

Literacy. Mahwah, NJ.

99

Sadovnick, A.D. (2009) 'European Charcot Foundation Lecture: The natural history of

multiple sclerosis and gender', Journal of the neurological sciences 286(1), pp. 1-5.

Saithna, A., Ajayi, O.O. and Davis, E.T. (2008) 'The quality of internet sites providing

information relating to hip resurfacing', Surgeon-Journal of the Royal Colleges of

Surgeons of Edinburgh and Ireland 6(2), pp. 85-87.

Salaun, Y. and Flores, K. (2001) 'Information quality: meeting the needs of the

consumer', International Journal of Information Management 21(1), pp. 21-37.

Sandvik, H. (1999) 'Health information and interaction on the internet: a survey of

female urinary incontinence', British Medical Journal 319(7201), pp. 29-32.

Saounatsou, M., Patsi, O., Fasoi, G., Stylianou, M., Kavga, A., Economou, O., Mandi, P.

and Nicolaou, M. (2001) 'The influence of the hypertensive patient's education in

compliance with their medication', Public Health Nursing 18(6), pp. 436-442.

Schafer, A. (2002) 'Spinning the web of hate:Web-based hate propagation by extremist

organizations', Journal of Criminal Justice and Popular Culture 9(2), pp. 69-88.

Sciamanna, C.N., Clark, M.A., Houston, T.K. and Diaz, J.A. (2002) 'Unmet needs of

primary care patients in using the Internet for health-related activities', J Med Internet

Res 4(3), pp. E19.

Seematter-Bagnoud, L. and Santos-Eggimann, B. (2007) 'Sources and level of

information about health issues and preventive services among young-old persons in

Switzerland', International Journal of Public Health 52 pp. 313-316.

Seidman, J.J., Steinwachs, D. and Rubin, H.R. (2003) 'Design and testing of a tool for

evaluating the quality of diabetes consumer-information Web sites', J Med Internet Res

5(4), pp. e30.

Shankar, G. and Watts, S. (2003) A relevant, believable approach for data quality

assessment. In: Proceedings of 8th International Conference on Information Quality.

Shanks, G. and Corbitt, B. (1999) Understanding data quality: Social and cultural

aspects. In: the 10th Australasian Conference on Information Systems.

Silberg, W.M., Lundberg, G.D. and Musacchio, R.A. (1997) 'Assessing, controlling, and

assuring the quality of medical information on the Internet - Caveant lector et viewer -

Let the reader and viewer beware', Jama-Journal of the American Medical Association

277(15), pp. 1244-1245.

Sim, J. and Wright, C. (2000) Research in health care: concepts, designs and methods.

Nelson Thornes.

100

Skinner, H., Biscope, S. and Poland, B. (2003) 'Quality of internet access: barrier behind

internet use statistics', Social Science & Medicine 57(5), pp. 875-880.

Slade, A., Tennant, A. and Ford, H. (2003) 'Meeting the information needs of people

with multiple sclerosis', International Journal of Therapy and Rehabilitation 10(5), pp.

211-217.

Somerset, M., Peters, T.J., Sharp, D.J. and Campbell, R. (2003) 'Factors that contribute

to quality of life outcomes prioritised by people with multiple sclerosis', Quality of Life

Research 12 pp. 21-29.

Somerset, M., Sharp, D. and Campbell, R. (2002) 'Multiple sclerosis and quality of life:

a qualitative investigation', J Health Serv Res Policy 7(3), pp. 151-9.

SPICH. (1999) Wired for health and wellbeing:The emergence of interactive health

communication. Department of Health and Human Services, U.S. Government Printing

Office. Available at: http://classweb.gmu.edu/gkreps/820/023.pdf (Accessed

22/12/2009).

Spink, A., Jansen, B.J., Blakely, C. and Koshman, S. (2006) 'A study of results overlap

and uniqueness among major Web search engines', Information Processing &

Management 42(5), pp. 1379-1391.

Stvilia, B., Mon, L. and Yi, Y.J. (2009) 'A Model for Online Consumer Health

Information Quality', Journal of the American Society for Information Science and

Technology 60(9), pp. 1781-1791.

Sukamolson, S. (2005) Fundamentals of quantitative research. Language Institute.

Available at: http://www.culi.chula.ac.th/e-Journal/bod/Suphat%20Sukamolson.pdf

(Accessed 04/08/2010).

Ta-Min, R. 2006a, 'Comprehension of Online Cancer Information: A Propositional

Assessment of Readability, Inferences, and Coherence', University of Waterloo,

Waterloo,.

Ta-Min, R. 2006b, ' Comprehension of Online CancerInformation: A Propositional

Assessment of Readability, Inferences, and Coherence', University of Waterloo,

Tang, R. and Solomon, P. (1998) 'Toward an Understanding of the Dynamics of

Relevance Judgment: An Analysis of One Person`s Serch Behavir', Information

Processing and Managment 34 pp. 237-256.

Tayi, G.K. and Ballou, D.P. (1998) 'Examining data quality', Communications of the

Acm 41(2), pp. 54-57.

Taylor, H. and Leitman, R. (2002) Cyberchondriacs continue to grow in America: 110

million people sometimes look for health information online. Health Care News.

101

Available at:

http://www.harrisinteractive.com/news/newsletters/healthnews/HI_HealthCareNews2002

Vol2_Iss09.pdf (Accessed 15/7/2010).

The Health On the Net Foundation. (2010) The Health On the Net Foundation.

Available at: http://www.hon.ch/ (Accessed 26/07/2010).

The Joint Commission. (2007a) What did the doctor say? Improving health literacy to

protect patient safety. The Joint Commission,. Available at:

http://www.jointcommission.org/nr/rdonlyres/d5248b2e-e7e6-4121-8874-

99c7b4888301/0/improving_health_literacy.pdf (Accessed 27/07/2010).

The Joint Commission. (2007b) What did the doctor say? Improving health literacy to

protect patient safety. The Joint Commission. Available at:

http://www.jointcommission.org/newsroom/presskits/health_literacy/ (Accessed

18/07/2010).

The National Collaborating Centre for Chronic Conditions. (2004) MULTIPLE

SCLEROSIS National clinical guideline for diagnosis and management in primary and

secondary care. The National Collaborating Centre for Chronic Conditions. Available

at: http://www.rcplondon.ac.uk/pubs/contents/409ed7a2-ae44-472f-ba3b-

ab7164229493.pdf (Accessed 14/09/2010).

The National Literacy Act. (2007) The National Illiteracy Action Project 2007 – 2012.

The National Literacy Act. Available at:

http://www.ahrq.gov/downloads/pub/evidence/pdf/literacy/literacy.pdf (Accessed

28/07/2010).

Thomas, S.L., Williams, R., Williams, T. and Hall.a, A.J. (2009) Prevalence Of

Multiple Sclerosis In The United Kingdom - Study Estimates Now 100,000 People With

MS In UK. Available at: http://www.medicalnewstoday.com/articles/153854.php

(Accessed 9/7/2010).

Thorne, S.E., Harris, S.R., Mahoney, K., Con, A. and McGuinness, L. (2004) 'The

context of health care communication in chronic illness', Patient Education and

Counseling 54(3), pp. 299-306.

Tongco, D.C. (2007) 'Purposive Sampling as a Tool for Informant Selection',

Ethnobotany Journal 15 pp. 148-158.

Toulmin, S. (1958) the use of argument. Cambridge UK: Cambridge University Press.

Trochim, W.M.K. (2006) Research Methods Knowledge Base. William M.K. Trochim.

Available at: http://www.socialresearchmethods.net/kb/index.php (Accessed 09/08/2010).

Tu, H.T. and Hargraves, J.L. (2003) 'Seeking health care information: most consumers

still on the sidelines', Issue Brief Cent Stud Health Syst Change (61), pp. 1-4.

102

U.S. Department of Health and Human Services. (2000) Healthy People 2010:

Understanding and Improving Health. Washington, DC: U.S: Government Printing

Office.

U.S. Department of Health and Human Services. (2002) Healthy People 2010:

Understanding and Improving Health. Washington, DC: U.S: Government Printing

Office.

Varkevisser, C.M., Mwaluko, G.M.P. and Le Grand, A. (2001) 'Research in action: the

training approach of the Joint Health Systems Research Project for the Southern African

region', Health Policy and Planning 16(3), pp. 281-291.

Varkevisser, C.M., Pathmanathan, I. and Brownlee, A. (1991) Designing and

Conducting Health Systems Research Projects. Amsterdam: KIT Publishers.

Vaus, D.A.D. (2003) Research design in social research London: SAGE.

Vaus, D.d. (2001) Research Design in Social Research. London: SAGE.

VICKI S. FREIMUTH. (1979) 'Assessing the Readability of Health Education

Messages', Public Health Reports 94(6), pp. 568-570.

Vickrey, B.G., Shatin, D., Wolf, S.M., Myers, L.W., Belin, T.R., Hanson, R.A., Shapiro,

M.F., Beckstrand, M., Edmonds, Z.V., Delrahim, S. and Ellison, G.W. (2000)

'Management of multiple sclerosis across managed care and fee-for-service systems',

Neurology 55(9), pp. 1341-1349.

Vivian, J. (2002) The media of mass communication. Boston, MA: Allyn & Bacon.

Voelker, R. (1998) 'Going online in rural health', Jama-Journal of the American

Medical Association 279(3), pp. 183-183.

von Wagner, C., Semmler, C., Good, A. and Wardle, J. (2009) 'Health literacy and self-

efficacy for participating in colorectal cancer screening: The role of information

processing', Patient Education and Counseling 75(3), pp. 352-357.

Walsh, T.M. and Volsko, T.A. (2008) 'Readability Assessment of Internet-Based

Consumer Health Information', Respiratory Care 53(10), pp. 1310-1315.

Wang, R.Y. and Strong, D.M. (1996) 'Beyond accuracy: what data quality means to data

consumers', J. Manage. Inf. Syst. 12(4), pp. 5-33.

Webster, R. and Williams, P. (2005) 'An evaluation of the NHS direct online health

information e-mail enquiry service - Quality of health information on the internet', Aslib

Proceedings 57(1), pp. 48-62.

103

Widman, L.E. and Tong, D.A. (1997) 'Requests for medical advice from patients and

families to health care providers who publish on the World Wide Web', Archives of

Internal Medicine 157(2), pp. 209-212.

Willard, H.F. and Ginsburg, G.S. (2008), 'Online Health Information Retrieval by

Consumers and the Challenge of Personal Genomics', in Genomic and Personalized

Medicine, Volumes 1 Vol. 2, Academic Press, pp. 252-257.

Williams, J., Clemens, S., Oleinikova, K. and Tarvin, K. (2003) The skills for life survey:

a national needs and impact survey of literacy, numeracy and ICT skills. UK.

Department for Education and Skills. Available at:

http://www.education.gov.uk/research/data/uploadfiles/RR490.pdf (Accessed

29/07/2010).

Williams, M.V., Baker, D.W., Honig, E.G., Lee, T.M. and Nowlan, A. (1998a)

'Inadequate literacy is a barrier to asthma knowledge and self-care', Chest 114(4), pp.

1008-1015.

Williams, M.V., Baker, D.W., Parker, R.M. and Nurss, J.R. (1998b) 'Relationship of

functional health literacy to patients' knowledge of their chronic disease - A study of

patients with hypertension and diabetes', Archives of Internal Medicine 158(2), pp. 166-

172.

Williams, M.V., Parker, R.M., Baker, D.W., Parikh, N.S., Pitkin, K., Coates, W.C. and

Nurss, J.R. (1995) 'INADEQUATE FUNCTIONAL HEALTH LITERACY AMONG

PATIENTS AT 2 PUBLIC HOSPITALS', Jama-Journal of the American Medical

Association 274(21), pp. 1677-1682.

Williams, P., Nicholas, D., Huntington, P. and McLean, F. (2002) 'Surfing for health:

user evaluation of a health information website. Part one: background and literature

review', Health Information and Libraries Journal 19(2), pp. 98-108.

Wilson, P. (2002) 'How to find the good and avoid the bad or ugly: a short guide to tools

for rating quality of health information on the internet', British Medical Journal

324(7337), pp. 598-600.

Wilson, R., Kenny, T., Clark, J., Moseley, D., Newton, L., Newton, D. and Purves, I.

(1998) PIL‟s Summary Report: Ensuring the Readability and Understandability and

Efficacy of Patient Information Leaflets. Health Centre for Health Informatics.

Available at: http://www.ecls.ncl.ac.uk/publications/Clark%5E1998-

ensuring_readability_Summary.pdf (Accessed 29/07/2010).

Wilson, R.A. (2007) 'Combining Historical Research and Narrative Inquiry to Create

Chronicles and Narratives', The Qualitative Report 12(1), pp. 20-39.

104

Wolf, M.S., Davis, T.C., Shrank, W., Rapp, D.N., Bass, P.F., Connor, U.M., Clayman,

M. and Parker, R.M. (2007) 'To err is human: Patient misinterpretations of prescription

drug label instructions', Patient Education and Counseling 67(3), pp. 293-300.

Zafar, S.Y., Alexander, S.C., Weinfurt, K.P., Schulman, K.A. and Abernethy, A.P.

(2009) 'Decision making and quality of life in the treatment of cancer: a review',

Supportive Care in Cancer 17(2), pp. 117-127.

Ziebland, S., Chapple, A., Dumelow, C., Evans, D., Prinjha, S. and Rozmovits, L. (2004)

'How the internet affects patients' experience of cancer: a qualitative study', British

Medical Journal 328(7439), pp. 564-+.

Zivadinov, R. and Bakshi, R. (2004) Brain and spinal cord atrophy in multiple

sclerosis. New York: Nova Biomedical Books.

105

Appendix A: PageRank Technology

PageRank Technology: PageRank performs an objective measurement of the

importance of web pages by solving an equation of more than 500 million variables and 2

billion terms. Instead of counting direct links, PageRank interprets a link from Page A to

Page B as a vote for Page B by Page A. PageRank then assesses a page's importance by

the number of votes it receives.

PageRank also considers the importance of each page that casts a vote, as votes from

some pages are considered to have greater value, thus giving the linked page greater

value. Important pages receive a higher PageRank and appear at the top of the search

results. Google's technology uses the collective intelligence of the web to determine a

page's importance. There is no human involvement or manipulation of results, which is

why users have come to trust Google as a source of objective information unspoiled by

paid placement.

106

Appendix B: Consent for Using MS Information Completeness Tool

Re: Multiple Sclerosis Tool Permission‏

From: J A Harland (lip05jah@sheffield.ac.uk)

Sent: Thursday, May 06, 2010 12:54:41 PM

To: Mohammad Khasawneh (m59awneh@hotmail.com)

Hi Mohammad,

Thanks for your message. Of course you can use it. Good luck with your

work

and let me know if you need any further information.

Best Wishes

Juliet

Quoting Mohammad Khasawneh <m59awneh@hotmail.com>:

>

> Hi

>

> My Name is Mohammad Khasawneh, I am postgraduate student studying MSc

Health

> Informatics in Swansea University.

> I will start my dissertation soon. I decided to evaluate the quality

and

> readability of multiple sclerosis websites in UK. While I was looking

for

> tools to evaluate the quality of multiple sclerosis websites I came

over the

> nice tool you developed. I am asking for your permission to use this

tool in

> my dissertation please.

>

> Thank You

107

Appendix C: Multiple Sclerosis Evaluation Tool

This tool is used to assess the comprehensiveness of the information provided on a

website in relation to the reported needs of people with MS (PWMS) and their families.

Sites will be given an overall score and individual scores for how well they meet the

needs of:

• Newly diagnosed or currently being diagnosed

• Post-diagnosis

• Family and carers.

They are also awarded a score for the way in which they present information in relation

to

The preferences reported in the literature.

Tool

Website address: …………………..

Name of the person or organization that produced the website: ……………………….

Date website accessed: …………………..

Section 1: information for newly diagnosed PWMS

1- Is information given about the physiology of the disease?

2- Is there a section of the website specifically for people who are being, or who

have just been, diagnosed with MS?

3- Is there information about how MS is diagnosed?

Are the following topics discussed?

4- Course of the disease?

5- Physical symptoms?

6- Symptom management?

7- Drug treatments?

8- Exercise?

9- Diet?

10- Role of an MS specialist nurse?

11- How to tell family and friends?

Section 2: post-diagnosis phase

12- Does the website have separate sections on medical information about MS and

practical information about how to live with the disease?

13- Does the website allow users to make contact with fellow MS sufferers, e.g.

through a bulletin board?

Does the site contain information about:

14- Balance and walking problems?

15- Bowel and bladder disturbances?

16- Emotional changes, including depression?

17- Memory problems?

18- Fatigue?

108

19- Visual disturbances?

20- Spasticity?

Does the site provide balanced advice and information about:

21- Different drug treatments available and how they compare?

22- Complementary medicines?

23- Nutrition?

24- Research into MS?

25- Physiotherapy?

26- Pregnancy and MS?

27- Sex and MS?

Does the website provide practical information on:

28- Access and transport?

29- Aids and appliances?

30- Benefits and entitlements?

31- Home modifications?

32- Leisure activities and holidays?

33- Occupational therapy?

34- Working and MS?

Section 3: information for family or carers of PWMS

35- Is there a section specifically for families or carers of PWMS?

36- Is there general advice on caring?

Does the site provide information on how to cope with:

37- Emotional challenges?

38- Practical challenges?

39- Does the site report the personal experiences of carers or family members of

PWMS?

40- Is information provided on support groups or useful contacts for the family and

carers of PWMS?

41- Is there a means to contact other carers or family members of PWMS (e.g.

specific bulletin board for carers/family)?

Section 4: overall content/presentation of information on the site

42- Are there any sections written by PWMS?

43- Is the information presented in a positive manner?

44- Does the information help to inspire or motivate PWMS?

45- Is the information presented in a succinct manner (e.g. long paragraphs are

summarized with bullet points)?

46- Can the web page be adjusted to allow it to be more easily read by disabled users

(e.g. can text size be increased)?

47 Are contact addresses and telephone numbers provided for people to get in touch

with organizations that support PWMS?

48 Would you recommend this website for people wanting to find out more about

multiple sclerosis?