Appropriateness measurement: application to advice-givingin community pharmacies

P. Bissell*, P.R. Ward, P.R. Noyce

Drug Usage and Pharmacy Practice Group, School of Pharmacy and Pharmaceutical Sciences, University of Manchester, Oxford

Road, Manchester M13 9PL, UK

Abstract

Awareness of variations in the delivery of medical care has resulted in considerable research activity focused ondeveloping measures to assess the appropriateness of health service provision both internationally and within GreatBritain. As in other areas of health service provision there is evidence of variation in advice provided alongside sales

of non-prescription medicines and variation in response to requests for advice about the treatment of minorailments within community pharmacies in Great Britain. However, there is little research which has explored theextensive methodological problems associated with developing criteria to assess the appropriateness of these twoactivities.

Following a critical review of relevant existing research, this paper describes a methodology and empirical ®ndingsfrom a study which aimed to develop criteria to measure the appropriateness of advice provided in communitypharmacies. Firstly, details of advice-giving episodes occurring between consumers and pharmacists or medicines

counter assistants were captured and documented using a combination of audio tape-recording and non-participantobservation. Secondly, the nominal group technique was used to develop a set of explicit criteria for assessing theappropriateness of advice. Thirdly, an assessment instrument was developed in order to operationalise the criteria.

The devised criteria include both process and output components. We discuss the utility of these criteria inrelation to developments in self-medication practice a�ecting community pharmacy and the deregulation ofmedicines within the UK. The criteria have been subject to rigorous statistical testing to establish standards of

validity and reliability (Ward, Bissell & Noyce, 2000a [Ward, P. R., Bissell, P. & Noyce, P. R. (2000a). Criteria forassessing non-prescription drug therapy in community pharmacy, Annals of Pharmacotherapy (in press).]). Thedeveloped criteria will allow us to identify dimensions of both appropriate and inappropriate advice provided incommunity pharmacies and provide the basis for education and training initiatives identi®ed as a result of the

research. In addition, we suggest that this research is highly relevant to informing the content, structure andoperationalisation of protocols and/or guidelines associated with the management of minor ailments and the sale ofmedicines through community pharmacies. 7 2000 Elsevier Science Ltd. All rights reserved.

Keywords: Appropriateness; Community pharmacy; Variation; Advice-giving; Consensus techniques

Introduction

Within Great Britain (GB), alongside their core ac-

tivity of dispensing prescription medicines, community

Social Science & Medicine 51 (2000) 343±359

0277-9536/00/$ - see front matter 7 2000 Elsevier Science Ltd. All rights reserved.

PII: S0277-9536(99 )00458-X

www.elsevier.com/locate/socscimed

* Corresponding author. Tel.: +44-161-275-4538; fax: +44-

161-275-2416.

E-mail address: pbissell@fs1.pa.man.ac.uk (P. Bissell).

pharmacy sta� have long been expected to provide freeadvice on the treatment of minor ailments and to

guide the public towards making safe and e�caciouschoices in their selection of non-prescription medicines(Trease, 1964; Mays, 1994). These activities have long

gone unrecognised and overlooked by researchers.However, the rapidly changing health care environ-ment has resulted in a surge of interest in evaluating

the role and performance of community pharmacysta� in carrying out these latter two activities(Cunningham-Burley & MacLean, 1987; Smith, Sal-

kind & Jolly, 1990; Bissell, Ward & Noyce, 1997a;Hassell, Noyce, Rogers, Harris & Wilkinson, 1998). Atthe same time, a growing number of studies haveidenti®ed variations in the delivery of health services

Ð including those services provided by communitypharmacy sta�: awareness of these variations hasprompted research into the development of measures

designed to assess the appropriateness of health serviceprovision (Buetow, Sibbald, Cantrill & Halliwell,1996). This paper makes a contribution to the litera-

ture on appropriateness measurement in relation toadvice provided alongside requests for non-prescriptionmedicines and advice about the treatment of minor

ailments.

Aims of paper

This paper has three main aims. Firstly, we describe

the changing social and political context of health careprovision in GB and its impact in terms of focusinginterest on assessing advice-giving within community

pharmacies. We also describe the response of the phar-macy profession to these developments. Secondly, weprovide a critique of the ways in which researchershave de®ned and operationalised the concept of appro-

priate health care within health services research, fol-lowed by a review of the few pharmacy-based studieswhich have explored this area. Thirdly, on the basis of

this critique and the application of existing methods tocommunity pharmacy advice-giving, we describe ourmethod and ®ndings from a recent study which sought

to produce explicit judgement criteria for assessing theappropriateness of advice-giving in community phar-macies. The strengths and weaknesses of this study

and implications for policy and further research arealso discussed.

The role and advisory activities of community pharmacy

sta�

In Great Britain, community pharmacies are aresource within primary care (Hassell et al., 1998) usedby consumers in di�erent ways and at di�erent points

in care pathways (Hassell, Noyce, Rogers, Harris &Wilkinson, 1997). In brief, the major health-related ac-tivities performed in community pharmacies are the

dispensing of prescribed medicines, providing advicealongside sales of non-prescription medicines and pro-viding advice on action to take regarding the treatment

of minor ailments. It is the appropriateness of theselatter two activities that we focus on in this paper. For

simplicity, these will hereafter be referred to by thegeneric term advice-giving throughout this paper.Advice-giving activities in community pharmacies are

carried out not only by pharmacists but by medicinescounter assistants (MCAs). The term MCA is used tosignify individuals whose work regularly includes the

sale of non-prescription medicines (Royal Pharmaceu-tical Society of Great Britain, 1999). We will use thegeneric term pharmacy sta� for the remainder of this

paper in order to denote advice-giving activities carriedout by both pharmacists and MCAs.

The legal position on the sale of non-prescriptionmedicines from community pharmacies in Great Brit-ain is that they must be sold under the supervision of

a pharmacist, which means that the pharmacist shouldbe aware Ð and in a position to intervene Ð in anysales of a such a product. There is also a professional

requirement that all MCAs need to have undergone arecognised course of training and work within a proto-col for medicines sales. One of the most widely used

(though by no means universal) protocols involves theuse of the mnemonic WWHAM2 (Blenkinsopp & Pax-ton, 1995) designed to assist and guide pharmacy sta�

in their questioning of consumers and to facilitateappropriate advice and medicines sales. Protocolswithin individual pharmacies are formulated within the

context of a code of ethics and professional standardswhich states that in relation to requests for medicines

by name: ``The pharmacist or assistant should obtainsu�cient information to allow an assessment to bemade that the medicine is likely to be appropriate for

the person concerned. This will normally include infor-mation about whether other medication is being usedwhich could interact adversely with the product

requested or make that product inappropriate. Thepharmacist or assistant must provide any advice whichis considered appropriate to the product and the

intended consumer'' (Royal Pharmaceutical Society ofGreat Britain, 1999).

Under community pharmacy protocols, MCAs areexpected to deal only with those consumer demandswith which they are competent to deal, otherwise they

2 WWHAM is a mnemonic which assists pharmacy sta� in

their questioning of consumers. Its stands for Who is the

patient?, What are their symptoms?, How long have the

symptoms been present?, Action already taken? and Medi-

cation already taken?

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359344

should refer the consumer to the pharmacist. In prac-tice, MCAs undertake unaided the majority of non-

prescription medicines sales (Ward, Bissell & Noyce,1998). But, as expected, the level of pharmacists' invol-vement varies according to the presenting condition

(Bissell, Ward & Noyce, 1997b). For instance, pharma-cists deal with the majority of requests for advice ontreatment of eye conditions, whereas MCAs deal with

the majority associated with the treatment of uncom-plicated cough (Bissell et al., 1997b). However, genericappropriateness criteria for advice-giving apply equally

to MCAs and pharmacists. What will di�er is theknowledge base needed to provide the appropriateadvice for ailments ranging in familiarity and complex-ity. The more straightforward and more common can

be adequately managed by MCAs whilst others requireadvice from the pharmacist or referral to anotherhealth care professional.

The challenge in developing criteria for assessing theappropriateness of advice-giving in community phar-macies is that they need to address the di�erential

roles and responsibilities of both pharmacists andMCAs. In addition, community pharmacy sta� haveto manage a diverse range of consumers, some of

whom will be ®rst time users of medicines, whilstothers will be regular or experienced users (Hassell etal., 1997; Bissell, Hibbert, Ward & Noyce, 2000),known individually to pharmacy sta�. Any appropri-

ateness criteria must take into account these factors.These tasks represent a considerable research endea-vour.

Factors stimulating research into advice-giving in

community pharmacy

The expansion of research activity in this area is a

product of the complex social, political and pro-fessional changes a�ecting the delivery of primaryhealth care services (Harding & Taylor, 1997). An im-

portant change is the re-classi®cation of an increasingnumber of medicines from prescription-only (POM) tonon-prescription (pharmacy-only or general sales list)

status (Blenkinsopp & Bradley, 1996; Thomas &Noyce, 1996). The UK Medicines Act (1968) speci®esthree classes of medicines that are available from com-munity pharmacies: prescription only medicines

(POMs), available on a prescription from an author-ised practitioner (e.g. general practitioners, nurses, den-tists); pharmacy (P) medicines, purchasable only from

a community pharmacy and general sales list (GSL)medicines, available from any retail outlets (RoyalPharmaceutical Society of Great Britain, 1999). Since

1982, over 50 medicines have made the transition fromprescription to non-prescription status, including treat-ments for ailments as diverse as cold sores, hay fever,

migraine, dyspepsia and vaginal thrush. For some for-mer prescription-only medicines, such as ibuprofen, the

process has gone further, with these now being madeavailable through retail outlets such as newsagents andservice stations as GSL medicines. In essence, medi-

cines deregulation provides the `consumer' of healthproducts with a choice over whether to self-medicateor go to the doctor when experiencing minor ailments

(Payne, Ryan-Woolley & Noyce, 1998).At the same time, alongside explicit changes in drug

classi®cation, cultural and social developments have

fuelled movements emphasising autonomy and sover-eignty within the health care sector (Coulter, 1997;Lupton, 1997). These developments involve a re-orien-tation of the relationship between lay people and

expert knowledge at the end of the twentieth century(Gabe, Kelleher & Williams, 1994; Williams & Calnan,1996). Based on the notion that the `consumer' of

health-care services is increasingly knowledgeable, re-sponsible and capable of making rational and appro-priate decisions about their own health, deregulation

actively feeds into a `consumerist' agenda. Lay knowl-edge and self-medication practices are thus seen asincreasingly important dimensions of health care ac-

tivity in their own right (Levin, 1990; Hoog, 1992;Chewning & Sleath, 1996).Alongside the rise of consumerism within health

care, other factors have stimulated research into

advice-giving within community pharmacies (Tully,Hassell & Noyce, 1997). For example, technologicalchange coupled with the widespread adoption of orig-

inal pack dispensing for both prescription and non-prescription medicines have meant that the profession'straditional compounding function is largely redundant.

One outcome of this is the renewed attention whichhas been focused on traditional areas of advising onthe management of minor ailments and choice of phar-maceutical products (Mays, 1994; Harding & Taylor,

1997).

Awareness of variation in advice-giving in community

pharmacies

Studies suggesting widespread variation in howmedicines are sold and advice communicated havebecome increasingly familiar to researchers working in

the ®eld (Bissell, et al., 1997a,b; Bissell, Ward &Noyce, 1997c). Research suggests that the level ofadvice varies according to the type of product sold

(Bissell, Ward & Noyce, 1997d) and the involvementof the pharmacist varies according to the productbeing sold (Bissell et al., 1997a). A number of other

studies have reported on practice variation and goneon to suggest that some pharmacy services may beeither de®cient or sub-optimal (Smith et al., 1990;

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359 345

Goodburn, Mattoshino, Mongi & Waterston, 1991;

Consumers' Association, 1991, 1994, 1996, 1999;Anderson & Alexander, 1993; Krska et al., 1994;Krska & Kennedy, 1996). In particular, the Consu-

mers' Association has become a trenchant critic,emphasising de®ciencies in advice, questioning andreferral of consumers to other health professionals. On

the basis of their research, doubts were voiced aboutthe wisdom of further switches of medicines from pre-

scription to non-prescription status (Consumers' As-sociation, 1996). More recent research by theConsumers' Association has shown that many relevant

questions are still not being asked when consumersrequest speci®c medicines Ð particularly if the sale is

managed by a MCA (Consumers' Association, 1999).The services provided by community pharmacies are

not, of course, alone in being perceived as variable.

Variability has become a highly salient feature of thedebate over standards in health care provision (Smith,1998; Wennberg, 1998). Variations in health service

delivery are both ubiquitous and pervasive, anddetected in diverse areas of health service activity

(McPherson, 1994). A cursory review of published stu-dies suggests that they are manifest in referrals to sec-ondary care providers (Wilkin, 1992; McColl, 1994)

and within institutions such as accident and emergencyclinics (Maitra & Choyce, 1994). Services have been

shown to vary by both volume and type and acrossgeographical boundaries (Chassin et al., 1987; Leape,Park, Solomon, Chassin, Koseco� & Brook, 1990) and

are evident in the indications used to guide decisionsabout the use of speci®c surgical interventions (Bern-stein, McGlynn, Siu, Roth, Sherwood & Keesey, 1993;

Bickell, Earp & Evans, 1995; Scott & Black, 1991).Variability is reported in chronic medication prescrib-

ing (Hanlon et al., 1992; Beers, Ouslander, Rollinger,Rueben, Brooks & Beck, 1991; Wilcox, 1994) andwithin primary care prescribing more generally (Audit

Commission, 1994; Buetow et al., 1996). Studies ofself-medication practices have also identi®ed variationsin the type of medicines used to treat minor ailments

(Charuptanapong, 1994; De Almeida Neto, Benrinoj,Gomel & Fois, 1996; Paxton & Chapple, 1996).

Evidence of variation may be widespread, but callsfor explanation have met with only limited success.Factors such as case mix and the age distribution of

patients only partially account for the variation inreferrals from general practitioners to secondary care

(Wilkin, 1992). Similarly, within the practice of phar-macy, variations in sales practice (Bissell et al., 1997a),advice-giving (Bissell et al., 1997b) and referral to

other health professionals (Bissell et al., 1997c) cannotbe explained adequately by reference to structural fac-tors such as ownership patterns, geographical location,

socio-economic background of the population or theage and gender of pharmacy sta�.

The need to understand the nature and cause of vari-ation remains paramount since consistency of service

for its own sake may be pointless (McPherson, 1994;Buetow, Sibbald, Cantrill & Halliwell, 1997). The pri-ority accorded to understanding variation has resulted

in calls for information systems to clarify practitioners'decision making and focused attention on uncertaintyabout the best way to practise medicine (Buetow et al.,

1997). This uncertainty, it is claimed, has been exacer-bated by the sheer pace of change in medical care,coupled with the huge increase in the volume of infor-

mation that is now available to guide medical decision-making (Brook, 1994). These factors have encourageda discourse based on an `epistemology of appropriate-ness in health care' (Buetow et al., 1997: p. 261). We

critically review this literature in the section below.

Assessing health service variation: measuring

appropriateness

Studies of appropriateness have typically attemptedto assess the extent of variation within service deliveryand provide an evaluation of whether this is leading to

de®cient or sub-optimal care. In other words, appro-priateness studies aim to detect, measure and accountfor variations in practice. Interest in assessing appro-

priateness has been stimulated by evidence from stu-dies claiming that between 20 and 60% of all healthcare may be less than appropriate (Winslow, Koseco�,Chassin, Kanouse & Brook, 1988). Such studies have

generated considerable interest within health servicesresearch, with one of their foremost advocatessuggesting they represent the `next frontier' in the

development of clinical practice (Brook, 1994, p. 214).In contrast to the attention given to the subject gen-

erally within health services research, pharmacy prac-

tice research has had only limited engagement with theissue of explaining practice variation through measuresof the appropriateness of care. Whilst there may be

limited agreement amongst researchers on the existenceof variation per se Ð as in other health services Ðthere has been an absence of empirical research onhow appropriateness might be measured in practice

(Tully et al., 1997). It is to this area that this paperturns.

Conceptualisation and measurement of appropriateness

It has been suggested that appropriateness measureshave two distinct strengths. Firstly, in those areaswhere clinical trial data or meta-analyses are weak or

absent, a synthesis of practitioner expertise can ®ll thatgap. The systematic application of practitioner exper-tise, it is argued, can help to both detect and account

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359346

for variations in practice. Secondly, appropriateness

measures also seek to encompass more than individualattributes of care (e.g. e�ciency, cost, e�ectiveness,acceptability) and seek to combine these into a coher-

ent model which link together these many dimensions:the result being a measure of the appropriateness ofcare (Buetow et al., 1997). The apparent simplicity of

this idea, however, obscures its manifest complexity inpractice. The Oxford English Dictionary refers to

appropriateness as ``suitable or proper for a particularpurpose'' a relatively vague de®nition implying littlemore than a sense of `rightness' for a given situation.

Berwick, in a review of appropriateness research,suggests that the term refers to the extent to whichthere is provision of ``what works'' (Berwick, 1989, p.

761). Unfortunately, this de®nitional simplicity has notbeen translated into agreed upon categories for

research purposes.The most widely referenced studies of appropriate-

ness within health services research are probably the

Health Services Utilisation Studies (HSUS) carried outby the RAND corporation in the US (Kahn et al.,

1988; Chassin, 1989). They de®ned appropriate healthcare in the following way: ``the de®nition we use is apurely medical one. For a procedure to be appropriate

in any given set of clinical circumstances we mean thatits expected bene®ts should be greater than its negativeconsequences by a su�ciently wide margin to make

the procedure worth doing'' (Chassin, 1989, p. 21).Critics have pointed out that adoption of a purely

medical perspective marginalises individual, societal oreconomic perspectives Ð each of which may beequally important to determining appropriate care.

For example, in relation to the selection of a prescrip-tion or non-prescription medicine, a di�erent productmight be equally `medically' appropriate for two

patients with the same condition, but the selectionmight vary according to patient preference. The HSUS

method takes no account of this. Liberati (1996) hasdiscussed the importance of incorporating the patient'sperspective and concludes that there are few measures

which do this successfully. He suggests that includingpatient preferences is likely to contribute to more ade-quate explanations of variations in care. By the same

token, a Working Group set up by the Director ofResearch & Development NHS Management Executive

in England was critical of the HSUS de®nition andproposed their own Ð much longer Ð version,emphasizing both the individuality of patient needs and

the availability of su�cient health care resources(Working Group, 1993). However, whilst helpful inhighlighting the complexity of the concept of appropri-

ateness, Buetow et al. (1997) have suggested that theWorking Groups' (1993) 198-word de®nition would be

di�cult to operationalise in practice.The HSUS technique has been used to assess the

appropriateness of use of a number of surgical inter-

ventions (e.g. coronary artery bypass surgery). A moredetailed description of the method and its empirical®ndings can be found in Park, Fink and Brook (1986),

Chassin (1989), Kahn et al. (1988) and Buetow et al.(1997). In brief, the method combines an expert panelmethodology in conjunction with a literature review of

e�cacy, indications, cost and use for the procedurebeing assessed. The expert panel then rates the indi-

cations using a nine-point scale (1 Ð certainly appro-priate to 9 Ð certainly inappropriate). Measures ofagreement and disagreement are calculated. The indi-

cations and their ratings are then applied to abstractsof medical records in di�erent settings to generate

measures of appropriateness.In practice, the HSUS studies have been used to

assess interventions that have already taken place and

thus ignore those instances where there may have beena failure to intervene, but which might constitute inap-propriate care. Critics have also pointed out that re-

liance on medical records alone is likely to be aninsu�cient guide to the quality of care experienced by

a given patient. Using medical records alone precludesany assessment of the individuality and situationalspeci®city of the doctor±patient encounter and its

e�ect on decision-making (Kassirer, 1993). However,this is a consequence of the material on which judge-

ments of appropriateness are made (medical records)rather than a limitation of the overall method ofassessment itself and could be recti®ed by providing

detailed, empirically based examples of the actualencounter occurring between professionals andpatients.

Perhaps more signi®cantly, the Working Group(1993) also argued that how judgements of appropri-

ateness are made in practice using the HSUS methodremains opaque. As Hicks (1994) suggests, in practice,judgements of appropriateness are likely to be context

speci®c, and in the HSUS method, they are not basedon explicit criteria. Hicks (1994) argues that explicit

criteria would more concretely anchor judgements ofappropriateness. Not surprisingly, levels of disagree-ment between panelists rating perceptions of risk have

varied considerably in studies (Kahan et al., 1996;Mulley & Eagle, 1988). Fink, Koseco�, Chassin andBrook (1984) have pointed out that in practice, dis-

agreement is endemic to consensus panels with agree-ment often being reached only on bland generalities.

Whilst a pertinent criticism, it is also true that theHSUS method does not require complete agreementand the consensus panel can uncover areas of disagree-

ment and produce detailed recommendations (Merricket al., 1987). This goes some way towards providing anunderstanding of di�erences in the interpretation of

appropriate care and by implication, the causes of vari-ations. The value of this is that consensus panels can

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359 347

produce their own gold standard in de®ning good andbad practice (Hall, 1994). Overall, we suggest that the

principles of being able to de®ne good and bad prac-tice as well as being able to generate information onthe extent of (dis)agreement between assessors are im-

portant factors which represent a strength of thisapproach to measuring appropriateness. With the ad-dition of more detailed, empirically based data on the

interaction between patient and health professional,plus the adoption of explicit judgement criteria, wewould argue that the HSUS approach has considerable

strengths as a method of appropriateness measure-ment.

The relevance of research assessing general medical

consultations

The reliance of the HSUS method on medicalrecords as the source of assessment data has already

been noted. Within general medical practice there hasbeen an overt focus on assessing the consultation itself,as a site of interaction between professional andpatient (see Ong, de Haes, Hoos & Lammes, 1995 for

a review of these studies). Consultation data has beenproduced using either audio-tape recording (Maguire& Faulkener, 1988; Stiles & Puttnam, 1989) or a com-

bination of video and audio-tape recording (Blanchard,Ruckdeschel, Blanchard, Arena, Saunders & Malloy,1983; Roter, Hall & Katz, 1988; Simino�, Fetting &

Abelo�, 1989; Henbest & Stewart, 1990) or direct ob-servation (Kraan, Crijen, Zuudweg, van der Kleuten &Imbos, 1989). The consultation data are then analysed

using a detailed coding frame which permits a metho-dical identi®cation, categorization and quanti®cationof what have been de®ned as the salient features ofdoctor±patient communication. Ong et al. (1995) refer

to these as `interaction analysis systems'. They identifytwo main types of interaction analysis systems: thosewhich are meant to assess task-centered or instrumen-

tal behaviour, and those which measure a�ective(socio-emotional) behavior. As well as di�ering byobservational strategy, Ong et al. (1995) point out that

these di�er from one another with regard to their clini-cal relevance, their reliability and validity and the typeof communicative behavior they are attempting toassess.

Whilst useful in alerting us to the range of strategiesused to assess communication within general practicesettings and di�erent modes of data capture, the rel-

evance of these techniques to the study of advice-giv-ing in community pharmacies is limited. Research hasonly recently began to explore the nature and content

of the pharmaceutical consultation (Tully et al., 1997;Bissell et al., 1997b; Hassell et al., 1997, 1998). Itseems important to resist the possibility of imposing

an inappropriate framework by borrowing techniquesfrom research on the GP consultation. A crucial di�er-

ence between GP and pharmacy consultation is therelatively large number of GP consultations whichfocus on multi-dimensional problems encompassing a

complex intermingling of physical, psychological andsocial elements (Howie, Heaney, Maxwell & Walker,1998). This does not appear to be the case in advice-

giving episodes in community pharmacies (Bissell etal., 1997b; Hassell et al., 1998). Thus, simple compari-sons between the two types of interactions may not be

valid. In addition, the focus of the interaction analysissystems has often been on `communication' as a pro-cess in itself. In the work reported on here, we did notwish to con®ne ourselves simply to exploring the

appropriateness of communication between consumersand health professionals: we wished also to explore theimportance of other dimensions of advice-giving,

including outputs such as product recommendationsand referral to other agencies. On these grounds, theinteraction analysis systems were not considered

further.Bearing in mind the strengths and limitations of the

approaches described above, we now review the few

published studies within pharmacy practice researchwhich have engaged with this issue.

Studies exploring the appropriateness of advice-giving in

community pharmacies

In a review of research, Tully et al. (1997) reportthat few studies have explored the quality or appropri-

ateness of service delivery in community pharmacy. Aswe have suggested, research undertaken by health pro-fessionals (Smith et al., 1990; Goodburn et al., 1991;Anderson & Alexander, 1992; Krska & Kennedy,

1996) and particularly by outside bodies (ConsumersAssociation, 1994, 1996, 1999) has been overtly criticalof pharmacists' performance along a number of dimen-

sions. The Consumers' Association studies are prob-ably the most widely quoted. However, not only hastheir work provoked critical debate from the pharmacy

profession and outside of it concerning the use of cov-ert research techniques Ð researchers/actors posing asconsumers presenting at pharmacies with ailments orrequesting non-prescription medicines (Alexander,

1991, 1992; Dingwall, Watson & Aldridge, 1992), theConsumers' Association studies are limited in otherrespects. The method of assessment is only brie¯y and

uncritically described (Anon, 1991) and any di�cultiesassociated with conceptualizing or measuring standardsof care are not referred to. Furthermore, there is no

account of disagreement between assessors (Bissell etal., 1997a). The absence of explicit criteria againstwhich (in)appropriate care was judged also exacerbates

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359348

the methodological frailty of these studies as does the

lack of comment on statistical validation of those cri-teria.Use of covert research techniques also precluded a

consideration of the individual speci®city of the consu-mer±pharmacy sta� interaction. Although we might

not disagree with the general thrust of their ®ndings,the methodological limitations compromise the valueof the Consumers' Association studies. Perhaps just as

importantly objections raised about the ethics of covertresearch (with its demonstrable lack of informed con-sent) have diverted attention from meaningful debate

within the profession about the ®ndings of the studiesand instead focused attention mainly on the adequacy

of the method used.Smith et al. (1990) approached the question of asses-

sing practice variation by developing nine explicit cri-

teria through a small group of community pharmacistsand general practitioners (two of each). Smith's workagain con®rms an impression of suboptimal care.

Although claiming to assess quality, rather than appro-priateness per se, their work represents a signi®cant

landmark in practice research. At the same time, thiswork raises a number of methodological and concep-tual issues. Smith et al. (1990) used audio-recording to

capture advice-giving between pharmacists and consu-mers in a sample of pharmacies in London. To dothis, a microphone was attached to the pharmacist for

a period of 3 h in 64 pharmacies. All advice-giving in-teractions were then transcribed. This data collection

technique meant that Smith et al. (1990) excludedadvice-giving provided by MCAs. Given that recentresearch has shown that four-®fths of deregulated

medicines sales are carried out by MCAs in theabsence of any formal input from the pharmacist(Ward et al., 1998), this, in hindsight, represents a sig-

ni®cant omission. Indeed, the latest Consumers' As-sociation (1999) study has emphasised the discrepancy

between advice provided by MCAs and the pharma-cist, with the advice provided by the latter generallyheld to be adequate. In addition, Smith et al. (1990)

did not provide contextual information to assessorswhich is likely to be relevant when measuring thequality or appropriateness of advice (for example,

whether the consumer was known by sta�, how busythe store was, whether the consumer seemed in a

hurry, etc).Finally, the dimensions identi®ed by Smith et al.

(1990) used to assess quality were not devised using

explicit consensus development techniques, such asthe Nominal Group Technique or Delphi technique.

Neither were they subject to rigorous statistical vali-dation. Despite the detailed criticisms of this work,Smith's contribution to research into community

pharmacy represents the ®rst attempt to assessadvice-giving, rather than simply document it. Thus,

Smith's work has been seminal in terms of focuss-

ing attention on methodologies for assessing advice-giving and her conceptualisation of advice-givingunderlies the approach taken in this study.

More recently, Krska, Greenwood and Howitt(1994) and Krska and Kennedy (1996) developed

`self audit' techniques to assess `response to symp-tom' presentations within community pharmacies. Inthese studies, community pharmacists completed

questionnaires indicating the advice they thoughtshould be given in each of twelve devised scenariosthat could be presented at community pharmacies.

The six scenarios with the highest level of consensuswere then re-enacted and audited by researchers

using semi-covert means (pharmacy sta� were awarethat a researcher would anonymously requestadvice/purchase a medicine at a time of the

researcher's choosing). Researchers recorded the in-teractions occurring within the study communitypharmacies. Whilst these studies employed a poten-

tially useful audit technique to explore the qualityof pharmacist's advice, their use of a quasi-covert

method means that the individuality of the phar-macy sta�-consumer relationship is not explicitlyexplored and the method did not capture any dis-

agreement between those setting the standards toaudit.Morrow, Hargie, Donnelly and Woodman (1993)

have used video-recording to explore the advisoryfunction of the community pharmacist, again focus-

ing only on the activities of the pharmacist ratherthan all pharmacy sta�. These authors were notconcerned with assessing appropriateness or quality

per se, rather they undertook a communicationaudit during which they analysed the number, typeand nature of questions asked by community phar-

macists. Although this approach provides details ofthe visual element of communication, similar criti-

cisms to that of audio-tape recording alone apply,in that video-recording cannot capture all relevantcontextual and background material. In addition,

within the community pharmacy environment, the®eld of vision of a video camera may not be wideenough to capture the activities of sta� involved in

the sale of medicines, provision of advice and o�er-ing general assistance given that Bissell et al.

(1997a) report that sta� often move around thestore in practice when providing services and advice(e.g. moving to quiet areas when providing personal

advice, helping on the selection of medicines indi�erent areas of the store, moving to the dispen-sary).

Bearing in mind this critique of studies from second-ary care, primary care and pharmacy practice, we now

describe the rationale for the choice of methods usedin this study.

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359 349

Methods used in this study

The present study used a three-stage research pro-cess. Stage one focused on capturing data on advice-giving in a sample of ten pharmacies using audio-tape

recording supplemented by non-participant obser-vation. In stage two, the Nominal Group Technique(NGT) was used to develop explicit judgement criteria

for assessing the examples of advice-giving collected instage 1. Finally, in stage three, an expert group ofpractising community pharmacists was convened to

ensure the practical relevance, feasibility and oper-ational utility of the explicit criteria. Each of thesestages is now described in more depth.

Stage 1. Capturing the pharmacy sta�±consumer

interaction

Full details of the data collection method are

reported elsewhere (Bissell et al., 1997a). In brief, datacollection took place in ten community pharmacies inone health authority located in the North-West ofEngland. An audio tape-recorder was positioned on

the front counter of each pharmacy, accompanied by aposter informing consumers they were being tape-recorded. Recording of pharmacy sta�±consumer in-

teractions took place over ®ve working days in eachcommunity pharmacy. All tape-recorded data weretranscribed. These data, consisting of 624 real-life

examples of advice-giving occurring within communitypharmacies, formed the backdrop against which theexplicit judgement criteria were developed. Local

Research Ethics Committee approval was grantedalongside Local Pharmaceutical Committee approval.Given the practical limitations of using video-record-

ing within community pharmacies (discussed above),

this study used audio-recording in conjunction withnon-participant observation to capture examples ofadvice-giving (see Appendix A for an example of a

transcript). Audio tape-recording provided a verbatimaccount of the interaction taking place between consu-mers and pharmacy sta�. At the same time, a non-par-

ticipant observer (a social scientist researcher) madeextensive ethnographic ®eld notes of the interactionstaking place in the pharmacy. Observation complemen-ted the audio-recorded data and allowed the researcher

to gather data on pertinent visual and contextualdetails of the advice-giving. For example, theresearcher could determine (if unclear) what medicines

had been requested; the type of symptoms presented;whether pharmacy sta� had prior knowledge of theconsumer; whether the shop was busy or quiet; a lim-

ited assessment of whether the consumer appeared in ahurry and which member(s) of sta� was involved ineach consumer interaction.

The dual data collection strategy allowed us to pro-duce a record of the words used in each interaction

occurring between consumers and pharmacists/MCAs,whilst observational ®eldwork provided details aboutthe context of the interaction in the pharmacy and

provided data on the speci®city of the encounterbetween consumer and professional. However, due tothe practical and logistical constraints of researching

what were often busy community pharmacies, wecould not obtain data on exactly what was knownabout each consumer Ð we could only ascertain

whether or not the consumer was previously known topharmacy sta�. In addition, there was no opportunityto collect data on any cognitive insights into pharma-cist or MCA decision making. In other words, we

could not document the reasoning behind the vari-ations in response by pharmacists and MCAs to thepresentations of symptoms or requests for non-pre-

scription medicines by consumers in the study pharma-cies.One of the problems which is sometimes seen as a

major problem for researchers conducting observa-tional research is the `Hawthorne' e�ect, wherebythose being studied modify their behaviour in some

way as a reaction to being investigated. Within thisstudy, we attempted to minimise any potential`Hawthorne' e�ect by conducting one long period ofobservation (5 days) in each pharmacy, rather than

undertaking short bursts of research activity, as it hasbeen suggested that the `Hawthorne' e�ect is likely tomanifest itself most strongly in the earliest phase of

the research and decline over time (Cornwell, 1984).However, recent research in a sample of pharmacieshas failed to describe any clear relationship between

observation and changes in practice (Savage, 1996).

Stage 2. Developing explicit criteria

Given the limitations of implicit criteria, this study

sought to develop explicit criteria for assessing theappropriateness of advice-giving. We took a decisionto use the nominal group technique (NGT), as it has

been successfully used within HSR to develop consen-sus guidelines for a wide variety of conditions (Jones& Hunter, 1995; Cantrill, Sibbald & Buetow, 1996;Hunter, McKee, Sanderson & Black, 1994; McKee &

Black, 1993). In addition, Buetow et al. (1996) usedthe NGT to develop indicators for the appropriatenessof prescribing. The NGT utilises and builds upon

expert opinions which, it has been suggested, shouldbe called upon ``whenever it becomes necessary tochoose among several alternative courses of action in

the absence of an accepted body of theoretical knowl-edge that would clearly single out one course as thepreferred alternative'' (Helmer, 1966, p. 11). This was

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359350

the case when attempting to identify or develop criteria

for assessing the appropriateness of advice-giving incommunity pharmacy. Thus, we felt that the NGT wasan appropriate method to use in this stage of the

study.Brie¯y, the NGT consists of a highly structured

meeting used to gather information from relevantexperts (usually about 8±12 in number) on a givenissue. The group is tightly controlled by a facilitator

and discussion restricted to the latter stages of thegroup process. It usually consists of a number ofrounds whereby ideas are individually generated and

then fed back to the facilitator. Each idea is then dis-cussed by the group. A priority ranking of ideas can

be undertaken, the results of which can be fed back tothe group, and re-voting can take place if necessary. Amore detailed description of the nominal group tech-

nique can be found elsewhere (Tully & Cantrill, 1997)and a description of alternative approaches to develop-ing consensus can be found in Cantrill et al. (1996).

Although there are modi®cations to the nominalgroup technique (Hunter et al., 1994), the approach

described above was used in this study. It has beensuggested that the NGT yields far more ideas than aconventional, unstructured expert group, or focus

group meeting (Cantrill et al., 1996). Its chief advan-tage lies in its ability to limit problems associated withprofessional hierarchies where group dynamics might

serve to inhibit freedom of expression. Its eschewal of`interaction' between group members was seen as its

primary advantage in this study. Given the absence ofstudies in this area which had sought to develop expli-cit assessment criteria, we believed that the nominal

group's speci®c structure for eliciting ideas had signi®-cant advantages over simple expert or focus groups(Delbecq, van de Ven & Gustafson, 1975). In addition,

as in other research which has used this method togenerate data (Tully & Cantrill, 1997), it is possible to

combine methods using the NGT alongside otherapproaches (such as the Delphi technique or validitysurveys). Information on validity assessment of the

identi®ed criteria is reported in a forthcoming paper.Alongside the advantages of using the NGT, there

are also a number of potential weaknesses inherent innominal groups. One of the weaknesses concerns whatis known as the `self-ful®lment prophecy' (Desselle &

Rappaport, 1997). When experts are asked to generatecriteria for assessing advice-giving, it may be di�cultfor them to disassociate criteria based on day to day

practice within community pharmacy from thosewhich may be seen to be the `ideal'. In other words, it

may be di�cult for experts to generate criteria toassess what happens in reality rather than what shouldhappen in theory. A further problem with the NGT

relates to validity. The main aim of the NGT was todevelop the criteria and to generate content and face

validity, but more speci®cally to generate a level ofconsensual validity, which may be present when ``a

number of experts agree that a measure is valid''(Streiner & Norman, 1994). However, one of the pro-blems with using consensual validity to justify the use

of an instrument is that di�erent groups of expertsmay di�er in their consensus of opinion. Thus, had weconvened di�erent set of experts, there may have been

some variation in the criteria that emerged. As a result,the next stage of the study involved a large scale sur-vey to assess the validity and reliability of the criteria.

It was hoped that this survey would provide a statisti-cally signi®cant level of validity which could not beachieved through the use of a nominal group. It hasalso been noted that a problem with the NGT, along

with all methods of developing consensus, is that ofreaching agreement or consensus. According to Fink etal. (1984), agreement is only ever reached on bland

generalities that represent the lowest common denomi-nator of debate. However, in this study, complete con-sensus was not a prerequisite, rather we wanted to

uncover areas of disagreement in addition to areas ofagreement, similar to the HSUS studies (Merrick et al.,1987).

The nominal group technique Ð selection of participants

Studies using consensus methods infrequently reporton the problems and processes associated with identify-ing relevant `experts' which seems surprising given thatthe composition of the group is likely to signi®cantly

shape its subsequent output (Cantrill et al., 1996).Similarly, the credibility and quali®cations of membersof expert groups largely go unreported (Brook et al.,

1990). Cantrill et al. (1996) point out that `expert' sta-tus is usually synonymous with achieving high statusin a professional group. They suggest that techniques

such as theoretical sampling may well be appropriatefor selection of experts.For this study, the research team conducted a tele-

phone survey of a theoretically sampled group of aca-demics and practitioners in the ®eld to produce a listof (ten) potential participants. Those surveyed by tele-phone were asked to nominate potential group mem-

bers on the basis of perceived expertise in eitherpractice, academic or policy areas of community phar-macy. Each nominated participant was contacted by

post and asked to take part. Eight participants agreedand were provided with a brie®ng pack detailing thepurpose of the study, background reading on appropri-

ateness measurement and transcripts of advice-givingcollected during the ®rst stage of the study.

The nominal group Ð description of process

Before the start of the nominal group, participants

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359 351

were asked to consider what constituted appropriateadvice-giving from their perspective as experts in either

academic, practice or policy areas of pharmacy and toidentify factors they felt were important when assessingwhat might constitute appropriate or inappropriate

advice-giving. However, it was made clear that whereother perspectives (e.g. that of the consumer, generalpractitioner, carer or family member) were felt to be

important by individual participants, they should beincluded in the discussion. Thus, whilst the expertswere asked to focus on their professional perspective

as pharmacists, they were also asked to bear in mindhow other important stakeholders might view appro-priate advice-giving. Although this may have facilitateda more rounded approach to developing criteria on the

appropriateness of advice-giving, one of the problemswith this approach was that we were not completelyaware of the perspective being taken when each item

was generated.Participants were asked to consider the nominal

question and to write down phrases or words (items)

they considered important in answering the question.The nominal question (devised by researchers) was:``How would you assess whether a consultation between

pharmacy sta� and a consumer was (in)appropriate?''.Participants then contributed one `item' until all par-

ticipants' ideas had been exhausted. Each item was dis-cussed by the group to clarify its relevance to the

nominal question. Some were subsequently rejected bythe group as unimportant, or subsumed within others.The decision to include or exclude a single item was

made on the basis of a majority show of hands.

The nominal group Ð discussion and ranking of items

In total, the group identi®ed 73 individual itemswhich were believed to be important in answering the

nominal question. These were then grouped into four-teen speci®c criteria by the nominal group participantswhich were reduced to ten criteria following detailed

discussion.Participants were then tasked with prioritizing the

criteria in terms of their relative importance to answer-

ing the nominal question. However, there were a num-ber of di�culties with this. Firstly, given that thecriteria would be used to assess examples of advice-giv-ing within community pharmacies, the nominal group

participants excluded any assessment of the outcomeof the consultation beyond the immediate actions orrecommendations made by sta� within the pharmacy.

Secondly, participants were concerned about the degreeof interdependence between the identi®ed criteria,which they felt made it di�cult to prioritise them.

Thus, prioritisation of criteria was deemed to beneither possible nor necessary by the group partici-pants. A (unanimous) group decision was taken that

each separate criterion should be assigned similarweight.

One week after the conclusion of the nominal group,the criteria were circulated to NG participants forassessment and consideration. They were asked to con-

sider the criteria and comment on their suitability. Theparticipants reported a high level of satisfaction withthe criteria.

Stage 3. Assessing the feasibility of the criteria

The third stage of this study was concerned with

ensuring the practical relevance and face validity of theidenti®ed criteria in the eyes of practising communitypharmacists given that it would be this group which

would be asked to assess actual examples of advice-giving. Researchers convened a group consisting of sixpractising community pharmacists randomly selectedfrom one health authority. Participants were asked to

comment on the validity of the criteria and to identifythe most appropriate assessment scaling tool. In ad-dition, they were tasked with ensuring that the criteria

were able to be operationalised in practice. Thisinvolved testing the e�cacy of three di�erent types ofvisual analogue rating scales, using a ®ve-item, seven-

item and nine-item scale.It was recognised that two of the nominal group de-

rived criteria could not be assessed in practice (theserelated to an assessment of the overall layout and or-

ganisation of the pharmacy). As a result, the ten cri-teria developed by the original nominal groupparticipants were reduced to eight explicit judgement

criteria.This group applied the revised criteria to ten tran-

scripts of actual advice-giving episodes collected during

stage 1 of the study. Of the three visual analoguescales, the ®ve-point scale was unanimously acceptedas the favoured means of assessment. The ®ve-point

scale was found to provide pharmacists with su�cientscope to record qualitative di�erences between thetranscripts on each of the eight criteria. The seven andnine point scales provided no advantage to pharma-

cists in terms of discriminant power and they found itdi�cult to di�erentiate between transcripts to such ahigh degree.

Description of criteria

The eight explicit criteria are presented below. Each

criterion is presented with a title, a descriptor, andexamples of the individual statements supporting eachcriterion.

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359352

Criterion 1 Ð General communication skills

This involves an assessment of the general communi-cation skills used in the advice-giving. This includes inparticular, the listening and responding skills of the

pharmacy sta� involved in the interaction. Forexample:

. encouraging continuous feedback by consumers;

. engendering honesty and trust between the consumerand pharmacy sta�;

. not being intimidating;

. being sensitive and tactful towards consumers.

Criterion 2 Ð What information is gathered bypharmacy sta�?

This is concerned with what kind of information isgathered by the pharmacy sta� and whether this was

su�cient to enable advice to be given and appropriaterecommendations to be made by pharmacy sta�involved. For example, did pharmacy sta�:

. ®nd out who the product was for?

. gather su�cient information in order to make a rec-ommendation?

. establish previous use of the product?

. investigate the consumer's expectation of the out-come?

. use appropriate questions in terms of seriousness ofthe illness?

. assess the consumer's levels of knowledge?

. address consumer's explicit requests?

Criterion 3 Ð How is the information gathered by thepharmacy sta�?

This criterion is concerned with how the informationwas gathered by pharmacy sta�. In other words, the

speci®c communication skills used to gather infor-mation from a consumer. For example:

. the e�ciency with which information was gathered;

. using information gathering e�ectively;

. using open and/or closed questions where appropri-ate;

. not missing information.

Criterion 4 Ð Issues to be considered by pharmacy sta�before giving advice

Once the information is gathered, there are otherissues which need to be taken into account before giv-

ing advice or making recommendations to the consu-mer. These include:

. addressing the beliefs and expectations of the consu-mer;

. taking account of consumer choice;

. assessing advice received from other sources;

. ensuring value for money for the consumer;

. assessing the e�ect on the consumer's quality of life;

. taking into account whether or nor the consumer isknown to pharmacy sta�.

Criterion 5 Ð Rational content of advice given by

pharmacy sta�

This section is concerned with what advice wasgiven by the pharmacy sta� to the consumer. This is

an assessment of the advice given by the pharmacysta� for both short term and longer term follow-up.For example, did the pharmacy sta�:

. provide accurate information?

. provide speci®c instructions?

. give advice on contributing factors to symptoms?

. explain about any longer term follow up?

. give an explanation of the product or action rec-ommended?

. where a product sale is refused, the risks/bene®ts areexplained to the consumer.

Criterion 6 Ð How is the advice given?

This section is an assessment of the communicationskills used to advise the consumer. For example:

. the timing or staging of the information given;

. the clarity of the explanation;

. making sure that the terminology was suitable forthe consumer;

. ensuring achievable goals;

. con®rmation of consumer understanding;

. avoiding information overload.

Criterion 7 Ð Rational product choice made by

pharmacy sta�

This section is concerned with the appropriateness

of products sold by the pharmacy sta�. This is onlyapplicable in consultations where a product sale wasmade. For example:

. giving the correct product quantity;

. making an evidence-informed product selection;

. ensuring safety;

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359 353

. not contravening product license indications;

. not forcing a sale on consumers.

Criterion 8 Ð Referral

This section is concerned with the appropriatenessof referrals made to other health professionals (includ-ing the pharmacist if the advice is provided by aMCA), including assessing those cases where a referral

should have been made but was not. Where referraldid take place, assessment should take into account:

. that the referral is to a relevant expert;

. that the referral takes account of consumer choice;

. the con®dentiality of the referral.

Summary and discussion

In this paper, we have argued that health servicesresearchers have become increasingly interested in

assessing the appropriateness of health care provisionas a result of a growing perception that much care isvariably delivered. Within pharmacy practice research,

whilst variation in patterns of service delivery areincreasingly reported, there have been few attempts tocarry out structured research into understanding vari-

ation and assessing appropriateness.Our review identi®ed two theoretical strengths of an

appropriateness measure: ®rstly, synthesising existing

practitioner expertise where the evidence base is weakor absent and secondly, linking together di�erent attri-butes of care (cost, e�ciency, e�cacy, acceptability,etc.) into one macro model. Despite the seeming sim-

plicity of the idea, however, it is clear there have beendi�culties operationalising appropriateness in practice.Notwithstanding this, existing studies have helped to

highlight good and inadequate practice in many areasof health service delivery and have demonstrated theimportance of understanding the processes in¯uencing

(in)appropriate care.Given the di�erences between the pharmaceutical

and the general medical consultation, we felt there wasa requirement for pharmacy speci®c assessment cri-

teria, as opposed to importing criteria from researchon general practice consultations which might haveonly peripheral relevance to community pharmacy.

The literature review noted the importance ofemploying a data collection strategy which capturednot only the advice-giving activities of both pharma-

cists and medicines counter assistants, it also stressedthe importance of providing background informationon the context of advice-giving encounters in the phar-

macy. The literature review also identi®ed the import-

ance of making explicit the perspective taken byparticipants involved in devising assessment criteriaand making judgements of appropriateness. To this

end, expert participants charged with developing thecriteria were required to provide not only their pro-fessional perspective as pharmacists, but to bear in

mind the perspective of other stake-holders whendevising judgement criteria. The Nominal Group Tech-

nique was used to derive the resulting criteria whichhas the advantage over expert groups and focus groupsof limiting potential bias introduced through pro-

fessional hierarchies. Nonetheless, we remain awarethat the composition of the nominal group (derived in

this study through telephone polling of a theoreticallysampled group of academics and practitioners in the®eld) is likely to considerably in¯uence the resultant

output. The techniques used for sampling expert/nom-inal group participants are rarely alluded to in research(Tully & Cantrill, 1997). We suggest that more

research is needed in order to explore the relationshipbetween group composition and resulting output. The

NGT as used in this study generated eight explicit cri-teria which have been subject to statistical validationand reliability testing. This will be reported on in a

subsequent paper (Ward, Bissell & Noyce, 2000a).Re¯ecting on the output of the nominal group, it

generated criteria focusing both on the process (e.g.communication skills, gathering information) and theoutput of the advice-giving (e.g. rational advice, pro-

duct selection and referral). The focus of the derivedcriteria on instrumental, rather than psycho-social orsocio-emotional dimensions may be a re¯ection of a

professional perspective that sees such factors at theheart of the encounter between pharmacy sta� and

consumers. Whilst such an emphasis may chime withmany pharmacy consumers' day-to-day experiences ofaccessing pharmacies, less widely acknowledged dimen-

sions of the pharmaceutical consultation (i.e. thoserelating to psycho-social care, and personal, socio-emotional support) need to be recognized as forming a

part of the service provided by community pharmacies.These under-researched dimensions may need to be in-

corporated into measures of the appropriateness ofadvice-giving, given that there is some evidence thatthese activities are highly valued by pharmacy consu-

mers (Bissell et al., 1997b). Furthermore, emphasisingonly the technical and clinical aspects of appropriate

advice-giving may lead to an overly technocratic andone-dimensional view of the community pharmacy,which obscures those factors less amenable to quanti®-

cation such as the informal and personal aspects ofadvice and care (Bissell, Ward & Noyce, 1996;Stromme & Haugli, 1996).

At the same time, we remain acutely aware that thefocus of many pharmacy consumers is unlikely to

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359354

extend beyond wanting to obtain a medicinal product

and complete what is essentially a commercial trans-action promptly and e�ciently. Consumers may wellwish to avoid the interference of being asked what are

considered to be unnecessary and intrusive questionswhen purchasing medicines (Hassell et al., 1998). Forexample, it has been argued that young people do not

expect to be asked questions when buying medicines inthe pharmacy (Gray, Cantrill & Noyce, 1998). Cer-

tainly, there is evidence that medicines counter assist-ants are aware of consumer resistance to being asked`unnecessary' questions (Ward et al., 1998). The expla-

nation for this resistance may be based on di�erentialnotions of the risks and bene®ts of non-prescription

medicines, with consumers viewing them as little di�er-ent to other items of commerce. Certainly, researchhas highlighted the relevance of lay expertise as a key

factor in shaping consumer resistance to the impositionof questioning in the pharmacy (Bissell et al., 1997,2000; Hassell et al., 1998; Ward, Bissell & Noyce,

2000b). It may be that lay people's beliefs about non-prescription medicines and con®dence to self-treat

minor ailments signi®cantly limits the extent to whichpharmacists and MCAs can operationalise questioningtechniques associated with medicine sales in the phar-

macy without incurring widespread consumer resist-ance. Thus, the focus of the criteria developed in thisstudy on gathering consumer information and asking

questions may well re¯ect a professional perspectivesomewhat at odds with that of the pharmacy consu-

mer. As Liberati (1996) has pointed out, the challengefor researchers developing measures of appropriatecare (and practice guidelines) is to do so with reference

to consumer preferences which in¯uence medical de-cision-making and thus shape practice variation in the®rst instance. The preceding discussion highlights some

of the di�culties in devising measures of the appropri-ateness of advice-giving which accurately re¯ect the

dynamic needs, requirements and demands of the con-sumer of pharmacy services. At present, within phar-macy practice research, there seems insu�cient

evidence to determine Ð with any precision Ð thecontours of consumer preferences in this area (Tully et

al., 1997). Given the absence of robust evidence ofconsumer preferences, the absence of valid and reliableoutcome measures (Mays, 1994) and the rapidly chan-

ging self-medication environment, it seems crucial toproduce (at the very least) professionally derived cri-teria which can serve as indicators of professional stan-

dards of appropriate care. This study has taken a stepin this direction. Subsequent research can explore the

acceptability of the established criteria to pharmacyconsumers and research should now be instigated tofurther re®ne these criteria through consumer involve-

ment in this area (Coulter, 1997; Liberati, 1996).The development of appropriateness criteria will

allow us to identify, for the ®rst time, the scale of bothinappropriate and appropriate advice-giving in com-

munity pharmacy. The analytical power to discrimi-nate is one of the obvious advantages of this work.For example, given that the developed criteria can be

applied to examples of advice-giving regardless ofwhich member of sta� (pharmacist and/or MCA) isproviding the advice, the instrument could be used to

determine the extent to which advice provided byMCAs is more or less appropriate than that providedby pharmacists. This is currently highly relevant given

that recent research suggests that pharmacists currentlyprovide more advice to consumers than MCAs,although they manage a much smaller proportion oftotal sales (Bissell et al., 1997a,b; Consumers' Associ-

ation, 1999). Our criteria will allow us to explore theappropriateness of this advice-giving. The developedinstrument could go some way to providing further

understanding of the appropriateness of the currentskill mix within community pharmacies.In addition, alongside an overall assessment of

(in)appropriateness, we will also be able to identifyspeci®c criteria (e.g. information gathering or rationalproduct choice) which have been rated as (in)appropri-

ate. Where sub-optimal levels of care are evident andwhere there is disagreement over what constitutesappropriate advice-giving, recommendations can bemade for education, training and further research.

Thus, not only will this work shed light on the appro-priateness of advice-giving per se, it will also be able todemonstrate which particular aspects of the advice-giv-

ing process are deemed to be ¯awed, and provideinsights into the reasons behind variations in practice.Given that the route adopted for improving practice in

this area has been protocol and guideline development,this work is relevant to improving and informing prac-tice associated with the management of minor ailmentsand the sale of medicines through community phar-

macy. Furthermore, given the renewed emphasis inUK Government health policy on auditing standardsof care across all sectors of the NHS, this work has

considerable practical and policy utility in contributingto the development of clinical governance.

Acknowledgements

We would like to thank Dr Marjorie Weiss for facil-itating the Nominal Group referred to in this study.Funding for the study was obtained from Sefton

Health Authority and the NHS E. In particular, theauthors would like to extend a huge debt of gratitudeto those pharmacists and medicines counter assistants

who took part in the study and subjected themselvesto detailed scrutiny: without their help, this researchwould not have been possible. The authors are also

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359 355

grateful to Dr Derek Hibbert for comments on draftsof this paper.

Appendix

The following transcript was recorded during the

®rst stage of the study. The abbreviations are as fol-lows: C=consumer, P=pharmacist, MCA=medicinescounter assistant.

Transcript 1A male, middle aged customer entered the phar-

macy. He presented a prescription for sleeping tablets

and proceeded to ask for advice about his cough. Hewas unknown to the medicines counter assistant andthere were no other consumers in the pharmacy at the

time.C: Have you got a cough medicine love for a . . .

(coughs) . . . this cough and catarrh here, you know

that keeps stickingMCA: Are you taking anything else at all?C: Just these (shows the sleeping tablets)

MCA: Just at night, those?C: Yes, just something to break it upMCA: Is it worse during the day?C: Errmm . . . yes, and at night love and you're . . .

(coughs) . . . all the time to clear it, you know what Imean(MCA goes into dispensary to ask P)

C: It's when I was in hospital when I did this, a fort-night ago . . . you know with laying in bed for 3 or 4days . . .

MCA: Hmmm . . . it's irritating?P: You don't really want anything to loosen it up,

because that'll just make your cough more . . . so just agood cough mixture just to stop the cough

C: Yes, yes . . . thanks yesMCA: (Points to pholcodeine linctus) That one?C: Yes, whatever you think . . . how much will it be

love?MCA: Do you pay for your prescriptions?C: Yes love (pays for prescription and pholcodeine

linctus)MCA: Thank youC: Thanks, bye.

References

Alexander, A. (1991). The agent provocateur study as a

research tool. Pharmaceutical Journal, 247, 154±155.

Alexander, A. (1992). Confessions of a covert researcher.

Pharmaceutical Journal, 248, 573.

Anderson, C., & Alexander, A. (1993). Response to dysme-

norrhoea: an assessment of pharmacists' knowledge and its

application in practice. International Journal of Pharmacy

Practice, 2, 180±183.

Anon (1991). Pharmacists' advice: not good enough, says the

Consumers Association. Pharmaceutical Journal, 247, 748.

Audit Commission (1994). A prescription for improvement

towards more rational prescribing in general practice.

London: HMSO.

Beers, M. H., Ouslander, J. G., Rollinger, I., Rueben, D. B.,

Brooks, J., & Beck, J. C. (1991). Explicit criteria for deter-

mining inappropriate medication use on nursing home resi-

dents. Annals of Internal Medicine, 151, 1825±1832.

Bernstein, S. J., McGlynn, E. A., Siu, A. L., Roth, C.,

Sherwood, M. J., & Keesey, J. (1993). The appropriateness

of hysterectomy: a comparison of care in seven health

plans. Journal of the American Medical Association, 269,

2989±2992.

Berwick, D. M. (1989). Health services research and quality

of care. Assignments for the 1990's. Medical Care, 27,

761±763.

Bickell, N. A., Earp, J., & Evans, A. T. (1995). A matter of

opinion about hysterectomies: experts' and practicing com-

munity gynecologists Ð ratings of appropriateness.

American Journal of Public Health Medicine, 85(1), 1125±

1128.

Bissell, P., Hibbert, D., Ward, P. R., & Noyce, P. R. (2000).

Transforming the work of the pharmacy: the challenge of

lay expertise, Sociology of Health and Illness (submitted).

Bissell, P., Ward, P. R., & Noyce, P. R. (1996). Advising the

public. In A report to Sefton Health Authority.

Bissell, P., Ward, P. R., & Noyce, P. R. (1997a). Variations

within community pharmacy: I Requesting over the coun-

ter medicines. Journal of Social and Administrative

Pharmacy, 14, 1±15.

Bissell, P., Ward, P. R., & Noyce, P. R. (1997b). Variations

within community pharmacy: II Presenting symptoms.

Journal of Social and Administrative Pharmacy, 14, 105±

115.

Bissell, P., Ward, P. R., & Noyce, P. R. (1997c). Variations

within community pharmacy: III Referrals to other health

professionals. Journal of Social and Administrative

Pharmacy, 14, 116±123.

Bissell, P., Ward, P. R., & Noyce, P. R. (1997d). Self-medi-

cation and the communication of risk: the case of deregu-

lated medicines. In Report to NorthThames NHSE.

Blanchard, C. G., Ruckdeschel, J. C., Blanchard, E. B.,

Arena, J. G., Saunders, N. L., & Malloy, E. D. (1983).

Interactions between patients and oncologists during

rounds. Annals of Internal Medicine, 99, 694±698.

Blenkinsopp, A., & Bradley, C. (1996). Over the counter

drugs: patients, society and the increase in self-medication.

BMJ, 312, 629±632.

Blenkinsopp, A., & Paxton, P. (1995). Symptoms in the phar-

macy. A guide to the management of common illnesses.

Oxford: Blackwell Scienti®c.

Brook, R. H. (1994). Appropriateness: the next frontier.

British Medical Journal, 308, 218±219.

Brook, R. H., Kamberg, C. J., Mayer-Oakes, A., Beers, M.

H., Raube, K., & Steiner, A. (1990). Appropriateness of

acute medical care for the elderly: an analysis of the litera-

ture. Health Policy, 14, 225±242.

Buetow, S. A., Sibbald, B., Cantrill, J. A., & Halliwell, S.

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359356

(1996). Prevalence of potentially inappropriate long term

prescribing in general practice in the United Kingdom,

1980±95: systematic literature review. British Medical

Journal, 313, 1371±1374.

Buetow, S. A., Sibbald, B., Cantrill, J. A., & Halliwell, S.

(1997). Appropriateness in health care: application to pre-

scribing. Social Science & Medicine, 45(2), 261±277.

Cantrill, J., Sibbald, B., & Buetow, S. (1996). The Delphi and

nominal group techniques in health services research.

International Journal of Pharmacy Practice, 4, 67±74.

Charuptanapong, N. (1994). Perceived likelihood of risks in

self-medication practices. Journal of Social and

Administrative Pharmacy, 11, 18±28.

Chassin, M. (1989). How do we decide whether an investi-

gation or procedure is appropriate? In Hopkins,

Appropriate investigation and treatment in clinical practice.

London: Royal College of Physicians.

Chassin, M., Koseco�, J., Park, R. E., Winslow, C. M.,

Kahn, K. L., Merrick, N. J., Keesey, J., Fink, A.,

Solomon, D. H., & Brook, R. H. (1987). Does inappropri-

ate use explain geographic variation in the use of health

care services? A study of three procedures. Journal of the

American Medical Association, 258, 2533±2537.

Chewning, B., & Sleath, B. (1996). Medication decision mak-

ing and management: a client centred model. Social

Science & Medicine, 42(3), 389±398.

Consumers' Association (1991). Pharmacists: how reliable are

they? Which? Way to Health, December, 191±194.

Consumers' Association (1994). Over-the-counter medicines.

No prescription necessary. Which?, June, 38±41.

Consumers' Association (1996). Pharmacists in crisis. Which?,

January, 18±21.

Consumers' Association (1999). Counter advice. Which?,

March, 22±25.

Cornwell, J. (1984). Hard-earned lives. London: Tavistock

Press.

Coulter, A. (1997). Partnerships with patients: the pros and

cons of shared clinical decision making. J. Health Serv.

Res. & Policy, 2(2), 112±121.

Cunningham-Burley, S., & MacLean, U. (1987). The role of

the chemist in primary health care for children with minor

complaints. Social Science & Medicine, 24, 371±377.

Delbecq, A. L., van de Ven, A. H., & Gustafson, D. H.

(1975). Group techniques for programme planning, a guide

to nominal group and Delphi processes. Glenview, IL: Scott,

Foresman.

De Almeida Neto, A., Benrinoj, S., Gomel, M., & Fois, R.

(1996). Inappropriate Self-medication practices: a phar-

macy-based intervention. Journal of Social and

Administrative Pharmacy, 13(3), 131±138.

Desselle, S., & Rappaport, H. M. (1997). The identi®cation of

Pharmaceutical Care Practice Standards in the Community

Pharmacy Setting. Journal of Pharmaceutical Care, 1(3)

http://198.79.220.3/pharmacy//jpc/jp10103.htm.

Dingwall, R., Watson, P. J., & Aldridge, A. (1992). Covert

research Ð poor ethics and bad science. Pharmaceutical

Journal, 249, 182±183.

Fink, A., Koseco�, J., Chassin, M., & Brook, R. H. (1984).

Consensus methods: characteristics and guidelines for use.

American Journal of Public Health, 74, 979±983.

Gabe, J., Kelleher, D., & Williams, G. H. (1994). Challenging

medicine. London: Routledge.

Goodburn, E., Mattoshino, S., Mongi, P., & Waterston, A.

(1991). Management of childhood diarrhoea by pharma-

cists and parents: is Britain lagging behind the Third

World? British Medical Journal, 302, 440±443.

Gray, N. J., Cantrill, J. A., & Noyce, P. R. (1998). Mass

media health information available to young adults in the

United Kingdom: (2) Television and radio. International

Journal of Pharmacy Practice, 6(4), 188±195.

Hall, M. A. (1994). Letter to the Editor. New England Journal

of Medicine, 330, 433.

Hanlon, J. T., Schmader, K. E., Samsa, K. E., Weinberger,

M., Uttech, K. M., Lewis, I. K., Cohen, H. J., & Feusser,

J. R. (1992). A Method of assessing drug therapy appro-

priateness. Journal of Clinical Epidemiology, 45, 1045±

1051.

Hassell, K., Noyce, P. R., Rogers, A., Harris, J., &

Wilkinson, J. (1997). A pathway to the GP: the pharma-

ceutical consultation as a ®rst port of call in primary

health care. Family Practice, 14(6), 498±502.

Hassell, K., Noyce, P. R., Rogers, A., Harris, J., &

Wilkinson, J. (1998). Advice provided in community phar-

macies: what people want and what they get. Journal of

Health Services Research and Policy, 3(4), 219±226.

Harding, G., & Taylor, K. (1997). Responding to change: the

case of community pharmacy in Great Britain. Sociology

of Health and Illness, 19(5), 547±560.

Helmer, O. (1966). In Social technology (p. 11). New York:

Basic Books.

Henbest, R. J., & Stewart, M. A. (1990). Patient-centredness

in the consultation: does it really make a di�erence?

Family Practice, 7, 28±35.

Hicks, N. R. (1994). Some observations on attempts to

measure appropriateness of care. British Medical Journal,

309, 730±733.

Hoog, S. (1992). The self-medication market Ð a literature

study. Journal of Social and Administrative Pharmacy, 7(4),

164±169.

Howie, J. G. R., Heaney, D. J., Maxwell, M., & Walker, J. J.

(1998). A comparison of a Patient Enablement Instrument

(PEI) against two established satisfaction scales as an out-

come measure of primary care consultations. Family

Practice, 15(2), 165±171.

Hunter, D. J. W., McKee, C. M., Sanderson, C. F. B., &

Black, N. A. (1994). Appropriate indications for prosta-

tectomy in the UK B results of a consensus panel. Journal

of Epidemiology and Community Health, 48, 58±64.

Jones, J., & Hunter, D. (1995). Consensus methods for medi-

cal and health services research. British Medical Journal,

311, 376±380.

Kahan, J., Park, R., Leape, L., Bernstein, S. J., Hilborne, L.

H., Parker, L., Kamberg, C. J., Ballard, D. J., & Brook,

R. H. (1996). Variations by speciality in physician ratings

of the appropriateness and necessity of indications for pro-

cedures. Medical Care, 34, 512±523.

Kahn, K. L., Koseco�, J., Chassin, M. R., Flynn, M. F.,

Fink, A., Pattaphongse, N., Solomom, D. H., & Brook,

R. H. (1988). Measuring the clinical appropriateness of the

use of a procedure. Can we do it? Medical Care, 26, 415±

422.

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359 357

Kassirer, J. P. (1993). The quality of care and the quality of

measuring it. New England Journal of Medicine, 329, 1263±

1264.

Kraan, H., Crijen, A., Zuudweg, J., van der Kleuten, C., &

Imbos, T. (1989). Evaluating undergraduate training Ð a

checklist for medical intervewing skills. In M. A. Stewart,

& D. L. Roter, Communicating with medical patients.

Newbury Park: Sage Publications.

Krska, J., & Kennedy, E. (1996). Expectations and experi-

ences of consumers purchasing over-the-counter medicines

in pharmacies in the north of Scotland. Pharmaceutical

Journal, 256, 354±356.

Krska, J., Greenwood, R., & Howitt, E. (1994). Audit of

advice provided in response to symptoms. Pharmaceutical

Journal, 252, 93±96.

Leape, L., Park, R., Solomon, D., Chassin, M. R., Koseco�,

J., & Brook, R. H. (1990). Does inappropriate use explain

small-area variations in the use of health care services.

Journal of the American Medical Association, 263, 669±672.

Levin, L. (1990). Re-orienting perspectives on self-medication.

Journal of Social and Administrative Pharmacy, 7(4), 123±

127.

Liberati, A. (1996). Assessing appropriateness of care. J.

Health Serv. Res. Policy, 1(1), 53±54.

Lupton, D. (1997). Consumerism, rel¯exivity and the medical

encounter. Social Science & Medicine, 45(3), 371±381.

Maguire, P., & Faulkener, A. (1988). How to do it.

Communicating with cancer patients: I Handling bad news

and di�cult questions. British Medical Journal, 297, 907.

Maitra, A. K., & Choyce, M. Q. (1994). A critical look at

accident and emergency review clinics. Journal of Accident

and Emergency Medicine, 11, 243±245.

Mays, N. (1994). Health services research in pharmacy: A criti-

cal personal review. University of Manchester: Pharmacy

Practice Research Resource Centre.

McColl, E. (1994). An agenda for change in referral Ð con-

sensus from general practice. British Journal of General

Practice, 44, 157±162.

McKee, M., & Black, N. (1993). Junior doctors work at

night: what is done and how much is appropriate? Journal

of Public Health Medicine, 15(1), 16±24.

McPherson, K. (1994). How should health policy be modi®ed

by the evidence of medical practice variations? In M.

Marinker, Controversies in health care policies. Challenges

to practice (pp. 55±74). London: BMJ Publishing.

Merrick, N., Fink, A., Park, R., Brook, R. H., Koseco�, J.,

Chassin, M. R., & Solomon, D. H. (1987). Deviation of

clinical indications for carotid endarterectomy by an

expert panel. American Journal of Public Health, 77, 187±

190.

Morrow, N., Hargie, O., Donnelly, H., & Woodman, C.

(1993). ``Why do you ask?'' A study of questioning beha-

viour in community pharmacist-client consultations.

International Journal of Pharmacy Practice, 2, 90±94.

Mulley Jr, A. G., & Eagle, K. A. (1988). What is inappropri-

ate care? Editorial. Journal of the American Medical

Association, 260, 540±541.

Ong, L. M. L., de Haes, J. C. J. M., Hoos, A. M., &

Lammes, F. B. (1995). Doctor±patient communication: a

review of the literature. Social Science & Medicine, 40,

903±918.

Park, R. R., Fink, A., & Brook, R. H. (1986). Physician rat-

ings of appropraite indications for six medical and surgical

procedures. American Journal of Public Health, 17, 766±

772.

Paxton, R., & Chapple, P. (1996). Misuse of over-the-counter

medicines: A survey in one English county. Pharm. J., 256,

313±315.

Payne, K., Ryan-Woolley, B. M., & Noyce, P. R. (1998).

Role of consumer attributes in predicting the impact of

medicines deregulation on NHS prescribing in the UK.

International Journal of Pharmacy Practice, 6, 150±158.

Roter, D. L., Hall, J. A., & Katz, N. R. (1988). Patient-phys-

ician communication: a descriptive summary of the litera-

ture. Patient Education and Counselling, 12, 99±106.

Royal Pharmaceutical Society of Great Britain (1999).

Medicines, ethics and practice: A guide for pharmacists.

London: RPSGB.

Savage, I. T. (1996). Observing pharmacists at work: quanti-

fying the Hawthorne E�ect. Journal of Social and

Administrative Pharmacy, 13, 8±19.

Scott, E. A., & Black, N. (1991). Appropriateness of cholecys-

tectomy Ð a consensus panel approach. Gut, 32, 1066±

1070.

Simino�, L. A., Fetting, J. H., & Abelo�, M. D. (1989).

Doctor patient communication about breast cancer adju-

vant therapy. Journal of Clinical Oncology, 7, 1192±1197.

Smith, F. J., Salkind, M. R., & Jolly, B. C. (1990).

Community pharmacy: a method of assessing quality of

care. Social Science & Medicine, 31(5), 603±607.

Smith, R. (1998). All changed, changed utterly. British

Medical Journal, 316, 1917±1918.

Stiles, W. B., & Puttnam, S. M. (1989). Analysis of verbal

and non-verbal behaviour in doctor±patient encounters. In

M. A. Stewart, & D. L. Roter, Communicating with medi-

cal patients. Newbury Park, CA: Sage.

Streiner, D. L., & Norman, G. R. (1994). Health measurement

scales. A practical guide to their development and use.

Oxford: Oxford Medical Publications.

Stromme, H. K., & Haugli, A. (1996). Communication and

interaction between consumers and pharmacy personnel in

two Norwegian pharmacies Ð an observational study.

International Journal of Pharmacy Practice, 4, 209±213.

Thomas, D. H. V., & Noyce, P. R. (1996). The interface

between self-medication and the NHS. British Medical

Journal, 312, 688±691.

Trease, G. E. (1964). Pharmacy in history. London: Balliere,

Tindall & Cox.

Tully, M. P., & Cantrill, J. A. (1997). The use of the Nominal

Group Technique in pharmacy practice research: processes

and practicalities. Journal of Social and Administrative

Pharmacy, 14(2), 93±104.

Tully, M. P., Hassell, K., & Noyce, P. R. (1997). Advice-giv-

ing in community pharmacies in the UK. Journal of

Health Services Research and Policy, 2(1), 38±50.

Ward, P. R., Bissell, P. R., & Noyce, P. R. (1998). Medicines

counter assistants: roles and responsibilities in the sale of

deregulated medicines. International Journal of Pharmacy

Practice, 6(4), 207±216.

Ward, P. R., Bissell, P., & Noyce, P. R. (2000a). Criteria for

assessing non-prescription drug therapy in community

pharmacy, Annals of Pharmacotherapy (in press).

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359358

Ward, P. R., Bissell, P., & Noyce, P. R. (2000b). Insights into

the uncertain world of the consumer: the case of deregu-

lated medicines. In P. Taylor-Gooby, Risk, trust and wel-

fare. London: Macmillan Press (in press).

Wennberg, D. E. (1998). Variation in the delivery of health

care: the stakes are high. Annals of Internal Medicine, 128,

866±868.

Wilkin, D. (1992). Patterns of referral: explaining variation.

In M. Roland, & A. Coulter, Hospital referrals (pp. 76±

91). Oxford: Oxford University Press.

Wilcox, S. M. (1994). Inappropriate drug prescribing for the

community-dwelling elderly. Journal of the American

Medical Association, 272, 292±296.

Williams, S., & Calnan, M. (1996). Modern medicine: Lay per-

spectives and experiences. London: UCL Press.

Winslow, C. M., Koseco�, J., Chassin, M. R., Kanouse, D.

E., & Brook, R. H. (1988). The appropriateness of per-

forming coronary artery bypass surgery. Journal of the

American Medical Association, 260, 505±509.

Working Group. (1993). What do we mean by appropriate

health care? Quality in Health Care 2, 117±123. Working

Group for the Director of Research and Development in

the NHS Management Executive.

P. Bissell et al. / Social Science & Medicine 51 (2000) 343±359 359