What Do Primary Care Practitioners Want to Know? A Content Analysis of Questions Asked at the Point...

Original Research

What Do Primary Care Practitioners Want to Know? AContent Analysis of Questions Asked at the Point of Care

LISE M. BJERRE,1,2,4 MD, PHD, MCFP; NICHOLAS R. PATERSON,1 BSCH; JESSIE MCGOWAN,1,3 MLIS, PHD, AHIP;WILLIAM HOGG,1,2,3,4 BSCH, MSC, MCLSC, MDCM, CCFP, FCFP; CRAIG M. CAMPBELL,5 MD, FRCPC; GARY VINER,1

MD, MED, CCFP, FPFP; DOUGLAS ARCHIBALD,1,2 PHD

Introduction: Assessing physician needs to develop continuing medical education (CME) activities is an integralpart of CME curriculum development. The purpose of the present study was to demonstrate the feasibility of iden-tifying areas of perceived greatest needs for continuing medical education (CME) by using questions collectedelectronically at the point of care.

Methods: This study is a secondary analysis of the “Just-in-Time” (JIT) information librarian consultation servicedatabase of questions using quantitative content analysis methods. The original JIT project demonstrated thefeasibility of a real-time librarian service for answering questions asked by primary care clinicians at the point ofcare using a Web-based platform or handheld device. Data were collected from 88 primary care practitioners inOntario, Canada, from October 2005 to April 2006. Questions were answered in less than 15 minutes, enablingclinicians to use the answer during patient encounters.

Results: Description of type and frequency of questions asked, including the organ system on which the questionsfocused, was produced using 2 classification systems, the “taxonomy of generic clinical questions” (TGCQ), andthe International Classification for Primary Care version 2 (ICPC-2). Of the original 1889 questions, 1871 (99.0%)were suitable for analysis. A total of 970 (52%) of questions related to therapy; of these, 671 (69.2%) addressedquestions about drug therapy, representing 36% of all questions. Questions related to diagnosis (24.8%) andepidemiology (13.5%) were also common. Organ systems questions concerning musculoskeletal, endocrine, skin,cardiac, and digestive systems were asked more than other categories.

Discussion: Questions collected at the point of care provide a valuable and unique source of information on thetrue learning needs of practicing clinicians. The TGCQ classification allowed us to show that a majority of questionshad to do with treatment, particularly drug treatment, whereas the use of the ICPC-2 classification illustrated thegreat variety of questions asked about the diverse conditions encountered in primary care. It is feasible to useelectronically collected questions asked by primary care clinicians in clinical practice to categorize self-identifiedknowledge and practice needs. This could be used to inform the development of future learning activities.

Key Words: clinical questions, point-of-care learning, evidence-based medicine, family practice/education, ed-ucation, medical, continuing/methods, continuing medical education, continuing professional development, self-assessment, performance improvement CE

Disclosures: The authors report this study was funded by a competitive grantawarded by the Academy for Innovation in Medical Education (AIME), Fac-ulty of Medicine, University of Ottawa. AIME played no role in the studydesign; in the collection, analysis, or interpretation of data; in the writingof the report; or in the decision to submit the report for publication. Theresearchers are independent of the funder.

Affiliations: 1Department of Family Medicine, University of Ottawa, Ot-tawa, Canada; 2C.T. Lamont Primary Health Care Research Centre, BruyereResearch Institute, Ottawa, Canada; 3Institute of Population Health, Univer-sity of Ottawa, Ottawa, Canada; 4Department of Epidemiology and Com-munity Medicine, University of Ottawa, Canada; 5Office of ProfessionalAffairs, Royal College of Physicians and Surgeons, Ottawa, Canada.

Introduction

Continuing medical education (CME), a component of con-tinuing professional development (CPD), plays an integralrole in the professional lives of health care workers. These

Correspondence: Lise M. Bjerre, C.T. Lamont Primary Health Care Re-search Centre, Bruyere Research Institute, 43 Bruyere St. Room 330Y, Ot-tawa (Ontario) Canada, K1N 5C8; e-mail: [email protected].

© 2013 The Alliance for Continuing Education in the Health Professions,the Society for Academic Continuing Medical Education, and the Councilon Continuing Medical Education, Association for Hospital Medical Educa-tion. • Published online in Wiley Online Library (wileyonlinelibrary.com).DOI: 10.1002/chp.21191

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS, 33(4):224–234, 2013

What Do Primary Care Practitioners Want to Know?

educational activities, which often take the form of annualrefresher courses or more focused live conferences, serve toinform physicians and other primary care practitioners onthe latest evidence and developments relevant to their prac-tice and assist in addressing gaps in knowledge, skill, andpractice. CME is also a significant time investment for med-ical professionals with some family practitioners document-ing close to 100 hours each year.1 Maximizing engagementin activities that are evidence-based and designed with thespecific identification of needs is a critical first step to en-suring positive changes in physician practice.2 Because as-sessing physician needs to develop CME activities is an in-tegral part of CME curriculum development, the depth andquality of needs assessment is an important issue. Surveys ofphysicians or planning committees are often a key compo-nent of needs assessment for CME activities.3 While theseare relatively easily generated, they may overrepresent is-sues that are topical and interesting, or given prominent cov-erage in the medical or general media, and underrepresentthe actual learning needs of the practicing physician. Thelack of specific identification of individual learning needs inCME events may be a reason why traditional CME offer-ings often fall short of attendee’s expectations for maintain-ing competence.3

The present study aimed to assess the self-perceived learn-ing needs of primary care practitioners by analyzing clinicalquestions stimulated by their daily practice. Using point-of-care questions to analyze primary care physicians’ needs hasalready been done successfully in a smaller study reportedby Ebell et al.3 In this study, the researchers’ goal was toidentify the clinical questions that health care professionalshave at the point of care and explore whether these ques-tions could be used to drive a needs assessment for continu-ing education programs. Five hundred sixty-three questionswere collected by researchers who asked clinicians “to re-port all questions—everything from clear-cut questions tovague and fleeting uncertainties—and to evaluate their im-portance (low, moderate, or high priority)” after a patientencounter.3 The authors concluded that more than two thirdsof physician questions fell into 1 of 5 categories: cause ofa finding, test selection, prevention, treatment selection, andprognosis.3

In order to characterize learning needs in a way that bet-ter reflects the diversity of questions asked at the point ofcare, we proposed using 2 parallel strategies within a de-ductive framework to classify a larger bank (n = 1889) ofpoint-of-care questions using both a question-type (treat-ment, diagnosis, management, etc; Ely classification) andan organ system–type (cardiologic, musculoskeletal, psycho-logical, etc; ICPC-2) classification. We felt that 2 classifica-tion strategies were needed in order to adequately describethe 2 “dimensions” clinical questions can be ascribed to—onthe one hand, generic question type (reflective of the undi-

agnosed presentations common in primary care), and on theother hand, the organ system–based problems.

While collecting point-of-care questions has, in the past,been considered time consuming and resource intensive,3

our approach overcomes those limitations as it draws upon abank of self-reported questions used to establish the feasibil-ity of a medical librarian information service.4 We intendedto assess the feasibility of using information generated in thecontext of what could become a “routine” clinical informa-tion source to better inform and guide providers of CME asto the needs of their participants.

Methods

A bank of 1889 questions was analyzed using 2 parallelclassification schemes. These questions were obtained dur-ing the randomized controlled trial phase of the JIT project,which used a real-time medical librarian consultation ser-vice to answer clinical questions asked by 82 physicians, 4nurse practitioners, 1 nurse, and 1 family medicine resident;the details of this trial have been published elsewhere.4–6

On the one hand, the questions were coded using a strategythat focused on the generic type of each question. We chosethis approach because this coding mechanism reflects thepresentation-based approach in primary care, where the diag-nosis or disease is unknown and the focus is on the patient’spresenting signs and symptoms. The “taxonomy of genericclinical questions” (TGCQ), developed by Ely,7 uses 4 lev-els of classification and a total of 64 subcategories to identifythe type of question being asked (see TABLE 1 for the first 2levels of classification and examples of questions). The high-est, overarching level consists of 6 broad areas, including di-agnosis, treatment, management, epidemiology, nonclinical,and unclassified questions.

The other coding strategy involved classifying questionsby subject matter, using the framework of the InternationalClassification for Primary Care, version 2 (ICPC-2).8 Wechose this scheme because it was developed specifically foruse in primary care; furthermore, it is widely used for admin-istrative coding in primary care in a number of countries.9–11

This classification uses a letter to represent the location in thebody or human organ system (for example, S = Skin) andsubsequent numbers ranging from 1 to 99 to address variouscomplaints, symptoms, diagnoses, or diseases relating to thathuman system.

Questions were exported from the original Accessdatabase (obtained from J.M.G., lead author of the originalJust-in-Time study,5 and coauthor of the present study) to anExcel spreadsheet and 2 raters (N.R.P., L.M.B.) conductedcoding for the first 100 questions under each classificationscheme. If there was uncertainty as to the appropriate clas-sification, it was discussed by the primary coder (N.R.P.)and the principal investigator (L.M.B.), a practicing family

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—33(4), 2013 225DOI: 10.1002/chp

Bjerre et al.

TABLE 1. TGCQ Classification—Frequency of Question Types (Primary and Secondary Level)

Number of Percent of

Primary Classification Secondary Classification questions (n) total (%)

1 Diagnosis 1.1 Cause/interpretation of clinical finding 463 24.75

139 7.43

1.2 Criteria/Manifestation 38 2.03

1.3 Test (Lab, skin test, biopsy, imaging, etc.) 241 12.88

1.4 Name finding 4 0.21

1.5 Orientation 36 1.92

1.6 Inconsistencies 1 0.05

1.7 Cost 1 0.05

1.8 Not elsewhere classified 3 0.16

2 Treatment 978 52.27

2.1 Drug prescribing 671 35.86

2.2 Not limited to but may include drug prescribing 299 15.98


3 Management (not specifying

diagnostic/ therapeutic)

978 52.27

3.1 Condition/Finding 86 4.60

3.2 Other providers 6 0.32

3.3 Doctor-patient communication 4 0.21


4 Epidemiology 252 13.47

4.1 Prevalence/Incidence 61 3.26

4.2 Etiology 125 6.68

4.3 Course/Prognosis 63 3.37


5 Nonclinical 47 2.51

5.1 Education 39 2.08

5.2 Administration 4 0.21

5.3 Ethics 0 0.00

5.4 Legal 2 0.11

5.5 Frustration 1 0.05


Unclassified 35 1.87

Total 1871 100.0

TGCQ = taxonomy of generic clinical questions.

physician, to see if consensus could be reached; if consensuscould not be reached, the physician’s decision would prevail.Given the number of choices for each of the 2 schemes (64 forTGCQ and hundreds for ICPC-2), a Cohen’s kappa test12 for

interrater reliability is mathematically not suitable.13 How-ever, since chance played little role in the determination ofagreement (because for each question there were only a few,not many, truly plausible choices), the percent agreement in

226 JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—33(4), 2013DOI: 10.1002/chp


FIGURE 1. Representation of Primary Taxonomy of Generic Clinical Questions Classification

fact roughly equals the Cohen’s kappa, which could thereforebe used. If more than 1 code applied to the question, each wasdocumented in order of significance. Only the primary codewas used during analysis.

Results

Interrater Agreement

Results from the first 100 questions were compared anddemonstrated agreement rates of 70% for the Ely classifica-tion and 79% for the ICPC-2, thus falling in the 0.6 to 0.8 or“strong agreement” range.

TGCQ/Ely Classification

Of the 1889 questions, a total of 1871 (99.0%) could be in-cluded for classification in the present study. Eighteen ques-tions were excluded because they were either incomplete, outof scope, or duplicates. There were 119 questions that hadsecondary codes. The breakdown of the TGCQ for the 1871study questions was as follows: 52% Treatment, 25% Diag-nosis, 14% Epidemiology, 5% Management, 3% Nonclini-cal and 2% Unclassified (see TABLE 1 and FIGURE 1). Thetop 10 most frequently asked question types made up 67.7%of the total questions (TABLE 2). Questions related to ther-apy were most common, with 970 (52%) questions falling inthat category; of these, 671 (69.2%) addressed questions re-lating to drug therapy, which represents 35.9% of all ques-tions (TABLE 3). Questions relating to diagnosis (24.8%)and epidemiology (13.5%) were also frequently identified.

This classification system was extremely successful at cov-ering the wide breadth of questions asked by physicians, withonly 18 out of 1871 questions being excluded as unclassifi-able. Of the 64 TGCQ categories, 57 had at least 1 question.

ICPC-2

The results of the ICPC-2 were fairly evenly distributedacross the different human organ systems (TABLE 4). Sev-eral questions fell under the General and Unclassified cate-gories (32% and 9% of questions, respectively) within eachof the main categories (diagnosis, prognosis, etc). Thoseaside, musculoskeletal, endocrine, skin, cardiac, and gas-trointestinal questions each made up a greater proportionof questions asked by primary care physicians than othercategories, a combined 32% of questions. There were 267(14.3%) questions with secondary codes and another 14(0.7%) with tertiary codes.

We further characterized the questions by looking at thenumber of questions asked for each disease category, as de-fined not only by the anatomic ICPC-2 code (letter) as inTABLE 4, but also by the more specific, 2-digit numericcode that follows. The results of these analyses are presentedin TABLE 5, which lists conditions for which 10 or morequestions were asked. From this table, it can be seen thatconditions/issues with the most questions included healthmaintenance and prevention (10.3% of all questions), un-classified (8.6%), adverse effects of medical agents (5.4%),abnormal results of investigations (4.8%), concern/fear ofmedical treatment (3.0%), bladder disorder (2.2%), infec-tious diseases not otherwise specified (NOS) (1.6%), general


Bjerre et al.

TABLE 2. Top 10 Questions Asked by Physicians at Point of Care

TGCQ Number of Percent of total

Rank code Section questions (n) questions (%) Examples of question type

1 2.2.1.1 Treatment 232 12.4 How should I treat finding/condition y (given situation z)?

2 2.1.2.1 Treatment 196 10.5 Is drug x (or drug class x) indicated in situation y or for condition y?

3 1.3.1.1 Diagnostic 166 8.9 Is test x indicated in situation y? (includes screening tests)

4 2.1.2.2 Treatment 129 6.9 Should this kind of patient get prophylactic drug x to prevent condition y?

5 4.2.1.1 Epidemiology 116 6.2 Is x a risk factor for condition y?

6 2.1.3.1 Treatment 109 5.8 Could finding y be caused by drug x?

7 1.1.3.1 Diagnostic 106 5.7 What is the cause of test finding x?

8 3.1.1.1 Management 86 4.6 How should I manage condition/finding/situation y?

9 4.3.1.1 Epidemiology 63 3.4 What is the usual course (or natural history) of condition y?

10 2.1.3.3 Treatment 63 3.4 Is drug x safe to use in situation y?

disease (1.5%), risk factors Not Otherwise Specified (NOS)(1.3%), osteoporosis (1.2%), lipid disorders (1.2%), compli-cation of medical treatment (1.1%) and musculoskeletal con-ditions (1.1%).

We did a cross-tabulation of all 64 TGCQ and 349ICPC-2 codes using a pivot table (data are available as sup-plemental material [APPENDIX S1] available in the onlineversion of this article). The combinations of codes with morethan 10 questions each are listed in TABLE 6. All of the mostfrequent combinations had codes in the ICPC-2 “Generaland Unspecified” (group A) category, or in the “unclassified”category. We also examined the organ-system topics (as ex-pressed by the ICPC-2 codes) that were most common for thetop 10 generic types of questions (see TABLE 7). There wereno evident patterns of topic-by-question-type clustering thatemerged, however, the preponderance of questions relating

to medication is notable (these are highlighted in gray in thetable).

Discussion

Both theory and empirical evidence indicate that the firststep to designing effective CME content is an evidence-basedneeds assessment.2,14 Surveys are used as tools to elicit thelearning needs of practicing clinicians. Unfortunately, thisapproach is often limited by poor response rates and by biasintroduced by the survey itself.15 Additionally, there is strongevidence to suggest that physicians have a limited ability toassess themselves, their skills, and their learning needs.15,16

The current study’s analysis of questions asked by primarycare practitioners at the point of care represents a feasible ap-proach to assessing their perceived needs for learning. Few

TABLE 3. Number of Questions Pertaining to Drugs

Number As percent of total

Category (n) questions (%) Example questions

Drug prescribing and

medication use

671 35.9 What is the shortest effective course of antibiotics for a bacterial

sinusitis in an otherwise healthy person?

Not limited to, but may include

drug prescribing as part of

treatment

299 16.0 In a 69-year-old man with leg pains, probably claudication, what are the

implications for a 50% stenosis in the R iliac artery? Are surgical

interventions warranted or treat with medication?

Total of all questions 970 51.8



TABLE 4. ICPC-2 Classification

Number Percent of

Organ System (n) total (%) Examples

A—General 600 32.1 • Is there any evidence linking Lipitor with Alzheimer’s disease?

• What is the recommended exclusionary period from contact sports following

mononucleosis?

• What is the management of breast milk jaundice in a 10-day-old infant?

• What immunizations are required for travel to Shenyang (Liaoning Province) in China?

• What is involved in post-test HIV counseling?

B—Blood 33 1.8 • What is “prothombin gene heterozygous positive” clotting tendency?

D—Digestive 113 6.0 • What is the differential diagnosis of a swollen salivary gland in the submandibular area?

F—Eye 15 0.8 • What are the criteria for surgery for redundant eyelid?

H—Ear 14 0.7 • What are the causes of pulsatile tinnitus?

K—Cardiac 112 6.0 • What is the proper workup for a 66-year-old woman with apparent new onset of atrial

fibrillation?

L—Musculoskeletal 135 7.2 • How frequent is Osgood-Schlatter disease found unilaterally in teens?

N—Neurologic 80 4.3 • What is the differential and workup for upper motor neuron lesions?

P—Psychiatric 87 4.6 • What are the DSM-4 criteria for Asperger’s syndrome?

R—Respiratory 71 3.8 • Do children with pneumonia require follow-up chest x-rays?

S—Skin 113 6.0 • What is the likely cause or precipitant of pityriasis rosea rash and what is the average

duration it will last?

T—Endocrine/Metabolic 120 6.4 • For a patient with a normal hemoglobin but elevated ferritin level, what is the next best test

to determine if he has hemochromatosis?

U—Urologic 49 2.6 • Is there any correlation between exophytic renal cancer and renal cysts?

W—Pregnancy/Contraception 87 4.6 • How long must a female wait postpartum before having an IUD inserted for contraception?

X—Female genital 52 2.8 • What is the rate of recurrence after enterocele (rectocele) repair?

Y—Male genital 29 1.5 • How significant are retractile testes in a 2-year-old?

Z—Social 1 0.1 • Is there a recommended screening test for a patient who was possibly exposed to “agent

orange” in Vietnam many years ago?

UNC—Unclassified 160 8.6 • Is there a table of narcotics (other than the CPS) that includes all dosage formulations and

equivalent potencies?

• What is thimerosal and where is it found? Does fluviral vaccine have this ingredient?

• How does Tamoxifen work at the cell level?

• What is the dosing range of Sorbitol required to have a laxative effect? How does this

translate into how many candies, sticks of gum, or chocolates are required?

• What kind of activities are considered “weight bearing” when counselling patients in

prevention of falls and osteoporosis?

ICPC = International Classification for Primary Care; HIV = Human Immunodeficiency Virus; DSM = Diagnostic Statistical Manual; IUD =Intra-Uterine Device; CPS = Compendium of Pharmaceuticals and Specialties.


Bjerre et al.

TABLE 5. Most Frequently Asked Questions by Clinical Problem (forProblems With 10 or More Questions Each)

ICPC-2 Number of % total

code Description questions questions

A98 Health maintenance/prevention 192 10.26

UNC Unclassified 160 8.55

A85 Adverse effect medical agent 101 5.40

A91 Abnormal result investigation NOS 89 4.76

A13 Concern/fear medical treatment 55 2.94

U13 Bladder symptom/complaint other 40 2.14

A78 Infectious disease other/NOS 30 1.60

A99 General disease NOS 27 1.44

A23 Risk factor NOS 24 1.28

L95 Osteoporosis 22 1.18

T93 Lipid disorder 22 1.18

A87 Complication of medical treatment 21 1.12

L99 Musculoskeletal disease, other 21 1.12

P76 Depressive disorder 16 0.86

A21 Risk factor for malignancy 15 0.80

K78 Atrial fibrillation/flutter 15 0.80

S99 Skin disease, other 15 0.80

W14 Contraception other 15 0.80

T90 Diabetes, non-insulin-dependent 14 0.75

W71 Infection complicating pregnancy 14 0.75

A94 Perinatal morbidity other 13 0.69

R99 Respiratory disease, other 13 0.69

U71 Cystitis/urinary infection other 13 0.69

D70 Gastrointestinal infection 12 0.64

U95 Urinary calculus 12 0.64

X99 Genital disease female, other 11 0.59

A90 Congenital anomaly NOS/multiple 10 0.53

D99 Disease digestive system, other 10 0.53

K84 Heart disease other 10 0.53

K99 Cardiovascular disease, other 10 0.53

L76 Fracture, other 10 0.53

L82 Congenital anomaly musculoskeletal 10 0.53

L87 Bursitis/tendinitis/synovitis, other 10 0.53

N04 Restless legs 10 0.53

N99 Neurological disease, other 10 0.53

P06 Sleep disturbances 10 0.53

S74 Dermatophytosis 10 0.53

W84 Pregnancy high risk 10 0.53

questions were unclassifiable (1.0%), leaving 99.0% of thequestions sorted into categories useful for determining learn-ing needs of physicians based on their personal perceptionof need. Furthermore, there was good agreement betweenour results and those from a smaller study by Ebell,3 where,for example, the top 9 categories made up 64.3% of ques-tions. In addition, if the JIT information librarian consul-tation service4 were to be broadly implemented in primarycare practices, these questions, which can be seen as an ex-pression of learners’ self-perceived needs, could be used byCME providers to frame content for group learning, insteadof requiring additional surveying.

One of the more interesting results of our study, realizedusing the TGCQ classification system, was the sheer vol-ume of treatment-related questions. Over half of all questionswere treatment-related, and the majority of these focused onelements regarding drug prescriptions. Questions that weremost popular were “How should I treat finding/condition y(given situation z)?”; “Should this kind of patient get prophy-lactic drug x to prevent condition y?”; and “Could finding ybe caused by drug x?” This is in keeping with previous stud-ies, which showed that these same treatment questions areoften of great concern among physicians.3,7 Whether this em-phasis on treatment and management of disease is addressedin sufficient detail in current CME activities is unknown.

The ICPC-2 results did not highlight a particular areaof need but rather the great diversity of questions pertain-ing to different human systems asked during primary careencounters, the complexity of which has been highlightedelsewhere.17 Recognizing and characterizing the breadth oftopics primary care clinicians encounter in practice is impor-tant to the creation of appropriate CME geared to the learn-ers’ needs, and as such the ICPC-2 has an important role toplay in informing CME planning decisions. Nonetheless, theICPC-2 and its predecessor have been criticized in the pastfor being incomplete and missing many clinically importantcategories.10 This was perhaps evident in the number of un-classified questions our group could not match to appropri-ate codes. However, our results were consistent with severalother studies, demonstrating musculoskeletal, cardiac, diges-tive, and endocrine questions to be among those most oftenseen in the primary care clinical setting.3,18,19

One of the strengths of this study’s findings is that usingpoint-of-care data to inform providers of CME of the ques-tions physicians have developed that express their need forinformation is likely be a very economical way of conductingeffective learning needs assessments. While the widespreadimplementation of a JIT information consultation service hasbeen projected to be cost effective to health systems for itsintended use,6 our approach promotes the use of the practice-stimulated questions as a low cost by-product to inform thecontent of formal CME activities. This means the questions



TABLE 6. Most Frequent Combinations of Question Types (TGCQ Classification) and Organ-System Topics (ICPC-2 Classification)

Frequency (number of questions

with each combination of)

TGCQ Example of question ICPC-2 Description TGCQ and ICPC-2 codes)

2.1.2.2 Should this kind of patient get prophylactic

drug x to prevent condition y?

A98 Health maintenance/prevention 56

2.1.3.1 Could finding y be caused by drug x? A85 Adverse effect medical agent 49

1.1.3.1 What is the cause of test finding x? A91 Abnormal result investigation NOS 46

1.3.1.1 Is test x indicated in situation y? A98 Health maintenance/prevention 35

2.1.3.3 Is drug x safe to use in situation y? A13 Concern/fear medical treatment 16

1.3.1.1 Is test x indicated in situation y? UNC Unclassified 16

4.2.1.1 Is x a risk factor for condition y? UNC Unclassified 14

2.1.1.3 When (timing, not indication) or how should

I start/stop drug x?


2.1.3.1 Could finding y be caused by drug x? A13 Concern/fear medical treatment 12

4.2.1.1 Is x a risk factor for condition y? A85 Adverse effect medical agent 12

2.1.3.3 Is drug x safe to use in situation y? A85 Adverse effect medical agent 10

2.1.4.1 Is it OK to use drug x with drug y? A13 Concern/fear medical treatment 10

2.2.1.1 How should I treat finding/condition y (given

situation z)?

UNC Unclassified 10

TGCQ = taxonomy of generic clinical questions; ICPC = International Classification for Primary Care.

could be used to inform content development on an ongo-ing basis. These questions, raised by the physicians who usethe service, may be reflective of the general learning needsof physicians who do not use the service and therefore bemore generally relevant to a population of physicians. In ad-dition, the fact that primary care practitioners receive an-swers to the questions they ask within 15 minutes or lessmeans they would likely see the service as useful. This is incontrast to Ebell and colleagues’ 2011 study,3 where physi-cians were asked to submit their questions with no immedi-ate reward to them, giving them less incentive to ask con-sistently and contribute to a bank of data than the currentstudy.

Another important advantage of using questions askedat the point of care for learning needs assessments is thatthese questions are very likely to be highly practice rel-evant, and potentially practice changing. Clinicians wouldbe unlikely to devote valuable clinical time to ask ques-tions that were not immediately relevant to the care of theirpatients.

This study also has some inherent limitations. First, nei-ther classification system is perfectly refined to usefullycategorize all questions. We can see this by how manyquestions, for the ICPC-2 classification, fell into the classifi-

cation of “(A) General” or those simply unclassifiable in theICPC-2. Since the “General” section covers such a vast rangeof topics (screening, prescribing medication, adverse effectsof treatment, bacterial/viral infections, immunizations, cer-tain paediatric conditions and individual symptoms such asfever, swelling, pain, etc.) there was a very high incidenceof questions, in seemingly unrelated areas, classified in thiscategory. The overrepresentation of “General” questions hasalso been documented in large-scale audits of health care sys-tems using the ICPC-2.19 One approach that could be use-ful to CME planners is to report the ICPC-2 “General” cat-egory in more detail, that is, at the 2-digit level. This couldimply reporting on almost 50 subcategories; however, onecould possibly group some related subcategories to facilitateinterpretation.

Second, despite having a strong interrater agreementthroughout the samples for both classification matrices, themajority of questions were coded by a single rater; this coderwas a medical student at the time this work was done andhad less content expertise than an experienced physician. Asthe basis of coding questions is certainly under the influenceof the rater’s interpretation of the question, the results couldhave been slightly different had the questions all been codedby a physician or a medical librarian.


Bjerre et al.

TABLE 7. Most Common Organ-System Topics (ICPC-2 Classification) for the Top 10 Question Types (TGCQ)

Rank Code Example of question ICPC-2 code Description No. of questions

1 2.2.1.1 How should I treat finding/condition

y (given situation z)?

UNC Unclassified 10

S99 Skin disease, other 5

D83 Mouth, tongue, lip disease 4

K92 Atherosclerosis/PVD 4

L76 Fracture: other 4

N04 Restless leg 4

R05 Cough 4

R74 Upper respiratory infection, acute 4

R99 Respiratory disease, other 4

U95 Urinary calculus 4

2 2.1.2.1 Is drug x (or drug class x) indicated

in situation y or for condition y?

A99 General disease, NOS 9

UNC Unclassified 9

A78 Infectious disease, other/NOS 6

D70 Gastrointestinal infection 5

S70 Herpes zoster 5

T93 Lipid disorder 5

K78 Atrial fibrillation/flutter 4

L95 Osteoporosis 4

N91 Facial paralysis/Bell’s palsy 4

3 1.3.1.1 Is test x indicated in situation y? A98 Health maintenance/prevention 35

UNC Unclassified 16

T99 Endocrine/metabolism/nutritional disease other 8

A91 Abnormal result investigation NOS 6

A21 Risk factor for malignancy 5

U71 Cystitis/urinary infection other 5

4 2.1.2.2 Should this kind of patient get

prophylactic drug x to prevent

condition y?


A85 Adverse effect medical agent 4

5 4.2.1.1 Is x a risk factor for condition y? UNC Unclassified 14


A23 Risk factor NOS 7

6 2.1.3.1 Could finding y be caused by drug x? A85 Adverse effect medical agent 49

A13 Concern/fear medical treatment 12


7 1.1.3.1 What is the cause of test finding x? A91 Abnormal result investigation NOS 46

UNC Unclassified 9


(Continued)



TABLE 7. Continued

Rank Code Example of question ICPC-2 code Description No. of questions

8 3.1.1.1 How should I manage condition/ A91 Abnormal result investigation NOS 7

finding/situation y? A78 Infectious disease, other/NOS 4

A94 Perinatal morbidity other 4


A99 General disease, NOS 4

9 2.1.3.3 Is drug x safe to use in situation y? A13 Concern/fear medical treatment 16



UNC Unclassified 4

10 4.3.1.1 What is the usual course (or natural

history) of condition y?


ICPC = International Classification for Primary Care; TGCQ = taxonomy of generic clinical questions.

Third, because the data collection period did not extendover a full calendar year, the questions collected do not re-flect the full range of problems seen in primary care, manyof which are seasonal in nature. Nonetheless, although thedata were collected between October 2005 and April 2006,and this time period would not include questions on topicalevents that occurred after this period (eg, avian flu H5N1),we have assumed that the distribution of question typeswould still be quite similar today, for the same portion of theyear.

An additional shortcoming of our study is that the origi-nal JIT trial encouraged participants to formulate questionsusing the PICO format (Patient, Intervention, Comparison,Outcome). However, physicians have other CME needs, suchas general review of subjects or practicing techniques, suchas cardiopulmonary resuscitation, that they do not get to usesufficiently frequently to maintain competence. Furthermore,they also need to know about rare entities, even though ques-tions concerning such entities will be unlikely to appear inquestions asked at the point of care. Therefore, these needsare unlikely to be captured by the questions that form the ba-sis of our study.

Future research related to this project will set out to iden-tify how well this sample of point-of-care questions is inconcordance or discordance with contemporaneous CME an-nual refresher courses or more focused live conferences forfamily physicians. Because the TGCQ and the ICPC-2 aretools that were developed for other purposes, an additionalavenue for future work might be to optimize the currentlyavailable classification systems, or develop new ones, to pro-

vide better tools for identifying learning needs at the point ofcare.

Conclusion

This study has shown the feasibility of using 2 classificationschemes to categorize questions asked at the point of careby primary care clinicians using online/electronic means,with a view to using these questions to inform providers ofCME with an expression of the needs of family physicians.Questions collected at the point of care provide a valuableand unique source of information on the true learning needsof practicing clinicians. Questions concerning therapy weremost common, representing more than half of all questionsasked; of these, questions concerning drug therapy were mostfrequent, representing 70% of therapy-related questions and36% of all questions.

A librarian consultation service that has been demon-strated to be cost effective6 has the ability to simultaneouslyclassify point-of-care questions and provide summaries ofthe subject areas and types of questions physicians are pursu-ing; this would supply providers of CME with a reflection ofneeds that complements other needs assessment strategies.The JIT information librarian consultation service4 can bea means for the routine collection and coding of banks ofquestions. This system has the potential to be cost effective,current, and practice relevant and to require relatively littlemaintenance. It would address a critical component of ef-fective CME content development, which is identifying theactual learning needs of practicing clinicians.


Bjerre et al.

Lessons for Practice

• Continuing Medical Education (CME) plan-ning does not routinely rely on needs anal-ysis to identify knowledge gaps of potentiallearners. Practitioner questions collected atthe point of care are a valuable and uniquesource of information on the needs of prac-ticing clinicians.

• This study demonstrated the feasibility ofusing questions collected electronically atthe point of care to assess learning needs ofclinicians. Questions on therapy were mostcommon, particularly questions concerningdrug therapy. The Just-in-Time (JIT) infor-mation librarian consultation service couldbe used routinely to provide needs assess-ments to guide CME planning.

Supporting Information

Additional supporting information may be found in the on-line version of this article:

APPENDIX S1: Cross-Tabulation of All 64 TGCQ and349 ICPC-2 Codes

As a service to our authors and readers, this journal pro-vides supporting information supplied by the authors. Suchmaterials are peer reviewed and may be reorganized for on-line delivery, but are not copy edited or typeset. Technicalsupport issues arising from supporting information (otherthan missing files) should be addressed to the authors.

Acknowledgments

The authors wish to acknowledge the valuable contributionof Megan James and Chandra Landry, who provided invalu-able help with various aspects of manuscript preparation aswell as assistance with the submission process.

References

1. Nielsen JE, Lous J, Adeler HF, Olesgaard P, Maagaard RR, Ole-sen F. General practitioners’ continuing medical education: a prospec-tive study from the County of Aarhus. Scand J Prim Health Care.2002;20(4):198–200.

2. Norman GR, Shannon SI, Marrin ML. Learning in practice education.BMJ. 2004;328:999–1001.

3. Ebell MH, Cervero R, Joaquin E. Questions asked by physicians as thebasis for continuing education needs assessment. J Contin Educ HealthProf. 2011;31(1):3–14.

4. McGowan J, Hogg W, Rader T, et al. A rapid evidence-basedservice by librarians provided information to answer pri-mary care clinical questions. Health Info Libr J. 2010;27(1):11–21.

5. McGowan J, Hogg W, Campbell C, Rowan M. Just-in-time informationimproved decision-making in primary care: a randomized controlledtrial. PloS one. 2008;3(11):e3785.

6. McGowan J, Hogg W, Zhong J, Zhao X. A cost-consequences analysisof a primary care librarian question and answering service. PloS one.2012;7(3):e33837.

7. Ely JW, Osheroff JA, Gorman PN, Ebell MH, Chambliss ML, Pifer EA.Information in practice A taxonomy of generic clinical questions: clas-sification study. BMJ. 2000;321:429–432.

8. WONCA International Classification Committee. ICPC-2-R: Inter-national Classification of Primary Care. 2nd ed. Oxford, England:Oxford University Press; 2005. See http://qicpd.racgp.org.au/media/57417/icpc-codes.pdf

9. Gebel R, Okkes I. ICPC-2-NL, International Classification of Pri-mary Care. 2nd ed. Utrecht: Nederlands Huisartsen Genootschap;2000.

10. Botsis T, Bassøe C-F, Hartvigsen G. Sixteen years of ICPC use inNorwegian primary care: looking through the facts. BMC Med InformDecis. 2010;10(11):1–10.

11. Boot CRL, Meijman FJ. Classifying health questions asked by the pub-lic using the ICPC-2 classification and a taxonomy of generic clinicalquestions: an empirical exploration of the feasibility. Health Comm.2010;25(2):175–181.

12. Cohen J. A coefficient of agreement for nominal scales. Educ PsycholMeas. 1960 ;20(1):37–46.

13. Hallgren KA. Computing inter-rater reliability for observationaldata: an overview and tutorial. Tutorials Quant Methods Psych.2012;8(1):23–34.

14. Mazmanian PE, Davis DA, Page P. Guide to the evidence. JAMA.2002;288(9):1057–1060.

15. Klein D, Staples J, Pittman C, Stepanko C. Using electronic clini-cal practice audits as needs assessment to produce effective contin-uing medical education programming. Med Teach. 2012;34(2):151–154.

16. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE,Perrier L. Accuracy of physician self-assessment compared with ob-served measures of competence: a systematic review. JAMA. 2006;296(9):1094–1102.

17. Katerndahl DA, Wood R, Jaen CR. A method for estimating rela-tive complexity of ambulatory care. Ann Fam Med. 2010;8(4):341–347.

18. Cooke G, Valenti L, Glasziou P, Britt H. Common general practice pre-sentations and publication frequency. Aust Fam Physician. 2013;42(1–2):65–68.

19. Moth G, Olesen F, Vedsted P. Reasons for encounter and disease pat-terns in Danish primary care: changes over 16 years. Scand J PrimHealth Care. 2012;30(2):70–75.


What Do Primary Care Practitioners Want to Know? A Content Analysis of Questions Asked at the Point...

Documents

Transcript of What Do Primary Care Practitioners Want to Know? A Content Analysis of Questions Asked at the Point...