1

Literature Review: Evaluating Library Services Using the Critical Incident Technique

Janet Marsden

August 2010

The critical incident technique (CIT) is a set of procedures used to identify and analyze

critical incidents from a behavioral-cognitive perspective. The technique was developed during

WWII by Col. John C. Flanagan to better understand effective and ineffective behaviors for

aviation training [17]. The technique is essentially a semi-structured interview or questionnaire

which asks the participant to describe a critical incident in detail, defined by Flanagan as “any

observable human activity that is sufficiently complete in itself to permit inferences and

predictions to be made about the person performing the act” [17, p. 237]. The participant may be

an actor in the incident or an observer. In contrast to other research methods that may examine

normal practices and procedures, CIT focuses on specific incidents which are unusual or outside

of the norm to quickly identify problems or opportunities. According to Butterfield et al “The

CIT started out as a task analysis tool and, although it is still used as such within industrial and

organizational psychology, it has expanded its use in counseling psychology, nursing, education,

medicine, and elsewhere to also become an investigative and exploratory tool” [8, p.489]. Since

Flanagan’s original publication in Psychology Bulletin, his article has been cited more frequently

by organizational and industrial psychologist than any other [8]. CIT has become a standard

technique in counseling psychology, and over five decades of use the technique has expanded to

measure and improve job performance and professional practice in a wide range of disciplines,

including healthcare, education and business [1, 8, 10, 28]. It has proven to be a robust and

reliable research method. In the more than 50 years since its development, the technique has

been applied hundreds of times in dozens of fields, with relatively few and minor changes to

Flanagan’s original procedure [8]. In addition to its original application for task analysis and

2

training improvement, today it is also recognized as an effective exploratory and investigative

tool [8, p.475].

CIT has also been used in studies of information seeking behavior [3, 4, 6, 7, 11, 14] and

particularly for research on services, including information services [2, 5, 29]. The use of CIT for

investigating library services appears to begin in the 1990’s with Radford’s study of academic

reference interactions [32]. Radford’s findings in her initial study include a lack of user-centered

approaches in earlier investigations and an ongoing gap between research and practice.

According to Radford, use of CIT allowed participants to describe the specific events that they

considered most significant and meaningful for characterizing their interactions with reference

librarians, thereby linking research results to user-focused recommendations for improving

practice. Similar studies using CIT to investigate educational performance and improvement

from the points of view of teachers and students separately and jointly have been conducted [12,

13, 18, 19, 20, 21]. Subsequent to Radford’s 1996 study, she and others have used CIT to further

investigate the role of information and digital literacies for improving library and information

services [9, 27, 31, 33, 34, 36, 37, 38, 39, 40] and to characterize links between library and

information services and academic performance [22, 24, 25, 26, 35, 44, 46].

Todd and Kuhlthau’s large-scale 2002-2003 study of how students benefit from school

libraries, involving over 13,000 students and more than 800 teachers in 39 Ohio school libraries,

suggests a strong correlation between effective school libraries and student learning and

achievement [46, 47]. Gilstrap and Dupree conducted a mixed methods investigation of

information literacy competency in which they used the Brookfield Critical Incident

Questionnaire to produce both qualitative and quantitative results. The qualitative data was

analyzed to develop constructs which were then assessed statistically (using regression analysis).

3

Other investigations into the value of library services have contributed to combined qualitative

and quantitative (mixed method) approaches. Marshall’s 1991 investigation (the so-called

Rochester study) is a seminal study on the linkage between health care delivery and library

services [16, 28, 9]. Marshall’s research findings of a direct correlation between hospital library

information services and patient care outcomes in hospitals has led to studies of the value of

library services in other contexts [16, 28], such as in K-12 [41, 42, 43, 45,47] and secondary

education [48]. Although Marshall used a questionnaire to survey physicians and utilized

statistical analysis to develop her results, rather than CIT per se, her survey included open-ended

questions and subjective accounts of outcomes from participants. In a reflective article in 2007,

Marshall stated that “A multiple methods approach that produces both quantitative and

qualitative data is needed to fully address value and impact research questions” (29, p.10).

Butterfield et al, in their 2005 retrospective of the CIT, point out that although the CIT is a

qualitative method, it is firmly rooted in empirical research methodology. Butterfield et al trace

CIT’s evolution as a research method from these roots to its current post-modern incarnation.

Furthermore, they ascribe its wide usage and methodological advancement to its inherent

flexibility and credibility; suggesting that Flanagan and later practitioners have demonstrated that

the method’s inherently robust processes can successfully address questions of validity,

generalizability and reliability [29].

4

Annotated bibliography: critical incident technique

http://en.wikipedia.org/wiki/Critical_Incident_Technique

~1000 articles retrieved using the following search criteria: include the term “critical

incident”; include at least one of the following: student, library, librarian, school; exclude:

military, police, firefighter, safety, health, nurse, nursing, medical, healthcare; within the

date range: 2000 to 2010.

Plus author searches: Ross Todd, Marjorie DeVault, Rebecca Reynolds, Marie Radford,

Keith Lance and Becky Russell, Joanne Marshall.

Reviewed titles of all, reviewed abstracts of ~400, selected [46] articles below.

1. Angelides. P. “Using critical incidents to understand school cultures”, Improving

Schools, 4:1, 2001.

This paper addresses the need to find new ways of examining workplace cultures quickly and effectively, in order to facilitate school improvement. Emphasizing the importance of school culture, it illustrates how critical incidents can be analyzed so as to help teachers to understand the factors which shape their practice. It is argued that the proposed method has the potential to go beyond systems of external monitoring, in such a way as to enable schools to develop procedures for self-review.

2. Armstrong, C., Fenton, R., Lonsdale, R., Stoker, D., Thomas, R. and Urquhart, C. “A study of the use of electronic information systems by higher education students in the UK”, Program, 35:3, 2001.

This paper reports findings from the first annual cycle of a three-year research project on the provision and use of electronic information systems (EIS) within higher education in the UK. The project, JISC User Surveys: Trends in Electronic Information Services (JUSTEIS), was funded by the Joint Information Systems Committee (JISC) and undertaken at the University of Wales, Aberystwyth (UWA). Students, academics and library staff in 25 universities were surveyed using critical incident and critical success factors methodologies to ascertain the range and nature of EIS use. Provision of these systems by higher education institutions was also investigated via an analysis of their library websites. The findings reported in this paper focus on student use and the purposes for which EIS are employed, and reveal the limited array of EIS used and the ad hoc nature of search strategies adopted across undergraduate and postgraduate bodies within a range of disciplines. There appears to be little or no variation in the pattern of EIS use by the various student groups studied – the effect of the Internet on information seeking by students is hugely significant and the more formal resources, such as JISC-negotiated resources are little used. There is little evidence of coherent search strategies used by students. Recommendations for both the JISC and higher education are offered.

3. Arnone, M. and Reynolds, R. “Empirical Support for the Integration of Dispositions in Action and Multiple Literacies into AASL’s Standards for the 21st Century Learner”, American Library Association, 2009.

AASL’s Standards for the 21st Century Learner are based on a number of common beliefs, including the importance of reading and technology skills, and the acknowledgement that the concept of information literacy has indeed become more complex since the last century. The data provided in this article support the importance of considering the dispositions-in-action component of the new standards when planning

5

instruction. Specifically, this article investigates the contributions of perceived competence in information and digital literacy skills, perceived competence in reading, the disposition to read for enjoyment, and the disposition of curiosity, towards actual performance in an information and digital literacy skills knowledge test. Study participants included more than twelve hundred eighth grade students from twenty states. The study is grounded in historical literature on the construct of curiosity and on self-determination theory. Implications for curriculum design are discussed. The research was supported by a National Leadership Grant from the Institute of Museum and Library Services.

4. Becker, N. “Google in perspective: understanding and enhancing student search

skills”, New Review of Academic Librarianship, 9:1, 2003 This paper describes a study of undergraduate student search behavior, examines the results through the lens of various disciplines, and provides new insights that will increase our understanding and facilitate the development of more effective instructional programmes. Perspectives and research results drawn from multiple disciplines are used to explore the role of mental models, reference groups and habits, and intellectual development in the search behaviour demonstrated by the undergraduate students. During interviews conducted as part of the study, many students were able to articulate the importance of source evaluation and describe electronically-appropriate methods for assessing the authority and reliability of Webbased information resources. In practice, however, these students frequently abandoned source evaluation altogether and, following the path of least resistance, relied exclusively on basic Google searching. This approach both compromised the quality of their search results and contributed to frustration with the research process. This may not be extraordinarily unusual behaviour, but it is cause for considerable concern among information literacy programme planners and instructors. Discussion of the study results and related research is followed by pragmatic suggestions for modifying ineffective search behaviour through enhanced instructional programmes.

5. Bishop, A., Neumann, L., Star, S., Merkel, C.,Ignacio, E. and Sandusky, R. “Digital Libraries: Situating Use in Changing Information Infrastructure”. Journal of the American Society for Information Science. 51:4, 2000.

How users meet infrastructure is a key practical, methodological challenge for digital library design. This article presents research conducted by the Social Science Team of the federally funded Digital Libraries Initiative (DLI) project at the University of Illinois. Data were collected from potential and actual users of the DLI testbed— containing the full text of journal articles—through focus groups, interviews and observations, usability testing, user registration and transaction logging, and user surveys. Basic results on nature and extent of testbed use are presented, followed by a discussion of three analytical foci relating to digital library use as a process of assemblage: document disaggregation and reaggregation; information convergence; and the manner in which users confront new genres and technical barriers in information systems. The article also highlights several important methodological and conceptual issues that frame research on social aspects of digital library use.

6. Bowler, L. “The Metacognitive Knowledge of Adolescent Students During the Information Search Process”. Proceedings of the 36th Annual Conference of the Canadian Association for Information Science (CAIS), 2008

The study used naturalistic methods to investigate the metacognitive knowledge of 10 adolescents as they searched for, selected, evaluated, and used information for a school-based, inquiry project. The study identified thirteen attributes of metacognitive knowledge related to the information search process: Knowing your strengths and weaknesses, knowing that you don’t know, building a base, scaffolding, communicating, changing course, balancing, understanding curiosity, understanding time and effort, understanding memory, pulling back and reflecting, connecting, and parallel thinking. The results contribute to the understanding of adolescent information seeking behavior and have implications for information literacy instruction.

6

7. Bowler, L.”The Self-Regulation of Curiosity and Interest During the Information Search Process of Adolescent Students”. Journal of the American Society for Information Science and Technology, 61:7, 2010.

In a world of increasing information and communications possibilities, the difficulty for users of information systems and services may not lie in finding information but in filtering and integrating it into a cohesive whole. To do this, information seekers must know when and how to effectively use cognitive strategies to regulate their own thinking, motivation, and actions. Sometimes this is difficult when the topic is interesting and one is driven to explore it in great depth. This article reports on a qualitative study that, in the course of exploring the thinking and emotions of 10 adolescents during the information search process, uncovered patterns of behavior that are related to curiosity and interest. The larger purpose of the study was to investigate the metacognitive knowledge of adolescents, ages 16–18, as they searched for, selected, and used information to complete a schoolbased information task. The study found that the curiosity experienced by adolescents during the search process was accompanied by feelings of both pleasure and pain and that both feelings needed to be managed in order to navigate a pathway through the search process. The self-regulation of curiosity and interest was a clear and distinct metacognitive strategy fueled by metacognitive knowledge related to understanding one’s own curiosity and the emotions attached to it.

8. Butterfield, L., Borgen, W., Amundson, N. and Maglio, A. “Fifty years of the critical

incident technique: 1954-2004 and beyond”. Qualitative Research, 5:475, 2005.

It has now been 50 years since Flanagan (1954) published his classic article on the critical incident technique (CIT) – a qualitative research method that is still widely used today. This article reviews the origin and evolution of the CIT during the past 50 years, discusses CIT’s place within the qualitative research tradition, examines the robustness of the method, and offers some recommendations for using the CIT as we look forward to its next 50 years of use. The focus of this article is primarily on the use of the CIT in counselling psychology, although other disciplines are touched upon.

9. Chin, K., Pun, K., Leung, W. and Lau, H. “A quality function deployment approach for improving technical library and information services: a case study”, Library Management, 22:4/5, 2001.

Quality function deployment (QFD) stresses customer-driven planning, continuous improvement and people participation. This paper elaborates a six-stage QFD approach based on an empirical study of the provision of the library and information services (LIS) in a technical organisation in Hong Kong. Customer requirements are identified through the focus-group brainstorming and critical-incident interviews with users. The study identified 45 critical incidents, and translated them into 23 satisfaction items. Two iterations of house of quality (HOQ) were finally constructed for meshing the quality improvement efforts. The paper identifies the key quality dimensions and illustrates how the QFD approach can lead to effective quality deployment in the provision of LIS. The approach provides a practical means that helps organisations to identify customer satisfaction criteria, and integrate improvement strategies and plans into management decision-making processes.

10. Cifuentes, L. and Ozel, S. “Resources For Attending to the Needs of Multicultural Learners” Knowledge Quest, 35:2, 2006.

School librarians know they must vary instruction to meet the needs of all learners. As the student body in the United States has become more diverse, and educators have become more aware of culturally responsive teaching (Gay 2000), the mandate for differentiated instruction has intensified. To add to the complexity of this challenge, people with different backgrounds have different expectations and attitudes toward educational strategies (Ziguras and Rizvi 200i) and the computerbased instructional interfaces that they encounter (Marcus and Gould 2000). These expectations and attitudes affect the way students

7

learn as well as the way teachers and librarians teach. What do librarians need to know in order to design instruction that addresses cultures in respectful and responsive ways?

11. Copeland, C. “Information Seeking Behaviors of Children and Youth: Challenges and Implications for Information Literacy Instruction”. A Review of the Literature, University of South Carolina, 2007.

Research indicates that technological advances are changing the ways people are searching for, finding, and using information for personal, professional, and educational purposes (Bernier, 2007; Heath, 2007; Perrault, 2007). Human information behavior encompasses people’s information needs, information seeking behaviors, information contexts, patterns of information access, retrieval, processing, and use (Todd, 2003). An emerging area of human information behavior is the information seeking practices of youth. Research suggests the need for educators to adapt instruction and refine students’ Information Seeking skills and foster advanced information literacy (Branch, 2003; Dixon & Shenton, 2005; Kulthau, 1994, 2003; Hultgren and Limberg, 2002; Laverty, 2003; Leu and Kinzer, 2000; and McGregor, 1994).

12. Cowan, J., “Facilitating development through varieties of reflection”. The Higher Education Academy, 2003.

Teachers in higher education will harness the power of reflection in their curricula to better effect if they differentiate between purposes and hence forms of reflection, and set out to facilitate these different forms in different ways. 'Once you have learnt how to ask questions - relevant and appropriate and substantial questions - you have learnt how to learn,and no one can keep you from learning whatever you want or need to know' (Postman and Weingartner, 1971) I firmly believe that we can improve our students' learning and the development of their capabilities through structured reflection, as Hinett (2002) (Improving learning through reflection - part I and part II) has recently and thoroughly described in the ILTHE Members' Resource Area. I also hold the view that teachers are only beginning our journey of discovery to find out how to do this effectively. For that reason, I hope it may initiate useful discussion to set out a summary of my current thinking and practice. I make no assertions which are not in accordance with my recent and often replicated experience. I reference where I can, but have not been inhibited to state firmly held views when lacking evidence beyond my own experience. Finally, I write in the first person plural, not from editorial or regal arrogance, but to give proper credit to those with whom I have shared in development work and our own reflection thereon.

13. Douglas, J., McClelland, R., Moores, J., Davies, J., Sudbury, L. “Using critical

incident technique (CIT) to capture the voice of the student”. The TQM Journal, 21:4, 2009.

Purpose – The aim of this paper is to compare the use of critical incident technique (CIT) for gathering student feedback in higher education (HE) with the more traditional and commonly used questionnaire survey method. Design/methodology/approach – The investigation involved a critical evaluation of the standard self-completion, multi-question “tick-box” quantitative survey questionnaire traditionally employed to collect student feedback in HE, against the more qualitative critical incident technique that was tested within the HE context. This evaluation was supported by a review of the extant literature to determine the advantages and disadvantages of both feedback methods and a comparison of the data gathered from university students using both survey instruments. Conclusions were then drawn regarding the value of both methods. The criteria used for the comparison were the design and administration of the survey instruments, analysis and quality of the data collected, and finally, the potential usefulness of the data to HE managers. Findings – The main issue regarding suitability of approach is resource utilisation. The CIT questionnaire is much quicker and easier to design than the traditional questionnaire, asking only a small number of questions. However, completion, input and analysis of the CIT questionnaire take longer than the

8

standard tick-box questionnaire. The richness of the data more than compensates for these drawbacks. In principle, the qualitative critical incident technique should be used to complement the existing methods of gathering student feedback in order to find out what is significant to students. However, in practice, it is more likely that managers within HE will continue to use the more traditional survey questionnaire, because of the limited resources available to them. Research limitations/implications – Not only is CIT a method that can be used by researchers in the education sector nationally and internationally, to gather rich and useful data about the student experience but it may also be useful for gathering information from other stakeholders. Originality/value – The paper is the first to use CIT to gather feedback from students on their university experience. It proposes that, in order to obtain valid and reliable data on which to base service provision decisions, university management should consider using this qualitative technique in combination with more traditional quantitative methods of gathering student feedback.

14. Dresang, E. “The Information-Seeking Behavior of Youth in the Digital Environment”. Library Trends, Volume 54, Number 2, Fall 2005.

The theory of Radical Change, which is based on the digital age principles of interactivity, connectivity, and access, is suggested as a lens to reexamine existing research on youth information-seeking behavior in the digital environment. After a brief review of research meta-analyses, which often point to defi cits in youth information-seeking behavior, questions that emerge from this research are suggested. Meta-analyses of gender and information behavior studies find that some recent research disputes former conclusions. Radical Change is applied to an examination of specific facets of contemporary research in order to demonstrate how new perspectives can be gained. This analysis addresses commonalities between information-seeking behavior related to the handheld book with hypertextual qualities and digital materials, the social nature of information seeking, and emerging issues of access. It is noted that the public library as a setting for research has rarely been used, even though its less structured nature might provide insights that do not surface in schools. A look at directions for youth information-seeking behavior research in the future proposes how brain research might shed further light on behavioral observations. Conclusions note existing research and summarize some new points of view and areas for investigation.

15. Druin, A., Elizabeth Foss, E., Hutchinson, H., Golub, E. and Hatley, L. “Children’s Roles Using Keyword Search Interfaces at Home”. CHI 2010.

Children want to find information about their world, but there are barriers to finding what they seek. Young people have varying abilities to formulate multi-step queries and comprehend search results. Challenges in understanding where to type, confusion about what tools are available, and frustration with how to parse the results page all have led to a lack of perceived search success for children 7-11 years old. In this paper, we describe seven search roles children display as information seekers using Internet keyword interfaces, based on a home study of 83 children ages 7, 9, and 11. These roles are defined not only by the children’s search actions, but also by who influences their searching, their perceived success, and trends in age and gender. These roles suggest a need for new interfaces that expand the notion of keywords, scaffold results, and develop a search culture among children.

16. Dunn, K., Brewer, K., Marshall, J. and Sollenberger, J. “Measuring the value and impact of health sciences libraries: planning an update and replication of the Rochester Study”. Journal of the Medical Library Association, 97:4, October 2009.

In 2007, the National Network of Libraries of Medicine (NN/LM), Middle Atlantic Region (MAR), formed a planning group to explore the possibility of replicating a landmark study on the value of hospital libraries and their impact on clinical care, popularly known as ‘‘the Rochester Study’’ [1]. The Rochester study was among the first studies to relate information services provided by librarians to patient care outcomes, and it continues to be cited as evidence of the value of library services. The purpose of this paper is to update the library community on the progress of the proposed value of libraries study.

9

17. Flanagan, J. “The Critical Incident Technique”. Psychological Bulletin, 51:4, 1954.

During the past ten years the writer and various collaborators have been engaged in developing and utilizing a method that has been named the "critical incident technique." It is the purpose of this article to describe the development of this methodology, its fundamental principles, and its present status. In addition, the findings of a considerable number of studies making use of the critical incident technique will be briefly reviewed and certain possible further uses of the technique will be indicated.

18. Fraser, W. and Killen, R. “The perceptions of students and lecturers of some factors

influencing academic performance at two South African universities”. Perspectives in Education, 23:1, March 2005.

This article presents the results of two recent empirical investigations at universities in South Africa (University of Pretoria and University of South Africa) where an attempt was made to identify the pre- and post-enrolment factors that lecturers and students perceived as having the most important influence on students' success in their university studies. The opinions of lecturers were compared with those of various groups of undergraduate students within and across the two universities. Different genders, different years of study, different home languages, different languages of instruction, and different modes of study (contact and distance education) were considered. The investigations revealed a strong level of agreement between lecturers and students concerning most factors that were identified as being likely to contribute to students' academic success. However, there was considerable diversity in the opinions of lecturers and students concerning the factors that were identified as being likely to contribute to students' failure at university. These differences were more pronounced at the distance education institution than at the contact university. At both universities the results of the studies pointed to a number of instructional practices that seem to be limiting the opportunities for students to develop the levels of understanding and insight that lecturers expect of undergraduates. The studies also showed that some students were taking counter-productive approaches to their study.

19. Fraser, W. and Killen, R. “Factors influencing academic success or failure of first-year and senior university students: do education students and lecturers perceive things differently?” South African Journal of Education, 23:4, 2003.

South African universities are changing in several very important ways; their entry standards are changing, their programmes are focusing more specifically on the outcomes that learners are required to achieve, and their student populations are becoming more diverse. These changes are occurring in a climate of increased accountability. It is, therefore, important for universities to be concerned about the standards of their academic programmes and about the success rates of students. Together, these factors have produced a range of opinions about what needs to happen in universities if the diverse range of students is to be successful in outcomes-based programmes that maintain high standards. Some educators argue that entry standards are the most important determinants of success at university; others maintain that non-academic factors must also be considered. There is considerable evidence that the views and expectations about success held by lecturers and students are not always consistent. This article presents the results of a recent empirical investigation at the University of Pretoria that attempted to identify the post-enrolment factors that lecturers and students perceived as having important influences on students'success in their university studies. The study investigated the different expectations of first-year students, senior students and lecturers and identified numerous important similarities and inconsistencies. Most notably, there was a high and significant correlation between the rankings the three groups (lecturers, first-year students and senior students) gave to 52 factors linked to successful university study. There was also a high and significant correlation between the rankings of lecturers and senior students on 55 factors suggestive of unsuccessful university study. However, the perceptions of first-year students were not strongly correlated with either lectures or senior students. This suggests that First-year students may have unrealistic expectations about the non-academic factors that could reduce their chances of successful study. The results of the research could be used in three ways. University administrators could provide more supportive learning environments to enhance the chances that students will be successful and lecturers could also use the information to enhance the influences

10

of positive factors on student learning. The influence of negative factors could also be minimised accordingly. Finally students could also be assisted and supported to approach university studies in a way that will increase their chances of success.

20. Gilstrap, D. and Dupree, J. “Assessing Learning, Critical Reflection, and Quality Educational Outcomes: The Critical Incident Questionnaire”. College & Research Libraries, 69:5, 2008.

This research study incorporates Brookfield's Critical Incident Questionnaire (CIQ) as a qualitative instrument to assess the ACRL Information Literacy Competency Standards for Higher Education in one library's instructional curriculum. A sample (n=348) of English Composition II students was studied over the course of two semesters during a four-session instructional program. A methodological framework of critical reflection, incidents, and events was incorporated, as well as reflection on practice. Results of the study showed the CIQ was effective in supporting qualitative methods for assessment of critical reflection in general and the ACRL Standards specifically during the research and learning process.

21. Gilstrap, D. and Dupree, J. “A Regression Model of Predictor Variables on Critical Reflection in the Classroom: Integration of the Critical Incident Questionnaire and the Framework for Reflective Thinking”. The Journal of Academic Librarianship, 34:6, 2008.

This research study investigates the influence of independent variables on students' critical reflection scores in a library instruction program. A student sample (n= 321), enrolled in English Composition II courses, participated in a four-session library instruction curriculum. Brookfield's Critical Incident Questionnaire was used as the main instrument to collect critical reflection data, and the Sparks-Langer et al. Framework for Reflective Thinking was used as a scoring instrument. Multiple regression analysis was conducted on 4 major theoretical constructs (academic achievement, semester, gender, age) and 6 theoretical assumptions (year in school, previous library instruction, transfer student, librarian, day of week, and time of day). Significance was found for academic achievement (p<0.01), gender (p<0.001), semester p<0.001), and year in school (p<0.05) as predictors of critical reflection.

22. Hart, G, “The Information Literacy Education Readiness of Public Libraries in Mpumalanga Province (South Africa)”. Libri, 56, 2006. This study examines if public libraries in a province in South Africa are ready to assume an enhanced responsibility for information literacy education, specifically that of students, and, if so, what inhibiting and facilitating factors might exist. The public libraries in the rural province of Mpumalanga provide the case site. “Readiness”, at one level, refers to physical capacity and, on a second level, to more subjective attributes such as staff attitudes and beliefs. The paper reports on the first phase of the study – in which both quantitative and qualitative data were gathered by means of a questionnaire/interview survey of 57 public librarians in 46 sites. The study finds that Mpumalanga public libraries are indeed heavily engaged in serving school learners. Short comings in certain physical facilities, such as the lack of space and absence of retrieval tools, are inhibiting factors with the heritage of apartheid still impacting on the availability of and quality of service. The low level of professional education of public library staff is found to impede innovation in library programming. The prevailing information literacy education model largely comprises one-to-one support, although there is a fair amount of source-based group library orientation. Moving towards information literacy education will depend on a shift in conceptions of the educational role of public libraries. In the absence of recognition of their curricular role by public library authorities and educators, many public librarians are not sure that their services to school learners are legitimate. There is, however, dawning recognition that present approaches are not meeting the needs of school learners and that more effective communication with educators is required. This recognition comes from public librarians’ frustrating encounters with learners rather than from insight into information literacy education

11

theory and experience. The fundamental conclusion is that sustainable information literacy education in public libraries will depend on more dynamic leadership and on a vision of a new model of public library.

23. Hjørland, B. “Epistemology and the Socio-Cognitive Perspective in Information Science”. Journal of the American Society for Information Science and Technology, 53:4, 2002.

This article presents a socio-cognitive perspective in relation to information science (IS) and information retrieval (IR). The differences between traditional cognitive views and the socio-cognitive or domain-analytic view are outlined. It is claimed that, given elementary skills in computer-based retrieval, people are basically interacting with representations of subject literatures in IR. The kind of knowledge needed to interact with representations of subject literatures is discussed. It is shown how different approaches or “paradigms” in the represented literature imply different information needs and relevance criteria (which users typically cannot express very well, which is why IS cannot primarily rely on user studies). These principles are exemplified by comparing behaviorism, cognitivism, psychoanalysis, and neuroscience as approaches in psychology. The relevance criteria implicit in each position are outlined, and empirical data are provided to prove the theoretical claims. It is further shown that the most general level of relevance criteria is implied by epistemological theories. The article concludes that the fundamental problems of IS and IR are based in epistemology, which therefore becomes the most important allied field for IS.

24. Jacoby, J. and O’Brien, N. “Assessing the Impact of Reference Services Provided to

Undergraduate Students”. College & Research Libraries, July 2005. This article describes a study assessing the impact of reference services on undergraduate students. The study targeted undergraduates receiving nondirectional reference assistance, yielding sixty-nine survey responses and five follow-up interviews. Three outcomes were examined: (1) Do undergraduate students perceive the reference staff as being friendly and approachable? (2) Do they learn something during the course of the reference interaction? and (3) Do they feel more confident about their ability to find information after the reference interaction than they did before? Our findings suggest that reference services can play a significant role in helping students become confident, independent information seekers. Correlations between variables and a multiple regression model further indicate that friendliness of the reference staff was one of the best predictors of students’ confidence in their ability to find information on their own. These outcomes are particularly salient in a college and university environment where building skills for independent information exploration is a primary goal.

25. Kwon, N. “A Mixed-Methods Investigation of the Relationship between Critical Thinking and Library Anxiety among Undergraduate Students in their Information Search Process”. College & Research Libraries, March 2008

This study investigated the nature of associations between critical thinking dispositions and library anxiety among 137 undergraduate students. The study was conducted by administering standardized survey instruments and by analyzing the contents of student essays on critical incidents of their library use experience. The results of these quantitative and qualitative investigations consolidated each other by revealing negative associations between the two variables. An interactive model of critical thinking and library anxiety emerged from the investigation, showing how they interact with each other during the library use process. Both theoretical and practical implications of the findings were discussed within the frameworks of affective information behavior and information literacy, respectively.

26. Lance, K. and Russell, B. "Scientifically Based Research on School Libraries and Academic Achievement." Knowledge Quest 32, no. 5 (2004): 13-17.

12

Our current literature is filled with discussions of scientifically based research (SBR). How much of this type of research has been done to document that quality school library programs help to improve student academic achievement? Does school library research meet the exacting requirements of SBR, and, if so, to what extent? Which scientifically based requirements does existing school library impact research not meet—or not meet as well as it might? How can the arguments for quality school library programs be made stronger by doing a better job of meeting high research standards? To answer these questions one must understand SBR and understand how it is being applied.

27. Lee, S. “Vroom’s expectancy theory and the public library customer motivation model”. Library Review, 56:9, 2007.

Purpose – The puropse of this paper is to understand the components of Vroom’s expectancy theory; to create or develop a public library customer motivation model using Vroom’s expectancy theory; to suggest appropriate public library services marketing mindset which public libraries can employ to enhance customers’ perceived expectancy and instrumentality of public library services to motivate customers to use public library services more frequently based on the proposed public library customer motivation model; and to suggest appropriate public library services marketing strategies to motivate customers to use public library services more frequently based on the proposed public library customer motivation model. Design/methodology/approach – Research paper based on expectancy theory. Findings – Customer-centered mindset is the most important factor to motivate public library customers. Furthermore, the suggested marketing strategies can be also achieved through a customer-centered marketing mindset. In conclusion, public libraries should continuously focus on the recognition of customers’ needs and deliver long-term value to customers. Originality/value – There were few studies that focused on library users’ motivations for using library products and services. In addition, there was a lack of developed theory in library and information science field.

28. Marshall, J. “Measuring the value and impact of health library and information

services: past reflections, future possibilities”. Health Information & Libraries Journal, 24:s1, 2007.

Objectives: To summarize the context, history and results of research studies conducted on the value and impact of

health library and information services by the author since 1975 and to use this as a basis for examining ongoing

developments related to evaluation research. To provide a comprehensive bibliography of library value and impact

studies.

Methods: Literature review and background based on personal involvement in the studies under discussion.

Results: The author’s studies demonstrate an ongoing evolution of value and impact studies since the mid-1970s. In

health sciences libraries, the approach taken to measuring value and impact has been strongly influenced by the type

of research being conducted in the health sciences field as a whole. As a result, health sciences library researchers

have become early adopters of methods that incorporate outcome and impact measures and rigorous research

designs, and the concept of evidence-based library and information practice. The paper recommends that a range of

research approaches from various disciplines be used to guide future evaluation research.

Conclusions: Value and impact studies will continue to be important resources for evidence-based practice as health

information professionals deal with evolving user needs and new ways of delivering information to a variety of

audiences.

29. Neale, D., Dunlap, D., Isenhour, P. and Carroll, J. “Collaborative Critical Incident Development”. Proceedings of the 40th annual meeting of the Human Factors and Ergonomics Society, 2000.

We have been using critical incidents to research the design and evaluation of computer-mediated collaborative and communication technologies for use in K-12 education. A variety of methods have generated a number of critical incidents identified during classroom evaluations. To enhance our analysis

13

and understanding of these incidents, we developed a Web-based forum called the collaborative critical incident tool (CCIT). Users (teachers) and researchers collaboratively post, rate incident criticality, and elaborate on critical incidents through sustained dialog, contributing to the understanding of underlying conditions, causes, and implications. In this paper we describe the tool and its impact on evaluation. Flanagan (1954) intended critical incidents to be identified in situations where it was clear what transpired and what the effects were as a result of the event. However, the use of critical incidents to explain human error for a number of complex systems has shown that multiple errors often lead to incidents, and these errors sometimes appear to be independent of each other (Malone, 1990). In the case of distributed groupware systems, causes of critical incidents and their consequences often are not proximal to the environment where they manifest themselves. The causes leading to critical incidents in complex systems cannot be assumed to be obvious or easily accessible. The CCIT allowed critical threads to evolve that contained temporally remote causes distributed across situations of group interaction. These temporally removed and distributed causes can be understood by reconstructing events from different frames of reference (data) and by enlisting users in their interpretation. Carroll et. al. (1993) described critical threads as higher-order abstractions. Taken together, sets of causally related threads make up major usability themes. Major usability themes having interconnected issues developed out of the CCIT use.

30. Ozkaramanli, E. “Librarians' Perceptions of Quality Digital Reference Services by Means of Critical Incidents”. Unpublished dissertation, University of Pittsburgh, 2005.

This research is an effort to understand chat reference services through librarians' perceptions of successful and less successful chat reference service. Ten Academic libraries in Ohio and Pennsylvania which offered chat reference services were identified and 40 librarians were interviewed in order to address the research questions of this study. The main methodology used was the Critical Incident Technique (CIT) that is based on collecting and analyzing the most memorable experiences of human behavior in order to evaluate and identify ways to increase effectiveness of service. On-site, personal interviews were conducted with librarians who provide chat reference services. The subjects were initially asked to define chat reference service and compare it with traditional reference services. Following CIT procedures, they were then asked to recall and describe successful and less successful chat reference interactions and to make suggestions for better chat reference services. The interviews were transcribed and entered into a database for coding and content analysis of the collected data using qualitative data analysis software (MAXqda). Coded data were transformed into categories to determine and describe librarians' perceptions of chat reference services. The six major themes that emerged from this study were: 1-Characteristics of chat reference, 2-Attitudes of librarians and users, 3-Efficiency of reference interview and question negotiation, 4-Service improvement and management issues, 5-Training and review, and 6-Publicity and user awareness. These themes were discussed throughout the study. Findings from the recorded critical incidents indicate the importance of “attitudes” of librarians and users, the role of question “negotiation” and “type”, and the availability of resources in successful chat reference service. The defining characteristics of chat reference, that it is online, remotely available and delivered through software, raised issues of “technology”, “chat software”, “service location” and “service hours” which were also significant in defining service success. Furthermore, investigation of reference service quality criteria, evaluation measures and methods were explored by comparing literature on traditional and chat reference services and study findings. This study provides practical evaluation criteria for providing successful chat reference services in three categories based on: librarians' performance, chat software, and marketing issues. Further research for developing comprehensive digital reference evaluation criteria is recommended.

31. Radford, M. “Communication Theory Applied to the Reference Encounter: An Analysis of Critical Incidents”. The Library Quarterly, 66:2, 1996.

Dimensions of interpersonal communication perceived to be integral to success or failure in academic reference interactions are identified through the use of the critical incident technique. Forty-seven critical incidents recalled by twenty-seven library users and nine librarians are analyzed from the theoretical perspective of interpersonal communication developed by Paul Watzlawick, Janet Beavin, and Don Jackson. This study is the first to be published that applies the heuristically rich relational theory from the

14

communication field to library interactions. Content and relational dimensions that have an impact on librarian and user critical incidents are identified, such as attitude, relationship quality, information, knowledge base, and approachability. Interpersonal relationships are major themes in incidents recalled by library users. Librarians give relatively more weight to content dimensions dealing with quality of information given but also perceive relationship qualities to be important. The value of applying communication theory to study of the reference interaction is demonstrated. In addition, this research also illustrates the importance of incorporating the user's point of view. Evidence presented argues for a new model of the librarian user reference interaction that recognizes the vital importance of relational messages that are communicated in the reference encounters along with the information transferred. 32. Radford, M. “Encountering Users, Encountering Images: Communication Theory and

the Library Context”. Journal of Education for Library and Information Science, 42:1, Winter 2001.

This article explores the importance of applying communication theory in the library context, especially in encounters between librarians and library users. Libraries and the work of librarians and informational professionals have undergone significant and rapid change in the past twenty years with every indication that transformational changes are to continue. Among these changes are increased diversity of users, increased use and sophistication of information retrieval systems, and increased time pressures and other stressors on library staff. Librarians continually engage in interactions with users who are confused and disoriented by a complex and expanding array of electronic, as well as traditional print, resources. Two applications of interpersonal communication theory to the library and information science (LIS) field are discussed: a) research in librarian-user encounters and media images of librarians and libraries, and b) graduate education. A research program is discussed that uses the theoretical foundations provided by Watzlawick, Beavin, and Jackson; Mehrabian; and Foucault. An underlying philosophy for integration of communication theory in core courses such as Information Sources (basic reference) and Online Searching and Services, as well as the creation of an advanced course, Interpersonal Communication for Information Professionals, are discussed. Examples of the range of communication courses taught in LIS programs in North America are provided.

33. Radford, M. and Connaway, L. “Seeking Synchronicity: Evaluating Virtual Reference

Services from User, Non-User, and Librarian Perspectives”. Proposal for a research project to the National Leadership Grants for Libraries program of the Institute of Museum and Library Services (IMLS), submitted February 2005.

This document proposes a project to study and evaluate the sustainability and relevance of virtual reference services, which are human-mediated, Internet-based library information services. The goal is to study VRS users, non-users, and librarians to create a fuller understanding of their behaviors, needs, and preferences in virtual environments, in order to improve libraries' ability to respond to increased demand on libraries to provide reference services online. The project will develop a theoretical model for VRS that incorporates interpersonal and content issues and will make research-based recommendations for library staff to increase user satisfaction and attract nonusers. It will also make recommendations for VRS software development and interface design and produce a research agenda for user-centered VRS.

34. Radford, M. “The Critical Incident Technique and the Qualitative Evaluation of the Connecting Libraries and Schools Project”. Library Trends, 55:1, Summer 2006.

This article describes Flanagan’s Critical Incident Technique (CIT) for those seeking guidance in its application. Examples in the libraryand information science (LIS) field are discussed, including an in-depth example of a CIT study conducted as part of a qualitative evaluation of the Connecting Libraries and Schools Project (CLASP) in New York City. The CLASP study analyzed critical incidents from 2,416 fifth and seventh grade students regarding their perceptions of interactions with urban public librarians and library staff. For both positive and negative critical incidents, the most important factor

15

in these preadolescent’s perception of successful library visits is the attitude of the librarian or staff member they encounter.

35. Radford, M. and Connaway, L. “Screenagers” and Live Chat Reference: Living Up to the Promise”. Scan 26:1, February 2007.

Today’s 12-18 year old members of the Millennial Generation have been referred to as “screenagers” because of their affinity for electronic communication via computer, phone, television, etc. screens (see Rushkoff, 1996). These young Millennials are at home in the instant messaging and chat environment. It is well known that their communication and information-seeking behaviors are distinctly different from those of other age cohorts and radically different from those of the baby boomer generation. Libraries are providing Web-based virtual reference services (VRS) as alternatives to traditional face-to-face (FtF) reference services to meet the information needs of virtual as well as FtF library users. This paper presents the revealing results of an international study of communication and information-seeking including a series of three focus group interviews with 12-18 year olds and analysis of a random sample of 431 live chat reference transcripts drawn from an international population. Focus groups were conducted with the cooperation of public and school librarians in collaboration with public school teachers. Analysis of these focus group interviews with groups of urban, suburban, and rural screenagers reveals their communication and information-seeking preferences. These groups have revealed that they use IM for socializing and collaborative homework, yet perceive library VRS differently then these other virtual encounters; they also express a preference for FtF encounters with librarians. Implications of the findings for school librarians are discussed.

36. Radford, M., Connaway, L., Agosto, D., Cooper, L., Reuter, K. and Zhou, N. “Behaviors and Preferences of Digital Natives: Informing a Research Agenda”. Sponsored by the Special Interest Group on Information Needs, Seeking and Use and the Special Interest Group on Digital Libraries, 2008.

Prensky (2001) asserts that because of their total emersion in the digital information environment from birth “…today's students think and process information fundamentally differently from their predecessors” (np). This panel highlights ground-breaking research on information-seeking behaviors of Digital Natives, those born after 1989. Mounting evidence confirms the need for innovative approaches to understand their behaviors and their radically different use of electronic sources, services and systems. An urgently needed research agenda is proposed that draws attention to the need to identify the characteristics and preferences of this tech savvy group that surprisingly lacks basic skills in information evaluation and retrieval. Three studies that explore various aspects of the proposed research agenda are discussed below, including research in the areas of virtual reference services, selection of digital library resources, and collaborative information behavior in online environments. This panel will be an interactive discussion with a substantial audience participation component. It is hoped that the audience for this session will consist of both researchers and library practitioners in virtual environments, as well as those interested in servicing digital natives.

37. Radford, M. and Connaway, L. “Cordial Connections: Evaluating Virtual Reference from User, Non-User, and Librarian Perspectives using the Critical Incident Technique”. Institute of Museum and Library Services; Rutgers, State University of New Jersey and OCLC, 2008.

Live chat virtual reference services (VRS) have become mainstream access points for seekers of online help from librarians over the past 10 years (Sloan, 2006). This paper reports results from the final phases of a multi-year grant project and compares perceptions of VRS users, non-users, and librarians by using the Critical Incident Technique (CIT) (Flanagan, 1954). Results indicate that VRS users and non-users find the following to be important to perceptions of successful encounters: accuracy of answers/information, in addition to the librarian demonstrating knowledge of sources and systems, a

16

positive attitude, and good communication skills. All groups indicate that a combination of relational and content factors is critically important to perceptions of what determines successful and unsuccessful reference interactions. Users and non-users were found to highly value the attitude and personal qualities of the VRS librarians, their knowledge, as well as their ability to quickly and accurately provide requested information. Librarians greatly value the positive attitudes of users and were attuned to negative attitudes of users in recounting negative CIs. Librarians were also interested in the provision of information in positive CIs, and reported a high number of CIs in which lack of information caused an encounter to be unsuccessful.

38. Radford, M. and Connaway, L. “E-Valuating E-Reference: Transforming Digital Reference through Research and Evaluation”. Institute of Museum and Library Services Final Performance Report, 2008.

This panel presents three different approaches, methods of analysis and data collection techniques to evaluate several digital reference formats. Radford and Connaway evaluate criteria important for perceptions of success across different generations of digital reference users, non-users, and librarians. They compare Baby Boomer vs. Millennial generations through 497 in-depth online surveys. Both content delivery and interpersonal communication were found to be important to successful chat encounters. Pomerantz presents a project to evaluate the Slam the Boards effort on Yahoo! Answers (YA) which is the largest web-based community answer board. The volume of questions and answers on YA is massive with librarian-provided answers comprising a small amount. Studies of YA users are required to determine if questioners find value in having questions answered by librarians. New practices, metrics, and organizational partnerships must be developed to make future quality evaluation possible. Mon discusses the use of new technologies in digital reference education and practice, and explores information behavior at the Internet Public Library (IPL). The IPL is building an online learning community site to support teaching, learning and research in digital reference. This paper reports the result of research that informs the implementation of digital technologies within the learning community site.

39. Radford, M. and Connaway, L., Pomerantz, J., Mon, L., Janes, J. “Seeking Synchronicity: Evaluating Virtual Reference Service from User, Non-user, and Librarian Perspectives”. Institute of Museum and Library Services Final Performance Report, 2008.

“Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-user, and Librarian Perspectives” (Radford & Connaway, 2005) was undertaken with the goals of evaluating the practice, sustainability, and relevance of virtual reference services (VRS). VRS are human-mediated, Internet-based, synchronous library information services. The rapidly increasing use of remotely accessed, digital reference resources has increased the demand for librarians to provide reference services online. VRS users, non-users and librarians provided data for this research in focus group interviews, online surveys, telephone interviews, and VRS session transcripts. The project provided findings regarding user and librarian preferences, process and usability issues related to technology, and content issues related to information accuracy and query negotiation. In addition, the project results inform the understanding of computer-mediated communication, the impact of virtual relationships, and generational differences in information seeking using live chat technology.

40. Reynolds, R. and Caperton, I. “Comparison of Middle School, High School and

Community College Students’ Wiki Activity in Globaloria-West Virginia (Pilot Year-Two)”. Poster for World Wide Workshop Foundation (www.worldwideworkshop.org), 2009.

Constructionist-learning researchers have long emphasized the epistemological value of programming games for learning and cognition. This study reports student experiences in a program of game design and Web 2.0 learning offered to disadvantaged West Virginia middle, high school and community college

17

students. Specifically, the poster presents findings on the extent of student use of the Wiki for project management, teamwork and selfpresentation of game design attributes, comparing results across 13 school pilot locations. Also presented are students’ self reported recommendations for possible improvements to the wiki. Results indicate that some locations were more active in their wiki use; the poster addresses location-specific implementation context factors that may have played a role in the variant results.

41. Shenton, A. and Dixon, P. “The nature of information needs and strategies for their investigation in youngsters”. Library & Information Science Research, 26, 2004. Little research has investigated young people’s information needs. This is despite the fact that such work can play a key role in informing the development of information services for this group. There appear to be two main barriers to investigators. The first problem lies in defining the nature of information needs in such a way that they can be researched. Second, in the absence of universal agreement, the investigator must develop a strategy for exploring information needs that is trustworthy and appropriate to the overall project aims. This article considers the ways in which past researchers have attempted to clarify the term ‘‘information needs’’ and examines the different techniques that have been employed for their scrutiny. The latter have included measurements of youngsters’ use of library materials and analysis of their reference questions in libraries, as well as interviews with both young people themselves and adults professionally involved in their welfare.

42. Shenton, A. “Use of school resource centre-based computers in leisure time by teenage pupils”. Journal of Librarianship and Information Science 40:2, 2008.

Little research has investigated young people’s use of school computers outside lessons, a deficiency that prompted the writing of this paper. The ICT related behaviour of pupils within the Resource Centre of an English high school is scrutinized, with data collected through observation and online questionnaire. The computers, which were especially popular before and after school, were heavily employed for private e-mailing, and their exploitation for personal purposes was greater than that for school work. E-mailing was particularly common among girls, whilst many boys spent much time playing games. Game playing had plummeted by Sixth Form level, however, and academic use was more prevalent among this age group.

43. Tice, M. “Queens Borough Public Library and the Connecting Libraries and Schools Project”. Journal of Youth Services in Libraries, 14:3, Spring 2001.

Acquiring literacy is a continuing problem in our society today. While schools assume the major responsibility for teaching reading and information literacy skills to children, meeting the literacy challenge requires the cooperation of parents, caregivers, teachers, community groups, and librarians. In New York City, the three public library systems-New York Public Library, Brooklyn Public Library, and Queens Borough Public Library-cooperate with the elementary and intermediate schools through the Connecting Libraries and Schools Project (CLASP). By encouraging greater community use of the public library, CLASP and its coordinated program of activities make family literacy a reality in New York City. CLASP introduces students, parents, and teachers to opportunities for lifelong learning provided by New York City's three library systems.

44. Todd, R. “ Adolescents of the Information Age: Patterns of Information Seeking and Use, and Implications for Information Professionals”. School Libraries Worldwide 9:2, 2003.

This article provides an overview of the field of human information behavior as it shapes and affects the provision of quality information services and products to children and adolescents. It is a diverse, dynamic, and complex field and one shaped by many situational, personal, social, and organizational factors. This review sets the theme for this issue's focus on adolescents' information seeking and use. It

18

briefly explores some of the key themes, theories, and challenges and explores how these shape the professional responsibilities and actions of school librarians.

45. Todd, R. and Kuhlthau, C. “Student Learning Through Ohio School Libraries: Background, Methodology and Report of Findings”. OELMA (http:/www.oelma.org/studentlearning.htm), 2004.

This study of 39 effective school libraries across Ohio involved 13,123 students in Grades 3 to 12, and 879 faculty. The research study sought to understand how students benefit from school libraries through elaborating “conceptions of help”, and providing some measure of the extent of these “helps”, as perceived by students and faculty. The study portrays effective school libraries across Ohio as dynamic agents of student learning and student achievement.

46. Todd, R. and Kuhlthau, C. “Listen to the Voices: Ohio Students Tell Their Stories of School Libraries”. Knowledge Quest 33:4, March/April 2005.

Only a child, hand probably clutching an iPod, ears plugged with miniature headphones and jiving to the thump, thump, thump of some contemporary composition, would walk into a school library and say, “Yeah, the school library rocks.” Yet this was only one of the thousands of candid comments collected from 13,123 students from grades three through twelve who participated in the Student Learning through Ohio School Libraries research study from October 2002 through December 2003. Amid the quantitative data about how helpful students perceive their school library is to them, these collective student voices represents a powerful, compelling, challenging, and, at times, confronting picture of the role and value of school libraries and school librarians in the experiences of school students. The purpose of this article is to give school librarians and educators an opportunity to hear some of these stories, reflect on their substance, and commit to actions that enable us to provide the best learning opportunities through school libraries. A full report of this study is available at the Ohio Educational Library Media Association (OELMA) Web site.

47. Urquhart, C., Light, A., Thomas, R., Barker, A., Yeoman, A., Cooper, J., Armstrong, C., Fenton, R., Lonsdale, R, and Spink, S. “Critical incident technique and explicitation interviewing in studies of information behavior” Library & Information Science Research 25, 2003.

This article discusses two related techniques, critical incident technique (CIT) and explicitation, which are used in a variety of social science research settings, and critically reviews their application to studies of information behavior. The current application of both techniques is compared with Flanagan’s early guidelines on the CIT and is discussed in relation to recent experience in the use of (1) the CIT in the JUSTEIS and VIVOS projects and (2) explicitation in projects concerned with text entering on interactive Web sites. The JUSTEIS project identifies trends, and reasons for those trends, in the uptake and use of electronic information services in higher education in the United Kingdom;this article examines experience gained over the first two cycles—1999 to 2000 and 2000 to 2001. The VIVOS project evaluated virtual health library services. Comparison of the experiences gained on the various projects suggests that critical incident methods could usefully be extended and enriched by some explicitation methods, to elicit the degree of evocation required for current and future studies of Internet use.