Teaching the use of complex IT in specific domains: Developing, assessing and refining a curriculum...

Education and Information Technologies 7:2, 137–154, 2002. 2002 Kluwer Academic Publishers. Manufactured in The Netherlands.

Teaching the Use of Complex IT in SpecificDomains: Developing, Assessing and Refininga Curriculum Development Framework

JOHN P. DOUGHERTY ∗Department of Computer Science, Haverford College, Haverford, PA 19041, USAE-mail: [email protected]

NED F. KOCKDepartment of Management Information Systems, Temple University, Philadelphia, PA 19122, USAE-mail: [email protected]

CHERYL SANDAS and ROBERT M. AIKENDepartment of Computer and Information Sciences, Temple University, Philadelphia, PA 19122, USAE-mail: sandas;[email protected]

Abstract

Information technology holds the promise of increased productivity. However, rapidly evolving tools require aprofessional able to incorporate these tools into their careers effectively, which signals the need for IT curriculumdevelopment initiatives that incorporate the use of complex, domain-specific IT applications in specific profes-sional fields. This paper reports on a study that addresses this need, by developing, assessing and refining acurriculum development framework. The Information Technology Fluency (ITF) framework is a methodologyfor constructing components (case studies) for inclusion into existing or newly proposed courses to help studentsdevelop the skills needed for this challenge. Results obtained using the framework are reported, compared tosimilar work at a different institution, and used to suggest improvements to the framework.

Keywords: information systems, information technology education, lifelong learning, fluency with informationtechnology, FITness

Introduction

Information is a key component at the onset of the 21st century. Tools used to processinformation, collectively referred to as information technology (IT), are essential. Theemergence of IT has been so rapid that many non-technical professionals are reluctant toincorporate IT into their activities. Those individuals who do incorporate IT often believethat they are underutilizing or misapplying these tools. Still others are simply uncomfort-able with IT, having been denied any reasonable opportunity to learn how to utilize ITeffectively (CSTB, 1999).

∗ Corresponding author.

138 DOUGHERTY ET AL.

The above scenario signals the need for IT curriculum development initiatives that incor-porate the use of complex, domain-specific IT applications in specific professional fields(Aiken et al., 2000). This paper reports on a study that attempts to fill this gap, by de-veloping, assessing and refining a curriculum development framework. The framework isaimed at the development of curricula that provide a strong foundation in IT that is em-bedded in hands-on-use of the technology in different complex, research and information-rich environments for students who are not information systems or computer science ma-jors.

The Information Technology Fluency (ITF) framework is a set of guidelines for the de-velopment of course components that provides an IT learning experience within an existingor newly designed course; computing may or may not necessarily be the primary subject.A set of non-trivial case studies acts to immerse students into projects where various skillsand concepts related to IT can be developed. The goal of this paper is to develop, empiri-cally assess, and refine the ITF framework. The case studies involve complex IT usage in aspecific set of problem domains, yet assume that students only possess basic IT knowledgea priori.

This paper is organized as follows. The “Research Background” section reviews theo-ries and previous course development efforts that address IT education issues and that arerelevant for the design of the ITF framework. This is followed by “The ITF Framework”section, which outlines development guidelines for the different components. The sectionoutlines the procedures involved in case study design using the framework, showing whichconclusions from previous course development efforts and theoretical principles should beimplemented, and why. The next section, “Evaluating Implementations of the ITF Frame-work”, discusses three implementations of the ITF framework in a large university settingand compares them to relevant work at a small liberal arts college. The implementationsinvolved the development of case study modules about the use of complex and specializedIT in the fields of anthropology, chemistry, and sociology. The three implementations areevaluated from quantitative as well as qualitative perspectives. The section “Refining theITF Framework” follows, providing a discussion of the quantitative and qualitative analy-ses performed in the previous section vis-à-vis our previous discussion of former coursedevelopment efforts and theoretical considerations. These analyses are used as a basis togenerate a refined version of the ITF framework. Finally, the “Conclusion” section identi-fies future research opportunities and challenges.

Research Background

There have been many efforts to identify the skills necessary to utilize IT successfully,as well as ways to develop these skills. There are also a number of theoretical modelsdescribing how IT competence is realized. This section covers related theoretical workin IT learning and in IT fluency, as well as surveys recent work to design and implementcurricular materials and courses to promote IT fluency.

TEACHING THE USE OF COMPLEX IT IN SPECIFIC DOMAINS 139

Relevant theoretical work

Several theories have been developed, particularly since the 1970s, to explain, predictand help design IT learning materials, courses and curricula, as well as use IT to teachother subjects. Conversation Theory proposes that IT learning occurs through discussionsabout the IT used in a specific context – these conversations serve to make knowledgeexplicit (Pask, 1975). Symbol Systems Theory argues that different communication mediainvolve distinct levels of mental processing when used to convey subject matter content(Salomon, 1979, 1981; Salomon et al., 1991). GOMS Theory (goals, operators, methods,selection rules) looks at the cognitive skills required for human-computer interaction (Cardet al., 1983). Programming-Facilitated Learning Theory argues that children need to oper-ationalize concepts through writing computer programs in order to retain an understandingof those concepts in a variety of domains (Papert, 1980, 1993). Cognitive Flexibility The-ory is primarily concerned with the transfer of IT concepts and skills beyond the initiallearning situation – the theory concentrates on the very nature of learning in complex andpoorly-structured domains (Spiro and Jehng, 1990; Spiro et al., 1992).

Two remaining theories are reviewed separately and in more detail due to their relevanceto the work discussed in this paper. Minimalist Theory places particular emphasis on thedesign of IT instruction materials (Carroll, 1990, 1998; Van Der Meij and Carroll, 1995).The theory proposes five main principles for IT education: meaningfulness, application,improvisation, recovery, and realism. The principle of meaningfulness implies that all ITlearning tasks should be self-contained and significant to the learners. The principle ofapplication proposes that learners should be given realistic IT projects as soon as possiblein their learning curve. The principle of improvisation implies that IT instruction shouldpermit self-directed reasoning and extemporizing. The principle of recovery states that ITtraining materials and activities should allow for error recognition and the chance to ad-dress those errors. Finally, the principle of realism implies that there should be a closelinkage between the training and the actual use of the system. The ITF framework im-plemented the principles of application and realism from minimalist theory; the principlesof improvisation and recovery can be incorporated into the resulting ITF modules givenenough lab time, availability of expertise to answer student questions, and other support.Meaningfulness is difficult to achieve because no assumptions can be made about the back-grounds or interests of the students a priori due to the diversity of the student population;the only common characteristic is that they are not computing majors.

Andragogy Theory emphasizes adult learning of IT (Knowles, 1975, 1984a, 1984b).The theory assumes that adults are self-directed and are expected to take a proactive rolein, and responsibility for, IT learning decisions. Andragogy theory advocates that IT in-struction for adults needs to focus more on process and less on content. Such approachesas case studies, role-playing, simulations, and self-evaluation are useful in achieving thisgoal. Andragogy theory argues that instructors should adopt the role of facilitators ratherthan lecturers or graders. The theory can be summarized as four main principles: de-sign involvement, experiential learning, immediate relevance, and problem solving. Theprinciple of design involvement proposes that adults need to be involved in the planningand evaluation of their IT instruction. The principle of experiential learning implies that


practical experience, including mistakes, should provide the basis for IT learning activi-ties. The principle of immediate relevance states that adults are most interested in, andthus learn more efficiently, IT subjects that have immediate relevance to their professionalor personal life. The principle of problem solving implies that effective adult IT learn-ing is problem-centered rather than content-oriented. Experiential learning and problemsolving are andragogy principles that are used explicitly in the ITF framework. Design in-volvement can be included provided the students are quite motivated and time permits thefacilitator opportunities to discover student interests. ITF framework modules developedfor this report have not explored this principle. Immediate relevance is a principle diffi-cult to include for the same reasons as the principle of meaningfulness from minimalisttheory.

Fluency with IT

The National Research Council (NRC) convened a committee in response to the increasingimportance and ubiquity of IT in daily life. The report of this group (CSTB, 1999) articu-lated a set of skills, concepts and capabilities that all citizens should know and understandabout IT to empower them throughout the information age. While skills will need to beupdated periodically, concepts and capabilities are timeless.

The term “fluency” was introduced to denote a higher level of understanding than theterm “literacy” which has been used in the past. A person capable of evaluating, distin-guishing, learning and using new IT is characterized as “fluent with IT”, or FIT. “FITness”is a measure of the degree of IT fluency, supporting the notion that fluency is developedgradually.

The report was explicit about the role of IT fluency in terms of life long learning:

In short, FITness should not be regarded as an end state that is independent of do-main, but rather as something that develops over a lifetime in particular domains ofinterest and that has a different character and tone depending on which domains areinvolved. Accordingly, the pedagogic goal is to provide students with a sufficientlycomplete foundation of the three types of knowledge that they can “learn the rest ofit” on their own as the need arises throughout life. (CSTB, 1999, p. 3)

The report also suggests a project-based approach to develop IT fluency. This approachchallenges the student to coordinate information and skills, and to make decisions thatconsider multiple dimensions of a specific problem. The scope and scale of the projectneed to be sufficiently complex to make intellectual integration necessary for completion;however, the project must be accessible to the target population.

The NRC report has started a debate about the distinction between IT literacy and flu-ency, and has opened the door to non-computing professionals. However, weaknesses ofthe report have been noted, including: levels of competence are defined by concepts ratherthan by standards of action; a lack of discussion regarding different levels of competence


for different skills/concepts/capabilities; and a report panel that consisted of computer sci-ence academics and researchers only (Denning, 2000).

The ITF Framework

As noted in the introduction, many students approach IT as a “necessary evil”, or at leastas a required tool that is difficult to utilize. The ITF framework is targeted at these per-sons, designed to provide an effective means of introducing IT without overwhelming thestudent. A balance is sought to provide a relatively gentle introduction to IT concepts, yetpermit active and discovery types of learning. This section will outline the ITF framework,discuss the type of learning environments where the framework would be appropriate, andfinally describe the three case studies implemented using the framework.

The process of case study creation begins with a goal; namely, to identify the specificskills, concepts and capabilities that need to be realized in the students. There are sugges-tions presented in (CSTB, 1999). It is assumed that this initial set will change as the casestudy is further developed, but a goal is still encouraged.

The next phase involves selecting the subject area for the case study. The pervasivenessof IT in professional activity provides a rich set of topics for consideration. The primarycriteria involved are:

• a subject area where a set of IT tools is utilized;• the usage of these IT tools merits substantial thought and practice to truly challenge the

current abilities of the student, supporting the principle of meaningfulness from mini-malist theory;

• problem examples exist that are accessible to the student, as well as open questions, thelatter supporting the principle of problem solving from andragogy theory;

• the assignment facilitates learning of the capabilities, concepts and skills from the initialphase of the project as suggested by the principles of application from minimalist theoryand immediate relevance from andragogy theory;

• a well-defined closure point for the assignment exists.

Note that standard IT tools like word processors, browsers and spreadsheets are encour-aged, and integrated into the case study, but should not be the only level of IT utilized.This objective can be achieved by introducing a non-standard tool specific to the casestudy, or to use a standard tool in a non-stand fashion dictated by the case study. Forexample, a physics case study might introduce an IT tool specific to the discipline (i.e.,non-standard) to acquire data in real-time, and then use a standard spreadsheet applicationto process this acquired raw data.

The subject area is often dictated by a number of local factors to the learning situa-tion. The most important factor is typically the willingness of an expert in the area tobecome involved with the project. All of the case studies to date involve personnel fromthe subject area and others from computing. Other factors exist, including staffing issues,expected student demographics in the course, hardware and software availability, cost, andlevel of interest. It should be stated explicitly that the ITF framework does not assume


a particular type of course, but rather supports the seamless integration of IT fluency in-jected into any undergraduate course where non-computing students wish to develop ITcapabilities.

Following case study design is integration into the syllabus. Each study requires sometailoring, but some general techniques are useful. Group projects involving hands-on lab-oratory work are central to the case study. Closed labs and self-scheduled labs have beenused successfully, depending on local factors. Facilitated discussion of the case study andits implications regarding IT usage are also included as recommended by conversationtheory (Pask, 1975).

Three case studies have been designed and implemented. Each one is chosen from out-side of the field of computing and IT. The case studies are from the areas of anthropology,sociology, and chemistry, and are briefly described below.

“Modeling human behavior over time and space: Deforestation in tropical America” isa case study from anthropology. This case study examines the expansion of tropical for-est farmers and the accompanying deforestation in Central Panama during the time periodfrom 9000 to 2000 years ago through the use of simulations carried out in a GIS (Geo-graphic Information Systems) environment.

The sociology case study is entitled “Occupational and age cohort consequences of theindustrial transformation, 1980–1990”. The project examines and evaluates possible ex-planations for the shifts in occupational distribution that have occurred in the United Statesbetween 1980 and 1990. Data used in this case study are the one percent Public Use Sam-ple of the 1980 and 1990 Censuses. The principal software tool used is Excel.

The chemistry case study examines methods for correlating measured physical proper-ties of simple organic molecules with their structures. Entitled “Exploring structures oforganic molecules by computational methods”, it involves calculations performed withthe aid of a commercially available software application for molecular design calledAlchemy.

The initial three case studies were conducted in a university setting as part of a ded-icated course using the ITF framework to facilitate the learning objectives from (CSTB,1999). This course was the second in a series of computing courses for non-computingmajors. A set of six professors collaborated to conduct this course, sequentially alternat-ing bi-weekly lectures between a computing professor and a non-computing professor whoplayed the role of “case study expert”. The first week consists of two lectures that describethe purpose and flow of the course, review the syllabus and overview each component.After the first week, each professor covers his topic and try to links it to the course ingeneral and, if possible, to the material of the next professor. Other support personnel in-cluded a graduate assistant to coordinate the lectures and labs for both the students and theprofessors (Aiken et al., 2000).

The comparison course introduced IT concepts using an example from economics, andwas conducted as part of a computing survey for non-science majors at a small, liberalarts college. The project, known as “Does money buy happiness? An analysis of eco-nomic survey data”, develops models and performs a statistical analysis of raw survey datato correlate economic standing with emotional state. Raw data is downloaded from on-line economic databases and processed with a commercially available software application


called Stata. IT fluency was an explicit yet secondary goal to a breadth first treatment ofcomputing. After a lecture outlining the role of IT, a professor of economics provided asingle visiting lecture to outline the goal of the case study and to demonstrate the IT ap-plication used to achieve this goal. Students were then asked to try the tool independently.Class size and availability of lab support limited the required part of the case study. The fi-nal project for the course was extended to include an optional group project where studentswould utilize the IT tools presented to gather economic information and resolve a differenteconomic goal. The local factors of IT fluency as secondary and limited lab equipment andtime dictated the implementation of the case study.

The primary distinction between the project from the comparison course and the threecase studies involved implementation. The case studies had each expert provide two weeksof lecture and lab, whereas the comparison course could only utilize a single lecture fromthe economics expert, in conjunction with a lab demonstration and a student project. Also,the case studies involved more dedicated sessions for experimentation with the IT toolsand the problems.

Evaluating Implementations of the ITF Framework

Table 1 shows the questions (see also Appendix A: Questionnaire used for assessment)asked from students in the case studies as well as in the comparison course. The questionswere split into three main types. Questions in the range Q1–Q3 refer to attitudes towardIT prior to taking the course. Questions in the range Q4–Q7 refer to the impact of the casestudies (or comparison course) on attitudes toward IT. Questions in the range Q8–Q9 referto the impact of the case studies (or comparison course) on IT knowledge.

The questions were actually presented to the students as statements (see Appendix A) towhich the students responded by indicating their level of agreement with the statementspresented on a 5-point Likert scale ranging from 0 (Strongly disagree) to 4 (Strongly

Table 1. Questions-statements used for collection of quantitative data

Attitudes toward IT prior to taking the courseQ1. I already had a very good knowledge of Information Technology (IT) prior to taking this course.Q2. I intended to take as many IT courses as possible in college prior to taking this course.Q3. I was generally very attracted to IT issues prior to taking this course.

Case study impact on attitudes toward ITQ4. The case study made me feel like I should take more IT courses in the future, even if I don’t pursue a career

in IT.Q5. The case study made me feel like I should pursue an IT career.Q6. The case study improved my perception of IT’s potential for solving complex tasks.Q7. The case study improved my general perception of IT.

Case study impact on IT knowledgeQ8. I learned a lot from this case study about specialized IT applications.Q9. I learned a lot from this case study about IT issues in general.

Answers range: 0 = Strongly disagree; 4 = Strongly agree; Mid-scale = 2.


agree). Although questions Q4–Q9 were multiple-choice, they also included the sameopen-ended question “Why?” as follow-up. This follow-up question was added to moti-vate the students to think about their multiple-choice answers (i.e., not provide “mindless”answers) and thus reduce perception bias.

Two additional open-ended questions, Q10 and Q11, were also included. The first ofthese open-ended questions, Q10, asked them about the main positive aspects of the casestudies (or comparison course). The second open-ended question, Q11, asked the studentsabout the main negative aspects of the case studies (or comparison course). The answers tothese questions as well as to questions Q1–Q9 are analyzed below, the latter group comingfirst.

Quantitative analyses of the impact of the case studies on student perceptions (Q1–Q9)

Table 2 shows the mean answers on the quantitative scales provided for questions Q1 toQ9, as well as the significance levels (P’s) obtained for Chi-squared tests of the frequencydistributions of answers. The top part of Table 2, below the headings row, shows figuresobtained for the three case studies covering anthropology, sociology, and chemistry issues.Below, in the row indicated as “Mean”, are the means for the three cases. The bottom partof Table 2 shows the figures obtained for the comparison group, where answers Q1 to Q9referred to an entire generic non-majors computing course.

P ’s lower than 0.05 signify that the probability that the distribution of answers for eachquestion is due to chance is lower than 5%, and suggest significantly skewed distributionsof answers (or a significant degree of agreement among respondents). Shaded columnsindicate P ’s lower than 0.05 for at least two of the case studies.

Table 2. Mean answers and frequency distribution analysis (Chi-squared test)

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9

Anthropology (N = 12) 2.08 1.33 1.33 1.75 0.75 2.58 2.58 2.20 2.00

P (Chi-squared) < 0.05 0.13 0.43 0.71 < 0.05 0.13 0.13 < 0.05 0.09

Sociology (N = 18) 2.06 1.44 1.53 1.94 1.22 2.44 2.39 2.17 2.44

P (Chi-squared) 0.11 0.21 0.28 0.38 < 0.05 < 0.05 < 0.05 < 0.05 < 0.05

Chemistry (N = 14) 1.50 1.36 1.64 1.93 0.93 2.92 2.77 2.31 1.92

P (Chi-squared) 0.43 0.26 0.15 0.15 0.09 < 0.05 < 0.05 < 0.05 0.12

Mean 1.88 1.38 1.50 1.87 0.97 2.65 2.58 2.22 2.12

Comparison (N = 16) 1.56 0.25 1.44 1.81 0.38 2.50 3.06 2.00 2.81

P (Chi-squared) < 0.05 < 0.05 < 0.05 0.08 < 0.05 0.08 < 0.05 < 0.05 0.08

Range: 0 (Strongly agree)–4 (Strongly agree); Mid-scale = 2.


Column Q5 of Table 2 indicates a significant level of agreement that the case studiesdid not make the students feel that they should pursue an IT career. The same effect wasobserved in the comparison group, although the mean there (0.38) was lower than themean for the three case studies (0.97). However, it must be noted that the mean answerfor Q2 (I intended to take as many IT courses as possible in college prior to taking thiscourse) was also much lower for the comparison group (0.25) than for the three case studies(1.38).

Column Q6 of Table 2 supports the claim that students found the case studies improvedtheir perception of the potential for IT to solve complex tasks. The chemistry case studyhad the highest Likert average (2.92). The mean case study average (2.65) was higher thanthat of the comparison course (2.50). It should be noted that P was greater than 0.05 forthe comparison course, indicating less consensus among the respondents.

Column Q7 of Table 2 indicates a significant level of agreement that the case studies didimprove the students’ perceptions of IT. The same effect was observed in the comparisongroup, although to a higher degree there, which is indicated by a comparison of the meananswer to question Q7 for the comparison group (3.06) with the mean answer to the samequestion for the case studies (2.58).

Column Q8 of Table 2 indicates a significant level of agreement that the case studiesled the students to learn a lot about specialized IT applications (Q8). The same effect wasobserved in the comparison group, although to a lower degree there, which is indicated bya comparison of the mean answer to question Q8 for the comparison group (2.00) with themean answer to the same question for the case studies (2.22).

Quantitative analyses of bivariate correlations (Q1–Q9)

Table 3 shows significant correlation coefficients between answers to questions Q1 to Q9,which indicate co-variation of the perception-related variables represented by the Likert-scale-based answers to questions Q1 to Q9. The actual coefficients (shown in Appendix B)were calculated using the Pearson bivariate correlation test (the type of the test was “two-tailed”). To filter out weak effects, only correlations that met the following two criteriaare shown on Table 3: P lower than 0.01, and Pearson coefficient higher than 0.6. Allcorrelations shown are positive.

Table 3. Significant correlation coefficients (Pearson Bivariate Correlation Test)

Q4 Q5 Q6 Q7 Q8 Q9

Anthropology (N = 12) Q2, Q3Sociology (N = 18) Q2, Q3Chemistry (N = 14) Q2

Comparison (N = 16)

P < 0.01 and coeficient > 0.6; Q1–Q3: Prior atttitudes; Q4–Q7: Impact on attitudes; Q8–Q9: Impact onknowledge.


Intersections between the range Q1–Q3 and the range Q4–Q7 indicate co-variation ofmeasures of attitudes toward IT prior to taking the course and measures of case studyimpact on attitudes toward IT; these intersections would appear in any of the four columnsof Table 3 labeled Q4, Q5, Q6, and Q7. Intersections between the range Q1–Q3 and therange Q8–Q9 indicate co-variation of measures of attitudes toward IT prior to taking thecourse and measures of case study impact on IT knowledge; these intersections wouldappear in either of the final two columns of Table 3.

Table 3 shows the occurrence of intersections between the range Q1–Q3 and the rangeQ4–Q7 for the case studies, and the absence of intersections between those ranges for thecomparison group. These factors, along with the absence of significant negative correlationcoefficients between those two ranges, suggests that the case studies had overall a weakerimpact on attitudes toward IT than the comparison course. For example, the high corre-lation between Q4 and the pair Q2–Q3 for the Sociology Case Study suggests that thosestudents who already had a very good knowledge of Information Technology (IT) prior totaking the course (Q2), and who intended to take as many IT courses as possible in collegeprior to taking the course (Q3), were also the ones who thought that the case study madethem feel like they should take more IT courses in the future, even if they did not pursue acareer in IT (Q4). At the same time, the case studies seemed to solidify the negative andpositive attitudes toward IT to a higher degree than the comparison group.

Qualitative analyses of open-ended questions regarding positive and negative aspects(Q10–Q11)

Table 4 summarizes positive and negative aspects of the case studies and comparisoncourse as perceived by the students. Overall, the case studies evoked more negative coursestructure-specific comments (e.g., comments that are not related to the instructor) from thestudents than the comparison course, especially negative comments that the case studieswere not particularly relevant to the students. On the other hand, the comparison courseevoked negative comments related to it not being challenging enough and “scattered” in itscoverage of IT topics.

Table 4 suggests that the case studies had a positive impact on students’ perceptionsabout IT’s potential for supporting specialized tasks (PA1, PA2, PS1, PS2, PC1), and thatthe case studies evoked negative student perceptions regarding the relevance of this type ofknowledge for their chosen major or career, particularly in the anthropology and chemistrycases (NA1, NA3, NC1, NC2). In these two cases, difficulties with using features of thecomputer systems (NC3, NC4) and their malfunction (NA6, NC3, NC5) were also men-tioned as negative aspects, even though students also reported enjoying working with thesystems (PA1, PC3, PC4). While in the sociology case the computer system was seen assimple and easy to use (PS4), and no malfunctions were reported, the case itself was seen asusing complex and “boring” concepts and methods (NS1, NS2, NS3, NS4, NS5). The com-parison course was perceived as fun and easy (PO1, PO2), as well as helpful in that it taughtthe students what they perceived as important practical skills. However, the topic coverageof the comparison course was seen as scattered (NO1) and not challenging enough (NO2).


Table 4. Main aspects of the case studies and comparison course

POSITIVE NEGATIVE

Anthropology AnthropologyPA1: New computer software and new info. about maps NA1: Not relevant to career/major

development NA2: Material itself was very dryPA2: Saw how computers can be used to trace human NA3: I am just not interested in this

history NA4: Doesn’t seem to be applicable to other fields ofPA3: Well organized lectures and labs study

NA5: Needs to be generalized, show how applied to awide range of fields

NA6: Computer glitches that need to be worked out

Sociology SociologyPS1: Illustrated IT and gave me an idea of what IT can NS1: Did not sound interesting at all

do and IT’s potential NS2: It was somewhat boring and slowPS2: Good use of data to find trends NS3: Calculations were tediousPS3: Informative NS4: Complicated mathPS4: Ease of using Excel NS5: Accountant like dullness, need more dynamicPS5: Learn the job market that is arising and the way it involvement from class

may be in futurePS6: It showed us the future of the job industry

Chemistry ChemistryPC1: Amazed at how computers work to make difficult NC1: Case study did not relate to me leaving me

tasks easier cluelessPC2: Made me aware of IT courses to stay away from NC2: Irrelevance to major, satisfies no requirement,PC3: Liked working with Alchemy in the lab waste of timePC4: Interesting to use Alchemy in relation with Internet NC3: Waiting long for results and didn’t know what

to do with themNC4: Software was somewhat confusingNC5: Computers were slow

Comparison ComparisonPO1: It was fun NO1: It could have been a bit more challengingPO2: A full overview of IT issues NO2: I felt that the lectures were very scatteredPO3: Taught us important practical skills – HTML,

Javascript, etc.

Refining the ITF Framework

The empirical results from the previous section show that the case studies based on the ITFframework does impact the perceptions of students regarding the role of IT. This change inperception is an important feature because it is likely that a course using the ITF frameworkmight be the final place where an undergraduate receives formal exposure to the field.Lifelong learning begins with a basic understanding and appreciation of its role.

The empirical feedback received, both positive and negative, suggests steps that can beimplemented to refine the ITF framework. Results indicate that while this experience didnot promote a career change towards IT, it did improve appreciation of IT. The positivecomments from the qualitative analysis of Table 4 show that the principles of application


and experiential learning were valued. Hands-on IT usage, from spreadsheets to chemistrysoftware, seemed to receive the most positive feedback. Students enjoyed actively usingtools to solve problems, and enjoyed working in the lab and building skills that they wouldpotentially use in their capstone projects.

Negative comments indicate that other IT teaching principles need to be addressed (or insome cases, readdressed). Students at this level responded negatively to issues that surfacedwhile using the laboratory equipment. The principle of recovery describes recognizing andfixing errors that arise during the case study itself. However, problems with lab softwareand hardware act more as a distraction than a chance to learn. Therefore, substantial testingof IT tools is mandated for this student population.

Student feedback also indicates that the principle of immediate relevance requires moreattention in delivery of this material. Phrases such as “waste of time”, “somewhat boring”,and “just not interested” (each phrase taken from a different case study) demonstrate thatit is worth the effort to find meaningful material for the case studies. This issue is hard toresolve because student interests vary widely. One remedy might be to solicit input fromthe students according to the principle of design involvement. This input could come fromcurrent or past students. Local factors may promote or inhibit this approach. Moreover, itis difficult to properly test IT tools for a case study that has not been completely definedbeforehand.

Finally, course facilitators need to have the proper IT and other tools to conduct thecase studies. Software and hardware are often readily available; preparation, testing andevaluation time are just as important.

Conclusions

This paper documents our experience designing, developing and conducting a set of casestudies using the ITF framework. Case studies were constructed in the areas of anthro-pology, sociology and anthropology, with the goal of increasing student fluency with IT.Survey data of student perceptions has been used to gauge the impact of this set, as wellas compare it to student perceptions resulting from another learning environment that alsoendeavored to improve IT fluency. These results have also been utilized to refine the ITFframework that was used to generate the case studies.

The quantitative analysis from the completed questionnaire (Appendix A) indicates thefollowing conclusions: (i) the set of case studies motivate students to reflect upon the roleof IT somewhat more than the comparison course (Q6); (ii) the students surveyed indicatethat the case studies had a positive impact on their perceptions of IT, though not as muchas the impact reported by students in the comparison course (Q7); and (iii) students feltthey learned more about specialized IT from the case studies than the comparison course(Q8). Other results suggest that prior background with IT does correlate positively tothe resulting perceptions about IT after the case studies have been conducted. Furtherwork needs to explore why a similar positive correlation was not found in the comparisoncourse.


Qualitative survey data emphasize the importance of selecting and testing IT tools toavoid problems (e.g., “computer glitches”, “waiting long for results. . . ” from Table 4) forthey appear to be significant distractions. Negative comments regarding the perceived lackof usefulness with the case studies indicate that the ITF framework needs to emphasize theprinciples of meaningfulness and immediate relevance more strongly. These issues needto be addressed for the ITF framework to evolve into an important pedagogical tool thatcan provide students the proper foundation for effectively utilizing IT now and throughouttheir careers.

References

Aiken, R., Kock, N., and Mandviwalla, M. (2000) Fluency in information technology: A second course fornon-CIS majors. ACM SIGCSE Bulletin, 32(1), 280–284.

Card, S., Moran, T., and Newell, A. (1983) The Psychology of Human-Computer Interaction. Erlbaum, Hillsdale,NJ.

Carroll, J. M. (1990) The Nurnberg Funnel. MIT Press, Cambridge, MA.Carroll, J. M. (1998) Minimalism Beyond the Nurnberg Funnel. MIT Press, Cambridge, MA.Computer Science and Telecommunications Board (CSTB) (1999) Being Fluent with Information Technology.

National Academy Press, Washington, DC.Denning, P. J. (2000) A commentary on fluency in information technology. Inventio (online journal), 1(2),

www.doiiit.gmu.edu/Archives/spring00/pdenning_1.html.Knowles, M. (1975) Self-Directed Learning. Follet, Chicago, IL.Knowles, M. (1984a) The Adult Learner: A Neglected Species. Gulf Publishing, Houston, TX.Knowles, M. (1984b) Andragogy in Action. Jossey-Bass, San Francisco, CA.Papert, S. (1980) Mindstorms: Children, Computers and Powerful Ideas. Basic Books, New York, NY.Papert, S. (1993) Childrens’ Machines: Rethinking Schools in the Age of the Computer. Basic Books, New York,

NY.Pask, G. (1975) Conversation, Cognition, and Learning. Elsevier, New York, NY.Salomon, G. (1979) Interaction of Media, Cognition, and Learning. Jossey-Bass, San Francisco, CA.Salomon, G. (1981) Communication and Education. Sage, Beverly Hills, CA.Salomon, G., Perkins, D., and Globerson, T. (1991) Partners in cognition: Extending human intelligence with

intelligent technologies. Educational Researcher, 20(4), 2–9.Spiro, R. J. and Jehng, J. (1990) Cognitive flexibility and hypertext: Theory and technology for the non-linear

and multidimensional traversal of complex subject matter. In Cognition, Education, and Multimedia, D. Nixand R. Spiro (eds), Erlbaum, Hillsdale, NJ.

Spiro, R. J., Feltovich, P. J., Jacobson, M. J., and Coulson, R. L. (1992) Cognitive flexibility, constructivismand hypertext: Random access instruction for advanced knowledge acquisition in Ill-structured domains. InConstructivism and the Technology of Instruction, T. Duffy and D. Jonassen (eds), Erlbaum, Hillsdale, NJ.

Van Der Meij, H. and Carroll, J. M. (1995) Principles and heuristics for designing minimalist instruction. Techni-cal Communications, 42(2), 243–261.

Appendix A: Questionnaire Used for Assessment

For questions 1–9, the following scale was used in the first part of the questions:( ) Strongly disagree ( ) Disagree somewhat ( ) Neither disagree or agree ( ) Agree

somewhat ( ) Strongly agree


Demographics:

Sex: ( ) Male ( ) FemaleAge: _____First language: ( ) English ( ) Other ______________

Attitudes toward IT prior to taking the course

1. I already had a very good knowledge of Information Technology (IT) prior to taking thiscourse.2. I intended to take as many IT courses as possible in college prior to taking this course.3. I was generally very attracted to IT issues prior to taking this course.

Case study impact on attitudes toward IT

4. The case study made me feel like I should take more IT courses in the future, even if Idon’t pursue a career in IT. Why? (Explain your answer)5. The case study made me feel like I should pursue an IT career. Why? (Explain youranswer)6. The case study improved my perception of IT’s potential for solving complex tasks.Why? (Explain your answer)7. The case study improved my general perception of IT. Why? (Explain your answer)

Case study impact on IT knowledge

8. I learned a lot from this case study about specialized IT applications. Why? (Explainyour answer)9. I learned a lot from this case study about IT issues in general. Why? (Explain youranswer)

Positive and negative aspects of the case study:

10. What were the main positive aspects of this case study?11. What were the main negative aspects of this case study?

Appendix B: Correlation Tables

Teaching the use of complex IT in specific domains: Developing, assessing and refining a curriculum...

Documents

Transcript of Teaching the use of complex IT in specific domains: Developing, assessing and refining a curriculum...