JOURNAL OF RESEARCH IN SCIENCE TEACHING VOL. 30, NO. 3, PP. 303-317 (1993)

Promoting Skilled Problem-Solving Behavior among Beginning Physics Students

Jose P. Mestre, Robert J. Dufresne, William J. Gerace, and Pamela T. Hardiman

Deparfment of Physics & Astronomy, University of Massachusetts, Amherst, Massachusetts 01003

Jerold S. Touger

Science Division, Curry College, Milton, Massachusetts 021 86

Abstract

Beginning physics students were constrained to analyze mechanics problems according to a hierarchical scheme that integrated concepts, principles, and procedures. After five 1-hour sessions students increased their reliance on the use of principles in categorizing problems according to similarity of solution and in writing qualitative explanations of physical situations. In contrast, no consistent shift toward these expert-like competencies was observed using control treatments in which subjects spent the same amount of time solving problems using traditional approaches. In addition, when successful at performing the qualitative analyses, novices showed significant improvements in problem-solving performance in comparison to novices who directed their own problem-solving activities. The implications of this research are discussed in terms of instructional strategies aimed at promoting a deeper understanding of physics.

Two important goals of physics instruction are to help students achieve a deep, conceptual understanding of the subject and to help them develop powerful problem- solving skills. Until recently, little was known about effecting these goals except that they required substantial time, effort, and practice on the part of the student. However, during the past few years cognitive research studies have provided detailed knowledge concerning the differences between experts and novices, and as a result we can begin to speculate on ways of making the transition from novice to expert more efficient. The study reported here is an attempt to ascertain whether or not physics novices will manifest expert-like problem-solving behavior following a treatment in which they performed qualitative problem analyses that are structurally similar to those employed by experts.

The motivation for evaluating the effectiveness of a conceptual problem-solving approach derives from research findings indicating that novices do not normally take advantage either of the efficient concept-centered techniques used by experts to store and recall domain-specific knowledge, or of the powerful techniques by which experts integrate concepts and procedures to solve problems. Research in diverse domains

0 1993 by the National Association for Research in Science Teaching Published by John Wiley & Sons, Inc. ccc 0022-4308/93/030303-15

304 MESTRE ET AL.

such as chess (Chase & Simon, 1973; deGroot, 1965), computer programming (Ehrlich & Soloway, 1984), electrical circuits (Egan & Schwartz, 1979), and physics (Larkin, 1980; Larlun, McDermott, Simon, & Simon, 1980a, 1980b) indicate that experts possess hierarchically organized, integrated knowledge structures. The expert’s knowledge can be accessed quickly and efficiently through underlying “themes” (e.g . principles, concepts, heuristics, etc.) that are used to chunk related knowledge units (Glaser, 1992). In contrast, the typical novice’s knowledge is accessed in a somewhat sequential fashion through superficial attributes rather than global themes.

Manifestations of these expert-novice differences become apparent in studies of problem categorization. When asked to categorize problems into groups, with the problems in each group related by the approach that would be used to solve them, experts rely on the problems’ deep sfrucrure (i.e., principles and concepts that could be applied to solve the problem) as the classification criterion (Chi, Feltovich, & Glaser, 1981; Hardiman, Dufresne, & Mestre, 1989; Schoenfeld & Herrmann, 1982). In contrast, novices rely on the problems’ surface features (i.e., objects and terminology described in the problems).

It should be pointed out that although novices cue predominantly on surface features, and experts cue predominantly on deep structure, novices and experts do not use these attributes exclusively in making judgments about solution similarity. That is, to assume that surface features and deep structure form a dichotomy for describing novices and experts is incorrect (Smith, 1992). There is evidence that experts find surface features to be a major distraction when they are pitted against deep structure in certain similarity judgment tasks; conversely, novices are capable of choosing deep structure over surface features, even when these attributes are competing influences in the similarity judgment task (Hardiman et al., 1989).

For the expert, classifying a problem as belonging to a particular type suggests possible solution strategies (Hayes & Simon, 1976; Hinsley, Hayes, & Simon, 1977; Newel1 & Simon, 1972; Simon & Simon, 1978). After the initial problem classification, the expert generally performs a qualitative analysis of the problem by considering which principles or concepts are appropriate for its solution and how they could be applied (Larkin, 1981). After this analysis is completed the expert begins to execute the solution strategy. In contrast, novices tend to rely on “means-ends analysis” to solve problems, which consists of attempting to reduce the “distance” between a problem’s initial state and the goal state; in quantitative domains such as physics, this consists of manipulating equations until the desired quantity is isolated (Larkin et al., 1980a, 1980b). Again, these statements should not be interpreted as dichotomous characterizations of experts and novices, but rather as potentially useful idealizations.

Although there are significant differences between the ways novices and experts solve problems, there is some evidence that novices can benefit from the approaches used by skilled problem solvers to classify, analyze, and solve problems. The studies that have explored treatments designed to promote skilled problem solving are of two types. In the first type of study the treatments last a protracted period of time and cover a wide range of topics, while in the second type the treatments are relatively brief and cover a very narrow range of topics. One example of the first type of study in mathematics (Schoenfeld & Herrmann, 1982) resulted in considerable improvements across a wide range of math topics after a treatment consisting of a semester-long problem-solving course where students were specifically taught heuristics that are used by skilled problem solvers.

PROMOTING SKILLED PROBLEM-SOLVING BEHAVIOR 305

Two examples typify the second type of study in the domain of physics. In one study (Eylon & Reif, 1984) novices were presented with physics content organized in a hierarchical fashion; following treatment the novices exhibited improved recall and problem-solving performance. In the other study (Heller & Reif, 1984) novices were taught to solve problems involving Newton’s Second Law by first conducting qualitative analyses (similar to those conducted by experts); this treatment resulted in improved problem-solving performance on problems of this type.

The purpose of the present study was to investigate the changes in problem-solving behavior that resulted from a treatment in which physics novices practiced performing qualitative analyses of problems that integrate principles, concepts, and procedures. Unlike the two types of studies described above, the treatment used in our study was relatively brief, yet included most topics covered in a typical introductory calculus- based classical mechanics course.

The main intent of the treatment was to highlight the role of concepts and procedures in problem solving and in so doing to counter novices’ tendency to rely on formulaic problem-solving approaches. The treatment was implemented in a menu-driven computer environment that was designed as a “tool” for solving problems, rather than as a problem-solving instructional environment. Neither feedback nor coachng was provided to the subjects while they performed the qualitative analyses; subjects were simply exposed to the expert-like approach and allowed to practice using it.

We begin by describing the computer-based environment through which the focal treatment was administered, .as well as another environment used to implement one of the control treatments. Two experiments are then discussed comparing the effectiveness of the focal treatment to that of the control treatments. We conclude with a discussion of the instructional implications of the study.

Description of Computer-Based Treatments

Two of the treatments used in our studies were implemented in the computer- based environments described below. One computer-based environment incorporated a conceptual approach to solving problems; we call this environment the hierarchical analysis tool (HAT), and it was used as the focal treatment. The other incorporated a formulaic approach; this environment was called the equation sorting tool (EST), and was used as a control treatment.

Description of the HAT

The HAT uses a hierarchical framework to structure qualitative analyses of problems on the basis of principles and procedures. It is a menu-driven environment implemented on IBM PCs. The word “tool” in the name is meant to imply that the environment facilitates constructing a problem’s solution, rather than actually supplying the answer.

To analyze a problem, the user answers a series of well-defined questions by making selections from menus that ask qualitative questions about the problem under consideration. In the first menu, the user is asked to select one of four general principles that could be applied to solve the problem. Subsequent menus focus on ancillary concepts and procedures, and depend upon the prior selections made. When the analysis is complete, the HAT provides a set of equations that is consistent with the menu selections made during the analysis. If the analysis is carried out appropriately, then these equations could be used to generate a solution to the problem; however, the user

306 MESTRE ET AL.

must still manipulate the equations to isolate the desired quantity. If the analysis is carried out incorrectly, the final equations are consistent with the choices made, but inappropriate for solving the problem.

It is important to note that the HAT neither tutors nor provides feedback to the user; it merely constrains the type and order of questions that should be considered when performing a qualitative analysis of a problem. Further, the HAT’S functioning is independent of the problem being analyzed; the HAT has no knowledge of the problem being worked on by the user. The problems that subjects analyzed with the HAT were written on index cards. In short, the HAT can be thought of as an elaborate, hierarchical tree-like structure. The exact path that a user takes to analyze a problem depends on the selections made at each node along the way.

Figure 1 provides an example of the series of menus and choices that would appropriately analyze the problem in Table 1 (each menu is presented one at a time on the computer screen). Note that the “prompt line” at the bottom of each screen allows users the options to (1) back-up to some previous menu to change the selection made, (2) return to the first menu to restart the analysis, (3) enter a glossary to look up the definition of a term, (4) list all previous selections made along the way, and (5) quit.

If a problem is more complex than the one analyzed in Figure 1 and involves the sequential application of two (or more) principles, the HAT can still be used to perform a complete analysis of the problem. For example, if the block of the problem in Tdble 1 undergoes a totally inelastic collision with a second block at the bottom of the ramp (i.e., the two blocks stick together after the collision), then the user could select option 2 at the very last menu in order to return to the first menu and continue with the “momentum” portion of the analysis. (Momentum is conserved in the collision.) At the end of the momentum analysis, the user would be provided with two sets of equations, namely the equations from the energy portion of the analysis (menu 9 in Figure 1) and the equations resulting from the momentum portion of the analysis. Together, these equations would enable the user to solve the problem.

Description of the EST

The EST incorporated a formula-based approach for analyzing problems and was developed for use as a control treatment. It was designed to be consonant with the problem-solving approaches used by novices (Larkin et al., 1980a; Sweller, 1988). The EST contains a data base of 178 equations taken from a standard classical mechanics textbook (Resnick & Halliday, 1977). A user could sort this data base in three different ways: (1) by problem types, such as “inclined plane” and “falling bodies”; ( 2 ) by variable names, such as “mass” and “velocity”; and ( 3 ) by physics terms, such as “potential energy” and “Newton’s Second Law.” Sequential sorts can be performed to reduce the data base to a small, manageable number of equations that might be useful for solving a problem (e.g., two sequential sorts using the problem type “inclined plane” and the variable name “velocity” will reduce the data base to those equations appropriate for inclined planes which contain the variable “v”). The EST was designed to reflect novices’ tendency to cue on surface features when deciding how to attack a problem, and to focus their problem-solving efforts on finding equations that can be manipulated to yield an answer to the problem. The EST, like the HAT, provides no evaluation of the appropriateness of the selections made. Basically the EST can be

6 7 8

Des

crib

e th

e ch

ange

s in

pot

entia

l en

ergy

. 1.

Cha

nges

in

grav

itatio

nal

pote

ntia

l en

ergy

2.

Cha

nges

in

sprin

g po

tent

ial

ener

gy

3. C

hang

es i

n gr

avita

tiona

l an

d sp

ring

pot

entia

l en

ergy

Plea

se e

nter

you

r se

lect

ion:

[l]

jn

)ack

up

(M)a

in

men

u (G

)los

sary

(L

)ist

sele

ctio

ns

(Q)u

it D

escr

ibe

the

boun

dary

con

ditio

ns.

1. N

o in

itial

gra

vita

tiona

l po

tent

ial

ener

gy

2. N

o fin

al g

ravi

tatio

nal

pote

ntia

l en

ergy

3.

Ini

tial

and

final

gra

vita

tiona

l po

tent

ial

ener

gy

Plea

se e

nter

you

r se

lect

ion:

[2]

jn

)ack

up

(M)a

in

men

u (G

)los

sary

(L

)ist

se

lect

ions

(Q

)uit

Is t

here

ano

ther

bod

y in

the

sys

tem

whi

ch h

as n

ot been

exam

ined

? 1.

Yes

1 2

I 2.No

Whi

ch p

rinci

ple

appl

ies

to t

his

part

of

the

prob

lem

sol

utio

n?

1. N

ewto

n's

Seco

nd L

aw o

r K

inem

atic

s 2.

Ang

ular

Mom

entu

m

3. L

inea

r M

omen

tum

4.

Wor

k an

d En

ergy

Pl

ease

ent

er y

our

sele

ctio

n: [

4]

(B)a

ckup

(M

)ain

m

enu

(G)l

ossa

ry

(L)i

st

sele

ctio

ns

(Q)u

it D

escr

ibe

the

syst

em i

n te

rms

of i

ts m

echa

nica

l en

ergy

. 1.

Con

serv

ativ

e sy

stem

(co

nser

vatio

n of

ene

rgy)

2.

Non

-con

serv

ativ

e sy

stem

9 LO

Fig

ure

1.

HA

T A

naly

sis

of p

robl

em s

how

n in

Tab

le 1

.

Plea

se e

nter

you

r se

lect

ion:

[2]

jl

l)ac

kup

(M)a

in

men

u (C

)los

sary

(L

)ist

se

lect

ions

(Q

)uit

'T

he E

nerg

y Pr

inci

ple

stat

es t

hat

the

wor

k do

ne on

the

syst

em by

all

non-

co

nser

vativ

e fo

rces

is e

qual

to

the

chan

ge in

the

mec

hani

cal

ener

gy of

the

syst

em:

Acc

ordi

ng t

o yo

ur s

elec

tions

, W

, =

0

(Con

serv

ativ

e sy

stem

: M

echa

nica

l en

ergy

is c

onse

rved

) E+

= (1

RM

v2) a

nd E

i= (

Mgy

)

W,=

Ef-

Ej

Plea

se p

ress

any

key t

o c

ontin

ue

***W

ork

and

Ener

gy**

* 1.

Pro

blem

Sol

ved

2. Return

to M

ain

Men

u to

con

tinue

sol

utio

n 3.

Rev

iew

pre

viou

s so

lutio

n sc

reen

s

Plea

se e

nter

you

r se

lect

ion:

5

Plea

se e

nter

you

r se

lect

ion:

[l]

(B

)ack

up

(M)a

in

men

u (G

)los

sary

(L

)ist

se

lect

ions

(Q

)uit

Des

crib

e th

e bo

unda

ry c

ondi

tions

. 1. No

ini

tial

eans

latio

nal

kine

tic e

nerg

y 2.

No

final

tra

nsla

tiona

l ki

netic

ene

rgy

3. I

nitia

l an

d fin

al u

ansl

atio

nal

kine

tic e

nerg

ies

?lea

se e

nter

you

r se

lect

ion:

[l]

;R

)ack

up

(M)a

in

men

u (C

)los

sary

(L

)ist

se

lect

ions

(Q

)uit

308 MESTRE ET AL.

Table 1 Sample Problem Analyzed by HAT Menus Shown in Figure 1

A small block of mass M slides along a track having both curved and horizontal sections as shown. If the particle is released from rest at height h, what is its speed when it is on the horizontal section of the track? The track is frictionless.

thought of as a lengthy “formula-sheet’’ that is cross-referenced and accessible via a large list of terms.

Experiment 1

Experiment 1 compares the HAT treatment with two control treatments on three copt ive tasks. The three tasks assessed problem categorization, explanation of physical situations, and problem solving.

Method Forty-two students at the University of Massachusetts volunteered to participate

in the study. All subjects had completed a calculus-based, college-level introductory classical mechanics course with a grade of B or better. These subjects were randomly divided into three equal-sized groups, with each group undergoing different treatments. All three treatment groups solved the same 25 problems, 5 at a time in 1-hour sessions spread over approximately 3 weeks. The “HAT group” solved the treatment problems using the HAT, the “EST group” solved the treatment problems using the EST, and the “T group” solved the treatment problems in a “homework style” using a mechanics textbook. In addition to the five hours used to administer the treatment, five additional hours were used to administer the pre-, and postassessments. Subjects were paid for their participation and wee told nothing regarding the philosophy behind the treatments.

Problem Categorization Task Description of Task. Some of the clearest differences in expert-novice behavior

have emerged from problem categorization experiments. Given that novices predominantly cue on a problem’s surface features when deciding on a possible solution strategy and that experts cue on deep structure, careful manipulation of these variables should provide a sensitive measure of shifts toward expert-like behavior.

The problem categorization task developed for this study presented subjects with a model problem and two comparison problems, and asked them to judge which of the two comparison problems would be solved most like the model problem. Surface feature and deep structure similarity to the model problem were varied systematically. More specifically, comparison problems were constructed to match their corresponding model problem in one of four ways: (1) surface features, meaning that the objects and descriptor terms in the comparison problem matched those of

PROMOTING SKILLED PROBLEM-SOLVING BEHAVIOR 309

the model problem; (2) deep structure, meaning that the same physical principle that could be applied to solve the comparison problem could also be applied to solve the model problem; (3) both surface features and deep structure; and (4) neither surface features nor deep structure. These four types of comparison problems were termed S, D, SD, and N, respectively.

Comparison problems were paired together such that one and only one comparison problem in each pair matched the model problem in deep structure. This constraint allowed the construction of four comparison problem pairings: (1) S-D, (2) S-SD, (3) N-D, and (4) N-SD. The task contained a total of 20 items, 5 items in each of the four comparison problem pairings. An example of a set of five problems (model problem with S, D, SD, and N comparison problems) is given in Table 2. Further details on the task can be found elsewhere (Hardiman et al., 1989).

A novice-like Categorization strategy based strictly on surface features would result in the following performance pattern: (1) S-D, 0% deep structure choices; (2) S-SD, 50% deep structure choices (both choices are equally good in terms of matching the model problem on surface features); (3) N-D, 50% deep structure choices (either alternative is equally “bad” in terms of matching the model problem on surface features); and (4) N-SD, 100% deep structure choices (a surface feature match to the model problem will also mean a deep structure match). In contrast, an expert-like categorization strategy based srrictly on deep structure would result in 100% deep structure choices for all four types. Thus, a shift toward an expert-like categorization scheme should be evidenced by an increase in the percentage of deep structure choices. Since the initid decision that must be made in the HAT concerns the principle to be applied, we hypothesized that the HAT group would be more likely to focus on deep structure after treatment than either of the two control groups.

Table 2 Sample Model Problem and Comparison Problems Used in Problem Categorization Task

Model problem: A 2.5 kg ball of radius 4 cm is traveling at 7 d s on a rough horizontal surface, but not spinning. Some distance later, the ball is rolling without slipping at 5 m/s. How much work was done by friction? Surface feature ( S ) comparison problem: A 3 kg soccer ball of radius 15 cm is initially sliding at 10 m/s without spinning. The ball travels on a rough horizontal surface and eventually rolls without slipping. Find the ball’s final velocity. Deep structure (D) comparison problem: A small rock of mass 10 g falling vertically hits a very thick layer of snow and penetrates 2 meters before coming to rest. If the rock’s speed was 25 m/s just prior to hitting the snow, find the average force exerted on the rock by the snow. Surface feature and deep structure (SD) comparison problem: A .05 kg billiard ball of radius 2 cm rolls without slipping down an inclined plane. If the billiard ball is initially at rest, what is its speed after it has moved through a vertical distance of .5 m? No (N) match comparison problem: A 2 kg projectile is fired with an initial velocity of 1500 m/s at an angle of 30 degrees above the horizontal and height 100 m above the ground. Find the time needed for the projectile to reach the ground.

Nore. In the task the same model problem was used in four items, each with two comparison problems: S- D, S-SD, N-D, and N-SD.

3 10 MESTRE ET AL.

Results. The same categorization task was administered to subjects both before and after treatment. Each item was assigned one point if the comparison problem selected matched the model problem in deep structure, and zero otherwise (hence a maximum possible score of 20 for each subject). Table 3 shows the mean performance of the three treatment groups, where the entries indicate the fraction of deep structure choices. An analysis of variance (ANOVA) of the categorization scores indicates that the overall pre- and posttreatment difference (.60 vs. .62) was not significant, but there was a significant time X treatment interaction [F(2,39) = 4.28, MSe = .17, p = .02]. The HAT group was the only group to show any indications of a shift toward deep structure categorization, increasing from .56 to .66, while the performance of the EST and T groups declined slightly (from .61 to .60, and from .62 to .58, respectively). The . 10 improvement for the HAT group was statistically significant, F(1,13) = 5.20, p = .04, and comprised improvements across all four types of comparison problem pairings (S-D, .17; S-SD, .8; N-D, .9; N-SD, .6). The pre-to- post improvement was statistically significant for the S-D pairings, t(13) = 3.12, p = .03, in which surface features and deep structure are in direct competition.

Explanation of Physical Situations

Description of Task. Another well-documented difference between experts and novices is the ability to provide qualitative explanations. Experts are capable of both verbalizing problem-solving strategies and drawing on principles and related concepts in constructing explanations (Chi et al., 1981). In contrast, novices are generally poor verbalizers of problem-solving strategies and most often resort to equations when discussing approaches to solving problems. Hence a task requiring subjects to provide qualitative explanations has the potential for detecting differential levels of expertise. More specifically, we hypothesized that, in comparison to the two control treatments, the HAT treatment would increase subjects’ reliance on principles when providing qualitative explanations.

The explanation task we used consisted of two equivalent pre- and posttasks containing two questions each. Each question presented a physical setup and asked the subject to explain in writing what would happen when a particular change was made in the setup. For example, one setup consisted of a bullet striking and imbedding

Table 3 Pre- and Posttreahent Performance on Categorization Task

Pre Post Group M SD M SD ~ ~~~~~

HAT .56 .08 .66 .12 (N = 14) EST .61 .10 .60 .08 (N = 14) T .62 .09 .58 .ll (N = 14)

PROMOTING SKILLED PROBLEM-SOLVING BEHAVIOR 311

Table 4 Pre- and Posttreatment Performance on Explanations Task

Pre Post Group M SD M SD

HAT .22 .26 .28 .26 (N = 14) EST .39 .28 .16 .26 (N = 14) T .30 .33 .30 .26 (N = 14)

itself in a block hanging from a ceiling, and subjects were asked to explain any changes occumng in the behavior of the system if the setup were moved to the moon. Subjects were instructed to write out explicitly the reasoning behind their explanations. Each subject was free to choose which variables should be addressed and how to structure the explanation (e.g., how qualitative or quantitative it should be, and whether to emphasize concepts or formulas).

An analysis of the explanations given by subjects was performed, the purpose of which was to provide some quantitative measure of the degree to which subjects were employing concepts to discuss the physical situations. We chose to analyze the data in terms of the level of structured use of work-energy concepts, since these concepts could be appropriately used to discuss all four questions. A quantitative measure of subjects’ use of the work-energy concept, designated level of concept use (or LCU), was devised and consisted of the sum of the numerical scores in a number of different categories, which included mention of concept, use of concept in reasoning, justification of conclusions, and correct identification of initial and final states (Touger, Dufresne, Gerace, & Mestre, 1987).

Table 4 contains the mean pre- and postperformance on the LCU measure for each treatment group, reported as a fraction of the maximum possible LCU score (the maximum LCU score was 46 points). The HAT group displayed an increased reliance on concept use in their explanations following treatment, whereas the two control groups’ reliance on concept use either declined or remained the same following treatment. These data are consistent with the categorization results.

Results.

Problem Solving

Description of Task. One of the most widely accepted measures of expertise is problem-solving performance. Because problem solving requires the interplay of numerous cognitive factors (e.g., acquisition of a rich base of content knowledge and procedures, organization and retrieval of declarative and procedural knowledge, ability to identify what principle and procedure can be used to solve a problem, ability to execute a solution plan, and ability to evaluate both the solution plan and the answer for accuracy and reasonability), it has been difficult to effect major improvements in problem- solving performance following short treatments, even when coaching is provided.

312 MESTRE ET AL.

Table 5 Pre- and Posttreatment Performance on Problem-Solving Test for Experiment 1

Re Post Group M SD M SD

HAT .29 .20 .41 .18 (N = 14) EST .36 .26 .45 .26 (N = 14) T .32 .25 .44 .24 (N = 14)

To evaluate the effectiveness of the HAT for improving problem-solving performance, we constructed and administered pre- and postassessments. Two equivalent test forms were constructed, each resembling a final exam in a freshman level classical mechanics course. The order in which the two forms were administered was randomized, with a particular subject receiving one form as the preassessment and the other as the post- assessment. Each form contained seven problems, one each requiring the application of Newton's Laws, energy principles, momentum principles, and angular momentum principles, and three requiring the application of pairs of these four principles.

The tests were graded independently by two physicists (who were ignorant of the treatment group to which subjects belonged) in a style similar to that which would be used to grade a final exam. Each problem was evaluated on a basis of 10 points. Whenever a score on a test problem differed by two or more points, the solution was reevaluated and a grade was determined by consensus.

Table 5 contains the mean performance for the three treatment groups displayed as a fraction of the maximum possible problem-solving score. All three groups increased about . 10 point, with nearly all improvement due to performance on the four single-principle problems; the two-principle problems proved very difficult for the subjects. An ANOVA of these data revealed no main effect of treatment and no significant interaction with treatment.

Results.

Discussion of the Three Tasks of Experiment 1

Findmgs from Experiment 1 indicate that freshman students with reasonable aptitude in physics can benefit from hierarchically structured problem solving. In contrast to the two control treatments, use of the hierarchical approach promoted a greater reliance on principle use both in categorizing problems according to similarity of solution and in providing qualitative explanations of physical situations.

Exposure to the HAT did not appear to provide any greater benefit to problem- solving performance than either of the two control treatments. One possible explanation is that the HAT subjects were unsuccessful in using the HAT approach during the treatment sessions. That subjects were unsuccessful in using the HAT appropriately was confirmed by an analysis of the key-strokes made by subjects while solving the 25 treatment problems. HAT subjects were able to reach the appropriate equation screen (e.g., the equivalent of menu 9 in Figure 1) for only 49% of the treatment

PROMOTING SKILLED PROBLEM-SOLVING BEHAVIOR 313

problems. Out of all complete analyses (recall that subjects could perform more than one complete analysis for the same problem by restarting the analysis after reaching the equation screen) resulting in an equation screen, 72% were initiated with a correct selection of a principle at menu level 1. Among those analyses that were initiated with a correct selection at menu level 1, only 52% resulted in a correct equation screen. Thus, it appears that subjects could identify the initial principle with a reasonable degree of success but could not make appropriate selections in subsequent menus.

Although there were no group differences on the pre and post problem-solving task, the HAT group’s performance on the 25 treatment problems was lower than that of the two control groups (34% for the HAT group vs. 43% for the two control groups combined). Had the HAT group been able to use the HAT appropriately during treatment, subjects should have outperformed the two control groups on the treatment problems since the HAT would have provided the exact equations needed to solve the problems.

We believe that the increased reliance on principle use exhibited by the HAT group was the result of the emphasis given to principles and concepts near the top levels of the HAT hierarchy. Students appear to be capable of gleaning the importance of applying principles to solve problems from simply using the HAT approach. However a comparable improvement in problem solving would require a greater facility with the HAT approach. This was the subject of investigation in Experiment 2.

Experiment 2

In Experiment 2 the level of difficulty of some of the treatment problems was reduced in an attempt to increase the number of successful analyses performed by the HAT group. We hypothesized that if subjects were able to use the HAT approach with moderate success then they would display larger improvements on the problem-solving task following treatment than would a control group.

Method

Thirty students at the University of Massachusetts who had completed a calculus- based mechanics course with a grade of B or better volunteered to participate in the study. The subjects were equally divided into two treatment groups, designated the “HAT group” and the “Control (C) group.” Both groups solved 25 treatment problems over five 1-hour sessions. The HAT group solved the treatment problems using the HAT, while the C group solved the treatment problems without any aid whatsoever. The decision to use only one control treatment in Experiment 2 was based on the fact that there was little difference between the T and the EST groups’ performance in Experiment 1 . Further, the new control treatment avoided the possibility of acquiring content knowledge through the textbook (although it should be pointed out that the textbooks were almost never used by the T group during Experiment l ) , and it did not encourage subjects to use an approach they might not have normally used (e.g., the formulaic approach incorporated in the EST).

Some of the treatment problems used in Experiment 2 differed from those used in Experiment 1. The 25 treatment problems from Experiment 1 were evaluated for difficulty level based on subjects’ performance during Experiment 1. Those that were deemed difficult were replaced with easier problems covering the same topics. As in Experiment 1, subjects were paid for their participation in the study.

314 MESTRE ET AL.

R e and post problem-solving tests were used to evaluate the effectiveness of the treatments. Each test form contained the four single-principle problems used in Experiment 1; the three two-principle problems were dropped because performance on these problems was near floor-level in Experiment 1, both before and after treatment. Subjects were given 45 minutes to finish the four problems, and the test forms were randomized over subjects. The tests were graded on a basis of 10 points per problem by two physicists using the same criteria as in Experiment 1.

Resutts

The two test forms were combined for the analysis since there was neither a main effect, nor a significant interaction involving the form of the test. Table 6 contains the performance of the two treatment groups on the pre and post problem-solving test. As hypothesized, the improvement of the HAT group was significantly greater than that of the control group (a .53 point increase in performance for the HAT group vs. a .37 point increase for the control group), one tailed t(24) = 1.98, p = .03.

In order to ascertain whether the HAT group in Experiment 2 was more proficient at using the HAT than the HAT group in Experiment 1, the performance data on the treatment problems were compared to the corresponding data from Experiment 1 (treatment problems were graded on the basis of one point for a correct answer and zero for an incorrect answer), These data indicate that the HAT group in Experiment 2 was more proficient than the HAT group in Experiment 1 at using the HAT to solve the treatment problems; the HAT group’s performance on the 25 treatment problems in Experiment 1 was .34, compared to .59 in Experiment 2.

Discussion Taken together, the results from Experiments 1 and 2 indicate that the HAT

treatment improves problem-solving performance more than student-directed problem solving, as long as subjects are able to use the HAT with a modest degree of success. Our findings suggest that the level of difficulty of the treatment problems affects subjects’ ability to use and assimilate the HAT approach, which in turn affects subsequent problem-solving performance. The differences in improvement between the two groups suggest that conceptual approaches to solving problems can have a beneficial effect on problem-solving performance over and above benefits accrued from student-directed problem solving.

Table 6 Pre- and Posttreatment Performance on Problem-Solving Test for Experiment 2

Pre Post Group M SD M SD

HAT .35 .22 .88 .12 (N = 15) C .40 .28 .76 .26 (N = 15)

PROMOTING SKILLED PROBLEM-SOLVING BEHAVIOR 315

General Discussion and Educational Implications

The main thrust of this research was to evaluate the effectiveness of having talented novices actively perform conceptual, qualitative analyses of problems for promoting behavior observed in skilled problem solvers. Findings indicate that performing this type of qualitative analysis for a relatively short period of time results in statistically significant shifts toward problem-solving behavior observed in experts. These positive findings result despite the lack of coaching to ensure that the novices mastered the approach. We recognize that the subjects used (volunteers who had received a grade of B or better) did not represent a random sample of physics students nationwide, and so conclusions offered may not be generalizable to the “average student.”

What might these findings suggest for physics education? First, our findings indicate that, at least for students who have displayed some aptitude for the subject matter, problem-solving activities that integrate conceptual and procedural knowledge are more conducive toward producing skilled problem-solving behavior than traditional, student-directed problem-solving activities. Tradtional approaches for teaching problem solving in quantitative domains rely largely on assigning large numbers of problems to be solved with little guidance as to the general techniques that students should use to solve them. However, our findings, as well as those from other studies (Larkin, 1981, 1983; Larkin et al., 1980b; Simon & Simon, 1978; Sweller, 1988), suggest that student-directed problem-solving activities not only encourage the development of formulaic approaches to problem solving, but also are inefficient for promoting desirable problem-solving strategies.

Traditional approaches for teaching problem solving in physics may inadvertently result in students perceiving that solving physics problems is tantamount to manipulating equations. When worlung out a problem in front of a class a physics instructor may talk about the principles and procedures being applied but most often only writes the equations resulting from the application of the principles and procedures. Students erroneously conclude that qualitative information, such as principles and heuristics, are superfluous abstractions that do not help in solving a problem-manipulating equations, they reason, is what really gets you the answer.

Ironically, although physics students gravitate toward formulaic problem-solving approaches out of practical necessity, the HAT subjects in our study were not only capable of perceiving the value of the concept-based problem-solving approach in- corporated in the HAT, but also indicated their willingness to adopt it. These are conclusions based on responses to attitudinal questions asked of the EST and HAT subjects upon completion of Experiment 1. Most (1 1 out of 14) HAT subjects thought that their problem-solving skills had improved as a result of using the HAT, whereas fewer (6 out of 14) EST subjects thought likewise. The following representative responses to the question, “What did you like about the computer program?,” indicates the need to wean students from formulaic problem-solving approaches, but that this weaning may not be as hard as we anticipate:

EST subjects:

1 . “All the beautiful formulas” (emphasis added by student) 2. “Being able to simply look up equations instead of searching through a

book for them”

316 MESTRE ET AL.

HAT subjects:

1. “It [the HAT] is useful in that it requires a systematic approach to problem solving-develops good habits. This method exposes the fundamental principles involved in each problem. This is very instructive.”

2. “It [the HAT] made you think about the kinds of basic laws you’d be using. Also, it forced you to classify each type of problem.”

What classroom activities may help wean students from formulaic approaches for solving problems and help them integrate conceptual and quantitative knowledge? Problem categorization is a fairly simple activity that could be used to initiate classroom discussion on how to go about extracting the deep structure from a problem’s storyline. Asking students to perform qualitative analyses of problems and posing “what if” physical situations without resorting to writing equations can be used to ascertain students’ ability to draw on concepts to explain physical phenomena, and to promote classroom discussions on the role of principles and qualitative analyses in problem solving. Another activity that would encourage the integration of qualitative and quan- titative knowledge is to ask students first to write down a qualitative solution plan for a problem that describes the principles, concepts, and procedures that could be applied and then to execute the plan. In short, any activity where students actively engage in using conceptual knowledge as global “themes” and in tethering related information to these themes should help students structure their knowledge into cohesive chunks that facilitate the effective recall and use of the knowledge in problem-solving situations.

Work reported herein has been supported by grant BNS-8511069 from the National Science Foundation (NSF). The views expressed are those of the authors and do not necessarily reflect the position, policy, or endorsement of NSF. We would like to thank Ms. Shari Bell and Mr. Ian Beatty for their help in analyzing the data.

References

Chase, W.G., & Simon, H.A. (1973). Perception in chess. Cognitive Psychology,

Chi, M.T.H., kltovich, P.J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5 , 121-152.

de Groot, A. (1965). Thaught and choice in chess. The Hague: Mouton. Egan, D.E., & Schwartz, B.J. (1979). Chunking in recall of symbolic drawings.

Memory & Cognition, 7 , 149-158. Ehrlich, K., & Soloway, E. (1984). An empirical investigation of the tacit plan

knowledge in programming. In J. Thomas & M.L. Schneider (Eds.), Human factors in computer systems (pp. 113-134). Norwood, NJ: Ablex.

Eylon, B.S., & Reif, F. (1984). Effects of knowledge organization on task per- formance. Cognition & Instruction, 1, 5-44.

Glaser, R. (1992). Expert knowledge and processes of thinking. In D. Halpern (Ed.), Enhancing thinking ski& in the sciences and mthematics (pp. 63-75). Hillsdale, NJ: Lawrence Erlbaum Associates.

Hardiman, P.T., Dufresne, R., & Mestre, J.P. (1989). The relationship between problem categorization and problem solving among experts and novices. Memory & Cognition, 17, 627-638.

4, 55-81.

PROMOTING SKILLED PROBLEM-SOLVING BEHAVIOR 317

Hayes, J.R., & Simon, H.A. (1976). The understanding process: Problem isomorphs. Cognitive Psychology, 8, 165- 190.

Heller, J.I., & Reif, F. (1984). Prescribing effective human problem solving processes: Problem description in physics. Cognition and Instruction, 1, 177-216.

Hinsley, D.A., Hayes, J.R., & Simon, H.A. (1977). From words of equations: Meaning and representation in algebra word problems. In M.A. Just & P.A. Carpenter (Eds.), Cognitive Processes in Comprehension ( pp. 89- 106). Hillsdale, NJ: Lawrence Erlbaum Associates.

Larkin, J.H. (1980). Skilled problem solving in physics: A hierarchical planning model. Journal of Structural Learning, 6 , 27 1-297.

Larkin, J.H. (1981). Enriching formal knowledge: A model for learning to solve problems in physics. In J.R. Anderson (Ed.), Cognitive Skills and Their Acquisition (pp. 31 1-334). Hillsdale, NJ: Lawrence Erlbaum Associates.

Larkin, J.H. (1983). The role of problem representation in physics. In D. Gentner and A.L. Stevens (Eds.), Mental Models (pp. 75-98). Hillsdale, NJ: Lawrence Erlbaum Associates.

Larkin, J., McDermott, J., Simon, D.P., & Simon, H.A. (1980a). Expert and novice performance in solving physics problems. Science, 208, 1335- 1342.

Larkin, J., McDermott, J., Simon, D.P., & Simon, H.A. (1980b). Mcdels of competence in solving physics problems. Cognitive Science, 4, 3 17-345.

Newell, A., & Simon, H.A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.

Resnick, R., & Halliday, D. (1977). Physics. New York Wiley. Schoenfeld, A., & Herrmann, D.J. (1982). Problem perception and knowledge

structure in expert and novice mathematical problem solvers. Journal of Experimental Psychology: Learning, Memory and Cognition, 8, 484-494.

Simon, D.P., & Simon, H.A. (1978). Individual differences in solving physics problems. In R. Siegler (Ed.), Children’s thinking: What develops? (pp. 325-348). Hillsdale, NJ: Lawrence Erlbaum Associates.

Smith, M. (1992). Expertise and the organization of knowledge: Unexpected differences among genetic counselors, faculty, and students on problem categorization. Journal of Research in Science Teaching, 29, 179-205.

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257-285.

Touger, J., Dufresne, R., Gerace, W., & Mestre, J. (1987). Hierarchical organization of knowledge and coherent explanation in the domain of elementary mechanics. In J.D. Novak (Ed.), Proceedings of the Second International Seminar on Misconceptions and Educational Strategies in Science and Mathematics (pp. 517-530). Ithaca, N.Y .: Cornell University, Department of Education.

Manuscript accepted August 28, 199 1.