Operant Conditioning on Laboratory Specimen

Running head: OPERANT CONDITIONING ON NORWAY SPECIMEN1

Operant Conditioning on Norway Specimen

Wesley R. Harrington

Rollins College

OPERANT CONDITIONING ON NORWAY SPECIMEN 2

Abstract

An experiment was conducted on a lab rat in a Psychology class at

a small liberal arts undergraduate institution. The experiment

focused on continuous and intermittent schedules of reinforcement

in determining which may be more efficient. Operant

conditioning, a behavioral shaping technique, developed by B. F.

Skinner, was conducted in the experiment. The rat was placed

into an operant chamber and shaped to press a bar within the

chamber and was given water as a reward. The rat was water

deprived for 23 hours prior to the experiment as the establishing

operation. Motivation was linked to the availability of the

stimulus. In this present experiment, shaping a response is

exhibited, also, how various shaping processes affect the rate of

response. Successive approximations allow for a more efficient

way of shaping a behavior of bar presses by the subject over

time. Intermittent schedules cause a response burst due to the

inconsistent delivery of reinforcement.


Operant Conditioning on Norway Specimen

B. F. Skinner introduced operant behavior and conditioning

by his classic demonstration of shaping pigeons in his operant

chamber or ‘Skinner Box’. He is considered the father of operant

conditioning because of his hard work and findings in the

prevalent field of operant behavior. In contrast to operant

conditioning, there is also classical conditioning, the behavior

is modified by contingencies between different stimuli (Personal

Communication). Most of Skinner’s work was built upon that of


Edward Thorndike and his ‘law of effect’. From that, Skinner was

able to formulate the concept of reinforcement. Behavior that is

reinforced is much more strengthened than behavior that is not

reinforced—such an instance as training a dog.

Skinner was able to use a cumulative recorder in order to

automatically record the data he collected from experiments as

opposed to Thorndike’s early concepts of data by hand and trial

by trial. Edward Thorndike was able to formulate his law of

effect and learn from a cat in a puzzle box through trial and

error learning. “The law went through many revisions, but its

essence was that response probability could be raised by some

consequences and lowered by others” (Catania, 2013, p. 58).

Much of Thorndike’s work was a building block for that of

Skinner’s work in operant conditioning.

Thorndike used the cat in a puzzle box and through the

reward of food; the cat underwent trial and error learning in

order to escape from the box. The time the cat took to escape

the puzzle box during each trial decreased. Skinner’s operant

chamber was much more efficient because it removed the direct

interaction from experimenter as opposed to Thorndike’s earlier


methods. Cats were better suited towards Thorndike’s work.

Skinner was able to use more animals rather than just one.

Various schedules of reinforcement such as continuous and

intermittent reinforcement are used in operant behavior and

conditioning. Different schedules are associated through

intermittent reinforcement while continuous reinforcement (CRF)

contains a continual shaping process. Catania (2013), describes

intermittent reinforcement as a ‘game of chance’, which provides

consequences (Catania, 2013, p. 197). Intermittent schedules

include ratio schedules, responses reinforced depending on the

number of responses emitted, variable responses, responses

reinforced that depends on time, and time schedules, responses

reinforced depending on the rate and timing of previous responses

(Catania, 2013, p.197). Reinforcement schedules can be based on

the time elapsed since the last reinforcer (Personal

Communication).

Continuous reinforcement reinforces every response during a

trial, which is a commonly used schedule of reinforcement; there

are other schedules that are more resistant to extinction

(Personal Communications). Continuous reinforcement, according


to Catania (2014), “is the exception rather than the rule” when

compared to intermittent reinforcement (Catania, p.197).

“Skinner’s discovery of what he later dubbed shaping, and the

impact that discovery had on his outlook on the determinants of

complex human behavior” (Peterson, 2004). Certain animals can be

shaped to respond in a certain way and rewarded for that specific

behavior. Thus, the animal is reinforced through an operant

behavior, which in turn could be a multitude of responses.

“Skinner used the method of successive approximation to develop

differentiated operant lever-pressing behavior in rats”

(Peterson, 2004). Shaping refers "the reinforcement of

successive approximations to a goal behavior" (Driscoll, 2000).

Skinner (2005) explained successive approximations through

his Analysis of Behavior using pigeons, he would only reinforce the

pigeon’s position successively closer to the desired spot, when

the head is moved slightly forward, and finally when the desired

spot is achieved (Skinner, 2005, p. 92). Essentially, the desired

behavior gradually becomes strengthened. Catania defines shaping

as “an alternative way in which reinforcements can create classes

of responses” (Catania, 2013, p. 112).


The shaping process involves multiple occasions such as:

establishing operations (EO), habituation, magazine training,

differential reinforcement of successive approximations, response

burst, and finally extinction. Establishing operations is the

motivation linked to the availability of the stimuli to the

animal such as water deprivation (Personal Communication).

Habituation is an initial process within shaping. Once the

animal is introduced to the operant chamber, the repeated

presentation of the stimuli causes a reduction in responding

(Personal Communication).

Magazine training is an important element in shaping because

it pairs the sound of the magazine with the delivery of water to

the animal (Personal Communication). The pairing from magazine

training is a classical Pavlovian responder or also called

classical conditioning. The differential reinforcement of

successive approximation guides the animal by means of

reinforcement for slightly similar topographical behavior to the

desired behavior, which is distinguished by the experimenter from

the beginning of the trial (Skinner, 2005, p. 97). The

continuous reinforcement of the behavior strengthens the


likelihood of such behavior, which results in a change to the

threshold to the behavior required for reinforcement (Skinner,

2005). When successive approximations are applied to differential

reinforcement, the animal learns by simply “doing it” (Personal

Communication).

The final part of the shaping process includes extinction.

Extinction is when procedures terminate both the reinforcer and

the contingency; it ultimately decreases responding (Personal

Communication). A main side effect of extinction is response

burst induction (Personal Communication). Extinction produces

induction—it creates variability in responses from which to

select the next approximation (Personal Communication). Response

bursts repeat the same behavior, response induction engages in

other related behaviors (Personal Communication).

Satiation and spontaneous recovery are also key terms when

applied to schedules of reinforcement. Satiation occurs when the

rat has had enough of the food or water and no longer desires

them (Personal Communication). Spontaneous recovery can

sometimes occur during extinction and is when behavior reoccurs


at the beginning of a following session and at the end of a

previous session (Personal Communication).

In this present experiment, shaping a response is exhibited,

also, how various shaping processes affect the rate of response.

Successive approximations allow for a more efficient way of

shaping a behavior of bar presses by the subject over time.

Intermittent reinforcement schedules include a fixed-ratio

schedule of five (FR5) and a variable-ratio schedule of five

(VR5). It is predicted that the intermittent schedules should

cause a response burst due to the inconsistent delivery of

reinforcement.

Method

Subject

The subject (RL011) was a female Sprague Dawley Rat (rattus

norvegicus) or (Norway rat). The subject was approximately four

months old at the time of the experiment and was born on July 17,

2014. The average weight of the rat during the beginning of the

experiment was 237.2 grams.

Apparatus


Skinner’s operant chamber was used in this present study.

The Animal Behavior Environment Test System (ABET) is a computer

system, which was used to control the chamber in which the

subject was within. The computer system was also used to record

real live data of the subject.

Procedure

IACUC, which stands for the Institutional Animal Care and

Use Committee, was approved protocol (#1002C), which was required

before caring and handling of the subject. The establishing

operation for this experiment was a 23-hour water deprivation

schedule. The duration of each session lasted for 30 minutes.

The participant and their lab partner started experiments with

habituation and observing the subject only. The next two

sessions were magazine training. The other two sessions

consisted of shaping and two more sessions of bar press

maintenance were to follow.

The next two sessions consisted of extinction, which was

followed by a return to bar press (CRF). Finally, the

experiments ended with three sessions of intermittent schedules

consecutively: FR 5, FR 5, and VR 5 schedules of reinforcement.


The subject used in this present experiment was used in operant

chambers. Using rats that are hungry or thirsty is more ideal as

the subjects are more motivated to work for the reward until they

are satiated. The subject was shaped through successive

approximations in order to strengthen the desired response, which

was in this case getting the subject to bar press in return for

water.

Results

The ‘move’ behavior had an unconditional probability of 33%

while the ‘other’ behavior demonstrated an unconditional

probability of 30% (Figure 1). The lowest recorded unconditional

probability was 0% for bar presses. Within the 30-minute

observation session that was separated into five-minute bins, the

behavioral velocity peaked in the fourth five-minute interval,

followed by the lowest behavioral velocity within the fifth five-

minute interval (see Figure 2). In self-oriented behavior, the

subject only exhibited behavior within the fifth five-minute

interval.

Unconditional probability was .2 in the first five-minute

interval for inactive behavior (see Figure 3). The second five-


minute interval also showed an unconditional probability of .2.

The third five-minute interval had an unconditional probability

of .3. No inactive behavior was observed in the fourth five-

minute interval. The fifth and sixth five-minute intervals both

demonstrated unconditional probabilities of .1.

For object-oriented behavior, the first and second five-

minute intervals both had unconditional probabilities of .14 (see

Figure 4). In the third five-minute interval the unconditional

probability was .21. In the fourth and fifth five-minute

intervals, the unconditional probabilities were .14. The last

five-minute interval did not have any behavior.

Spatially oriented behavior in the first and the sixth five-

minute intervals both had unconditional probabilities of .18 (see

Figure 5). The unconditional probability of the second five-

minute interval was .16. The third and fifth five-minute

intervals were both .12. The unconditional probability of the

fourth five-minute interval was .21.

During the shaping process, the number of pairings during

magazine training was 244. During continuous reinforcement

(CRF), the highest number of bar presses was 106. The highest


number of bar presses across sessions was 269 and the lowest was

0 (see Figure 6). In the first extinction session, the number of

bar presses was 30 (see Figure 7) and the second session was

reduced to 26 (see Figure 8).

Discussion

The intermittent schedules caused a response burst due to

the inconsistent delivery of reinforcement, which corresponded to

our hypothesis. Throughout the experiment, successive

approximations allowed for a much more efficient way of shaping

the subject of bar pressing over time. The subject also showed

some spontaneous recovery during extinction, therefore the bar

press behavior returned when it was predicted to go extinct.

The subject exhibited a lot of ‘move’ when classified under

its behavioral hierarchy. The highest average number of

behavioral velocity occurred within the 16 to 20-minute range.

The subject in this experiment had a constant behavior of

smelling and burrowing its nose into the bottom part of the

operant chamber, which could have caused some of the results and

shaping to be skewed. Real world applications that could relate

to this experiment could include owning a pet, a dog for example.


The dog would understand when to go and get food or water for

itself when necessary.

The subject was placed on a 23-hour water deprivation

schedule, which was the establishing operation. This schedule

allowed for the subject to become more motivated for water within

the operant chamber. The shaping process of the subject involved

multiple occasions such as: establishing operations (EO),

habituation, magazine training, differential reinforcement of

successive approximations, response burst, and finally

extinction. Without the establishing operation, the data and

results would not be sufficient enough.

The subject had the most amount of bar presses during a

fixed ratio five schedule of reinforcement (FR5) with an amount

of 269 bar presses. This number decreased during the next fixed

ratio five schedule of reinforcement (FR5) with a number of 243

bar presses. The experiment ended on a variable ratio five

schedule of reinforcement (VR5) with 202 bar presses. The number

of bar presses decreased each time from the start of a fixed

ratio schedule of reinforcement (FR) to the end of the variable


ratio schedule of reinforcement (VR), which was significant

because the behavior became extinguished overtime.

In the two separate instances of extinction, there appeared

to be a gradual increase of the amount of bar presses as time

increased. Both instances of extinction contained spontaneous

recovery. The amount of bar presses was lower during the second

instance of extinction compared to the first instance of

extinction. This comparison demonstrates that extinction

gradually becomes stronger when it occurs more frequently.


References

Catania, A C. (2013). Learning 5th Edition. Cornwall-on-Hudson,

NY: Sloan Publishing.

Drisoll, M. P. (2000). Psychology of Learning for Instruction.

2nd. Needham Heights, MA: Allyn and Bacon.

Peterson, G. B. (2004). A day of great illumination: B. F.

Skinner’s discovery of shaping. Journal of the Experimental Analysis

of Behavior, 82, 317- 328.

Skinner, B. F. Operandum. (1962). Journal of the Experimental Analysis of

Behavior, 5, 224.

Skinner, B. F. (2005). Shaping and maintaining operant behavior.

Science and Human Behavior, 91-106. Cambridge, MA: The B. F.

Skinner.


Table 1

Behavioral Taxonomy

Behavior Definitionbar press Subject depresses the lever operandum

sufficiently to trigger a monitoring light diode.


bar touch Subject touches the lever operandum with nose or

paw(s).

magazine

entry

Subject breaks the plain of the dipper/wall

barrier with nose.

object

touch

Subject touches lights, top latch, or wall screws

with nose or paw(s).

rest Subject shows no movement, other than fibrissa,

for sustained period (>3 seconds).

freeze Subject shows no movement, including fibrissa,

for sustained period (>3 seconds).

groom self Subject licks self or paws, including movement of

paws over nose or divides fur and bites at self

or uses hind foot to scratch self

move Subject moves at least one hind paw, thus

changing locations and/or orientations in the

chamber.

explore Subject moves upper body, but not hind feet, thus

changing orientations and/or levels in the

chamber. One forepaw may be raised from the floor

in this activity, but not both at the same time.


rear Subject raises both forepaws off the floor in

upright exploration, but remains fixed in the

placement of both hind feet.

other Subject engages in a behavior that is not

defined above.

Figure 1. Behavior Hierarchy

The graph shows that the highest unconditional probability for

behavioral hierarchy is ‘move’ and the second highest is ‘other’.

0%

5%

10%

15%

20%

25%

30%

35%

Behavior

Unco

ndit

iona

l Pr

obab

ilit

y


0-5 min 6-10 min

11-15 min

16-20 min

21-25 min

26-30 min

01234567

Time (minutes)

Numb

er O

f Be

havi

ors

Figure 2. Behavioral Velocity

The graph shows number of behaviors by the subject for each five-

minute interval. The highest number of behaviors for this

subject occurred during the 16-20-minute interval.


0-5 min

6-10 min

11-15 min

16-20 min

21-25 min

26-30 min

-0.2

-1.66533453693773E-16

0.2

0.4

0.6

0.8

1

1.2

Time (minutes)

Unco

ndit

iona

l Pr

obabil

ity

Figure 3. Inactive Behavior

The graph shows inactive behaviors by the subject. The subject

had an unconditional probability of zero during the 16-20-minute

interval. Inactive behaviors included resting and freezing.


0-5 min

6-10 min

11-15 min

16-20 min

21-25 min

26-30 min

-0.2

-1.66533453693773E-16

0.2

0.4

0.6

0.8

1

1.2

Time (minutes)

Unco

ndit

iona

l Pr

obab

ilit

y

Figure 4. Object-Oriented Behaviors

The graph shows the unconditional probabilities of the subject on

object-oriented behaviors. Object-oriented behaviors included

bar pressing, bar touching, magazine entry, and object touching.


0-5 min 6-10 min

11-15 min

16-20 min

21-25 min

26-30 min

0

0.2

0.4

0.6

0.8

1

1.2

Time (mInutes)

Unco

ndit

iona

l Pr

obab

ilit

y

Figure 5. Self-Oriented Behaviors

The graph shows the unconditional probabilities of the subject

during self-oriented behaviors. Self-oriented behaviors

consisted of the subject self-grooming.


0-5 min

6-10 min

11-15 min

16-20 min

21-25 min

26-30 min

-1.66533453693773E-160.20.40.60.81

1.2

Time (minutes)

Unco

ndit

iona

l Pr

obab

ilit

y

Figure 6. Spatially-Oriented Behaviors

The graph shows unconditional probabilities of spatially-oriented

behaviors. Spatially-oriented behaviors included moving,

exploring, and rearing.


0 200 400 600 800 1000 1200 1400 1600 18000

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150

Time (seconds)

Bar Presses

Figure 7. Cumulative Recorder Graph

The graph above demonstrates successful shaping to bar press

prior to extinction.


0 200 400 600 800 1000 1200 14000

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150

Time (seconds)

Bar Presses

Figure 8. Extinction Session One

The removal of water delivery as a reinforcer.


Figure 9. Extinction Session Two

The removal of water delivery as a reinforcer.

0 200 400 600 800 1000 1200 14000102030405060708090100110120130140150

Time (seconds)

Bar Presses


CRF CRF CRF CRF CRF CRF EXT EXT CRF FR5 FR5 VR50

50

100

150

200

250

300

Session

Bar Presses

Figure 10. Bar Presses Across Sessions

After the extinction sessions, when returning to a continuous

reinforcement schedules, the amount of bar presses returned to

the approximate same amount of bar presses prior to extinction.

Operant Conditioning on Laboratory Specimen

Documents

Transcript of Operant Conditioning on Laboratory Specimen