the effect of conditioned stimuli signalling food upon the ...

ACTA NEUROBIOL. EXP. 1976. 36: 277-310

THE EFFECT OF CONDITIONED STIMULI SIGNALLING FOOD UPON THE AUTOCHTHONOUS INSTRUMENTAL RESPONSES IN DOGS

S. SOETYSIK, J. KONORSKI, A. HOEOWNIA and T. RENTOUL

Department of Neurophysiology, Nencki Institute of Experimental Biology Warsaw, Poland

Abstract. Food CSs, presented (i) concomitantly with SD controlling bar pressing for food, or (ii) on the background of non-discriminated bar pressing (FR 1/15), suppress instrumental performance but elicit undimi- nished conditioned salivation. This result supports Sdtysik-Konorski's model of CNS mechanism controlling food-oriented behavior which postulates drive inhibition by taste-consummatory neurons.

INTRODUCTION

The effect of cl'assically conditioned food stimuli (FCSs) on the instrumental conditioned responses rewarded with food (i.e., autochthonous instrumental behavior = motivated by the same drive, rewarded with the same food US) was a subject of theoretical considerations in two different conceptual contexts: (i) interrelations between classical and instrumental conditioning mechanisms, and (ii) mechanism of incentive motivation. We shall briefly summarize the basic hypotheses and predictions pertinent tc these two theoretical contexts.

Classical-instrumental interrelations

The controversy over the status of instrumenrtal conditioning as a separate learning mechanism started in the early thirties when Pavlov (34, 35) was confronted with the ideas of Konorski and Miller (23, 24, 31) on two types d conditioned reflexes. For Konorski and Miller the classically conditioned signals of food (FCSs) were basically different from the stimuli eliciting the instrumental response (we shall call them SDs after Skinner). FCS was assumed to activate the "alimentary center" whereas, the SD was believed to be inhibitory in respect to the same

2 78 S. SOLTYSIK ET AL.

center. For Pavlov, both kinds of stimuli, FCSs and SDs were excitatory and were addressed to the same alimentary center. The instrumental response rewarded with food was considered by Pavlov as a special case of classical conditioned reflex in which, after the kinesthetic stimuli generated by rewarded movements have become the FCSs, th'e excitation from the alimentary center could move "backwards" along the "conditioned connection" to the kinesthetic center and from there to motor neurons. Thus, the instrumental movements would be elicited whenever the alimentarv center is activated, 'by conditioned or unconditioned stimuli. Pavlov's view was supported by the following authors: Skipin (41), Wyrwicka (53, 54), Fedorov (15), Yalkovleva (55), Beritoff (7) and most consistently by Asratyan (2-4).

An alternative hypothesis was proposed by Soltysik (42) and develcp- ed later by Soltysik and Klonorski (47) and Konorski (19). According to their view, FCS and S D have different properties because they are addressed to different parts of the "alimentary brain". FCS is classically conditioned to activate the taste-consummatory center(s) whereas SD, parallelly to the connection to motor center of the instrumental movement, has established a classically conditioned "connection" activating the hunger-drive center(s). It was hypothesized from earlier data of KO- norski and Miller (22-24) that the two parts of the alimentary system are interconnected in such a way that the drive facilitates consummatory behavior but consummatory behavior (or the consummatory stimuli, taste US and FCSs) inhibits the drive. This idea worked well in an artificiaI nerve net model of Konorski and Gawroriski (20, 21) and the present experiment is an attempt to verify it in the animal subjects. Two different predictions arise from Pavlov's and Soltysik-Konorski's modeIs of classical-instrumental interrelation wtthin the alimentary s v s t ~ m . when FCS and SD are presented concurrently, the instrumental behavicr should be (a) facilitated according to Pavlov's model, because both stimuli have basically identical effects on the alimentary center; (b) inhibited according to Soltysik-Konorski's model, because they are eliciting different behaviors, consummatory by FCS and hunger-drive, or appetitive, by SD. Since consummatory activity inhibits drive, which is the eliciting force behind the instrumental behavior in Soltysik-Konorski concept, the rate and amplitude of instrumental responses to S D in presence of FCS are expected to diminish. The first experiment in this study was aimed a t testing these two conflicting predictions.

Incentive motivation

It has long been hypothesized that not only the deprivation of food, causing the increase of hunger-drive, but also the properties of the food reward itself influence the performance of an instrumental response.

EFFECT OF FOOD css ON INSTRUMENTAL FOOD CRS 3 79

Rlany formulations of the possible mechanism of this influence were reminiscent of Pavlov's view that the consummatory excitation feeds into the motor system (Hull 1941, 1950, Spence 1947, 1956, Seward 1950, 1951, Bindra 1969). A typical hypothesizing was as follows. A given movement is rewarded with a food US. Therefore movement-generated stimuli become conditioned FCSs. The feedback of these conditioned consummatory responses (called "fractional antedating goal responses" in Hullian language, or re-sG) adds to the drive and energizes the ongoing instrumental behavior. The important difference between Pavlov's theory of instrumental reflexes and Hull-Spence-Seward concept of incentive motivation is that in the former, the alimentary center is the only activator of the instrumental movement while in Hullian system it is both drive and consummatory CRs that contribute to the eliciting mechanism and

Schematic representation of three views on the central mechanisms underlying the classically and instrumentally conditioned food-oriented responses.

Inputs Organism, CNP Outputs (stimuli, states) (hypothetical variables, (responses)

structural constructs)

Food CSs -- - -+ "Alimentary center" -+ Salivation, chewing, etc. (taste neurons)

~inesihetic cortex Instrumental response (motor analyser) --> (voluntary movement) 1

(?), Reward expectancy -+ Antedating goal responses 1

.- -- - - - - - - --

Food US, Food CSs - + Taste-consummatory -+ Salivat~on, chewing, etc. 3 center b

(food CRs) 1 - . 8 a i s Z

I "Pp;

.- Y C .- r, 2 : i 5 2 k

Bodily signds of depriva- Appetitive behavior, % tion, hunger CSs, SDs -+Hunger drive center --+ cn

instrumental response

Y

Organismic state of Drive, central motive Overt behavior, instru- +

=- d 5 5 = rn

hunger need - -+ state mental response ,

2 80 S. SOLTYSIK ET AL.

the stress is on drive while the incentive variable K only supplements the drive 1. All three concepts are represented schematically on Table I.

It is not quite clear whether the predictions for the FCS's effect upon the instrumental behavior rewarded with food would be identical by Pavlov's and incentive-motivation theories. While Hullian concepts, particularly in Spence's and Seward's formulations, postulate an energizing effect of FCSs on the ongoing instrumental activity, Pavlov's theory, although postulating even stronger causative relation between the consummatory FCSs and instrumental movements, may also invoke the reciprocal inhibition between different motor responses to explain the contrary results (see 26).

The Soltysik-Konorski model, however, predicts only H suppressive effect of the FCSs upon the instrumental behavior, whether it be in- cipient or ongoing. Therefore, the second experiment of this study was designed specifically to test the effect of classically conditioned food CSs upon the ongoing instrumental behavior.

Rational and general design of the experiments

The consummatory food conditioned stimuli (and responses) are considered to be either facilitatory (incentive-motivation view) or elicitatory (Pavlov's view) or inhibitory (Soltysik-Konorski view) in respect to the autochthonously motivated instrumental conditioned responses 2. Tc test the above conflicting views, in well trained dogs classical FCSs eliciting measurable and stable salivary reactions were presented in conjunction with the discriminative stimulus SD controlling an instrumental response (Experiment I), or were applied during the performance of non-discriminated instrumental responses (Experiment 11). The vigor of instrumental performance in the presence of the food CSs was compared with the vigor of the same performance in the absence of the FCSs.

To achieve this aim the animals were trained in the following con- secu tive phases:

1 I t is interesting to note that in Pavlov's theory the central representation of hunger drive is not entirely absent but it has not been recognized as a topically and physiologically separate process from the taste-consummatory excitation; Pav- lov regarded central correlate of hunger s tate as a n increased excitability of the taste neurons.

Most probably, it is neither stimuli nor responses but the central processes linking the stimulus and response of the consummatory act that initiate the inhibitory process in the drive neurons. But since the taste stimuli and FCSs a r e causative agents of these central "consummatory" processes, and, the responses a re peripheral outcomes, or, for the experimenter, indices, of these processes, t h e cor- relations between FCSs o r FCRs and the suppression of instrumental responses (considered as a n index of central drive processes) a re expected to be very high.

EFFECT OF FOOD css ON INSTRUMENTAL FOOD CRS 281

Phase A and B: classical and instrumental food conditioned reflexes were trained in two separate phases. In Experiment I classical conditioning preceded instrumental learning to press the bar. In Experiment I1 the instrumental bar pressing was taught before the phase of classical conditioning.

Phase C consisted in retesting of the responses acquired in the phase A.

In phase D the effects of presenting a classical FCS in the context of instrumental behavior were analysed. In Experiment I, one of the FCSs was presented in the middle of an instrumental session and the "generalization" from SD to this FCS was measured. In Experiment I1 one of the FCSs was presented on the background of the dog's bar pressing and the change in the rate of pressing in the presence of the FCS was measured.

Phase E (only in Experiment I) consisted in superimposing an SD on the pcsitive or inhibitory classical FCS.

EXPERIMENT I THE EFFECT OF FCSs ON DISCRIMINATED INSTRUMENTAL

AUTOCHTHONOUS RESPONSES

Material and methods

Subjects: Five adult mongrel male dogs were used in this experiment. They were experimentally naive but had spent a few months in a viva- rium where they had been inoculated against rabies, cleaned from ecto- and intestinal parasites and subjected to surgery on the Stensen's duct (48) with the aim of obtaining a shortened duct fistula of the parotid gland. In one dog, Fred, a scar formation around the orifice of the fistula could not be controlled and eventually both Stensen's ducts were closed. This dog exhibited good consummatory responses and the data from hinl were included in tables; the cardiac and motor response data were not included, however, in pictures showing the group averages. The dogs became acquainted with the personnel and habituated to the experimental CR chamber. They were fed once a day after a training session.

Apparatus: The dog was placed in a sound-proofed chamber on a Pav- lovian stand equipped with an automatic feeder providing access to reward consisting of a 75 g portion of ground meat with broth-soaked bread crumbs. A paddle wheel mounted to the right of the feeder served as a manipulandum; a wheel turn of 1.5 cm at its circumference activated a microswitch and was counted as an instrumentaL response. The wheel

-

3 This experiment was conducted jointly by Konorski, Rentoul and Soltysik, and the results were also briefly discussed elsewhere (43).

282 S. SOETYSIK ET AL.

automatically turned 90" and was ready for the next response. With this manipulandum the dogs were able to perform series of bar presses but could not continuously press the paddle. The wheel turn responses (called henceforth bar-presses), salivation from the parotid gland and the electrocardiogram were recorded on the Alvar Reega-IVcc portable polygraph using a paper speed of 15 mmls. Saliva was collected from the fistula into a drop counter which activarted a pulse generator connected to one channel of the polygraph. The collecting capsule was attached to the skin surrounding the orifice of the fistula with the hot Mendeleev's wax. Electrocardiographic record was obtained from electrodes placed on the animals' legs. For purpose of analysis, mean heart rate was calculated by measuring the interdbeat intervals during the 10 s control period just prior to the onset of the CS, during the CS-US intervals and during the first 10 s of eating. Auditory, visual and tactile stimuli served as conditioned, positive and inhibitory stimuli and are described in the next section.

Training procedures and results

A. Acquisition and establishment of classical consummatory conditioned food responses

After habituation to the experimental situation, which involved feeder training, attachment of EKG electrodes to the legs and wax-sealing the metal receptacle to the orifice of the salivary fistula on the dog's cheek, the dogs underwent a 6 mo training with a set of stimuli with different reinforcement contingencies. Metronome (120 beatslmin), tone (800 Hz, 85 db, delivered through a loudspeaker) and light (a flashing 100 Watt bulb, at the rate of 21s were always reinforced with the food US. They will be referred to as Mf, Tf and L+, respectively. A buzzer (Bz*) was reinforced only 50°/o of time, i.e., it was presented twice a day but only once was it followed by food reinforcement, randomly either on the first or the second application during the session. In two dogs (Ringo and Wag) it was the metronome which was reinforced of FR I/:! (therefore denoted Mf) while the buzzer was reinforced continuously (Bz'). Two other stimuli were never reinforced with food: a dfferential tone (T-), presented through the same loudspeaker with the same intensity (85 db) had a higher frequency (1,200 Hz) than T+; the second inhibitory CS was a sound of bubbling water (Bu-) and was very different from all the remaining stimuli not only in quality of sound but also in location, being delivered from behind the dog, while the former stimuli were presented in front of the animal. This was done in order to minimize the generalization from Tf , Mf or Bz+ and, indeed, Bu- did not elicit from the very beginning any noticeable salivation and was considered to be a "primary inhiibitor" according to Konorski's terminology (25, 19).


At the beginning of training the stimuli were presented for 2 s and almost immediately reinforced with food US. Only very gradually in the course of 6 mo the CS-US interval (and the duration of the CS+, which was terminated 2 s after the presentation of food) was extended to 20 s. This relatively long CS-US interval, had some adverse effect on the salivary CR which was attenuated by the inhibition of delay but was necessary for the planned testing of the FCS's effect on instrumental behavior. Each daily session consisted of 8 trials: M+, T+ and L+ were presented once a day and the five remaining trials were designated for two presentations of Bz* and three presentations of Bu-. Throughout the training the ratio of reinforced to nonreinforced trials was 4 : 4. The stimuli were presented in a quasi-random order, avoiding tripple strings of inhibitory trials and never presenting two inhibitory trials at the beginning of the session.

After the dogs reached a 20-s CS-US interval the differential stimulus T- was introduced once in each session replacing one Bu- trial. Within a month this stimulus was clearly an inhibitory CS- in respect to salivary CR.

Thus, in each dog six conditioned stimuli were well established at the end of this first phase of the experiment:

Bz' and T+ which were assumed to be strong positive FCSs, L+ which was assumed to be a weak FCS, M I (or Bz* in some dogs) which was hoped to be also a weak FCS because of the partial reinforcement schedule (FR l l z ) , T which was a differential CS relative to T+ and which had an "excitatory" property before the repetetive nonreinforced presentation rendered it into an inhibitory CS, and Bu- which was never an elicitor of salivary CR and was considered a primary inhibitor in Konorskian sense.

The phase of classical conditioning lasted 21 wk in Wag (78 sessions1 and 24 wk in the four other dogs (85 or 86 sessions). Salivary and cardiac responses recorded in the last few weeks of this training will be presented together with the data obtained during phase C when the same stimuli were retested after the dogs acquired a discriminated instrumental bar- press response.

B. Acquisition and establishment of an instrumental multiple bar press response (FR = 0.1) to a tactile discriminative stimulus SD

When the dogs acquired stable and reliable salivary CRs and good differentiation to T, the classical conditioning was discontinued and a manipulandum was introduced close to the feeder, in front of the animal and slightly to his right side. The training started with shaping a bar response and the dogs learned quickly to press the paddle with continu-


ous reward. Consecutively, a tactile stimulus on the right side of the dog's chest was trained as a discriminative stimulus SD. A Pavlovian "skin stimulator" (a touchie) was used for this purpose; it consisted of a small metal cylinder attached by a belt to the chest and pneumatically movable piston could be remotely forced out of the cylinder to make a contact with the skin. The touching surface of the piston was covered with short and blunt metal bristles. The stimulus initially elicited an orienting response but in the course of further training the dogs responded promptly with bar pressing usually without a trace of orienting response towards the touchie. A reward was delivered after the first bar press, performed after the onset of the SD. When the latencies of these responses became short and stable and the intertrial responses declined considerably, the training was continued with a FR 'in. The ratio of responses to reward was gradually increased up to 10 : 1 and the dogs learned to press the bar in a quick succession 10 times with the total response time (i.e., SD-US interval) seldom exceeding 10 s. One dog (Star), however, failed to learn this multiple bar pressing and remained on a one press per trial schedule. Therefore, he was excluded from the final testing on generalization and the interaction between SD and FCSs.

The phase of instrumental conditioning lasted over 50 sessions (Fred = 50, Pedro = 52, Ringo = 56, Wag = 57) except for Star who needed for his single bar press response only 39 sessions.

C. Retesting of classical FCSs after the dogs acquired a discriminated multiple instrumental response

When the dogs mastered the instrumental 10-press response to a tactile SD, the manipulandum was removed and the classical FCSs were retested in eight sessions. As previously, each session consisted of 4 positive and 4 inhibitory trials with quasi-random order of trials. The results are presented in Fig. 1-3.

Salivary responses. Figure 1 presents the mean rate and standard errors of salivary responses to all stimuli in four dogs. White bars represent scores from the last period of classical conditioning in the phase A, before the instrumental training; black bars represent the scores from the phase C of retesting the FCSs. First bars (US) show the amplitude of the unconditioned salivary response (in drops per 10 s) measured in the first 10 s of eating. Next three pairs of bars represent the salivary CRs to the positive FCSs in the following order:

Bz+ (or M+) denote buzzer (or metronome in some dogs) which was a strong auditory stimulus, regularly (FR 1) reinforced with food; Tf denotes another strong auditory FCS which in contrast to the

EFFECT OF FOOD CSs ON INSTRUMENTAL FOOD CRs 2%

first one possesses an inhibitory counterpart in T-, and therefore might be, possibly, weaker FCS than Bz+ (or M f ) ; L+ denotes light, a weak visual FCS. It has been long known that visual FCSs elicit in dogs smaller salivary CRs than auditory stimuli.

To compare the rate of conditioned salivation with the control intertrial secretion, the next (5th) pair of bars, labeled IT shows the average amount of drops of saliva recorded in the 10 s period just preceding the onset of trial. The 6th pair of bars labeled M* or Bzk shows conditioned

Fig. 1. Salivary responses to unconditioned and conditioned stimuli. Ordinates: drops of saliva per 10 s. Two columns for each stimulus. White column - average salivation during the la te period of the phase of classical conditioning, i.e., prior to instrumental training; black column - average salivation during the classical conditioning sessions after the instrumental learning. Vertical bars a t the top of each column a r e standard errors of means. Salivation was measured and computed for this graph only during the second half (10 s) of the stimulus duration. US, unconditioned stimulus (food); Bz, buzzer; M, metronome; T+, 800 Hz tone; T-, 1,200 Hz tone; L+, flashing lamp; Bu-, sound of bubbling water; IT, intertrial interval, or more accurately, 10 s period just preceding the onset of conditioned stimuli. The sign + means continuous reinforcement, FR 1; the sign - means non-

-reinforcement, and the sign k means reinforcement on FR

salivary CRs to an auditory FCS which was reinforced only on 50'O/o of their presentations. The last two stimuli, T- and Bu- were never reinforced with food and represent a differential FCS (T-) to which the salivary CR, generalized from the Tf had to be extinguished in the process of differentiation, and a "primary inhibitor" (Bu-) which did not elicit any salivary CR from the very beginning of its presentation.

286 S. SOLTYSIK ET AL.

The comparison of white and black bars shows no consistent change in responses before and after the acquisition of an instrumental response. Two more facts should be noted, however, in the Fig. 1. First is that in all dogs the intertrial salivation seemed to be increased after the instrumental training. Second astonishing observation is that the assumed different values of consummatory FCSs' "strength" or "reaction potential" were poorly reflected in the average salivary .CR's scores. There was no apparent difference between positive FCSs and even Lf or FCS* show scores close to those of the strong auditory FCSsf.

Anm. '60

Ringo ITheorf mh

20

0 b b

Pedro I t t I /rheort rote

Fig. 2. Heart ra te responses to unconditioned and conditioned stimuli. Ordinates: differences between heart ra te during the stimuli and the heart ra te in the 10 s period just prior to the stimulus onset. The first point of each curve (a small horizontal bar) is a starting point a t a prestimulus level. The second and the third points correspond to the heart ra te changes in the first and second 10 s halves of the stimulus duration. Solid lines represent the reactions a t the late period of a classical conditioning phase of experiment. Broken lines represent the cardiac reactions to the same stimuli in the phase of retraining of classical CRs after the dogs had learned instrumental responses to SD. Abbreviations the same as in Fig. 1. Right- -hand insert on each graph shows the average intertrial heart ra te during the phase of .classical conditioning (solid line column) and during the phase of retraining (broken line column). Note that in all four dogs the heart r a t e reactions dropp- e d considerably after the dogs were trained to press a bar, whereas the mean

heart ra te in the intertrial intervals remained the same.

EFFECT OF FOOD css ON INSTRUMENTAL FOOD CRS 28 T

There was, however, a definite difference between positive and inhibitory FCSs and the latter did not elicit any salivation in excess of the control intertrial "spontaneous" secretion.

It has to be reminded that the salivation elicited by FCSs was measured only in the second half of their 20 s duration and that this might have obscured the differences relative to different latencies and slower rise of the salivary responses.

Cardiac responses. Figures 2 and 3 show the heart responses to the same stimuli in all five dogs. The pre-stimulus rate is presented as zero on the ordinates and the deviations from this value in the first and second 10-s periods of each stimulus are mar'ked as black circles for the data

/Theart rate

Fig. 3. Heart rate responses to unconditioned and conditioned stimuli. Heart rate responses in the 5th dog (Star-left graph) and combined averages from all five

dogs (right graph) are shown in the same way as in Fig. 2.

from the original classical conditioning phase, or, as open circles for the data collected during the retesting after the dogs learned instrumental response. Since there were some consistent change3 in cardiac responses between these two phases of experiment, the average heart rate data from all the dogs is included on the right-hand graph of Fig. 3. It is evident, that the cardiac responses to all stimuli were considerably reduced after the instrumental training, and that this occurred without any change in the intertrial heart rate level: see the right hand insert on all graphs.

Inspection of the cardiac responses in individual dogs shows that they are not a simple concomitant component response of a consummatory CR. They do not correlate with salivary responses and may be lower during eating than during the positive FCSs (Wag); they may be even larger to inhibitory FCSs than to positive ones (Wag and Pedro). Also individual differences between dogs should be noted. Thus, while in four dogs L+ elicited little or no cardioacceleration as expected from our earlier study (46) it did elicit quite large response in Fred, compar- able to the responses to other FCSsf or to US. These facts will be discussed later.


D. Testing FCSs for "generalization" from SD to elicit the instrumental response

In the next 30 sessions the manipulandum was again present and the daily session consisted of six "background" trials with SD eliciting a 10-bar press response rewarded with focd. On two remaining trials the classical FCSs were administered for 20 sec in the same way as in the 1st and 3rd phases of training. Positive FCSsf were reinforced with food US while the FCS- were not. The dogs were allowed to press the bar during the FCSs but this did not influence the outcome of the trial which was treated as a classical conditioning trial. The various FCSs were randomly chosen for a given day but always one positive and inhibitory FCS was used. The FCSs were also randomly placed between the SD trials avoiding only the first trial which was always an instrumental trial in order to secure a response set for bar pressing from the outset of a session. Table I1 shows on what proportion of such inter- polated FCS trials did the dogs displayed even a partial (1 press) instrumental response within the 20 s FCS's duration. The Table requires

Instrumental responses occurring to FCSs in the phase of testing for "Generalization" - - - - - -

Number Number of Number Number of

Dcg FCS of test trials with 1 Dog FCS of test trials with t r i~ l s a bar piess 1 trials a bar press

- - . -

B + 9 1 M - 9 0 T+ 8 T - 7 0

Riilgo L- I 0 I L ' 10 2 I h l 17 0 E - 3 17 T- 8 0 I T- 7 0

- - B ~ I - .- --

14 -

0 ' - -

Bu- 14 - -

10 -

Bk 8 3 M + 10 0 T+ 9. 2 T+ 8 2

Wag L- 8 8 1 Fred L+ 8 0 M + 15 Bi 17 2 T- 9 0 T- 8 0

- BL-

- 14 0 Bu- 13 0

--- - - - - - - -- -- --

Percsntage of trials with a bar press

- . --- - --

All FCSs 257 3 3 12.8 All All FCS;+ 170 23 13.5 Dcgs All FCSs- 8 7 10 11 5

Auditory FCSsf 137 12 9.0


a little comment. First of all, i t has to be stressed that on most occasions of instrumental responding during the FCS it was a sinple bar press or two, so the response was incomparable with the energetic repetitive pressing during the SD. Only in Wag were some of the responses during L" similar in vigor to those elicited by an SD. Thus, the percentage scores should not be taken at their face values because they refer to the occurences of, however weak, instrumental response and would be smaller by a factor of 10 or more if the vigor of the bar pressing was included in the calculation of the scores. Note, that only L+ in Wag and Bu- in Pedro had higher occurrences of instrumental bar presses. It has to be reminded that these scores were obtained with FCSs' duration of more than twice the average SD's duration necessary to complete a 10-bar press response. ,

E. Testing the responses during concurrent presentation of FCSs and SD

Following the tests for generalization a new series of tests was run in which superimposing of SD upon FCS was executed. The design of a daily session was exactly the same as in the previous phase. The session was started with an SD trial and the remaining seven trials were randomly intermixed five SD trials and two test trials. The test trial started with an FCS and after 10 s the SD was added. Both stimuli were continued for 10 s and then discontinued if the FCS was of inwbitory type or reinforced with food US if the FCS was a positive one. The instrumental behavior did not affect the outcome of the trial. The rate of bar presses during 10 s of simultaneous presentation of FCS and SD compared to the rate of instrumental responses to the SD presented alone is shown on Fig. 4. The scores are presented separately for each stimulus in each dog. The first bar ( A ) which shows the number of bar presses during the first 10 s of FCS is in all cases zero, except for L+ in Wag where on one of the three L+/SD trials a single bar press occurred. The second bar (B) shows the average rate of pressing during the concomitant presentation of SD and FCS (its second half, as shown on a diagram above the graph of Ringo). This middle column should be compared with a third (black) column representing the average rate of responding to SD alone on trials just preceding and just following the FCS/SD test trials. The number of test trials in this phase was much smaller than in the preceding test (see numbers above the middle columns) because the dogs were sHowing a deterioration in performance after a few days of testing. The results from these tests are fairly consistent in all dogs. Except for L+ in Wag and Pedro, and, perhaps, Tf in Fred, a profound inhibition of in-


strumental response by FCSs was evident in all dogs. This suppression of bar pressing to S D was complete for FCSs- in Wag and Fred and for FCS* in all dogs. It was very strong for most of the FCSsf. In no case, however, a facilitation of instrumental behavior was observed. The sta- tistics for individual stimuli with Ns ranging from 1 to 6 seemed to us pointless and the overall tendency is so clear that it does not require any statistical confirmation.

Fred

15 10 I -, T-f B ~ i , B u ,

5

O A B C A B C A B C A B C A B C A B C

Fig. 4. The effect of classical conditioned stimuli on the rate of bar pressing. Ordi- nates are the differences of rates of pressing during the conditioned stimulus and during the 10 s pre-stimulus period (latter usually zero). A, rate of bar pressing during the first half (10 s) of an FCS acting alone. B, rate of bar pressing during the second half (10 s) of the FCS acting together with the discriminative stimulus (SD): see the diagram above Ringo's graph. C, rate of bar pressing on the trials just preceding and just following the test trials with combined FCSISD presentation. Note the profound suppression of instrumental performance (middle white column B) when the SD was presented simultaneously with the FCS, as compared to the vigorous bar pressing when the SD was presented alone (right black columns C). Note also the failure to inhibit bar pressing by visual FCS L+ in Pedro and Wag.

Further explanation in the text.

EFFECT O F FOOD CSs ON INSTRUMENTAL FOOD CRs

Comparison of the effect of simultaneous presentation of FCS and SD on the salivary conditioned responses

Ringo, Pedro and Wag were both g d salivators and instrumental performers, so it was of great importance to compare the changes in the performance of bar pressing and consummatory salivary CRs when both SD and FCSs were concurrently presented. Figure 5 shows the data

14 k Rinqo Pedro Instrumen fa1 instruments/

Fig. 5. Salivary conditioned responses and instrumental bar-press responses on t h e test trials. Ordinates a re rates of instrumental activity (upper graphs) or ra tes of salivation (lower graphs) in presses or drops of saliva per 10 s. The first black column on each graph represents average response to the SD alone. Other pairs of columns show t h e average responses to the auditory positive conditioned stimuli (FCSs +), to the inhibitory conditioned stimuli (FCSs-), to t h e partially reinforced (FR ' 1 2 ) conditioned stimuli (FCSk) and to the visual conditioned stimulus ( L S ). White, left-hand column of each pair represents the responses to the FCS presented alone. Black, right-hand column of each pair shows the responses to the same stimulus presented together with the S D . Note that : (i) ba r pressing is absent when t h e FCSs a r e presented alone (white columns on the upper graphs) and also a r e severely suppressed when the SD is presented together with the F C S (black columns on the upper graphs). (ii) L + failed to suppress bar pressing in two dogs, Pedro and Wag. (iii) Conditioned salivary responses to the compound FCSISD are either the same as to the FCS alone or a r c increased as in the case

of FCE+, FCS* and L + i n Ringo, Lf in Pedro and FCS* in Wag.

rearranged in such a way that the comparison is possible. In the upper graph the rate of bar pressing and in the lower graph the rate of salivation is shcwr, s2parately for auditory FCSs '. FCS-, FCS*, and L'. For each category of the FCS the left white bar represents the rate (of presses or drops of saliva, respectively) during the FCS presented alone

392 S . SOLTYSIK ET AL.

in the sessions consisting mainly of SD trials (from the phase D). The right black bar shows the rate of responses during concomitant occurrences of both FCS and SD. The instrumental responses are absent on FCS alone trials and absent or greatly reduced on FCS/SD trials. The salivary CRs, on the contrary, are not suppressed on FCS/SD trials and in some cases like FCS+ and FCS* in Ringo or FCS* in Wag, are evidently increased while the instrumental responses are suppressed. FCSs- suppressed both the instrumental responses and salivary CRs. The L' is the only inconsistent FCSf and therefore is shown separately. In Ringo it inhibited bar pressing to SD but at the same time it trippled the rate of conditioned salivation which became equal to salivary CR accompanying the instrumental performance to SD. In Pedro the suppression of bar pressing was negligible and the increase of salivation was also substantial. And in Wag, who has shown previously a tendency to press a bar in the presence of L+ alone, no suppression was observed but only a slight increase of salivation was noted.

Co?nparison of FCSs as conditioned sialogogic stimuli and suppressors of bar pressing

A disparity was found in some dogs between the FCS's capacity to elicit sizable salivary responses and a relatively small tendency to inhibit the instrumental performance. For instance, in Pedro and Wag, the visual FCS (L+) although eliciting a good salivary response, failed completely to inhibit bar pressing when presented simultaneously with the SD. It has to be added, however, that performing a bar pressing act was rather incompatible with targetting the sight on the lamp, which was placed in front and above the dog. The other stimuli have been found to follow the rule of correspondence between the rate of conditioned salivation and suppression of bar pressing. In Ringo, all positive FCSs elicited good salivary responses and suppressed the instrumental behavior to a compar- able degree. Food CSs reinforced 50°/o of time (M* and Bz*) did not differ from other stimuli in their salivary responses and the same was true for their capacity to inhibit bar pressing.

Changes in heart rate responses to concomitant presentation of FCSs and SD

Figure 6 shows the averaged cardiac .and salivary responses from Ringo, Wag and Pedro, separately for each FCS. The three bars for each FCS represent the scores obtained respectively: black bar - increase of heart rate during the SD; white bar - increase of heart rate during the second half of the FCS alone; dotted bar - increase of heart rate during the concurrent presentation of FCS and SD. What this graph shows is

EFFECT OF FOOD CSs ON INSTRUMENTAL FOOD CRs 2 93

that the heart rate changes accompanying the instrumental and classical salivary CRs are not correlated with either instrumental or salivary CRs. For instance, instrumental and classical responses were practically absent to FCS- or to the compound FCS-ISD but the heart rate responses on these trials were not smaller than to FCSA or SD. Generally, the cardiac responses to FCS/SD compounds seemed to match the cardiac responses to SD alone. Also the presence or absence of instrumental activity seemed to make no difference for the amplitude of a heart rate response. . lnstr. CS

class. CS Hearf rote El lnsk CSt Class. CS

Fig. 6. Comparison of salivary and cardiac responses on test trials with concurrent presentation of FCS and SD. Ordinates: In the upper graph, difference in heart beat per minute between the stimulus and the 10 s pre-stimulus period. In the lower graph, a difference between the rate of salivation (in drops per minute) during the stimulus and the 10 s pre-stimulus period. Black column: average response rate to the SD alone. Middle white column: average response rate to the conditioned food stimulus. Dashed column: average response rate to the compound stimulus SDIFCS. Note that: (i) Salivation to the compound stimulus is larger than lo the FCS+ alone. (ii) There is no correlation between salivation

and heart rate. Abbreviations the same as in Fig. 1.

Behavioral observations during the FCSISD trials

Visual observations of the animals' behavior on the test trials with superimposing of SD upon the FCS revealed that there was little or no orienting behavior to the unusual CS combinations. Usually the dogs responded as if the SD was not noticed or shifted from the FCS behavior (watching the feeder) to SD type behavior (turning to the bar and pressing degree. The dogs became restless, performed erratically and sometimes it). Soon, however, some disorders in behavior appeared, first in Fred and then in other dogs, with Ringo seemingly beeing affected in the smallest

2 - A c t a Neurobiologiae Experimentalis

2 94 S. SOLTYSIK E T AL.

even refused to eat. The tests had to be disc'ontinued and the number of compounded trials for each FCS was small and ranged from 0 (for L+ in Fred) to 6 (for Bz' in Ringo).

F. Some additional tests in Ringo

Since one of our dogs, Ringo, suffered very little from clashing together of FCSs and SD, and performed well after the FCS/SD trials were discontinued, an attempt was made to apply the FCSs and SD in other temporal combinations. In one, the test trials started as previously by presenting a FCSf and after 10 s the SD was added. Following the 10 s of this compound, during which little or no bar pressing was observed, the FCS+ was terminated while the SD was continued. Even then the bar pressing was very slow (one per 4-5 s) showing that the inhibitory effect of FCS' remained strong after its termination. In another test, the trials started with the SD and only when the dog started pressing the bar at his usual pace (one per second or faster) the FCSf was added. Despite the fact that the animal was already in the midst of performing the multiple instrumental response, the FCS+ had an immediate inhibitory effect. The bar pressing was completely suppressed and for the next 20 s, i.e., till the moment of delivery of food, there were no bar presses observed.

EXPERIMENT I1

THE EFFECT OF FCSS ON NON-DISCRIMINATED INSTRUMENTAL AUTOCHTHONOUS RESPONSES

In the second experiment similar procedures and phases of training were repeated in two dogs with the following changes:

1. a non-discriminated multiple bar press response was used instead of a discriminated instrumental response to SD,

2. instrumental conditioning took place before the classical conditioning phase,

3. conditioned food stimuli were tested on the background of the ongoing instrumental response.

The first experiment of this study has shown that classical food CSs, either positive or inhibitory, exert a powerful suppressive influence on discriminated instrumental behavior. This suppression of bar pressing was nearly complete for all the FCSs; therefore it was difficult to correlate the "strength" of the FCSs with the degree of suppression. The second

4 This experiment is a part of Dr Holownia's study to be reported in his Ph. D. thesis.

EFFECT OF FOOD CSs ON INSTRUMENTAL FOOD CRs 29 j

experiment was designed in the hope that with non-discriminated bar pressing the inhibition by FCSs might be less effective and that some correlation between the rate of conditioned salivation and the inhibition of instrumental response would be observed. Also, classical conditioning was carried out after the instrumental training, for the purpose of weak- ening the inhibitory effect of FCSs upon instrumental behavior.

In addition to the above considerations, the second experiment was designed to meet the possible criticism that in the first experiment the SD was always (except for a few trials in Ringo) presented after the dogs had started responding to the classical FCS. In many formulations, such as Hull's from 1941, (16) or Seward's from 1951 (38) the incentive motivational effect of the classical consummatory CR was assumedly to energize the ongoing instrumental response. It was decided, therefore, that in order to give the incentive motivation hypothesis a fair test, the FCSs should be presented during the performance of an instrumental behavior.

Training ~ T O C ~ ~ U T ~ S

A. Initial instr~:mental training

Two dogs (El and E2) were habituated to the situation cnci k::! 53l i~z- ry duct fistulas prepared prior to the bcglnr.iqg of treinirg. Initi-lly each bar press was promptly rewarded with food. Gradually, ovcr sor?e 35 sessions (of 9 rewards each), the number of responses required for the reward was increased to 10. After a few weeks of customary summer interruption in training, during the following 70 sessions the number of bar presses required for reward was further increased to 20. The relriard consisted of 75 g of ground meat mixed with bread cubes soaked in broth. Normally, the dogs needed 10-15 s to consuming that amount of food and they immediately resumed bar pressing at a fairly fast rstc. The period starting with the first post-reward bar press and ending with eating after the last bar press was considered a trial even though no discrete SD was used. Each daily session consisted of 9 trials.

B. Classical conditioning

When the dogs' performance of instrumental responses on FR 1/20 was stable, the manipulandum was removed and classical conditioning of the following stimuli has begun:

TL - an 800 Hz, 85 db tone as a positive FCS; L+ - a flashing 100 W bulb, also as a pasitive FCS; Tci - tactile stimulus placed on the dog's chest as a positive FCS; STS+ - a "specific tactile stimulus" (11, 12) applied to the forepaw which performed the bar pressing. We wanted to compare the effect


of this stimulus with the Tcf, because it has been shown by Dobrze- cka and Konorski (11, 12) that i t possesses special properties due to cortico-cortical connections between cutaneous sensations and movements of the same paw; M- - metronome, Bz- - buzzer were used as differential non reinforced FCSs-.

Initially, the CS-US interval was 2 6, but in the course of 46 sessions its duration was gradually extended to 10 s. The duration of FCSs- was equal to the CS-US interval. M- was presented without reinforcement from the very beginning of classical conditioning, whereas the other inhibitory stimulus, Bz-, was introduced only on 27th session when the CS-US interval was 6 s. Each positive FCS was presented and reinforced once in every session and the inhibitory FCSs- were presented once each every session. The order of presenting the trials was random. This phase of classical conditioning lasted 130 sessions. Both heart rate and salivation were monitored in every session; the third phase of experiment started only when the dogs reached stable performance of salivary conditioned responses.

C. Retraining of instrumental responses

Following classical conditioning, the instrumental responses were re- trained for 2 wk; a stable performance was reached with FR 1/15. This fixed ratio of 15 bar presses per reward remained unchanged during the consecutive tests.

D. Testing the FCSs during the performance of bar pressing

Testing the effect of FCSs upon the instrumental responses was carried out in the following way. The dogs had sessions 7 days a week. In the first 2 days the manipulandum was removed and only FCSs were presented as in the phase B. During the following 4 days, instrumental responses were trained on FR 1/15. Once during the session, on 4th, 5th or 6th trial, a FCSf was presented after a 5th bar press for 10 s and followed by a reward regardless of dog's behavior. Every FCS' was tested once a week. On the seventh day there was a pure instrumental session and on the 8th day the weekly sequence was started again. Each positive FCS+ was tested 5 times. After this 5 wk testing of FCSs' was completed, the inhibitory FCSs- were tested in the following way. They were similarly applied for 10 s, once per session on one of the middle trials, after the dog performed 5 bar presses (out of 15 required for receiving a reward). However, in this case the reward was not presented until the dog completed the series of 15 bar presses. Each FCS- was tested five times.


In either case, FCSf or FCS-, the rate of bar pressing in the presence of FCS was compared with the rate of bar presses on the trials just preceding and just following the test trials. Of course, the rates of pressing for responses 6th through 15th on normal trials were used for comparison because the FCSs were applied only after the 5th bar press.

1. Salivary and cardiac responses to FCSs

Figure 7 shows the salivary responses to the classical FCSs in three situations: (a) left white bars - mean salivary responses before testing; (b) middle black bars - mean salivary responses on special sessions (day 1 and 2) during the tests with FCSs+; (c) right dashed bars - mean salivary responses on special sessions during the tests with F C S s .

Fig.7. Salivary responses to classical conditioned stimuli. Ordinates: drops of saliva per 10 s. Three columns for each stimulus. White column (left), salivary responses a t the completion of classical conditioning phase of training. Black column (middle), salivary responses on pure classical conditioning sessions during the tests with FCSsS. Dashed columns (right), salivary responses on pure classicai conditioning sessions during the tests with FCSs-. US, food unconditioned stimulus; T+, tone; Lf, flashing lamp; Tc+, tactile stimulus on the dog's chest; STS+,

tactile stimulus on the dog's right forepaw; M-, metronome; Bz-, buzzer.


Dog E2 was a poor salivator so the comments will concern mainly E l . As in the experiment I there are very small differences between responses to different FCSs+ but a substantial difference is seen between FCSs-- and FCSs-.

Cardiac reactions are presented on Fig. 8. In both dogs the heart rate increase during the Tf was the largest of all responses to FCSs and in E2 it was even larger than the heart rate response during eating. The responses to Lf and Tcf were much smaller while the STSf elicited an intermediate response. Cardiac responses to inhibitory FCS- were quite small in E2 but fairly large in E l .

Fig. 8. Cardiac responses to classical conditioned stimuli. Ordinates: differences between the heart rate during the stimulus (of 10 s duration) and the heart rate in the 10 s pre-stimulus period. The first point of each curve (a small horizontal "fwt") is starting point a t the pre-stimulus level of heart rate. The second point is a change in the heart rate during the stimulus. Solid lines show the data from the period of classical conditioning and the broken lines illustrate the same responses in the classical conditioning sessions during the testing phase of experi-

ment. Abbreviations as in Fig. 7.

EFFECT OF FOOD CSs ON INSTRUMEN'TAL FOOD CIis 3 99

2. The effect of food conditioned stimuli on instrumental behavior

Figure 9 illustrates the changes in the rate of pressing the bar in the presence of FCSs. Only presses from 6th to 15th were used in this comparison. The scores are means of 5 trials for each FCS and are ex-

20

I0

I T' L' Tc'

Fig. 9. The effect of classical conditioned food stimuli on the rate of instrumental bar pressing. Ordinates: number of bar presses per 10 s. Three columns for each stimulus. Black columns (middle) represent an average (from 5 trials) bar press rate in the presence of FCS. The two white columns represent rates of

pressing on trials just preceding and just following the t.est trial.

pressed in number of bar presses delivered in 10 s period. It is evident that in most instances a suppression of bar pressing was observed: middle black bars, representing the rate of pressing during the FCS, are smaller than the white bars representing the rates of pressing on trials just preceding and just following the test trial. The degree of suppression varied from very high in case of STS+ in E2 to none in case of Tc+. In most cases the pressing rate was a t least halved. In three instances, however, no suppression of response was observed: both tactile stimuli in E l an a tactile stimulus to the chest in E2 failed to reduce the rate of instrumental responding. Paradoxically, the other tactile stimulus (STS+) in E2 had the most pronounced inhibitory effect. In no case was a clearly facilitatory effect found.

DISCUSSION

Two principal findings of this study are: 1. A sup-ression of instrumental responses by either positive or in-

hibitory food conditioned stimuli, and 2. A suppression of the instrumental response concomitant with 1 2 0

change cr facilitation of the salivary conditioned reaction when the


classical food conditioned stimuli (FCSs-) and a discriminative instrumental stimulus (SD) are simultaneously presented.

In the following discussion we shall 'briefly review the data from other studies and try to the conflicting evidence with our model of the neuronal mechanism of food oriented behavior 5. Afterwards we shall concentrate on some weaknesses of our results.

In the studies devoted to relations between classical FCSs and food- rewarded instrumental behavior two basic designs have been used. In one design, separately trained FCSs are presented on a background of spontaneous or discriminated instrumental food responses. Konorski and Miller (24) were the pioneers in this design and were also the first to note that positive FCSs+ inhibit performance of iestrumental responses. In the following four decades this type of experiment on the "transfer of control", with food US used as reward and as reinforcing US 6 in separate phases of training, had been undertaken several times and yielded various results. Facilitation of instrumental responses by FCSsf were often observed when long (13, 14, 28) or variable (32) CS-US intervals were used in the classical conditioning procedure. In one study (29) long FCS+ did not affect instrumental responses, however, and in another ( 5 ) , in which the quantity of food delivered as reinforcing US in classical conditioning phase was much larger than the amount of food presented as a reward of instrumental response, a definite suppression of instrumental performance by FCSf was dbserved. When FCSs were conditioned with short CS-US intervals, the suppression of instrumental behavior was observed (22, 24, 28, 29) and labeled recently as a "positive conditioned suppression" by Azrin and Hake (5). However, when food CSs conditioned with short CS-US interval were tested on the background of instrumental activity by presenting them and omitting or delay- ing the food US, then also the facilitation of instrumental responses was observed; it occurred in the period when normally the food US would have been consumed (18, 22, 39). We shall return to this phenomenon of "late facilitation" in further discussion.

Another design for testing the effect of FCSs upon the autochthonous instrumental behavior is to train a discriminative instrumental response using a previously established FCS as an SD and compare the rate of acquisition and the final performance to this FCS-made-into-SD with the acquisition rate and asymptotic level of performance to other SDs of different or no previous history. Again, the early case of such a "transfer of learning" design has come from Konorski (26) who found a strong

For a more detailed review and discussion see Soltysik's earlier paper (43). The terms "reward" and "reinforcement" are selectively applied to instru-

mental learning and classical conditioning, respectively.

EFFECT OF FOOD CSs ON INSTRUMENTAL FOOD CRs 301'

negative transfer in both measures. In other studies of this kind either no transfer (i. e., no effect) was found for FCSs employed as SDs (6, 51) or quasi-positive results were claimed (9, 52) when the transfer for p,ositive and inhibitory FCSs was compared. For the lack of "indifferent" or non- pretrained SD in their designs, the conclusion of a positive transfer cannot be accepted. It could have been, that in the last two studies the negative transfer to inhibitory FCSs was larger than the negative transfer to positive FCSs.

Thus there are the following phenomena to be accounted for:

1. Suppression of instrumental responses by the conditioned signals. of reward;

2. Facilitation of instrumental responses by the FCSs when either long CS-US intervals, or very small reinforcements, or irregular schedules of reinforcement were used in a phase of classical conditioning;

3. Suppression followed by facilitation of instrumental responses when the FCS was presented and not reinforced in due time or discontinued without reinforcement.

Let us consider some of the explanations offered when it became evident that the simple Pavlov's or Hull's hypothesis - of FCS being facilitatory in respect to instrumental food-rewarded responses - had not found unequivocal experimental support.

(a) Drive-reward oscillation hypothesis. Neal Miller suggested in 1951 (30) that although a stimulus-situation cannot be, according to a drive- reduction hypothesis, simultaneously both a drive and a reward, it could be, however, that the two effects, namely, a secondary reward (which reduces drive and, by inference, would suppress instrumental behavior) and secondary drive (when the food is not forthcoming) "could be pro- duced in rapid succession" (p. 463).

(b) Optimum drive level hypothesis. Mowrer (33) contended that the stimulus may reduce drive and facilitate instrumental behavior at t he same time if, under this lower drive level, the behavior becomes more organized and focussed, and "giving the appearance of 'increased drive', when, in fact, it is just the reverse phenomenon" (p. 272). Obviously this hypothesis, although assuming a unitary effect of FCS - a reduction of drive, is capable of explaining both an increase and a suppression of instrumental behavior, because it proposes that vigor of response is not monotonically related to intensity of drive. Its predictive power depends on the a priori assessment whether the subject's drive level at the moment of FCS's onset is above or below the optimum point.

(c) Frustration drive hypothesis. There are several versions of t he hypothesis that non-reward in a situation and at the moment when it is


expected, elicits an unconditioned frustration response (1) or state ( lo ) , or simply "a special sort of excitement" or "unconsummated consummatory arousal" (40), which represents a stimulus for action, i.e., a drive- like excitation. This explanation would account only for the case of suppression-facilitation sequence as, for example, when the duration d FCS is extended or FCS discontinued without being reinforced by food. It has been contended, however, that the post-consummatory facilitation of instrumental behavior is more simply explained by the rebound of the original drive (i. e., hunger drive) without invoking a special frustrative response (44, 45).

(d) Hypothesis of Drive-Reward regulatory circuit. As stated in the introduction, this hypothesis is based on a hypothetical neural circuit consisting of drive (appetitive) and consummatory (taste) elements (see Table I). The drive elements of a circuit are assumed to prompt appetitive and instrumental behavior, and also facilitate consummatory reflexes. The consummatory elements are assumed to inhibit the drive elements, secur- ing thus two effects: (i) immediate and transient inhibition of drive, which constitutes a reward of any preceding instrumental behavior; and (ii) termination of appetitive or instrumental behavior while the organism is engaged in consuming food.

This hypothesis explains the existing data satisfactorily and, in fact, predicts all three basic phenomena observed in experiments with food reinforcement:

1. Suppression of instrumental responses by FUS and FCSs eliciting consummatory responses. In all experiments where the conditions were favorable to the formation of a stable consummatory CR, a suppression of instrumental responses was observed. Little comment is needed for this prediction, since the present study and other recent papers (28, 29) clearly confirm it.

2. Facilitation of instrumental responses by FCSs which were paired with the food US irregularly or with long CS-US intervals. In such conditions, these CSs are paired more with the appetitive behavior than with consummatory response and therefore become drive CSs rather than food CSs. That the length and constancy of CS-US interval might be a decisive parameter for the CS becoming either food or hunger CS, was hyp- thesized in our earlier writings (19, 44, 47). It is very probable, according to this idea, that conditioned classical drive reflexes are formed more easily than consummatory CR and that they are readily acquired with long CS-US intervals or with irregular schedules of reinforcement (VI or VR), whereas consummatory CRs rely on short and fixed CS-US intervals. It was also speculated (44) that this difference between drive and consummatory responses, in respect to the parameters of pairing the CS

EFFECT O F FOOD CSs ON INSTRUMENTi2L FOOD CRs 303

with US, need not be related to the biological nature of the responses (i.e., to the drive or consummatory character of the responses) but to their "time base"; consummatory reflexes, as a rule are stimulus bound and have short latency, while the drive processes develop and last over considerably longer periods of time. Therefore the associative optimal intervals are short for consummatory responses, such as leg flexion, eye blink, salivation, and could be much longer or variable for fear (e, g., CER) and particularly long for very slowly developing processes as in case of "bait shyness" (36). By the same token, a hunger drive, which belongs to the category of slow and long lasting processes, should be conditionable with long CS-US intervals. As a matter of fact, the studies in which the CSs were paired with food in an irregular manner and with long CS-US intervals (which should have minimized formation of a consummatory CR) have demonstrated that such "food" CSs were facilitatory in respect to alimentary instrumental responses (13, 14, 28, 32).

Although the facts, both in our and in other authors' studies, are dealt with satisfactorily by our neural circuit model, the other explanations cannot be dismissed lightly. Miiier's (30) drive-reward oscillation hypothesis could be actually incorporated in our model, because the postulated circuit has the ability to switch from drive to consummatory activity and back to drive. The inhibitory connection between consummatory and drive centers makes such switching possible when a consummatory FCS+ is discontinued without reinforcement or when its duration is extended beyond the normal CS-US interval. The termination of conditioned (i.e., "secondary") reward releases the drive center from inhibition and the appetitive behavior (including instrumental responses) reappears. The recordings of conditioned salivation and instrumental responses (23, 24) often show switching from one type of behavior to another. The dog, which from the onset of an FCSf was waiting for food and salivating, suddenly stops to salivate and makes an instrumental response or two and begins to salivate again. Such an upsurge of drive, as evidenced by sudden instrumental activity, is clearly evident after a small portion of food. The "priming" effect of such small and free reward could be at- tributed to the rebound of drive which was temporarily inhibited by taste US (44). Similar rebounds could occur when the conditioned food stimuli are discontinued or prolonged without reinforcement. Many features of this hypothetical model were demonstrated on electronic mcdel of "alimentary centers" by Konorski and Gawronski (20, 21). Undoubt- edly, the next step in elucidating these problems must involve direct monitoring the activity of "drive" and "consummatory" neurons after they are unequivocally identified and localized in the CNS.

The need of direct recording from the "drive" and "reward" neurons


is even more evident in the case of Mowrer's (33) hypothesis of an inverse U shaped relationship between the drive level and the vigor of instrumental performance.

Let us retulrn now to the results of the present study and see in what respect they deviate form what might have been expected. As stated above, these results were generally very corroborative with our model. However, one observation was unaccounted for and needs some discussion. It was the apparent lack of correlation between the rate of conditioned salivation and the intensity of suppression of instrumental responses.

It was hoped that the FCSs of different modalities and reinforced with different fixed ratios would be evidently different in their capacity of eliciting food consummatory CRs. It was also assumed that these differences would be clearly reflected in the rates of conditioned salivation which would provide a measure of the strength of each FCS. Assuming a close correlation between the rate of salivation and the excitation of a consummatory "center", a correlation was also expected between the sialogogic property of FCSs+ and their ability t o suppress the instrumental behavior.

These expectations were not fulfilled. In the first place, the salivary responses rather poorly corresponded to the physical nature of the FCSs or to the conditioning procedure. For example, a flashing lamp or a tactile stimulus are traditionally considered "weak" stimuli in comparison with the "strong" ones, such as a tone or buzzer. However, the average salivation to tactile STS+ and auditory T+ in El was practically the same. Similarly close were the responses to L+ and Mf in Pedro. Also, the stimuli reinforced only 50Vo of the time elicited almost as much salivation as other auditory stimuli reinforced on each presentation: compare B+ and Mf in Wag or M+ and Bf in Pedro. Thus, one has to conclude that the asymptotic rate of conditioned salivation is not a perfect index of the "consummatory" strength of a food CS. It is also apparent that salivartion cannot be such an index when one realizes that the salivary secretion reflects the physical and chemical properties of food more than the intensity of the consuming activity. Wet and soft food of mild taste may produce very little salivation although the vigor of consummatory act may be quite strong. Another reason for similar salivary responses to different FCSs may be that what the experimenter considers the stimulus is only a part of a compound 'percept for the dog: an atten- tive and motivated dog reacts to a FCS by focussing his gaze on the feeder and this visual stimulus, combined probably with olfactory cues emanating from the feeder (and those are controlled by the dog's sniff- ing), plus the CS itself -constitute a compound food CS which elicits the salivary response. The sight of feeder and the smell of food are the


common components of all FCSs and they might be strong enough to obs- cure the differences between the FCSs themselves. Still, the auditory stimulus may have an edge because the dogs may hear it even when looking only at the foodtray, whereas the lamp, unless it is placed very close to the feeder, may diminish considerably as a perceived stimulus in the same circumstances. A tactile stimulus seems to 'be more like an auditory one because its perception is not physically handicapped by looking in different direction, but there is another factor which should be taken into consideration. A dog on a Pavlovian stand is restricted by the harness, which is a source of constant cutaneous stimulation, so the tactile CS has to be discriminated from numerous other nonreinforced tactile stimuli and this may reduce its potency as an FCS. Obviously an auditory stimulus in a sound-proofed chamber is in a more adventageous position because it "competes" only with noises prcduced by the animal itself.

The above analysis applies also to the fact that some of the FCSsf did not inhibit the instrumental behavior. It was visual L+ in Pedro and Wag, and tactile FCSsf in El that failed to suppress instrumental bar pressing. That the factor of attentiveness of the subjects, or, better, the salience of the stimuli have been of prime importance for the occurrence of inhibition of instrumental responses, is suggested by our heart rate data. The stimuli that failed as suppressors of bar pressing elicited rather small or even negative (i.e., reduction of heart rate) cardiac reactions. On the contrary, the auditory FCSs+ which never failed to suppress the bar press responses elicited the largest and most reliable heart rate accele- ratory responses. Although the picture is not flawless and some stimuli do not comfort exactly to the assumed correlation 'between suppression of instrumental behavior and the amplitude of cardiac response (cf. L' in Ringo and STS+ in El, which inhibited bar pressing although elicited rather meagre cardioacceleratory reactions), the general tendency is clear. When the stimulus activates the part of CNS which affects the heart rate (presumably by inhibiting vagal tone) it also possesses the capacity to suppress operant behavior. Could it be that in order to inhibit motivated behavior the stimulus has to have an access to the "emotive brain"? Or, if a cardiac response is reflecting the "salience" of the stimulus, is it easier to disregard a stimulus if it does not affect the cardiovascular system, i.e., does not arouse the ergotropic part of the nervous system?

Undoubtedly, a further research into these problems is needed before more satisfactory answers are provided. In the next studies the attention- a1 aspects of dog's behavior must be more carefully monitored, and a special analysis should be conducted on the problem of mutual compatibility between instrumental response and the targeting responses (19) to the , CSs, manipulandum and feeder.


Leaving open these unanswered questions, we would like to reaffirm our belief that, on the whole, this study rather convincingly supported our theoretical considerations on the structure and functioning of the food oriented behavior in dogs. It disproved the incentive motivation concept, in which the signals of reward were thought to be drive in- stigators. However, our model does not entirely reject the concept of reward being indirectly involved in creating an appetitive motivation. As elaborated elsewhere (44), the inhibition of drive neurons by the taste stimuli (and their conditioned substitutes, FCSs+) may result in post- inhibitory rebound of drive which is more specific and reflects the pat- tern of inhibitory influences of the taste (goal) stimuli. If this "specific" drive, or appetite, becomes conditioned to stimuli, such as SDs or CSs paired with food irregularly and with long CS-US intervals, then these stimuli are adding to the basic drive level and facilitate instrumental behavior 7.

The last comment is addressed to the fact that inhibitory FCSs- were equally good suppressors of instrumental activity as were the positive FCSs+. They were also good suppressors of conditioned salivation. This should be so, according to our model, because stimuli paired with no-food become inhibitory in respect'to both, consummatory activity and drive (see 19). Only stimuli paired with sudden recurrence of drive, such as in the case of a small "priming" reward, or with unexpectedly non-reinforced FCS+, become drive CSs and are capable of facilitating instrumental behavior.

SUMMARY

Classical salivary and cardiac reactions and instrumental motor responses were studied in two groups of dogs subjected to (i) classical conditioning with food US, (ii) instrumental conditioning of bar pressing rewarded with the same food US (iii) combined presentations of conditioned stimuli eliciting salivation with the discriminative stimulus, SD, eliciting bar pressing, or (iv) presentations of classical conditioned stimuli during the performance of free instrumental 'bar pressing.

In Experiment I instrumental !bar presses were discriminated and xcurred only in the presence of tactile SD. Ten consecutive bar presses were required for a reward. In Experiment I1 bar pressing was trained without a discrete SD and with a fixed ratio of 15 bar presses per reward. Salivary and cardiac responses to classical food CSs were measured in

For further discussion on this subject the Reader is referred to a n excellent chapter on Incentive motivation in Mackintosh's book (27).

EFFECT OF FOOD css ON INSTRUMENTAL FOOD CRS 30 i

order to estimate the conditioned consummatory character of these stimuli.

Instrumental responses were inhibited when the conditioned CSs, positive or inhibitory, were applied concomitantly with the SD or on the background of ongoing bar pressing. Conditioned salivation on these test trials remained unchanged or increased, obviating thus the possibility that presenting the food CS with the SD might have resulted in mutual suppressive interaction between them.

The results of this study strongly support the hypothesis of Soltysik and Konorski that consummatory food conditioned stimuli inhibit appetitive instrumental behavior, or, in terms of the neuronal model, taste-consummatory neurons inhibit the hunger drive neurons.

The authors wish to express their gratitude to Dr. George Wolfe for his proof-reading and comments on the manuscript. Particular thanks a r e due to Mrs. Alicja Boguslawska whose expert help in training animals made this s tudy possible. Mrs. Maria Raurowicz and Mrs. Steiania hfakowska helped In prepdring salivary fistulas. The authors a r e also very grateful t o Dr. Eric W. Holman for his interest in this study and valuab!e comments on related papers in American literature.

REFERENCES

1. AMSEL, A. 1967. Part ia l reinforcement effects on vigor and persistance: Ad- vances in frustraiion theory derived from a variety of within-subjects experiments. I n K. W. Spence and J. T. Spence (ed.), The psychology of learning and motivation. Acad. Press, New York, p . 1-66.

2. ASRATYAN, E. A. 1971. Physiology of reinforcement of the conditioned reflex (in Russiar.) Zh. Vyssh. Nerv. Deyat. 21: 3-13.

3. ASRATYAN, E. A. 1973. Some features of the present development stage of the conditioned reflex thecry (in Russian). Zh. Vyssh. Nerv. Deyat. 23: 262-278.

4. ASRATYAN, E. A. 1974. Conditional reflex theory and motivational behavior. Acta Neurobiol. Exp. 34: 15-31.

5. AZRIN, N. H. and HAKE, D. F. 1969. Positive conditioned suppression: Condi- tioned suppression using positive reinforcers a s t h e unconditioned stimuli. J. Exp. Anal. Behav. 12: 167-173.

6. BACON, W. E. and BINDRA, D. 1967. The generality of the incentive motivational effects of classically conditioned stimuli in instrumental learning. Acta Biol. Exp. 27: 185-197.

7. BERITOFF (BERITASHVILI), I. S. 1965. Neural mechanisms of higher verte- brate behavior. Little, Brown and Co. Boston. 384 p.

8. BINDRA, D. 1969. The interrelated mechanisms of reinforcement and motivation, and the nature of their influence on response. In William J. Arnold and David Levine (ed.), Nebraska symposium on motivation. Current theory and research in motivation. Vol. 17: 1-33.

9. BOWER, G. and GRUSEC, T. 1964. Effect or" prior Pavlovian discriminatioil


training upon learning and operant discrimination. J. Exp. Anal. Behav. 7: 401-404.

10. BROWN, J. S. and FARBER, I. E. 1951. Emotions conceptualized as intervening variables - with suggestions toward a theory of frustration. Psychol. Bull. 48: 465-495.

11. DOBRZECKA, C. and KONORSKI, J. 1962. On the peculiar properties of the instrumental conditioned reflqxes to "specific tactile stimuli". Acta Biol. EX^. 22: 215-226.

12. DOBRZECKA, C., SYCHOWA, B. and KONORSKI, J. 1965. The effects of lesions within the sensori-motor cortex upon instrumental response to the "specific tactile stimulus". Acta Biol. Exp. 25: 91-106.

13. ESTES, W. K. 1943. Discriminative conditioning - I. Discriminative property of conditioned anticipation. J. Exp. Psychol. 32: 150-155.

14. ESTES, W. K. 1948. Discriminative conditioning - 11. Effects of a Pavlovian conditioned stimulus upon a subsequently established operant response. J. Exp. Psychol. 38: 173-177.

15. FEDOROV, V. K. 1952. Basic principles of mutual influences between various motor reactions (in Russian) Fiziol. Zh. SSSR 38: 559-565.

16. HULL, C. L. 1941. Fractional antedating goal reactions as pure stumulus acts. In Psychological memoranda, 1941-1944. Bound mimeographical material on file in the libraries of the University of Iowa, University of North Carolina, and Yale University, Oct. 24, 1941.

17. HULL, C. L. 1950. Behavior postulates and corollaries - 1949. Psychol. Rev. 57: 173-180.

18. HYDE, T. S., TRAPOLD, M. A. and GROSS, D. M. 1968. Facilitative effect of a CS for reinforcement upon instrumental responding as a function of reinforcement magnitude: A test of incentive-motivation theory. J. Exp. Psychol. 78: 423-428.

19. KONORSKI, J . 1967. Integrative activity of the brain. An interdisciplinary approach. Univ. Chicago Press, Chicago. 531 p.

20. KONORSKI, J. and GAWRONSKI, R. 1970 An attempi a t modelling of the central alimentary system in higher animals. I. Physiological organization of the alimentary system. Acta Neurobiol. Exp. 30: 313-331.

21. KONORSKI, J. and GAWRONSKI, R. 1970. An attempt at modelling of the central alimentary system in higher animals. V. Instrumental conditioned reflexes. Acta Neurobiol. Exp. 30: 397-414.

22. KONORSKI, J. and MILLER, S. 1930. L'influence des excitateurs absolus et conditionnels sur les reflexes conditionnels de l'analysateur moteur. C. R. Soc. Biol. 104: 911-913.

23. KONORSKI, J. and MILLER, S. 1933. Podstawy fizjologicznej teorii ruch6w na- bytych. Ruchowe odruchy warunkowe. Ksiqinica Atlas TNSW, Warsaw. 168 p.

24. KONORSKI, J. and MILLER, S. 1936. Conditioned reflexes of the motor analyser (in Russian with English summary, p. 285-288). Tr. Fiziol. Lab. I. P. Pavlova 6: 119-278.

25. KONORSKI, J. and SZWEJKOWSKA, G. 1952. Chronic extinction and restoration of conditioned reflexes. IV. The dependence of the course of extinction and restoration of conditioned reflexes on the "history" of the conditioned stimulus. (The principle of the primacy of first training). Acta Biol. Exp. 16: 95-113.


26. KONORSKI, J . and WYRWICKA, W. 1950. Researches into conditioned reflexes of the second type. I. Transformation of conditioned reflexes of the first type into conditioned reflexes of the second type. Acta Biol. Exp. 15: 193-204.

27. MACKINTOSH, N. J. 1974. The psychology of animal learning. Acad. Press, New York, 730 p.

28. MELTZER, D. and BRAHLEK, J. A. 1970. Conditioned suppression and conditioned enhancement with the same positive UCS: an effect of CS duration. J. Exp. Anal. Behav. 13: 67-73.

29. MICZEK, K. A. and GROSSMAN, S. P. 1971. Positive conditioned suppression: effects of CS duration. J. Exp. Anal. Behav. 15: 243-247.

30. MILLER, N. E. 1951. Learnable drives and rewards. In S. S. Stevens (ed.), Handbook of experimental psychology. John Wiley and Sons, Inc., New York, p. 435-472.

31. MILLER, S. and KONORSKI, J. 1928. Sur une forme particuliere des reflexes conditionnels. C. R. Soc. Biol. 99: 1155-1157.

32. MORSE, W. H. and SKINNER, B. F. 1958. Some factors involved in the stimulus control of operant behavior. J. Exp. Anal. Behav. 1: 103-107.

33. MOWRER, 0. H. 1960. Learning theory and behavior. John Viley and Sons, Inc., New York. 555 p.

34. PAVLOV, I. P. 1932. The reply of a physiologist to psychologists. Psychol. Rev. 39: 91-127.

35. PAVLOV, I. P. 1936. Physiological mechanism of the so-called voluntary movements (in Russian). Tr. Fiziol. Lab. I. P. Pavlova 6: 115-118.

36. REWSKI , S. and GARCIA, J. 1970. Learned associations over long delays. In G. Bower (ed.), The psychology of learning and motivation: advances in research and theory Vol. 4. Acad. Press, New York, p. 1-84.

37. SEWARD, J. P. 1950. Secondary reinforcement as tertiary motivation: a revision of Hull's revision. Psychol. Rev. 57: 362-374.

38. SEWARD, J. 1951. Experimental evidence for the motivating function of reward. Psychol. Bull. 48: 130-149.

39. SHAPIRO, M. M. and MILLER, T. M. 1965. On the relationship between conditioned and discriminative stimuli and between instrumental and consummatory responses. In. W. F. Prokasy (ed.), Classical conditioning: A symposium. Appleton-Century-Crofts, New York, p. 269-301.

40. SHEFFIELD, F. D. 1965. Relation between classical conditioning and instrumental learning. In. W. F. Prokasy (ed.) Classical Conditioning: A symposium. Appleton-Century-Crofts, New York, p. 302-322.

41. SKIPIN, G. V. 1947. On the mechanism of formation of conditioned food reflexes (in Russian). Izdat. Sovetskaja Nauka, Moscow. 139 p.

42. SOETYSIK, S. 1960. On the intercentral connections in the reflex arc of the alimentary conditioned reflexes (in Russian). In E. A. Asratyan (ed.), Tsentral'nye i perifericheskie mekhanizmy dvigatel'noi' deiatel'nosti zhi- votnykh. Izdat. AN SSSR, Moscow, p. 58-60.

43. SOETYSIK, S. 1971. Drive and consummatory response: A neurophysiclogical approach. In D. A. A. Primrose (ed.), Proceedings of the Secand Congress of the Internat~onal Association for the Scientific Study of Mental Defi- ciency (Warsaw, 1970). Polish Med. Publ., Warsaw, p. 538-539.

44. SOLTYSIK, S. 1975. Post-consummatory arousal of drive a s a mechanism of incentive motivation. Acta Neurobiol. Exp. 35: 447-474.

3 - Acta Neurobiologiae Experimentalis


45. SOETYSIK, S. and GASANOVA, R. 1969. The effect of "partial reinforcement" on classical and instrumental conditioned reflexes acquired under continuous reinforcement. Acta Biol. Exp. 29: 29-49.

46. SOETYSIK, S., JAWORSKA, K., KOWALSKA, M. and RADOM, S. 1961. Cardiac responses to simple acustic stimuli in dogs. Acta Biol. Exp. 21: 235-252.

47. SOETYSIK, S. and KONORSKI, J. 1966. Relations between classical and instrumental conditioning Proc. XVIII Int. Congr. Psychol., Symp. 4: Classical and instrumental conditioning. Izdat. Nauka, Moscow, p. 66-73.

48. SOETYSIK, S. and ZBROZYNA, A. 1957. The chronic fistula of shortened Sten- sen's duct in dogs. Acta Biol. Exp. 17: 339-344.

49. SPENCE, K. W. 1947. The role of secondary reinforcement in delayed reward learning. Psychol. Rev. 54: 1-8.

50. SPENCE, K. W. 1956. Behavior theory and conditioning. Yale Univ. Press, New Haven.

51. TRAPOLD, M. A., LAWTON, G. W., DICK, R. A. and GROSS, D. M. 1968. Transfer of training from differential classical to differential instrumental conditioning. J. Exp. Psychol. 76: 568-573.

52. TRAPOLD, M. A. and WINOKUR, S. 1967. Transfer from classical conditioning and extinction to acquisition, extinction, and stimulus generalization of a positively reinforced instrumental response. J. Exp. Psychol. 73: 517-525.

53. WYRWICKA, W. 1952. Studies on motor conditioned reflexes. 5. On the mechanism of the motor conditioned reaction. Acta Biol. Exp. 16: 131-137.

64. WYRWICKA, W. 1972. The mechanism of conditioned behavior. A critical look a t the phenomena of conditioning. Charles C Thomas, Springfield. 179 p.

55. YAKOVLEVA, Y. V. 1952. Establishment of conditioned reflexes to a complex kinesthetic stimulus (to the so-called involuntary movements) (in Russian). Zh. Vyssh. Nerv. Deyat. 2: 305-316.

Accepted 6 June 1975

Stefan SOLTYSIK, Mental Retardation Research Center, Neuropsychlatrlc Institute 58 - 242C. University of California, School of Medicine, Los Angeles. California 90024, USA.

the effect of conditioned stimuli signalling food upon the ...

Documents

Transcript of the effect of conditioned stimuli signalling food upon the ...