This article appeared in a journal published by Elsevier. The attachedcopy is furnished to the author for internal non-commercial researchand education use, including for instruction at the authors institution

and sharing with colleagues.

Other uses, including reproduction and distribution, or selling orlicensing copies, or posting to personal, institutional or third party

websites are prohibited.

In most cases authors are permitted to post their version of thearticle (e.g. in Word or Tex form) to their personal website orinstitutional repository. Authors requiring further information

regarding Elsevier’s archiving and manuscript policies areencouraged to visit:

http://www.elsevier.com/authorsrights

Author's personal copy

Behavioural Processes 95 (2013) 100–112

Contents lists available at SciVerse ScienceDirect

Behavioural Processes

journa l homepage: www.e lsev ier .com/ locate /behavproc

On the joint control of preference by time and reinforcer-ratiovariation�

Michael Davison ∗, Sarah Cowie, Douglas ElliffeThe University of Auckland, New Zealand

a r t i c l e i n f o

Article history:Received 13 August 2012Received in revised form 27 January 2013Accepted 1 February 2013

Keywords:Fixed interval performanceVariable-interval performanceTimingLocal choiceExtended choiceMatching

a b s t r a c t

Five pigeons were trained in a procedure in which, with a specified probability, food was either avail-able on a fixed-interval schedule on the left key, or on a variable-interval schedule on the right key.In Phase 1, we arranged, with a probability of 0.5, either a left-key fixed-interval schedule or a right-key variable-interval 30 s, and varied the value of the fixed-interval schedule from 5 s to 50 s across 5conditions. In Phase 2, we arranged either a left-key fixed-interval 20-s schedule or a right-key variable-interval 30-s schedule, and varied the probability of the fixed-interval schedule from 0.05 to 1.0 across8 conditions. Phase 3 always arranged a fixed-interval schedule on the left key, and its value was variedover the same range as in Phase 1. In Phase 1, overall preference was generally toward the variable-interval schedule, preference following reinforcers was initially toward the variable-interval schedule,and maximum preference for the fixed-interval schedule generally occurred close to the arranged fixed-interval time, becoming relatively constant thereafter. In Phase 2, overall left-key preference followedthe probability of the fixed-interval schedule, and maximum fixed-interval choice again occurred close tothe fixed-interval time, except when the fixed-interval probability was 0.1 or less. The pattern of choicefollowing reinforcers was similar to that in Phase 1, but the peak fixed-interval choice became morepeaked with higher probabilities of the fixed interval. Phase 3 produced typical fixed-interval scheduleresponding. The results are discussed in terms of reinforcement effects, timing in the context of alter-native reinforcers, and generalized matching. These results can be described by a quantitative model inwhich reinforcer rates obtained at times since the last reinforcer are distributed across time accordingto a Gaussian distribution with constant coefficient of variation before the fixed-interval schedule time,changing to extended choice controlled by extended reinforcer ratios beyond the fixed-interval time. Thesame model provides a good description of response rates on single fixed-interval schedules.

This article is part of a Special Issue entitled: SQAB 2012.© 2013 Elsevier B.V. All rights reserved.

Discriminative control over behaviour by differential stimulirequires differential reinforcement with respect to the stimuli.Commonly, research uses two or more discrete stimuli such ascolors, line orientations, or times since events. Less commonly,except in research on temporal control, differential reinforcementis changed with respect to a continuous change in a stimulus

� We thank the students enrolled in Psych 309 Learning in 2010 and 2011, whoconducted this experiment as their laboratory requirement, supervised by SarahCowie. We also thank the members of the Experimental Analysis of BehaviourResearch Unit who helped run this experiment out of term time, and Mick Sib-ley who looked after the pigeons. The research was approved by the University ofAuckland Animal Ethical Committee (Approval TR-909). Reprints may be obtainedfrom Michael Davison (email: m.davison@auckland.ac.nz).

∗ Corresponding author at: School of Psychology, The University of Auckland CityCampus, Private Bag 92019, Auckland 1142, New Zealand.Tel.: +64 9 373 7599x88540; fax: +64 9 373 7450; mobile: +64 2102675101.

E-mail address: m.davison@auckland.ac.nz (M. Davison).

dimension. In research into behavioural control by time, a numberof different procedures have been used: (1) conditional discrimi-nation (symbolic matching), in which one duration is mapped toone response, and the other duration to the other response—a 1:1procedure; (2) procedures in which a continuous change in elapsedtime is mapped onto two responses according to a criterion value(bisection procedure); and (3) procedures in which a continuouschange in elapsed time is mapped onto a continuous change indifferential reinforcement for two responses. Because differentialcontrol by stimuli requires differential reinforcement, the rein-forcers obtained (rather than arranged) in the procedure are critical,and cannot be simply assumed to be maximally differential. Thispoint was made forcefully by Davison and Nevin (1999) for condi-tional discrimination, and needs to be made equally forcefully forstudies of control by elapsed time. Davison and Nevin suggested aquantitative model that converted the arranged reinforcer contin-gencies with respect to both stimuli and responses into effectivereinforcer contingencies, those contingencies that are experienced

0376-6357/$ – see front matter © 2013 Elsevier B.V. All rights reserved.http://dx.doi.org/10.1016/j.beproc.2013.02.001

Author's personal copy

M. Davison et al. / Behavioural Processes 95 (2013) 100–112 101

by the subject. What affects the relation between arranged andeffective contingencies is the degree to which animals can discrim-inate the conditional stimuli, and the degree to which they candiscriminate the contingencies of reinforcement.

These considerations are highly pertinent to the study of controlof behaviour by elapsed time. The ability of animals to discrimi-nate elapsed times decreases with increasing time according to ascalar property: the error in estimating the elapsed time increaseswith the time so that the ratio of the error to the time (the coeffi-cient of variation, CV) remains approximately constant. Accordingto Davison and Nevin (1999), this will result in an animal’s percep-tion of the reinforcer contingencies becoming progressively lessaccurate as time elapses—and if this happens, the contingenciesarranged by the experimenter will become progressively less pre-cise across elapsed time. As a result, it is not possible to see datafrom timing experiments as representing pure timing with precisedifferential reinforcement because the arranged, precise, differen-tial reinforcement is not maintained. Data from experiments ontiming are always about both timing and reinforcement.

The relation between effective and arranged differential rein-forcers is dynamical, with effective differential reinforcement anddiscriminative control mutually affecting each other. Clearly, pro-viding differential reinforcement with respect to stimuli may notresult in differential stimulus control if either the differential rein-forcer contingencies or the differential stimuli are indiscriminable.In general, to the extent that either the contingencies or the stim-uli are difficult to discriminate, differential stimulus control willbe incomplete. Time since the last food on standard concurrentvariable-interval schedules does not usually signal any changesin obtained food ratios, except when changeover delays (COD:Herrnstein, 1961) are arranged—when these are used, the localobtained reinforcer ratio immediately after a food reinforcer on akey is infinite towards the just-productive alternative for the dura-tion of the COD. Given the scalar nature of timing, timing will beaccurate in this period, and we would expect local control overchoice. This has been reported (Davison and Baum, 2002).

The scalar nature of timing would suggest that reinforcer differ-entials at longer elapsed times would not control choice, and thishas recently been reported. Cowie et al. (2011) investigated theeffects of local reinforcer-ratio differentials across time since fooddelivery. They found that preference tracked imposed local differ-entials when these differentials were signalled by food delivery.In their experiment, the next reinforcer was always equally likelyto be on either key, but the probability that the next food wouldbecome available soon after the last food, or late, on each key wasmanipulated across conditions. Control was strongest, and lastedthrough the next inter-reinforcer interval, when any food signalledthat food would probably occur sooner on a particular key. It wasless strong when food on a key signalled likely sooner food on thejust-productive key, and less strong again when food on a key sig-nalled the next sooner food was likely on the not-just-productivekey. In both these latter cases, the post-food preference differentiallasted for only 10–20 s. Taking these results together, control overlocal choice by local reinforcer differentials across time since fooddepended on the complexity of the signal: control of choice acrosstime was most accurate when both reinforcers signalled the samecontingency changes across time, and accurate when each rein-forcer signalled opposing contingencies across time, but sooner onthe just-productive key; and less accurate again when each rein-forcer signalled opposing contingencies with the sooner reinforcerlikely on the not-just-productive key.

The present experiment investigated a situation in whichobtained food ratios following reinforcers were always infinitelytoward one alternative, as in concurrent schedules that arrangea COD. We arranged a procedure like that of Cowie et al. (2011),but after each food delivery, with a particular probability, the next

schedule was either a fixed-interval (FI) schedule on the left key, oran exponential (constant–probability) VI 30-s schedule on the rightkey. Thus, no left-key food deliveries could be obtained for the dura-tion of the FI schedule. In Phase 1, we investigated post-food choicewhen the FI-schedule values were varied from 5 s to 50 s, with VI30 s on the right key, and with the probability of the FI schedulealways 0.5. This procedure is similar to that used by Church et al.(1998) in their Experiment 2. In Phase 2, we arranged FI 20 s on theleft key, VI 30 s on the right key, and varied the probability that eachwould be presented after food. We investigated how both choiceand obtained-food ratios changed across time after food. In Phase3, we arranged extinction on the right key, and the same seriesof FI-schedule values as in Phase 1 on the left key, and measuredresponse rates on the FI schedules.

Some discussion is in order here about exponential VI schedules,sometimes called random-interval schedules. These are arrangedby querying a probability gate set at (in this case) 1/30 every 1 s,thus arranging a constant conditional probability of food becomingavailable. Such a schedule provides an exponentially decreasinglocal food frequency over time since food (see Elliffe and Alsop,1996, Fig. 7)—in the first 1 s, the probability is 1/30; in the secondit is (29/30) (1/30), and in general the probability of food avail-ability in the tth second is (1/30)e−t/30. Thus, the longer the FIschedule, the higher should be the FI-to-VI food ratio at the time atwhich the FI schedule finishes timing. This would not be the casewith an arithmetic VI schedule, which arranges a constant localfood frequency over time since food. Using single exponential andarithmetic VI schedules, Catania and Reynolds (1968) showed thatpost-food responding quickly reached a constant rate on exponen-tial schedules, but increased more slowly (but to a higher rate in 2of 3 pigeons) on arithmetic schedules, perhaps reflecting the higherconditional probability of food at longer times since the last food onarithmetic schedules. Church and Lacourse (2001) compared singleexponential and Wald VI schedule performance (a Wald scheduleprovides an increasing, then decreasing probability of food acrosstime since food). They showed clearly that local response rate afterfoods followed the probability of food deliveries. This result againpoints to a signalling effect of food reinforcers in conjunction withcontrol by time since food. Such post-food effects, it would seem,have not been explicitly investigated in concurrent schedules, butChurch and Lacourse’s Fig. 4 seems to imply that, in concurrentlyarranged Exponential VI Wald VI schedules, there would be a strongpreference toward the Exponential schedule just after food, fol-lowed by a preference toward the Wald schedule, followed againby indifference. However, the requirement of remembering the lastfood delivery on each key, and separately timing from both of these,might preclude precise control of post-food choice. A simpler pro-cedure, as used here and by Cowie et al. (2011), is to present eachschedule probabilistically, so that timing is simply from the lastreinforcer, on whichever key.

Some concurrent-schedule procedures will make such local con-trol unlikely by requiring independent timing from the last foodon each key, but some will promote such an effect by requiringonly timing from the last event. An instance of the latter wouldbe dependent concurrent VI VI scheduling in which the probabil-ity of food on a key is set at 0.5, but the average intervals on eachkey are varied across conditions. We believe that this procedurehas not been reported in the literature. If the two schedules wereVI 10 s and VI 120 s, then when 10 or 20 s have elapsed with nofood, the next food becomes more likely on the second alterna-tive. This result is functionally the same as Ferster and Skinner’s(1957) demonstration of “priming the ratio” in mixed schedules.Thus, on mixed shorter–longer fixed-ratio arrangements, animalsfirst respond appropriately to the shorter schedule, but if food isnot forthcoming, they then cease responding, and later commenceresponding appropriately to the longer schedule.

Author's personal copy

102 M. Davison et al. / Behavioural Processes 95 (2013) 100–112

Finally, as Church and Lacourse (2001) pointed out, respond-ing on a single schedule is a form of temporal-search foraging, theanimal’s task being to search at an appropriate time since the lastfood delivery. The present procedure adds a location aspect to thesearch: The animal’s task is to search efficiently in two patcheswhen the temporal characteristics of the patches are different—asin concurrent schedules. To what extent can and do environmentalevents such as food delivery lead to temporal and locational controlof responding following such events, and if and when these effectsoccur, how do they affect extended choice and the way in whichthe process of reinforcement is to be viewed?

1. Method

1.1. Subjects

Five pigeons, numbered 161–165, were maintained at85% ± 15 g of their ad-lib body weights by supplementary feedingof mixed grain at 9.30 am daily. These pigeons had served in aprevious experiment (Davison and Elliffe, 2010), but Pigeon 165’sdata were not reported there due to some problems maintainingits behavior. Grit and water were available in their home cages, inwhich the experiment was conducted, at all times.

1.2. Apparatus and procedure

The apparatus was the same as used by Davison and Elliffe(2010). The experimental chambers, which were also the pigeons’home cages, were 300 mm on all sides, with the rear and both sidessheet metal, and the top, front, and base open bars. On the rightpanel were 3 response keys that could be transilluminated variouscolors, with a food magazine containing wheat below the centerresponse key. Only the left and right keys were used in the presentexperiment, and these were illuminated with white light when thekeys were operative. The room in which they were situated hada shifted day/night cycle, with the room lights turned on at 12midnight, and off at 4 pm. The sessions for all pigeons started at2 am.

The experimental sessions were arranged using MED-PC IV soft-ware running on a remote PC compatible computer. The time atwhich every event (responses emitted and food deliveries obtainedon each key) at a time resolution of 0.01 s were collected for furtheranalysis. Each condition lasted for 40 daily sessions, and the dataused for all analyses were taken from the last 20 sessions of eachcondition.

Sessions commenced with the two side keys lit white. A proba-bility gate, set at the values given in Table 1, determined whetheran FI schedule would be arranged on the left key, or a VI scheduleon the right key. The various conditions contributed to 3 experi-mental phases: Phase 1, in which the probability of the FI schedulewas 0.5, and the FI-schedule value was varied over conditions from5 s to 50 s; Phase 2, in which the FI schedule value was 20 s and theprobability with which it occurred was varied over conditions from0.05 to 1.0; and Phase 3, in which the probability of the FI schedulewas 1.0 (no VI schedule reinforcers were arranged), and the FI-schedule value was varied from 5 s to 50 s as in Phase 1. Extinctionwas arranged on the right white key in Phase 3. Some conditionscontributed to more than 1 phase of the experiment (Table 1).The VI-schedule intervals were determined exponentially—every1 s, a probability gate was interrogated with a probability of 0.033(VI 30 s) to determine whether the VI schedule would arrange areinforcer for the next response. Following each food delivery, theprobabilities shown in Table 1 again determined whether the nextfood would be arranged after a fixed time on the left key, or after avariable time on the right key.

Table 1Sequence of experimental conditions for Phases 1, 2, and 3.

Condition Phase FI (s) VI (s) p(FI schedule)

1 1, 2 20 30 0.52 1 10 30 0.53 1 40 30 0.54 1 5 30 0.55 1 50 30 0.56 2 20 30 0.97 2 20 30 0.38 2 20 30 0.79 2 20 30 0.1

10a 1, 2 20 30 0.511 2 20 30 0.9512 2 20 30 0.0513 2,3 20 – 114 3 10 – 115 3 40 – 116 3 5 – 117 3 50 – 118 1, 2 20 30 0.5

a Conditions 10 and 18 were replications of Condition 1.

Reinforcement was 3-s access to wheat in the lighted hopper,during which the keys were darkened. Sessions ended after 80 suchdeliveries, or after 45 min, whichever occurred first.

2. Results

The data used in analyses were from the last 20 sessions of eachcondition. Generally, response and reinforcer numbers were col-lected in 2.5-s bins for all conditions except those arranging FI 5 s,for which 1.25-s bins were used. From these data, we calculatedlocal response and reinforcer rates in each bin, and analyzed onlydata obtained on trials in which VI food deliveries were arrangedin Phases 1 and 2. The reason for not using FI-trial data was that,of course, these data were truncated at, or after, the FI-schedulevalue, beyond which the schedule became continuous reinforce-ment. It is worth noting here that responding on FI trials beforethe arranged FI schedule had timed out was indistinguishable fromthis part of the performance on VI trials. However, in Phase 3, inwhich only FI schedules were arranged, we show the FI-trial data forcomparison with those from Phases 1 and 2. We will present dataaveraged across the 5 pigeons in the text, and show some individualdata in the Appendix. Generally, group data were representative ofindividual-pigeon data.

An important point to be made prior to the data analyses is thatoften foods arranged by the FI schedule were not collected as soonas the FI schedule completed timing. The proportions of total foodreinforcers obtained across time since food on the FI schedules areshown in Fig. 1. Longer FI-schedule values, and lower probabilitiesof FI 20-s schedule trials, resulted in proportionally more delayedFI-food deliveries. The proportions of delayed FI-food deliveriesvaried unsystematically in Phase 3. Thus, the accuracy of the FI-time signal was variable with a tail to the right that depended onboth the value and probability of the FI-schedule. This effect is dueto extended visits to the right, VI-schedule, key before switchingto the left key in Phases 1 and 2, and to visits comprising extrane-ous behaviour in all 3 phases (Davison, 2004), and has been foundbefore (Church and Lacourse, 2001). In general, the right tail in thetiming signal (that is, the time presented to the animal in a timingexperiment) will depend on the value of the payoffs associated withFI-schedule responding relative to the value of all other availableresponses, and this systematic error in stimulus presentation willoccur to some lesser or greater extent in all timing experimentsusing fixed-interval schedules.

Author's personal copy

M. Davison et al. / Behavioural Processes 95 (2013) 100–112 103

PHASE 1

0 20 40 60 80

0.0

0.2

0.4

0.6

0.8

FI 5 s

FI 10 s

FI 20 s

FI 40 s

FI 50 s

PHASE 2

20 25 30 35 40

FO

OD

PR

OP

OR

TIO

N

0.0

0.2

0.4

0.6

0.8

p(FI) = .05

p(FI) = .1

p(FI) = .3

p(FI) = .5

p(FI) = .7

p(FI) = .9

p(FI) = .95

PHASE 3

SECONDS AFTER FOOD (2.5-s BINS)

0 20 40 60 80

0.0

0.2

0.4

0.6

0.8

1.0

FI 5 s

FI 10 s

FI 20 s

FI 40 s

FI 50 s

Fig. 1. The proportion of all food deliveries received on the FI schedule as a functionof time since the last FI food delivery. For Phase 2 (FI 20 s), for clarity, the X axishas been scaled from 19 to 40 s. Because times since food were binned in 2.5-sintervals, foods were obtained in the bin FI interval to FI interval + 2.5 s, and plottedat FI interval + 1.25 s for all conditions except those for FI 5-s and FI 10-s schedules,where a 1.25-s bin was used. The data shown were averaged across the 5 pigeons.

We conducted two general analyses—local response rate as afunction of time since food, and local choice as a function of thesame variable.

2.1. Local response rates over time since food

Condition 1 was replicated in Conditions 10 and 18, and Fig. 2shows mean response rate on the left and right keys on VI trialsin all 3 conditions. Fig. A1 shows data from the individual pigeons.The data from Conditions 10 and 18 closely replicated those fromCondition 1, though there is some evidence of increasing accuracyof FI timing in Condition 18. The peak in FI-schedule respondingwas close to the arranged FI 20-s value for all 3 conditions.

Fig. 3 shows local response rates in Phase 1 in which the prob-ability of the FI schedule, p(FI), was 0.5, and the FI-schedule valueswere varied from 5 s to 50 s. Response rates on the FI key peakedat the FI-schedule values for values of 5, 10 and 20 s, but responserates for FI 40 s and FI 50 s increased progressively across theseintervals, showing that timing was becoming less accurate. Thus,timing was less accurate at FI 40 s and FI 50 s with ill-defined peaksat about 40 s for the FI 40-s schedule, and beyond 50 s for the FI50-s schedule. It must be noted here that the frequency with whichpigeons reached longer times after foods decreased with time since

COND 10FI 20 s

RE

SP

ON

SE

S/M

INU

TE

0

10

20

30

40

50

COND 18FI 20 s

SECONDS AFTER FOOD

0 20 40 60 80

0

10

20

30

40

50

B LEFT

B RIGHT

COND 1FI 20 s

0

10

20

30

40

50

Fig. 2. Replication conditions. Group mean response rates on the left (FI) and right(VI) keys in successive time bins following food deliveries on VI trials.

food (we used an exponential VI 30-s schedule on the right key),so that response-rate estimates become progressively less accu-rate with time since reinforcement. Thus, the response-rate peakin Condition 5 at around 72 s may be more apparent than real.

There was a reciprocal relation between FI-schedule and VI-schedule responding, with VI rates being lowered around theFI-schedule value. This is most evident in the FI 5-, 10-, and 20-sconditions.

Fig. 4 shows response rates in Phase 2 when the probability ofthe FI schedule was varied from 0.05 to 0.95. Condition 13 (Table 1)is potentially part of this phase, but as there were no VI trials inthis condition, there were no comparable data to plot from thiscondition. As the probability of the FI schedule was increased, FI-schedule temporal control became more accurate (no control wasevident at p(FI) of 0.05 and 0.1), and remained at close to 20 s. How-ever, FI responding after the FI-interval had elapsed also increasedprogressively with p(FI) and with the complementary decrease inp(VI). Of course, the decrease in p(VI) decreased the relative value ofthe sum of behaviors other than FI responding, which may accountfor the increase in FI responding after the FI interval. VI respondingdecreased progressively with the increase in p(FI), and the decre-ment in VI responding around the FI interval is again evident. Itshould be noted that the reliability of response-rate measures also

Author's personal copy

104 M. Davison et al. / Behavioural Processes 95 (2013) 100–112

COND 1FI 20 s

SECONDS AFTER FOOD

0 20 40 60 80

0

20

40

60

B LEFT

B RIGHT

COND 2FI 10 s

RE

SP

ON

SE

S/M

INU

TE

0

20

40

60

COND 3FI 40 s

COND 4FI 5 s

0

20

40

60

COND 5FI 50 s

SECONDS AFTER FOOD

0 20 40 60 80

Fig. 3. Phase 1. Group mean response rates on the left (FI) and right (VI) keys insuccessive time bins following food deliveries on VI trials.

decreased as p(FI) was increased—for instance, in Condition 11when p(FI) was 0.05, only 5% of all trials provided the measuresused here.

In Phase 3, there were no VI trials, so Fig. 5 shows responserates on FI trials as a function of time after food. Note that responserates before the FI schedule value are comparable with the data inFigs. 2–4, but response rates after the FI interval had elapsed arenot comparable because every response would produce food. Fig. 5shows the typical pattern of response rates increasing during theFI interval, reaching a maximum at the interval value. However,decreasing accuracy of timing with increasing FI value was evi-dent in increasing FI responding at times less than the FI interval,and changing curvature in the response-rate increase. Right-keyresponding was effectively absent.

2.2. Local choice over time since food

Figs. 6–8 show local left/right choice as a function of time sincefood delivery for, respectively, the 3 replications (Conditions 1, 10and 18), for the Phase 1 conditions, and for the Phase 2 conditions.Individual pigeon data from replication Conditions 1, 10 and 18are shown in Fig. A2. Choice measures are not meaningful for thePhase 3 conditions. Fig. 6 shows that replicated conditions closelyfollowed the pattern of choice change in Condition 1, though prefer-ence following the fixed interval was more toward the VI schedulein Condition 18, as we noted in Fig. 2. In all 3 conditions, peak choiceoccurred close to 20 s after food. Fig. 7 (Phase 1) shows that peak

COND 8p(FI) = .7

COND 7p(FI) = .3

RE

SP

ON

SE

S/M

INU

TE

0

20

40

60

COND 9p(FI) = .1

0

20

40

60COND 6p(FI) = .9

COND 10p(FI) = .5

SECONDS AFTER FOOD

0 20 40 60 80

0

20

40

60

COND 11p(FI) = .95

SECONDS AFTER FOOD

0 20 40 60 80

COND 12p(FI) = .05

0

20

40

60

B LEFT

B RIGHT

Fig. 4. Phase 2. Group mean response rates on the left (FI) and right (VI) keys insuccessive time bins following food deliveries on VI trials.

choice was accurately located at the FI value in FI 5, 10, and 20 s (andperhaps FI 40 s), but that the peak value fell progressively with FIvalue, and choice became more equally distributed over time sincefood, with increasing FI value. Similarly, Fig. 8 (Phase 2) showedthat peak choice became higher and more accurately located withincreasing FI-schedule probability.

2.3. Choice: extended analyses

Figs. 9 and 10 show log response ratios across conditionsin Phases 1 and 2 for each pigeon and for the mean of the 5pigeons. For Phase 1 (Fig. 9), choice is shown as a function ofthe FI value. For 4 of the 5 pigeons and the group, choice fellwith increasing interval size, and overall this decrease was signif-icant on a nonparametric trend test (replications averaged, N = 5,k = 5, z = −7.1; Ferguson, 1971). In Phase 2, the relative frequencyof food deliveries was varied between the left and right keys, sohere choice is plotted as a function of obtained log food ratios(Fig. 10). The fits of the straight lines were good, with between88 and 96% of the variance accounted for across individuals. Forall pigeons, and for the mean data, sensitivity to reinforcement(the slopes of the lines) was less than 1.0, and bias (the interceptof the fitted lines) was negative—that is, preference was biasedtowards the VI-schedule key (Trevett et al., 1972; Lobb and Davison,1975).

Author's personal copy

M. Davison et al. / Behavioural Processes 95 (2013) 100–112 105

COND 16FI 5 s

0

20

40

60

80

100

B LEFT

B RIGHT

COND 15FI 40 s

COND 14FI 10 s

RE

SP

ON

SE

S/M

INU

TE

0

20

40

60

80

100

COND 13FI 20 s

SECONDS AFTER FOOD

0 20 40 60 80

0

20

40

60

80

100

COND 15FI 50 s

SECONDS AFTER FOOD

0 20 40 60 80

Fig. 5. Phase 3. Group mean response rates on the left (FI) and right (VI) keys insuccessive time bins following food deliveries on VI trials.

3. Discussion

3.1. Some general considerations

There are two ways in which the schedules arranged herecan be viewed. First, they can be seen in an extended sense, inwhich we might expect, according to the generalized matchinglaw, that the extended allocation of behavior between the keyswould be a simple linear function of the extended log reinforcerratio between the keys. In Phase 1, the sessional obtained logreinforcer ratio was close to zero throughout, so this approachwould suggest that choice should be close to indifferent and notchange with changes in FI-schedule value. However, as Fig. 9showed, choice for the FI schedule significantly decreased as theFI-schedule interval was increased. In Phase 2 (Fig. 10), relativeobtained food frequencies on the 2 keys changed across condi-tions, and log response ratios did follow log obtained food ratiosas would be expected from generalized matching (Baum, 1974;Taylor and Davison, 1983). Generalized matching was thus aninaccurate description of overall choice in Phase 1, but a gooddescription in Phase 2, at least in terms of the change in choiceas a function of the change in relative reinforcers (the sensitiv-ity parameter in generalized matching). In this view, the changein FI-schedule values in Phase 1 would simply change the over-all reinforcer rate—for instance, with an FI 20-s schedule on theleft key, the overall time to reinforcers on both keys would be0.5 × 20 + 0.5 × 30 = 25 s. With FI 50 s, it would be 40 s. Under this

COND 10FI 20 s

LO

G L

EF

T/R

IGH

T R

ES

PO

NS

E R

AT

IO

-3

-2

-1

0

COND 18FI 20 s

SECONDS AFTER FOOD

0 20 40 60 80

-3

-2

-1

0

COND 1FI 20 s

-3

-2

-1

0

Fig. 6. Log left/right response ratios as a function of time following food delivery forthe 3 conditions in which FI 20 s was arranged on the left key. The horizontal lineshows equal preference.

approach, both left and right keys would have the same meantime between reinforcers in the present procedure so that, accord-ing to generalized matching, choice should be constant and closeto indifferent. But in Phase 1 we did find a bias to the VI sched-ule that was similar to that reported by Trevett et al. (1972) andLobb and Davison (1975) using standard concurrent VI FI sched-ules, in which times to reinforcers on the keys changed when theFI or VI schedules were changed. We also found the same VI biasin Phase 2, in which times to reinforcers on the keys did changeas we varied the probability of presentation of the FI versus VIschedule when the FI and VI schedules were kept constant. Thefinding of a bias towards a VI schedule and away from an FI isthus invariant across three different procedures. What is commonacross these procedures is differences in FI- versus VI-key inter-reinforcer-time distributions—there is always a longer, and fixed,minimum inter-reinforcer time on the FI-schedule key (see alsoElliffe and Alsop, 1996). If it is this that causes the generalized-matching bias, then the changes in preference with increasing FItime in Phase 1 when the minimum inter-reinforcer time on theFI key was varied (Fig. 9) is understandable. These results suggesta binary effect: the existence of a minimum inter-reinforcer inter-val biases choice a constant amount to an alternative that does nothave such a minimum. A further, and more general, interpretationmight be that if contingencies of reinforcement support differentialresponding with respect to time, choice will be biased away froman alternative on which the passage of time signals the absence of

Author's personal copy

106 M. Davison et al. / Behavioural Processes 95 (2013) 100–112

COND 1FI 20 s

SECONDS AFTER FOOD

0 20 40 60 80

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

COND 2FI 10 s

LO

G L

EF

T/R

IGH

T R

ES

PO

NS

E R

AT

IO

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

COND 3FI 40 s

COND 4FI 5 s

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

COND 5FI 50 s

SECONDS AFTER FOOD

0 20 40 60 80

Fig. 7. Phase 1. Log left/right response ratios as a function of time following fooddelivery. The horizontal line shows equal preference.

reinforcers on that alternative, and maybe toward an alternativeif the passage of time signals additional reinforcers on that alter-native. The latter occurs when the passage of time signals a localreinforcer differential, as in Cowie et al. (2011).

3.2. Control by elapsed time

We have shown clear control of response rate by time sincereinforcement on the FI-schedule key (Figs. 2–5) (see also Churchet al., 1998; Jozefowiez et al., 2005), with apparently com-plementary response rates on the VI-schedule key, and clearresultant temporal control over choice (Figs. 6–8). We havealso shown that overall choice is controlled by the relative fre-quency of obtained reinforcers on the two keys (Fig. 10). Ourtentative conclusion, then, might be that both extended con-trol over choice by relative food frequencies and local controlby time since food are active in the present data. But, is con-trol by elapsed time directly over choice, or is it over respondingon each key separately with a constraint on overall output ofresponses? If the latter, then it is conceivable that simple con-trol by time over individual-key responding, with constant totaloutput, could account for both bias (Phases 1 and 2) and sensitiv-ity to extended food frequencies in Phase-2 choice. However, thedata shown in Figs. 2–4 do not support a notion of constant totaloutput.

COND 8p(FI) = .7

COND 7p(FI) = .3

LO

G L

EF

T/R

IGH

T R

ES

PO

NS

E R

AT

IO

-3

-2

-1

0

1

2

COND 9p(FI) = .1

-3

-2

-1

0

1

2

COND 6p(FI) = .9

COND 10p(FI) = .5

SECONDS AFTER FOOD

0 20 40 60 80

-3

-2

-1

0

1

2

COND 11p(FI) = .95

SECONDS AFTER FOOD

0 20 40 60 80

COND 12p(FI) = .05

-3

-2

-1

0

1

2

Fig. 8. Phase 2. Log left/right response ratios as a function of time following fooddelivery. The horizontal line shows equal preference.

3.3. Timing and modeling

Timing, whether measured by peak response rates on the FI keyon VI trials, or as measured by peak choice to the FI key, was gener-ally accurate, more so when the FI schedules were shorter (Phases1 and 3) and when the FI schedule was more probable (Phase 2).Fig. 11 compares these two estimates and shows that they werevery similar in all but one case. Note that only response rates on FItrials are shown for Phase 3 because no VI schedule was arranged.Such peaks are not, however, informative about timing accuracy,which must take into account the height of the peak relative to thelevel of responding or choice at other times since food (Figs. 2–5and Figs. 6–8). Fig. 12 attempts to index the accuracy in FI-key peakresponding by plotting the ratio of the FI-key response rate at theFI value to the mean FI-key rates both before and for an equal timeafter, the FI interval.

In Phases 1 and 2, for all except the FI 5-s condition, the ratioof the peak response rate to the mean rate before the FI time wasgreater than that for the ratio of peak rate to rate after the FI time.This result shows that the distribution of rates around the FI timewas not symmetrical. This, in turn, might suggest that the dis-tribution of perceived times was non-normal, but would only doso if the timing signal was precisely located at the FI time—thatis, if all reinforcers were obtained exactly at the FI time. Giventhat they were not precisely located (Fig. 1; see also Church et al.,1998, Fig. 6; Church and Lacourse, 2001, Fig. 2), but had a tail tothe right, the asymmetrical distribution of response rates around

Author's personal copy

M. Davison et al. / Behavioural Processes 95 (2013) 100–112 107

PIGEON 161

-1.5

-1.0

-0.5

0.0

0.5

PIGEON 162

LO

G L

EF

T/R

IGH

T R

ES

PO

NS

E R

AT

IO

-1.5

-1.0

-0.5

0.0

0.5

PIGEON 163

FI-SCHEDULE VALUE (s)

0 10 20 30 40 50 60

-1.5

-1.0

-0.5

0.0

0.5

PIGEON 164

PIGEON 165

MEAN

0 10 20 30 40 50 60

Fig. 9. Phase 1. Log left/right response ratios as a function of the size of the arrangedFI schedule for each pigeon and for the data averaged across pigeons. All replicationsof Condition 1 (FI 20 s) are shown. The straight lines were fitted to the data by themethod of least squares to illustrate the general trend.

the FI time more likely reflects the distribution of food deliver-ies at and after the FI-interval time. Thus, while peak time may beaccurately located at the FI interval in most cases, timing—the per-formance on the FI key during VI trials—may be best modelled asthe distribution, or generalization, of the effects of food deliveriesat particular times. Thus, instead of time estimates being normallydistributed around a particular elapsed time as has often been sug-gested, perhaps more generally it is the effects of food deliveriesthat are normally distributed around the times at which they aredelivered. Such an approach may imply that food densities (rates)at particular times play a part in timing, as suggested by Bizo andWhite (1995).

We constructed a model based on these considerations. Weassumed that the measured rates of food delivery on a key ineach time bin would be distributed between this and adjacentbins according to a normal distribution in linear time, and that the“effective” food rate for a bin would be the sum of the distributedfood rates from all time bins. Thus, in general, the effective foodrate in a time bin would be most affected by the food rate in thatbin, and progressively less affected by food deliveries in increas-ingly distant bins, if food rates were similar for all bins. If the ratesacross bins were not similar, then the effective food rate in a par-ticular bin would be determined by the food rate in that bin andfood rates in other bins, normally distributed according to theirdistances from that bin. For instance, the apparent food rate in anFI schedule before the schedule has finished timing (local obtained

PIGEON 161

-1

0

1

PIGEON 162

LO

G L

EF

T/R

IGH

T R

ES

PO

NS

E R

AT

IO

-1

0

1

PIGEON 163

-1 0 1

-1

0

1

PIGEON 164

PIGEON 165

MEAN

-1 0 1

Y = 0.95X - 0.45(0.92)

Y = 0.79X - 0.71(0.91)

Y = 0.53X - 0.15(0.94)

Y = 0.74X - 0.45(0.88)

Y = 0.61X - 0.15(0.96)

Y = 0.64 - 0.43(0.87)

Fig. 10. Phase 2. Log left/right response ratios as a function of log left/right obtainedfood ratios for each pigeon and for the data averaged across pigeons. All replicationsof Condition 1 (FI 20 s) are shown. The origins of the graphs (0, 0) are shown bycrosses. Straight lines were fitted to the data by the method of least squares, andthe equations of the fitted lines are shown on each graph with the proportion ofvariance accounted for in parentheses below.

PHASE 2

PROBABILITY OF FI SCHEDULE

0.0 0.2 0.4 0.6 0.8 1.0

PHASE 3

SECONDS AFTER FOOD

0 10 20 30 40 50

0

10

20

30

40

50

PHASE 1

PE

AK

TIM

E (

s)

0

20

40

60

80

B RATE

CHOICE

FI VALUE

Fig. 11. The time at which peaks in FI-key response rate and in left/right choiceoccurred in all 3 phases (Phase-3 data show only response rates). The heavy linesshow the arranged FI values.

Author's personal copy

108 M. Davison et al. / Behavioural Processes 95 (2013) 100–112

PHASE 2

PROBABILITY OF FI SCHEDULE

0.0 0.2 0.4 0.6 0.8 1.0

PHASE 3

SECONDS AFTER FOOD

0 10 20 30 40 50

0

1

2

3

4

5

6

PHASE 1

PE

AK

/ME

AN

0

1

2

3

4

5

6

BEFORE FI

AFTER FI

Fig. 12. Peak FI-schedule response rates divided by the mean response rates beforethe FI-interval and for the same number of bins after the FI-interval time. Data forafter FI are not plotted for Phase 3 because a single FI schedule was arranged in thisphase. Replications are shown as unjoined points.

food rate = 0) would depend on the temporal distances to all foodrates obtained after the schedule had timed. We assumed that thestandard deviation of time estimation would increase with elapsedtime according to a single coefficient of variation (CV), and weallowed for different CV values for the FI- and VI-schedule perfor-mances. Fig. 13 shows the operation of the model. Obtained fooddeliveries in successive time bins are shown in the top graphs, andtemporal discriminabilities in the second pair of graphs. The fooddistributions and temporal discriminabilities for each of the FI andVI schedules are convolved and summed, according to the model,in the third pair of graphs for the two schedules, providing a pre-diction of choice (log FI/VI response ratios in each time bin) in thebottom graph. While all graphs are shown as continuous for didac-tic purposes, the calculations were done according to discrete timebins.

The proposed model, as described above, is closely related tothat proposed by Davison and Nevin (1999) to describe choice whenmultiple discriminative stimuli on a stimulus dimension signaldifferent reinforcer contingencies. In Davison and Nevin’s model,the discriminabilities between pairs of stimuli are free parametersdetermined by fitting the appropriate model, so many free parame-ters are required for complex discriminations. In the present modelfor timing, these multiple free parameters are replaced by a singleparameter, the coefficient of variation of the normal distribution.

This model was used to predict log FI/VI response ratios in binsin each of the Phase 1 and 2 conditions, incorporating a further freeparameter which was the overall bias between FI and VI responding(log c). In this analysis, we used only the response ratios on VI-schedule trials, and the food rates from the combination of FI andVI trials (total time in a bin as the food-rate denominator). Theanalysis indicated a clear problem with the model—if it was fittedjust to response ratios prior to the FI-schedule time, it describedthese data extremely well, but failed to predict accurately responseratios beyond this point. If it was fitted using all response ratios upto the 60th time bin (recall the FI 5-s data were collected in 1.25-sbins), it fitted poorly.

LO

CA

L F

OO

DR

AT

ES

0

5

10

15

20

25

DIS

TR

IBU

TE

D F

OO

DR

AT

ES

0.0

0.2

0.4

0.6

FI TOTALDISTRIBUTION

0 10 20 30 40 50 60

SU

MM

ED

FO

OD

RA

TE

S

0.0

0.5

1.0

1.5

2.0

2.5 VI TOTALDISTRIBUTION

0 10 20 30 40 50 60

SECONDS AFTER FOOD

0 10 20 30 40 50 60

LO

G C

HO

ICE

(L/R

)

-1.5

-1.0

-0.5

0.0

0.5

1.0

CV(VI) = 0.3

etc...

LEFT KEY FI 20 s RIGHT KEY VI 30 s

CV(FI) = 0.3

Fig. 13. The operation of the proposed model. The top line of graphs shows theobtained local rate of food deliveries as a function of time since the last food for theFI and VI schedules; the second line shows the assumed Gaussian temporal discrim-inabilities of times since food given a constant coefficient of variation; the third lineshows the summed convolution of food rates and temporal discriminabilities foreach of the schedules; and the bottom graph shows the predicted log FI/VI scheduleresponse ratios.

The fact that the model fitted the data well up to the end of theFI schedule suggested that the reallocation of reinforcer rates wascorrect for these limited data, but that performance after the timeat which the FI schedule primed may be controlled by a differentmechanism. Choice beyond this point became relatively constant(Figs. 6–8) and the data from Phase 1 suggested that choice mea-sures at these times after food followed the mean local food-rateratios across the sessions (thus, the extended food-rate ratio).

Given these findings, we modified the model for choice. In thismodel, we assumed full control over choice by a bias factor across allbins. To this factor was added the log ratio of normally redistributedreinforcer rates in each bin, raised to a power. The value of thispower was 1.0 prior to the time at which the FI schedule primed,implying that the redistributed log reinforcer ratios exerted fullcontrol up until this point. Beyond the point of FI priming, thepower fell according to the distance from the priming point, atten-uating control by redistributed food-reinforcer rates. This decrease

Author's personal copy

M. Davison et al. / Behavioural Processes 95 (2013) 100–112 109

FI 5 s

-3

-2

-1

0

PREDICTED

OBTAINED

FI 10 s

LO

G L

EF

T/R

IGH

T R

ES

PO

NS

E R

AT

IO

-3

-2

-1

0

FI 20 s

SECONDS AFTER FOOD

0 10 20 30 40 50 60

-3

-2

-1

0

FI 20 s (R)

FI 40 s

FI 50 s

0 10 20 30 40 50 60

Fig. 14. Phase 1. Obtained and predicted log response ratios for group data.

was modeled according to the right of the distribution around thearranged FI time, using the fitted FI standard deviation at that time,and normalized to 1.0 at the FI time. This approach assumes thatthe animal estimates the FI-schedule time in just the same way as itestimates the times to particular reinforcer-rate ratios. The model,therefore, is a type of generalized-matching model in which thereis control over local choice by both the distributed reinforcer ratesand the overall bias up to the FI-prime time, after which sensitivityto the distributed reinforcer rates falls to zero, leaving control byoverall bias, log c. This model therefore assumes that the control ofchoice shifts from joint control by distributed food rates and biasprior to FI priming to progressively increasing relative control byextended bias alone.

To test this model, we fitted it to the choice data, averaged acrosspigeons, up to the 60th bin for all conditions in Phases 1 and 2 (780data points in total). We used all (FI- and VI-schedule trial) data.We allowed different CV values for the FI and the VI schedules,and a different value of bias for each condition. The fit was goodfor the grouped data, with a median 92% (range over conditions78–100%) of the data variance accounted for, and a mean-squareerror of the fit of 0.05 in log terms. Graphs of predicted and obtainedlog response ratios for the 3 phases of the experiment are shownin Figs. 14–16. The median value of the CV for the FI schedule was0.28 (range 0.19–0.35), and for the VI schedule the median was0.38 (range 0.24–16). The value of the fitted bias was not signifi-cantly different from the ratio of the mean reinforcer rates averagedacross all bins—the slope of this relation was 1.10 (SE 0.11) and the

p(FI) = .05

-3

-2

-1

0

1

PREDICTED

OBTAINED

p(FI) = .1

LO

G L

EF

T/R

IGH

T R

ES

PO

NS

E R

AT

IO

-3

-2

-1

0

1

p(FI) = .3

SECONDS AFTER FOOD

0 10 20 30 40 50 60

-3

-2

-1

0

1

p(FI) = .7

p(FI) = .9

p(FI) = .95

0 10 20 30 40 50 60

Fig. 15. Phase 2. Obtained and predicted log response ratios for group data.

intercept was 0.07 (SE 0.06), with 93% variance accounted for.Therefore, this parameter in the fits ceases to be free, leaving just2 free parameters (the FI and VI CVs) for each condition. Sinceneither of the CVs appeared to change systematically across con-ditions, the median CV can describe all conditions. However, theranges of the parameters reported above indicate some problem-atic conditions. These were the FI 5-s condition in Phase 1, andthe two low-FI-probability conditions (0.05 and 0.1) in Phase 2—allprovided abnormally large values of the VI-schedule CV. This mayindicate some problems with the model when FI schedules are veryshort or very infrequent.

This model was also fitted to the response-rate data obtainedin Phase 3, using a single (FI) CV, but adding a simple multiplica-tive parameter that scaled reinforcer rates to response rates. Acrossthese 5 conditions, this model accounted for a median 99% of thevariance accounted for with a median CV of 0.26. The progressivelyincreasing multiplier across increasing FI-schedule values arisesbecause, while response rates remained at similar levels across FIschedules, reinforcer rates decreased (Table 2).

Overall, this modeling approach provided a very accuratedescription of the data obtained in this experiment. Indeed, theprinciples underlying the model are logical and reasonable: rein-forcer rates at times since reinforcers are progressively more widelydistributed with increasing times since the last reinforcer—that is,reinforcer rates control choice according to the scalar property oftime—and that when time exceeds the scalar estimate of the FI time,choice reverts to simple matching to extended, overall, ratios ofreinforcer rates. The model can thus be taken as a theory, ratherthan just a description.

Author's personal copy

110 M. Davison et al. / Behavioural Processes 95 (2013) 100–112

Table 2Results of fitting the model to choice in all conditions of the experiment using pooled data across the first 60 time bins after food. CV is the coefficient of variation, bias is theadditive constant in the model, and VAC is the proportion of variance accounted for. In Phase 2, multiplier converts local effective food rates to responses per minute.

FI (s) p(FI) CV FI CV VI Bias VAC

Phases 1 and 25.00 0.50 0.32 9.45 −0.82 0.9810.00 0.50 0.26 0.33 −0.58 0.9020.00 0.50 0.30 0.37 −0.23 1.0040.00 0.50 0.30 0.86 −0.31 0.8950.00 0.50 0.35 0.52 −0.34 0.9120.00 0.50 0.24 0.38 −0.42 0.9520.00 0.05 0.32 16.03 −0.94 0.9920.00 0.10 0.28 10.77 −0.83 0.9220.00 0.30 0.28 0.33 −0.43 0.8920.00 0.50 0.24 0.30 −0.35 0.9920.00 0.70 0.22 0.24 −0.23 0.9520.00 0.90 0.19 0.24 0.22 0.7820.00 0.95 0.25 0.58 0.44 0.84

Median 0.28 0.38 0.92

CV FI Multiplier VAC

Phase 35.00 0.26 8.80 0.95

10.00 0.21 59.32 0.8020.00 0.21 91.32 0.9940.00 0.31 89.52 0.9950.00 0.33 147.81 0.99

Median 0.26 89.52 0.99

Notice that the general model as used for Phases 1 and 2 is largelycompatible with generalized matching as found in concurrent VI VIschedules. Because there are few or no temporal discriminationsavailable in such schedules, sensitivity to local variations would be

FI 20 s

SECONDS AFTER FOOD

0 10 20 30 40 50 60

0

50

100

150

FI 10 s

RE

SP

ON

SE

S/M

INU

TE

0

50

100

150

FI 5 s

0 5 10 15 20 25 30

0

50

100

150

FI 50 s

SECONDS AFTER FOOD

0 10 20 30 40 50 60

FI 40 s

OBTAINED

PREDICTED

Fig. 16. Phase 3. Obtained and predicted log response ratios for group data.

zero and matching would occur throughout time after food deliv-ery. Generalized-matching sensitivity to reinforcement is usuallygreater in concurrent VI VI performance than found here (Fig. 10),but the present procedure has more in common with a multipleschedule (or a mixed schedule in which priming the ratio can occur;Ferster and Skinner, 1957)—and sensitivity is known to be lowin multiple schedules (Davison and McCarthy, 1888; McSweeneyet al., 1986). Indeed, “priming the ratio” is a reasonable way of view-ing the current procedure in Phases 1 and 2: before the FI intervalelapses, performance is largely controlled by the time-controlledlocal apparent food deliveries on the FI and VI schedules, but oncethe interval has elapsed and no FI food is forthcoming, this newstimulus condition eliminates the temporal control of local foodratios.

3.4. Timing and reinforcers

When food deliveries occur differentially with respect to somediscriminable stimulus event, performance may come to be con-trolled by the presence versus absence of that stimulus. Whether itdoes or not depends on the discriminability of the signalling stimu-lus (Davison and Nevin, 1999). This effect is generically called stim-ulus control, and it is well known that the passage of time can actas such a stimulus, whether as a pair of discrete stimuli as in exper-iments on temporal discrimination, or as a continuously changingvariable, as in experiments on timing. The notion of “apparentreinforcers” was introduced by Davison and Jenkins (1985) anddeveloped further by Davison and Nevin (see also Nevin et al., 2005)in order to understand the slippage between more or less discrim-inable stimuli and more or less discriminable payoffs. We usedthe same general notion here, because we assume, informed by anextensive literature, that time estimates become less accurate withincreasing elapsed time, resulting in food obtained at longer timesbecoming progressively less accurately controlled by that time, andmore spread across adjacent times. This approach is similar to thattaken by Gibbon (1977), but not identical because we assume that itis the food rates delivered at a time, rather than simply the time esti-mates, that get spread across adjacent times. Thus, it is our view thatinvestigations of the nature of the internal clock (e.g., Church and

Author's personal copy

M. Davison et al. / Behavioural Processes 95 (2013) 100–112 111

PIGEON 161

0

20

40

60

LEFT (FI)

RIGHT (VI)

PIGEON 162

RE

SP

ON

SE

S/M

INU

TE

0

20

40

60

80

PIGEON 163

0

20

40

60

PIGEON 161

PIGEON 162

PIGEON 163

CONDITION 1 CONDITION 10

PIGEON 161

PIGEON 162

CONDITION 18

PIGEON 163

PIGEON 164

0

20

40

60

PIGEON 165

SECONDS AFTER FOOD

0 10 20 30 40 50 60

0

20

40

60

PIGEON 164

PIGEON 165

0 10 20 30 40 50 60

PIGEON 164

PIGEON 165

0 10 20 30 40 50 60

Fig. A1. Responses per minute for each Pigeon on the left (FI) and right (VI) keys in Condition 1 and its replications, Conditions 10 and 18.

Lacourse, 2001) cannot proceed without an understanding of thecontribution made by differential payoffs with respect to elapsedtime, or at least the elimination of such payoff effects experimen-tally. The latter, though, is very difficult for the simple reason thatlocal payoff rates and ratios are often inaccurate representationsof overall, or locally arranged, rates and probabilities. For example,

Church and Lacourse (their Fig. 2) show systematic deviations ofobtained and expected probabilities of food delivery on exponentialschedules and on Wald schedules. In the latter, obtained probabili-ties were considerably and systematically less than arranged prob-abilities before the probability peak, and systematically greaterafter the peak. Thus, not only did food probability affect response

Author's personal copy

112 M. Davison et al. / Behavioural Processes 95 (2013) 100–112

PIGEON 161

-3

-2

-1

0

1

COND. 1

COND. 10

COND. 18

PIGEON 162

LO

G L

EF

T/R

IGH

T R

ES

PO

NS

E R

AT

IO

-3

-2

-1

0

1

PIGEON 163

SECONDS AFTER FOOD

0 10 20 30 40 50 60

-3

-2

-1

0

1

PIGEON 164

PIGEON 165

SECONDS AFTER FOOD

0 10 20 30 40 50 60

Fig. A2. Log FI/VI-key response ratios for all pigeons on trials that ended in FI-schedule food deliveries in Condition 1 and its replications, Conditions 10 and 18.

probability, but response probability affected food probability. Afurther example comes from local analyses of concurrent VI VI per-formance that arrange a changeover delay. If the changeover delayis arranged in standard fashion then, after a food delivery on analternative, the changeover delay remains satisfied after the deliv-ery. As a result, additional foods can be obtained on an alternativeimmediately after food on the just-productive alternative, and nofood deliveries can be obtained on the other alternative until afterthe changeover delay has elapsed, resulting in a local differencein obtained food ratios, which are unambiguously signalled by theprior food delivery, and unrepresentative of the overall or arrangedfood ratios and rates. This is likely the reason for the “preferencepulses” to the just-productive alternative, and extended visits tothe just-productive alternative, that have been demonstrated ina series of papers by Davison and Baum (2002, 2006; Baum andDavison, 2004), and this interpretation has been empirically inves-tigated by Boutros et al. (2011). These may not be a “reinforcement”effect, and may not be caused by the food delivery locally increasingthe probability of the response that has just been productive.

Such dynamical effects are, of course, what causes the frequentdelay between the priming of the FI schedule, and obtaining the FIfood delivery. The VI schedule promotes visits to the right VI key,which delays obtaining food on the FI key. Assuming that visits toundefined alternative responses occurred in Phase 3 (Herrnstein,1970; Davison, 2004), FI food deliveries in this phase would be sim-ilarly delayed, but because the relative value of the FI foods wouldbe increased in the absence of VI food deliveries, we would expectthat the delays to FI food delivery in Phase 3 would be less than inPhase 1, and indeed this was the case (Fig. 1).

It is the dynamical, two-way, control between responding andfood delivery that makes the experimental elimination of foodeffects difficult, and makes it necessary to consider obtained pay-off rates in the analysis of timing data. We believe that theoriesof timing should be more sensitive to differential reinforcementwith respect to time. Investigations of pure timing should takepains to eliminate such differential-reinforcement effects; and the-ories of behaving with respect to time must incorporate sucheffects. We know that stimulus discrimination cannot be studiedwithout taking into account reinforcer effects, and that reinforcereffects cannot be studied without taking into account stimuluseffects (Davison and Nevin, 1999). We make the same argu-ment for discriminative control by time: Pure control by elapsedtime can only be derived if obtained reinforcer effects are fullyaccounted.

Appendix A.

References

Baum, W.M., 1974. On two types of deviation from the matching law: bias andundermatching. J. Exp. Anal. Behav. 22, 231–242.

Baum, W.M., Davison, M., 2004. Choice in a variable environment: visit patterns inthe dynamics of choice. J. Exp. Anal. Behav. 81, 85–127.

Bizo, L.A., White, K.G., 1995. Biasing the pacemaker in the behavioural theory oftiming. J. Exp. Anal. Behav. 64, 225–235.

Boutros, N., Elliffe, D., Davison, M., 2011. Examining the discriminative and strength-ening effects of reinforcers in concurrent schedules. J. Exp. Anal. Behav. 96,227–324.

Catania, A.C., Reynolds, G.S., 1968. A quantitative analysis of responding main-tained by interval schedules of reinforcement. J. Exp. Anal. Behav. 11,327–383.

Church, R.M., Lacourse, D.M., 2001. Temporal memory of interfood interval distri-butions with the same mean and variance. Learn. Motiv. 32, 2–21.

Church, R.M., Lacourse, D.M., Crystal, J.D., 1998. Temporal search as a function ofthe variability of interfood intervals. J. Exp. Psychol. Anim. Behav. Process. 24,291–315.

Cowie, S., Davison, M., Elliffe, D., 2011. Reinforcement: food signals the time andlocation of future food. J. Exp. Anal. Behav. 96, 63–86.

Davison, M., 2004. Interresponse times and the structure of choice. Behav. Process.66, 173–187.

Davison, M., Baum, W.M., 2002. Choice in a variable environment: effects of blackoutduration and extinction between components. J. Exp. Anal. Behav. 77, 65–89.

Davison, M., Baum, W.M., 2006. Do conditional reinforcers count? J. Exp. Anal. Behav.86, 269–283.

Davison, M., Elliffe, D., 2010. Divided stimulus control: a replication and a quantita-tive model. J. Exp. Anal. Behav. 94, 13–23.

Davison, M., Jenkins, P.E., 1985. Stimulus discriminability, contingency discrim-inability, and schedule performance. Anim. Learn. Behav. 13, 77–84.

Davison, M., McCarthy, D., 1888. The Matching Law: A Research Review. Erlbaum,Hillsdale, NJ.

Davison, M., Nevin, J.A., 1999. Stimuli, reinforcers, and behaviour: an integration. J.Exp. Anal. Behav. 71, 439–482.

Elliffe, D., Alsop, B., 1996. Concurrent choice: effects of overall reinforcer rate andthe temporal distribution of reinforcers. J. Exp. Anal. Behav. 65, 445–463.

Ferguson, G.A., 1971. Statistical Analysis in Psychology and Education. McGraw-Hill,Tokyo.

Ferster, C.B., Skinner, B.F., 1957. Schedules of Reinforcement. Appleton-Century-Crofts, New York, NY.

Gibbon, J., 1977. Scalar expectancy theory and Weber’s law in animal timing. Psychol.Rev. 84, 279–325.

Herrnstein, R.J., 1961. Relative and absolute strength of response as a function offrequency of reinforcement. J. Exp. Anal. Behav. 4, 267–272.

Herrnstein, R.J., 1970. On the law of effect. J. Exp. Anal. Behav. 13, 243–266.Jozefowiez, J., Cerutti, D.T., Staddon, J.E.R., 2005. Timing in choice experiments. J.

Exp. Psychol. Anim. Behav. Process. 31, 213–225.Lobb, B., Davison, M., 1975. Performance in concurrent interval schedules: a system-

atic replication. J. Exp. Anal. Behav. 24, 191–197.McSweeney, F.K., Farmer, V.A., Dougan, J.D., Whipple, J.E., 1986. The generalized

matching law as a description of multiple-schedule responding. J. Exp. Anal.Behav. 45, 83–101.

Nevin, J.A., Davison, M., Shahan, T.A., 2005. A theory of attending and reinforcementin conditional discriminations. J. Exp. Anal. Behav. 84, 281–303.

Taylor, R., Davison, M., 1983. Sensitivity to reinforcement in concurrent arithmeticand exponential schedules. J. Exp. Anal. Behav. 39, 191–198.

Trevett, A.J., Davison, M.C., Williams, R.W., 1972. Performance in concurrent intervalschedules. J. Exp. Anal. Behav. 17, 369–374.