Temporal pitch mechanisms in acoustic and electric hearing

Temporal pitch mechanisms in acoustic and electric hearingRobert P. CarlyonMRC Cognition and Brain Sciences Unit, 15 Chaucer Road, Cambridge CB2 2EF, England

Astrid van WieringenLab. Exp. OtoRhinoLaryngology, KU Leuven, Kapucijnenvoer 33, 3000 Leuven, Belgium

Christopher J. Long and John M. DeeksMRC Cognition and Brain Sciences Unit, 15 Chaucer Road, Cambridge CB2 2EF, England

Jan WoutersLab. Exp. ORL, KU Leuven, Kapucijnenvoer 33, 3000 Leuven, Belgium

~Received 11 October 2001; revised 2 May 2002; accepted 3 May 2002!

Two experiments investigated pitch perception for stimuli where the place of excitation was heldconstant. Experiment 1 used pulse trains in which the interpulse interval alternated between 4 and6 ms. In experiment 1a these ‘‘4–6’’ pulse trains were bandpass filtered between 3900 and 5300 Hzand presented acoustically against a noise background to normal listeners. The rate of anisochronous pulse train~in which all the interpulse intervals were equal! was adjusted so that itspitch matched that of the ‘‘4–6’’ stimulus. The pitch matches were distributed unimodally, had amean of 5.7 ms, and never corresponded to either 4 or to 10 ms~the period of the stimulus!. Inexperiment 1b the pulse trains were presented both acoustically to normal listeners and electricallyto users of the LAURA cochlear implant, via a single channel of their device. A forced-choiceprocedure was used to measure psychometric functions, in which subjects judged whether the 4–6stimulus was higher or lower in pitch than isochronous pulse trains having periods of 3, 4, 5, 6, or7 ms. For both groups of listeners, the point of subjective equality corresponded to a period of 5.6to 5.7 ms. Experiment 1c confirmed that these psychometric functions were monotonic over therange 4–12 ms. In experiment 2, normal listeners adjusted the rate of an isochronous filtered pulsetrain to match the pitch of mixtures of pulse trains having rates ofF1 andF2 Hz, passed throughthe same bandpass filter~3900–5400 Hz!. The ratioF2/F1 was 1.29 andF1 was either 70, 92, 109,or 124 Hz. Matches were always close toF2 Hz. It is concluded that the results of both experimentsare inconsistent with models of pitch perception which rely on higher-order intervals. Together withthose of other published data on purely temporal pitch perception, the data are consistent with amodel in which only first-order interpulse intervals contribute to pitch, and in which, over the range0–12 ms, longer intervals receive higher weights than short intervals. ©2002 Acoustical Societyof America. @DOI: 10.1121/1.1488660#

PACS numbers: 43.66.Ba, 43.66.Fe, 43.66.Hg, 43.66.Ts@NFV#

exo

tioo

bilyathusoripdnl o

yrea

n,

5;

n-f-of

ing

lec-ingral

lop-ies.ndhichar-toeri-with

I. INTRODUCTION

For normally hearing listeners, the pitch of a complsound is usually dominated by the lower-numbered harmics ~Plomp, 1967; Mooreet al., 1985!. The frequencies ofthese harmonics may be encoded by their place of excitaon the basilar membrane, by the temporal patternauditory-nerve~AN! responses to them, or to some comnation of the two. In this article, we concentrate on ‘‘puretemporal’’ pitch perception, which we define as a pitch thcan only be derived from the temporal characteristics ofAN response. To do this we use two types of stimulacoustic pulse trains that have been bandpass filtered tmove spectral components that are resolvable by the peeral auditory system~cf. Hoekstra, 1979; Shackleton anCarlyon, 1994; Carlyon, 1997; Kaernbach and Dema1998!, and electric pulse trains applied to a single channea cochlear implant.

Although the pitch derived from these stimuli is in manways weaker than that derived by normal listeners fromsolved harmonics—for example showing larger discrimin

J. Acoust. Soc. Am. 112 (2), August 2002 0001-4966/2002/112(2)/6

n-

nf

-

te:re-h-

y,f

--

tion thresholds~Hoekstra, 1979; Shackleton and Carlyo1994; Kaernbach and Bering, 2001! that increase markedlyabove 300–600 Hz~Shannon, 1983; Tong and Clark, 198Townshendet al., 1987; McKay et al., 2000; Carlyon andDeeks, 2002!—our approach is of theoretical, and, potetially, practical significance. First, by excluding place-oexcitation cues, one can perform straightforward testsmore general pitch models that rely on temporal process~Pattersonet al., 1991; Meddis and O’Mard, 1997!. Second,users of most modern cochlear implants rely entirely on etric pulse trains for their sense of hearing. Understandhow the auditory system derives a pitch from the tempocharacteristics of pulse trains may help guide the devement of new cochlear-implant signal-processing strategFinally, by comparing the results obtained with electric aacoustic pulse trains, one can evaluate the extent to wthe acoustic stimuli provide an adequate simulation of heing by cochlear implant users. If the simulation is foundbe accurate, then it may prove possible to develop expmental procedures and/or signal-processing strategies

62121/13/$19.00 © 2002 Acoustical Society of America

e

reth

hr

ioobninth

na

ct

ga

ulw

rv

coela

ittrdniph-on.c.hems.

ak in-rterc-

is-

ndis

es, in

ain-re,

one

ortingals

ar-s3in-

arlyasorts

d

ose

redses

ses

-

eri-ig.

he

nd

the aid of normal listeners, before using the comparativscarce time of cochlear implant users~cf. Blamey et al.,1984!.

The interpulse intervals in the pulse trains used in pvious pitch studies were usually the same throughoutstimulus. The approach used here is to modify such isocnous pulse trains@Fig. 1~a!# in a way which will differentiatebetween three different theories of temporal pitch perceptAll three theories were developed to account for datatained with normal listeners, but make specific predictioabout the pitches of both electric and acoustic pulse traDifferences between the theories can be illustrated bystimulus shown schematically in Fig. 1~b!, in which the in-terval between successive pulses alternates between 4 ams. This stimulus was used in our first experiment, and wpresented both acoustically to normal listeners and elecally to cochlear implant users.

Autocorrelation theories, originally applied to hearinby Licklider ~1951!, assume that the auditory system anlyzes the intervals between each pulse and every other pThe largest peak in the autocorrelation of the stimulus shoin Fig. 1~b! is at 10 ms, because every second-order inte~between each pulse and the next-but-one! corresponds tothis value. There are also peaks at 4 ms and 6 ms. Wecentrate on an influential modern version of the autocorr

FIG. 1. Schematic representation of pulse trains used here and elsewThe ordinate is amplitude and the abscissa is time.~a! Periodic pulse train.~b! The ‘‘4–6’’ pulse train used in experiment 1.~c! Pulse train with regularsecond-order intervals and quasirandom first-order intervals. Arrows icate the regular second-order intervals.~d! Pulse train with a proportion ofregular first-order intervals, indicated by arrows.~e! Regular pulse trainfrom which a proportion of pulses has been deleted.~f! Mixtures of twopulse trains having ratesF1 Hz ~gray lines! andF2 Hz ~black lines!.

622 J. Acoust. Soc. Am., Vol. 112, No. 2, August 2002

ly

-eo-

n.-ss.e

d 6s

ri-

-se.n

al

n--

tion approach, originally described by Meddis and Hew~1991! and subsequently modified by Meddis and O’Ma~1997!. This model is similar to the original autocorrelatioapproach but takes into account important aspects of pereral processing such as filtering and half-wave rectificatiAn implementation of this model will be described in SeII B., but, for the time being we note that it too produces tlargest peak at 10 ms, plus two smaller peaks at 4 and 6The original version of the model~Meddis and Hewitt, 1991!assumed that the auditory system selects the largest pethe autocorrelogram~10 ms!, and, when two peaks have approximately equal size, the one corresponding to the shointerval is selected. According to this model, then, the ‘‘seond choice’’ is a period of 4 ms (pitch5250 Hz). The laterversion ~Meddis and O’Mard, 1997! uses a slightly morecomplicated decision metric, and its predictions will be dcussed in more detail in Sec. II B.

In contrast to autocorrelation models, Kaernbach aDemany~1998! have suggested that the auditory systemsensitive only tofirst-order intervals between successivpulses. This conclusion was based on a set of experimentone of which subjects could not discriminate between‘‘quasirandom’’ pulse train and one whose second-ordertervals were regular, but whose first-order intervals wesubject to certain constraints, random@Fig. 1~c!; arrows in-dicate regular second-order intervals#. Subjectscould, how-ever, discriminate between the quasirandom train andwith a small proportion of regular first-order [email protected]~d!, arrows indicate regular first-order intervals!. The pre-dictions of their theory for the stimulus shown in Fig. 1~b!are that the pitch should correspond to a period of either 46 ms. In fact, they describe informal observations suggesthat subjects favor the longer of the two first-order intervin pulse trains similar to those in Fig. 1~b!, a finding consis-tent with the results of a pilot experiment described by Clyon et al. ~2001!. Early research using filtered pulse train~e.g., Thurlow and Small, 1955; Small and McClellan, 196!was also consistent with pitch being based on first-ordertervals, but suggested that theshortestinterpulse intervalswere dominant. However, it should be noted that those eexperiments did not control for place-of-excitation cuesstrictly as is possible in more modern studies, and the repof ‘‘temporal separation pitches’’ as high as 2000 Hz~Thur-low and Small, 1955!—much higher than the few hundreHz at which temporal pitch perception breaks down~Burnsand Viemeister, 1976, 1981; Carlyon and Deeks, 2002!—suggests that place cues were in fact present in thexperiments.1

Yet another metric was proposed by Carlyon~1996,1997!, who suggested that the pitch of a bandpass-filtepulse train may correspond simply to the number of pulpresent during a fairly long~.100 ms! segment of the stimu-lus. For example, he found that deleting a number of pulat random from a pulse train@Fig. 1~e!# reduced its pitch.According to thismean ratehypothesis, the pitch of thepulse train shown in Fig. 1~b! should correspond to an interval of 5 ms~the average of 4 and 6 ms!.

The present study consisted of two experiments. Expment 1 presented electric pulse trains similar to those in F

re.

i-

Carlyon et al.: Acoustic and electric pitch

ndme

stiauf-

rythnth

inis

sb

llygesotthte

Iarraafeess

opi

wathw

smstumofuo

eeauoluulu

uae

o be7%

, athelse

r-nt.o-lantthea

oreachar-isand

6usse

peri-hehadro-b-

mereal

1aple-t

, 6,ub-s,

am.ofid

n-ntedthe

lse

insvia

nt. ofityso-

1~b! to a single channel of the LAURA cochlear implant, abandpass-filtered acoustic versions of the stimulus to norlisteners. An advantage of this dual approach is that it ovcomes some of the limitations inherent to purely acouexperiments.2 For example, the temporal response of theditory system to the pulses illustrated in Fig. 1 will be afected by peripheral auditory processes such as ringingauditory filters or adaptation at the inner hair cell/auditonerve synapse. Additionally, acoustic experiments requirepresence of background noise to mask combination to~CTs!, and this noise is often not included when passingexperimental stimuli through auditory models~e.g., Carlyon,1998; Kaernbach and Demany, 1998; Krumbholzet al.,2000!. Electric stimulation by-passes cochlear processand can be performed in the absence of background noand so if the same results are obtained with the two typestimulation, then a range of alternative explanations canexcluded. Experiment 2 was performed only with normahearing listeners, and used a different stimulus, consistina mixture of two isochronous pulse trains of different ratfiltered into the same frequency region. The results of bexperiments are consistent with pitch being derived fromfirst-order intervals present in the stimulus, with the greaweight being applied to the longest first-order intervals.

II. EXPERIMENT 1

A. Method

Two different techniques were used in experiment 1.experiment 1a, four normally hearing volunteers took pall of whom were members of the auditory research labotory at the University of Leuven. They adjusted the rate ofisochronous pulse train so that its pitch matched that o‘‘4–6’’ pulse train, whose first-order intervals alternated btween 4 and 6 ms@Fig. 1~b!#. The 400-ms pulse trains werconstructed at a sampling frequency of 44 100 Hz and pathrough a digital bandpass filter having cutoff frequencies3900 and 5300 Hz. The digital filter was designed to aproximate the analog eighth-order Butterworth filter usedexperiment 2. The attenuation at the cutoff frequenciesno more than 3 dB, relative to the level in the center ofpassband. The attenuation at half an octave below the locutoff, and at half an octave above the upper cutoff, waleast 24 dB relative to that at the cutoff frequencies. The rlevel was set to 54 dB SPL. Stimuli were presented againbackground of continuous pink noise, having a spectrlevel of 9.5 dB SPL at 4 kHz. They were turned on andwith 50-ms raised-cosine ramps. A total of 29 isochronopulse trains, with periods ranging from 2 to 14 ms in steps7%, with the period rounded to the nearest 0.1 ms, wgenerated and saved to separate waveform files. Duringrun of the matching test the subjects heard a triad of stimpresented repeatedly: the 4–6 pulse train, an isochronpulse train, and the 4–6 pulse train again. The interstimutime was 400 ms within, and 1500 ms between, triads. Sjects were required to adjust the pitch of the second stimuto match that of the first~and third!. The particular ‘‘startingvalue’’ for the isochronous pulse train was chosen in a qsirandom fashion, from run to run. The subject could th

J. Acoust. Soc. Am., Vol. 112, No. 2, August 2002

alr-c-

of-e

ese

ge,

ofe

of,hest

nt,-

na-

edf-ns

eeratsa

fsf

rechliuss

b-s

-n

increase or decrease the period of the isochronous train tpresented on the next trial, by either one, two or threesteps ~by pressing ‘‘---,’’ ‘‘--,’’ ‘‘-,’’ or ‘‘ 1,’’ ‘‘ 11,’’‘‘ 111’’ !. When s/he was satisfied with the pitch matchseparate button could be pressed which then initiatednext run. Each subject performed 30 matches. All putrains were played out of a portable PC~Toshiba, SatellitePro 420CDT! via a 16-bit PCMCIA card~WAVjammer, NewMedia Corp!, mixed ~Eurorack MX 1604A! with the analognoise from a CD player~Sony, CDP-209!, and presented tothe right earpiece of a TDH39 headset.

Experiment 1b was performed both with normally heaing listeners and five users of the LAURA cochlear implaIn pilot experiments we found that the pitch-matching prcedure was quite time-consuming, and one cochlear impuser produced quite variable results. We attributed this tofact that the 4–6 stimulus, by virtue of its irregularity, haddifferent timbre to the isochronous sounds. We therefadopted a constant-stimulus procedure in which, during eexperimental block, the pitch of the isochronous sounds vied substantially from trial to trial. We reasoned that thwould help subjects focus on the pitches of the sounds,that the timbral difference would ‘‘stick out’’ less.

During each block of 50 trials in experiment 1b, the 4–stimulus was paired, in random order, with an isochronopulse train having a period of 3, 4, 5, 6, or 7 ms. Thevalues spanned the range of pitch matches made in exment 1a. The interstimulus interval was 500 ms. After ttwo sounds were presented the subject indicated whichthe higher pitch. No feedback was given. Another isochnous train was selected at random for the next trial. All sujects except one completed 20 blocks of trials, each tiwith different randomizations of the paired stimuli, so thewere 200 trials for each isochronous stimulus. One normlistener~NH5! did only 50 trials per point.

Although nearly all of the matches in experimentwere to periods between 3 and 7 ms, we performed a supmentary experiment~1c! using the procedure of experimen1b but with isochronous pulse trains having periods of 48, 10, and 12 ms. This was done as an extra check for sjects hearing a pitch corresponding to a period of 10 mcorresponding to the largest peak in the autocorrelogrOne implant user and two normal listeners took part, bothwhom had participated in experiment 1b. Two of them d100 trials per point, and one normal listener~NH5! did 50trials per point.

For the normally hearing listeners the stimuli were geerated in the same way as in experiment 1a, and preseagainst the same continuous pink-noise background. Forimplant users the stimuli were 400-ms-long biphasic putrains, with a phase duration of 40ms, and the current wasgated on and off with 50-ms linear ramps. The pulse trawere applied to the stimulation channel shown in Table Ia DSP—TIC30 board, the sequencer of the computer~Com-paq Armada M700!, and the external coil of the LAURAdevice. Table I also provides brief details of each implauser. Subjects were implanted at the university ENT Deptthe St. Augustinus Hospital in Antwerp and the UniversHospital in Leuven. Based on closed-set vowel and con

623Carlyon et al.: Acoustic and electric pitch

ir age,

tslation

624 J. Acoust. S

TABLE I. Details of the cochlear implant users who participated in experiment 1b. The table shows theetiology, duration of deafness prior to implantation~DD, in years!, number of implanted years~CI years!,percentage-correct speech recognition results reported by van Wieringen and Wouters~1999! for vowels~SRV!and for consonants~SRC!, channel under test~CH! and current used for the 40ms/phase in these experimen@C~mA!#. In the LAURA device, channel 1 is the most apical, and channel 8 is the most basal. The stimuchannel with the largest dynamic range was chosen.

Listener Age AetiologyDD

~years!CI

~years! SRV ~%! SRC ~%! CH C~mA!

CI 1 20 Meningitis 3.5 7 53 45 2 600CI 2 44 Meniere’s 3 5 79 51 7 1000CI 3 33 Unknown 6 7 73 58 3 900CI 4 51 Congenital

progressive1 3 74 38 3 1050

CI 5 48 Congenitalprogressive

12 5 40 35 4 1200

sonntngceerersacs

ubt tll,heerrttheu

in

lua

fv

04,

oloalsuro

inare

ichpro-ossandombeffer-

asess,

itchho-arseametheer

ely,the

de-c-

allthe-

d

m

nants tests~‘‘SRV’’ and ‘‘SRC,’’ respectively, Table I!, theyare considered moderate to good performers~van Wieringenand Wouters, 1999!. They had all participated in previoupsychophysical studies and were paid for their participati

For a given subject, all electrical stimuli in experime1b were presented at the same current level, as showTable I. This level, measured using a test implant havin2.2-kV resistor between electrodes, was judged to producomfortable loudness for all stimuli. We chose not to pform formal loudness-balancing measurements in thisperiment for the different pulse rates, for two reasons. Fibecause this was a pitch preference task without feedbwe considered it unlikely that any small residual loudnedifferences would strongly and systematically influence sjects’ responses. Second, pilot experiments revealed tharesulting current levels differed only slightly or not at aeven between the shortest and longest periods usedThis is similar to the measurements of McKay and McDmott ~1998!, who showed that loudness matches for comfoably loud pulse trains varied by less than 0.5 dB overrange of rates used in experiment 1. However, becauseperiment 1c used a wider range of rates, we decided to loness balance all stimuli for the cochlear implant user~CI 2!who took part in that experiment. We did so by presentthe 4–6 stimulus (level51000mA) with each isochronoussound in an alternating fashion, and with an interstimuinterval of 400 ms. The subject could increase or decrethe level of the isochronous sound by 5, 20, or 50mA bypressing one of six buttons. Two matches were obtainedeach isochronous stimulus and averaged to obtain the leused in experiment 1c. These levels were 1000, 1005, 11040, and 1050mA for isochronous periods of 4, 6, 8, 10and 12 ms, respectively.

All stimuli ~electric and acoustic! in both parts of theexperiment were presented using theAPEX software/hardware package~Geurts and Wouters, 2000!.

B. Results and discussion

The pitch matches made by the four normal listenersexperiment 1a were pooled, placed in 0.5-ms bins, and pted in histogram form in Fig. 2. The geometric mean ofthe matches was 5.71 ms, with a standard error acrossjects of 0.23 ms. Psychometric functions describing the p

oc. Am., Vol. 112, No. 2, August 2002

.

inaa

-x-t,k,s-he

re.--ex-d-

g

sse

orels0,

ft-lb--

portion of trials in which each isochronous pulse trainexperiment 1b was judged higher than the 4–6 stimulusshown in Fig. 3~a! for the normal listeners and Fig. 3~b! forthe implant users. The point of subjective equality—at whthese functions cross the 50% point, was estimated by abit function. The geometric means of these values acrsubjects were 5.56 and 5.69 ms for the normal listenersimplant users, respectively. A summary of the results frboth parts of the experiment is shown in Table II. It canseen that the results are highly consistent both across dient procedures~experiments 1a vs 1b! and across the twogroups of listener~experiment 1b!.

The results of experiment 1c are shown in Fig. 3~c!. Itcan be seen that the psychometric functions in all three care monotonic, and do not show any discontinuity at 10 mas might be expected if subjects sometimes heard a pcorresponding to that value. The intercepts of these psycmetric functions with the 50% point were 5.9 ms for cochleimplant user CI2, and 5.7 ms for normal listener NH5. Thevalues are reasonably close to those observed for the ssubjects in experiment 1b. Subject NH1’s data intercept50% point at a period slightly longer than 6 ms, and longthan the 5.4 ms observed in experiment 1b. Unfortunatwe could not measure the intercept precisely, becauseprobit functions used successfully for the other functionsscribed here did not provide an accurate fit to NH1’s funtions in this condition~chi-squared,d f53, p,0.001!.

The results of experiment 1 have implications forthree models of temporal pitch perception described inIntroduction. Figure 4~a! shows the output of an autocorrelogram model ~Meddis and Hewitt, 1991; Meddis an

FIG. 2. Distribution of pitch matches falling into 0.5-ms bins, obtained froexperiment 1a.


re

neionto

se

pe

d-heth

ic-

hery

sgood

6heshese

an ithav-the

ds–6

thho

the

er-dard

d’s

orre-uli,the

ith a

O’Mard, 1997! in response to the 4–6 stimulus used he~The background noise was not used in this simulation!. Thelargest peak corresponds to a period of 10 ms, and thelargest to a period of 4 ms. As discussed in the Introductthe original version of the model would predict matchesthese values, which did not occur in the data. The reviversion~Meddis and O’Mard, 1997! differs not in the auto-correlogram itself but in the decision statistic applied. Scifically, it takes the summary autocorrelograms~‘‘SACFs’’ !of all ‘‘matching’’ stimuli, and calculates the squared Eucliean distance (D2) between each one and the SACF of tcomparison stimulus. When we applied this model to

FIG. 3. ~a! Proportion of trials in which isochronous stimuli, having perioshown on the abscissa, were judged as higher in pitch than the ‘‘4stimulus by the normal listeners in experiment 1a.~b! As in part~a!, but forthe cochlear implant users~of experiment 1b!. ~c! Results of experiment 1cfor one cochlear implant user and two normal listeners. An indication ofpulse rates corresponding to periods of the isochronous pulse trains is sat the top of each plot.


.

xt,

d

-

e

acoustic stimuli of experiment 1a, we observed the predtions shown in Fig. 4~b!. Because a perfectly harmonic~iso-chronous! matching stimulus has peaks at multiples of tperiod, a pulse train with a period of 5 ms provides a vegood ~in fact, the best! match to our 4–6 stimulus. This ibecause it has a peak at 5 ms, which does a reasonablyjob of matching the two peaks in the 4–6 SACF at 4 andms, respectively; it also has a peak at 10 ms which matcthe 10-ms peak in the 4–6 SACF exactly. Accordingly, tfunction relatingD2 to the period of the matching sound haa minimum at 5 ms. On either side of this minimum thfunction rises, but does so less towards shorter periods thdoes towards longer periods, because a matching sounding a period of 4 ms also produces a good match, due to

’’

ewn

TABLE II. Rate of an isochronous pulse train judged equal in pitch to‘‘4–6’’ pulse train of experiment 1. Results from experiment 1b~‘‘2IFCintercept’’! represent the point at which the psychometric functions intcepted the 50% point, as estimated from probit fits. Means and stanerrors are geometric.

Normal listeners Implant users

2IFC 2IFCSubject intercept Match Subject intercept

NH 1 5.40 ms 5.8 ms CI 1 5.55 msNH 2 5.69 ms 6.3 ms CI 2 5.40 msNH 3 5.53 ms 5.2 ms CI 3 6.20 msNH 4 5.62 ms 5.6 ms CI 4 5.58 msNH 5 5.7 ms CI 5 5.65 msMean 5.59 ms 5.71 ms 5.69 msS.E. 0.06 0.23 0.13

FIG. 4. ~a! Summary autocorrelogram produced by Meddis and O’Mar~1997! model in response to the ‘‘4–6’’ stimulus of experiment 1.~b!Squared Euclidean distances, according to Meddis and O’Mard’s autoclogram model, between the 4–6 stimulus and various isochronous stimas a function of their interpulse period. The open square on the right ofplot is the squared Euclidean distance for an isochronous stimulus winterpulse interval of 10 ms.


am

arra

ce

ines-bu-

a.3he

et

dileseaal ist-

ed

aoda

h

segaeer

aon-mpluenantivwtcilehad

ric

ofuslyspec-therd aeg-av-

eat

af-and

dwasor-

1

-ssionddset.iontoec--

4

l-av-onthetrialde-toldeyce-

tch,l of

eo-ub-

in

ias.HL

nc-

large peak in the SACF of the 4–6 stimulus. Additionminima occur for matching sounds having periods of 10@value shown as open square on the right of Fig. 4~b!# and at3.33 ms, both of which produce SACFs having secondpeaks at 10 ms. Unfortunately, the revised autocorrelogmodel ~Meddis and O’Mard, 1997! does not specify how todeal with situations where the squared Euclidean distanindicate the possibility of multiple pitch matches, but neithof the two most obvious options is seen in the data:~i! ifsubjects usually match to the stimulus producing the mmum D2, with the probability of matches to adjacent valubeing inversely proportional toD2, there should be a unimodal distribution of matches with a mean lower than 5 mshigher than 4 ms, or~ii ! if subjects match to stimuli producing a low D2, regardless of their period, there should bebimodal distribution of matches clustered around 5 and 3ms, and, in experiment 1c, there should be a ‘‘bump’’ in tpsychometric function around 10 ms.

The data are also inconsistent with the ‘‘mean rate’’ mric proposed by Carlyon~1996, 1997!, according to whichthe perceived pitch of the 4–6 stimulus should correspona period of 5 ms. The results could, however, be reconcwith Kaernbach and Demany’s conclusion that pitch is baon first order intervals, if one additionally assumes thlonger intervals receive a larger weight than shorter intervwhen pitch is calculated. In Sec. IV, we describe a modewhich pitch is derived from such a weighted sum of firorder intervals.

Finally, we should mention two related findings reportby other authors. First, Kaernbach and Demany~1998! men-tioned the results of an informal experiment in which, forsequence consisting of alternating 5-ms and 3-ms perisubjects sometimes heard a pitch corresponding to 5 mssometimes~although more rarely! to 3 ms. In contrast, theresults of experiment 1a indicate that subjects never matcto the shorter of our two intervals~4 ms!. We are not surewhether this is due to a difference between the stimuli uin the two experiments, or whether a more formal investition using Kaernbach and Demany’s stimuli would yield rsults more similar to our own. Second, McKay and McDmott ~1996! required cochlear implant users to performpitch-rating experiment using electrical pulse trains that csisted of ten pulses evenly distributed throughout each 10period, and with two of these pulses having a higher amtude than the others. The intervals between higher-amplitpulses alternated between two values, which, for differstimuli, were 1 and 9 ms, 2 and 8 ms, 3 and 7 ms, and 46 ms. Consistent with our results, they found that the relapitch varied in the same direction as the longer of the tinterpulse intervals. However, the exact value of this piwas not the primary focus of their study, and a more detainterpretation of their results is complicated by the fact tpitch would also have been affected by the lower-amplitupulses, in a way which varies across listeners~McKay et al.,1995; McKay and Carlyon, 1999!. We plan to report theresults of a study of the pitch of acoustic and electamplitude-modulated pulse trains in a future publication~vanWieringenet al., 2002!.


ls

ym

esr

i-

t

3

-

toddtlsn

s,nd

ed

d-

--

-s

i-detdeohdte

III. EXPERIMENT 2

A. Rationale and method

Carlyon ~1996! presented normal listeners with pairsinharmonically related pulse trains, presented simultaneoand passed through the same bandpass filter. When thetral components of the pulse trains were unresolved byperipheral auditory system, he reported that subjects heaunitary ‘‘crackle-like’’ percept, and could not extract thpitches of the two underlying pulse trains. He further sugested that the pitch of this mixture corresponded to theerage number of pulses in a fairly long~.100 ms! portion ofthe stimulus. This ‘‘mean rate’’ hypothesis received somsupport from the later finding that deleting some pulsesrandom from an otherwise isochronous pulse train couldfect its pitch, which dropped systematically as a greatergreater proportion of pulses was deleted~Carlyon, 1997!.However, the pitch of mixtures of inharmonically relatepulse trains had never actually been measured, and thisthe purpose of experiment 2. Only acoustic stimuli and nmal listeners were used in this experiment.

Mixtures of pulse trains having rates ofF1 andF2 Hzwere played concurrently out of two DACs of a CED 140plus laboratory interface~16-bit resolution, sampling rate520 000 Hz!, antialiased~Kemo VBF 25.01, cutoff 8300Hz, attenuation rate5100 dB/octave!, bandpass filtered between 3900 and 5400 Hz~one low-pass and one high-paKemo VBF 25.03 in series for each pulse train, attenuatrate 48 dB/octave!, mixed with a continuous pink noise, anpresented to the left earpiece of a Sennheiser HD250 heaThese mixtures had an overall level of 61 dB SPL, a duratof 400 ms, and were turned on and off abruptly priorfiltering. The continuous pink-noise background had a sptrum level of 6 dB SPL at 4000 Hz. On different pitchmatching runs the value ofF1 was either 70, 92, 109, or 12Hz, andF2/F1 was always 1.29~F2590, 119, 141, or 160Hz!. Each run started with one of the mixtures being folowed, 500 ms later, by a single isochronous pulse train hing a rate selected at random from a uniform distributiranging from 50 to 400 Hz. Subjects could then adjustrate of the single pulse train to be presented on the nextby pressing one of four buttons, which increased orcreased the value by a factor of 1.1 or 1.4. Subjects werethat, if they did hear more than one pitch in the mixture, thwere to match to the strongest pitch. The adjustment produre continued until the subject was satisfied with the maand s/he pressed a separate button indicating this. A tota12 pitch matches was obtained at each value ofF1 for eachof four normally hearing subjects. These 12 values were gmetrically averaged to obtain the pitch match for each sject and condition. In addition, six matches were obtainedeach condition to theF1-alone andF2-alone stimuli. Thiswas done in order to check for any systematic response bAll subjects had absolute thresholds of less than 15 dB~ANSI, 1969! at all audiometric frequencies.

B. Results

The symbols in each panel of Fig. 5~a! show the pitchmatches obtained for each listener in experiment 2, as a fu


rea

e

inaterd bydr-rsnd

anta.

heual

to

als,ri-ther

arebed

a

ch,osely,

ientist of

uls

o

noliib

tion of F1. Matches to the mixtures of two pulse trains ashown by upright triangles. In all cases, these matchesvery close to the matches obtained to theF2 stimulus alone~inverted triangles!. Mean data are shown by the dashed linconnecting open symbols in Fig. 5~b!, which also contains

FIG. 5. ~a! Pitch matches obtained by each listener in experiment 2 to ptrains ofF1 Hz ~diamonds!, F2 Hz ~inverted triangles!, and to mixtures ofF11F2 Hz ~upright triangles!. The geometric means and standard errorsthe matches are plotted as a function ofF1. ~b! Geometric means of thematches obtained from the four listeners in experiment 2. Solid lines withsymbols are the predictions of the model described in Sec. IV A 2. Ssquares indicate the predictions of the revised ‘‘mean rate’’ model descrin Sec. IV B 1.


re

s

two additional lines, whose meanings will be discussedSec. IV. The absence of any matches significantly grethanF2 argues against the mean rate hypothesis proposeCarlyon ~1996, 1997!, according to which subjects woulmatch toF11F2 Hz. However, some of the shorter intepulse intervals in the stimulus will be removed by factosuch as auditory filter ringing and neural refractoriness, aso it might be possible for a modified version of the merate model to produce matches more in line with the daThis possibility is examined further in Sec. IV.

The pattern of matches is also slightly different from tsummary autocorrelogram, which has peaks of roughly eqheight at the periods corresponding to 1/F1 and 1/F2; this isshown in Fig. 6, which plots Meddis and O’Mard’s SACFour stimulus whereF15109 Hz. In Sec. IV B we willpresent a quantitative model based on first-order intervwhich, we will argue, is consistent with the results of expement 2, as well as those of experiment 1 and of some oresults in the literature.

IV. DISCUSSION

A. Analysis based on first-order intervals

1. Overview

As discussed earlier, the results of our experimentsconsistent neither with the autocorrelogram model describy Meddis and his colleagues~Meddis and Hewitt, 1991;Meddis and O’Mard, 1997!, nor with Carlyon’s mean ratehypothesis~Carlyon, 1996!. In this section, we proposemodel in which ‘‘purely temporal’’ pitch is derived from aweighted sum of first-order interpulse intervals, and whiwe argue, can account not only for our own data but thfrom two other paradigms reported in the literature. Initialwe discuss the model in terms of intervals betweenstimuluspulses. This has the advantage of simplicity, and is sufficfor explaining the major features of the model. However, itclear that any pitch mechanism must operate on the outputhe auditory nerve~AN!, and that this will differ from the

e

f

oded

d in
FIG. 6. Summary autocorrelogram produced by Meddis and O’Mard’s~1997! model in response to the mixture of 109- and 141-Hz pulse trains useexperiment 2.

f-

y-

r

FIG. 7. ~a! Distribution of first-orderinterpulse intervals in the mixture o70- and 90-Hz pulse trains used in experiment 2. Parts~b!–~e! are similar,but show simulated ISIHs obtained bpassing the stimulus through a simulated auditory filter and low-pass fil-tering to mimic properties of the innehair cells~see the text for details!. F1is 70 Hz in part~b!, 92 Hz in part~c!,109 Hz in part~d!, and 124 Hz in part~e!.

-onut

erteactioe

b-therrbreeron

teva

inlel

tent

ert it

bydnic

yedth-

tchsesemmesval,ionnter-ht

ontateth-

st-ted

raw input due to basilar-membrane ringing~for acousticstimuli! and to refractory effects~for both acoustic and electric stimuli!. We then move on to incorporate the effectsthis peripheral processing for stimuli, such as those withmerous short interpulse intervals, where they are likelyhave a significant effect. Finally, for all stimuli, we considwhether the transformations imposed by the auditory sysup to and including the auditory nerve are sufficient tocount for the pattern of results obtained. This first subsecgives an overview of our general approach, which is thquantified in the next subsection.

At first sight, the results of experiment 2, in which sujects matched to the higher of the two rates present inmixture, seem at odds with those of experiment 1, whthey matched to the lower pitch~period close to 6 ms, rathethan to 4 ms!. However, this discrepancy can be resolvedconsidering the pattern of first-order intervals in the mixtuused in experiment 2; this is illustrated for the case whF1570 Hz in Fig. 7~a!. It can be seen that the most commfirst-order interval corresponds to 1/F2 (1/90 Hz511.1 ms), and that this is also the longest first-order inval present. The reason for the absence of first-order interat 1/F1 is apparent upon inspection of Fig. 1~f!. The intervalbetween successive pulses in theF1-Hz pulse train~graylines! is so long that there is always at least one intervenpulse from theF2-Hz stimulus~black lines!. Hence, a modewhich operates on first-order intervals and selectiv


f-

o

m-nn

ee

yse

r-ls

g

y

weights longer intervals will yield a match to about 1/F2. Asdiscussed in Sec. II B, this general scheme is also consiswith the results of experiment 1.

An additional attraction of the idea that longer first-ordintervals dominate purely temporal pitch perception is thacan qualitatively account for some recent data presentedPlack and White~2000b!. In one condition, they presentetheir subjects with a 40-ms unresolved ‘‘standard’’ harmocomplex, having a nominalF0 of 250 Hz, in which the fifthand all subsequent ‘‘pitch pulses’’ were advanced or delaby either 0, 1, or 3 ms—thereby either shortening or lengening a single interpulse interval. Subjects adjusted theF0 ofa perfectly periodic comparison sound so that its pimatched that of the standard. Delaying the last few pulproduced much larger pitch shifts than did advancing thby the same amount. This can be understood if one assuthat imposing a delay increases a single interpulse interwhich then receives a large weight in the pitch estimatprocess. Imposing an advance shortens one interpulse ival, but this shortened interval will receive a smaller weigwhen estimating pitch.3

Finally, it should be noted that a pitch analysis basedfirst-order intervals could, in principle, account for the dathat Carlyon~1997! interpreted as support for the mean ramodel. As he pointed out, deleting some pulses from an oerwise periodic pulse train will lengthen some of the firorder interpulse intervals, and it is clear that any weigh


hetion

ngap

fallha

unwec

he

tiot-aire

heese

er

ha

h

enrorv2%alp

llit

lee

dy,

obso

qui-

e-r-

orecalthe

ata’s

at-fortly

rick

to

ndal

bedl for

dednd

sum of such intervals will produce a lower pitch than is tcase for an uninterrupted pulse train. In the next subsecwe investigate whether a simple model can account quatatively for the reductions in pitch that resulted from deletipulses, as well as the effects reported in the present pand in the study described by Plack and White~2000b!.

2. General model

In this subsection we describe the general format omodel which estimates pitch based on first-order intervWe then describe some constraints imposed on that modethe data presented here and elsewhere, and considereffectively it could account for the data. We should statethe outset that we do not have sufficient data to specifyambiguously how such a model should work. However,believe that it is worthwhile to determine whether a modbased on a reasonable set of parameters can be reconwith data from a fairly wide range of paradigms.

As stated above in Sec. IV A 1, we will first describe tmodel in terms of first-order intervals betweenstimuluspulses, rather than between neural spikes. This simplificais likely to be fairly adequate for stimuli in which the firsorder intervals are fairly long—such as the 4–6 pulse trof experiment 1, but will require modification in cases whenumerous short interpulse intervals are present, and wthe effects of basilar-membrane ringing and refractorinare likely to be greatest. These modifications will be dscribed after the basic model has been introduced.

In general terms, we propose that pitch~P! will be de-termined by the first-order intervals present~t!, by theweights applied to intervals of different lengths (W(t)), andby some function of the relative proportions of those intvals (f (pt))

P5( t51

n W~t t! f ~pt!

( t51n W~t t! f ~pt!t t

. ~1!

For example, the standard stimulus of experiment 1equal proportions of 4-ms and 6-ms intervals, sot154 ms,t256 ms, andp15p250.5. The fact that the observed pitcwas much closer to 6 ms than to 4 ms reveals thatW(t) mustbe an expansive function, at least over the range betweand 6 ms. Specifically, given that the average intercept acthe ten listeners of experiment 1b corresponded to an inteof 5.64 ms, the 6-ms interval should receive about 8~~5.64–4!/~6–4!! of the total weights, and the 4-ms intervonly about 18%—a ratio of 4.56:1. This relationship is aproximated by the form ofW(t) shown by the thick solidline in Fig. 8, which is, as described below, also generaconsistent with the data of experiment 2, of Plack and Wh~2000b!, and of Carlyon~1997!. It approximates a power lawfor intervals between about 1 and 7 ms, but becomesexpansive at longer intervals, and reaches a plateau betw8 and 12 ms. Given the fact that very low repetition ratesnot elicit a pitch percept at all~e.g., Guttman and Pruzansk1962!, it seems probable thatW(t) will start to drop at evenlonger intervals. Here, we do not attempt to model datatained with stimuli containing such long intervals, andhave leftW(t) undefined fort.12 ms.


nti-

er

as.byowt-

eliled

n

n

res-

-

d

4ssal

-

ye

ssen

o

-

To account for the data of Plack and White~2000b!, weneed to specify the form off (pt). This was not an issue withexperiment 1, because the two interpulse intervals were eprobable. Using the form ofW(t) described in Fig. 8, wecould approximate their data by assuming thatf (pt)5pt

2.This ‘‘squaring’’ was chosen to fit the data, but has the dsirable feature of minimizing the influence of very rare intepulse intervals. This could render the pitch estimate mrobust to the influence of both external and physiologinoise. The results obtained using the model are shown byopen diamonds in Fig. 9, along with Plack and White’s d~solid line! and the predictions of Meddis and Hewitt~1991! autocorrelogram model~dotted lines!, as imple-mented by Plack and White~2000b, Fig. 7!. It can be seenthat the predictions of the present model, while underestiming the pitch shifts in some conditions, account quite wellthe general form of the data. The fit can be improved sligh

FIG. 8. Heavy solid line: a possible form of the functionW(t) in Eq. ~1!.Dashed line: function fitted to the mean auditory nerve data of Fitzpatet al. ~1999!. The function is zero for intervals~t! below 1 ms, equal to0.8410.63 log10(t/8) for 1 ms,t,8 ms, and to 0.910.15 log10(t/20) fort.8 ms. Faint line: fit to Kiang’s~1965! AN data, as a function of theinterpulse interval~t! of isochronous pulse trains. The function is equal0.7020.6* (log10(10/t)) for t,10 ms, and 0.9520.25* (log10(100/t)) fort>10 ms.

FIG. 9. The thick solid line shows the pitch shifts obtained by Plack aWhite ~2000b! to 40-ms 250-Hz pulse trains in which one interpulse intervhad been reduced~negative values on abscissa! or increased~positive val-ues!. Solid lines with diamonds show the predictions of the model descriin Sec. IV B. The unconnected square shows the predictions of the modea 3-ms reduction in interpulse interval, when refractory effects are inclu~see the text for details!. The dashed line shows the predictions of Plack aWhite’s implementation of Meddis and Hewitt’s~1991! autocorrelogrammodel.


of

blt t

hewls

iv

tllydts

thp

ec

derdei

inope

nsch

th

edyi

ofto

am

edio

nal-

ortrtu-

er-

ithnlyic-

noith

lels,ntthe

twe

dele

nuer

ter-

ofrvaltherp-

o

ch-ter-er-The

rc-

d-fit6the

ight

t 1

roic

esc-ig

if we follow Plack and White’s observation that in onetheir conditions, where the last few pulses were advanced3 ms, the resulting 1-ms interpulse interval would probanot be represented in the auditory nerve. They argued tharefractory properties of the auditory nerve~Kiang, 1965!made it likely that many fibers would not respond to tpulse marking the end of this short interval, and so, folloing their suggestion, we have replaced this 1-ms interpuinterval with one of 5 ms. Doing so introduces a negatpitch shift in the condition with a23-ms pulse shift~opensquare!, which is more in line with the data. With or withouthis modification, the fit, although imperfect, is substantiabetter than that obtained with the autocorrelogram mo~dotted line!, which performs very poorly, and which prediclarge positive shifts for the21-ms and13-ms delays, andlarge negative shifts for the23-ms and11-ms delays. Simi-larly unsuccessful predictions were generated by anomodel, based on a spectral analysis of simulated neural strains, that was considered by Plack and White~2000b!. Rea-sons for the failure of the autocorrelogram and ‘‘spike sptrum’’ models are discussed in detail in that article.

The present model can also account for the resultsCarlyon ~1997!, who asked subjects to adjust theF0 of apulse train from which a proportion of pulses had beenleted, so that its pitch matched that of a 200-Hz ‘‘standapulse train, which had either 15%, 30%, or 45% of pulsdeleted. By manipulating the proportion of deleted pulsesthe ‘‘matching’’ train, it was determined that a 10% droppulse probabilityre the standard produced about a 4% drin pitch, regardless of the overall proportion of pulses dleted from the standard stimulus. The model predictioshown in the right-hand column of Table III, produces pitshifts which are of a similar size to those obtained~middlecolumn!, and which, like the data, do not vary markedly withe overall proportion of pulses deleted in the Carlyon~1997!paradigm.

The stimuli of experiment 2 differ from those describso far in that, as shown in Fig. 7~a!, there are numerous vershort first-order intervals in the stimulus. These intervals wbe modified by auditory filter ringing, the time constantthe inner hair cell membrane potential, and neural refracriness. We have simulated peripheral processes by a ‘‘gmatone’’ ~Pattersonet al., 1988! auditory filter centered on4500 Hz, followed by an 800-Hz low-pass filter implementin order to simulate the receptor potential transfer funct~Holton and Weiss, 1983!. As can be seen in Figs. 7~b!–~e!,

TABLE III. The middle column shows the percentage change in pitch pduced by deleting an additional 10% of pulses from a pulse train in wheither 15%, 30%, or 45% of pulses had already been deleted. These rare taken from Carlyon~1997!. The right-hand column shows the preditions of the model described in Sec. IV B, using the function shown in F8.

% deleted in standard

% pitch change for 10% pulse reduction

Data Model

15 4 3.930 5.2 6.045 3.9 5.5


byyhe

-e

e

el

erike

-

of

-’’sn

-,

ll

--

n

this removes very short intervals and introduces additiointervals longer than 1/F2. However, the details of this simulation are unlikely to be crucial, because the form ofW(t)shown in Fig. 8 greatly reduces the influence of the shintervals that the simulation removes. This is perhaps fonate, given the preliminary report by Longet al. ~2002! thatmost cochlear implant users, like normal listeners, also pceive these mixtures as having a pitch close toF2 Hz. In-deed, we have measured the predictions of the model wand without this additional filtering, and have observed ominor differences in the predictions obtained. The predtions of the model are shown by the solid lines withsymbols in Fig. 5~b!, and show reasonable agreement wthe mean data, except whenF1570 Hz, where the matchedpitch is approximately 7% below the predictions.4

3. Summary

In this subsection we have shown that a fairly simpmodel, which selectively weights longer first-order intervacan account reasonably well for data from four differeparadigms. It is acknowledged that the parameters ofmodel were developed on apost hocbasis, in order to best fithe data, and it should also be stressed that the solutionarrived at is unlikely to be unique. Nevertheless, the morests on only two crucial assumptions, both of which wconsider reasonable. One is that the form ofW(t), althoughexpansive for short and moderate ISIs, should not contito expand indefinitely ast is increased to longer and longevalues. The other is that the functionf (pt) should also beexpansive, thereby reducing the influence of very rare inpulse intervals.

B. Possible physiological correlates and limitationson rate discrimination

1. Comparison to AN recovery data

The functionW(t) described by the solid line in Fig. 8could be interpreted as reflecting an increased probabilityneural firing in response to a stimulus pulse as the intesince the previous pulse is increased. To evaluate wheour weighting function could simply reflect refractory proerties of the AN, we have plotted functions derived from twsets of AN data in Fig. 8. The first of these~thin unbrokenline!, described by Kiang~1965!, shows the probability of acat AN fiber firing in response to any one pulse in an isoronous acoustic pulse train, as a function of interpulse inval; the duration of the pulse train was 60 s when the intpulse interval was 5 ms or greater, and 24 s. otherwise.second function~dashed line! is a fit to the data of Fitzpatricket al. ~1999!, who measured the probability of a cat AN fibefiring to the second of two equal-amplitude clicks, as a funtion of interpulse interval.

Neither of the two AN recovery functions seems aequate to account for the results. Recall that, in order tothe results of experiment 1, in which the pitch of the 4–stimulus corresponded to an average period of 5.64 ms,6-ms intervals had to receive more than 4.5 times the weof the 4-ms intervals. This is reflected in the form ofW(t),but both of the AN functions are steepest between abou

-hults

.


toeerik

te

ncicein

thte

lliif-

bise

pasv

ve

rata

seta

-mb

Nnxthnecho

atlder

ded

ver,chinssticableueureics.lvedcha-li-

-lved

l-igms

od

ics.f

mem

ed

hasa-her-

t 5.7twofmerre-Were are

n

re-dofthe

and 3 ms, and show at most a 1.2-fold increase from 4ms. Furthermore, our attempts to model the results of expment 2 using a revised ‘‘mean rate’’ approach, which opated not on stimulus interpulse intervals but on AN interspintervals simulated from Fitzpatricket al.’s ~1999! data, wereunsuccessful: The resulting pitch matches overestimathose observed by more than 40% at all values ofF1 studied@Fig. 5~b!, solid squares#.5 Finally, it is worth noting that it isunlikely that an account based purely on AN recovery futions could account for the results of experiment 1, whwere very similar for electric and acoustic stimulation dspite the fact that different peripheral processes werevolved in the two cases. This does not mean, of course,refractoriness effects at higher levels of the auditory syscould not produce a function more similar toW(t). In thisregard, it is worth noting that Fitzpatricket al.also measuredrecovery functions from the cochlear nucleus, inferior coculus, and cortex. Although the form of the functions dfered between these neural structures, in no case wasincrease between 4 and 6 ms as great as that shown byW(t).

2. Coding by first-order intervals

Given the above conclusion that the form ofW(t) can-not be accounted for by properties of AN fibers, a reasonainterpretation of our model is that a more central mechanperforms a weighted combination of first-order intervals btween spikes in the AN response to the stimulus. One apent problem with this approach is that representations baon first-order interspike intervals within a single neuron hathe undesirable characteristic of being level dependent~e.g.,McKinney and Delgutte, 1999!.6 However, the problem oflevel dependence can be overcome if the responses of seneurons, innervating separate hair cells, are combinedbeforethe pitch estimate is made. Indeed, Carlyon~1997! has ar-gued that such a combination of responsesmusttake place ifthe auditory system is to discriminate between a pulse tfrom which several pulses have been deleted, and an inpulse train that has been reduced in level: In both caindividual auditory-nerve fibers will ‘‘miss’’ some spikes, buwith deleted pulses the same spikes will be missed byfibers~leading to an ‘‘irregular’’ percept!, whereas an attenuated intact pulse train will sound smooth but quiet, presuably because different neurons miss different pulses, andcause the responses of several neurons will be summed.that this differs from the form of analysis used by Cariaand Delgutte~1996!, who showed that, for periodic completones, pitches estimated from first-order intervals inauditory-nerve firing patterns were strongly level dependeeven when the responses of multiple fibers were combinHowever, they calculated an interval distribution for eaand every auditory-nerve fiber, before combining eachthese distributions into a summary measure. We think this a priori very unlikely that a neural pitch mechanism wouactually perform this sort of calculation on a fiber-by-fibbasis.7


6ri--e

d

-h--atm

-

the

lem-r-

ede

ral

incts,

ll

-e-otei

et,d.

fit

C. Implications for more general models of pitchperception

Section IV B described one way in which a weightesum of first-order intervals can account for data obtainfrom a variety of paradigms. It should be stressed, howethat all of these paradigms involved purely temporal pitperception, in that they involved either electric pulse traapplied to a single channel of a cochlear implant, or acoupulse trains whose spectral components were unresolvby the peripheral auditory system. We do not wish to argthat a similar process necessarily applies to the pitch of ptones or of complex tones containing resolved harmonIndeed, there is substantial evidence that pitches of resoand unresolved harmonics are extracted by separate menisms: That of unresolved harmonics leads to differencemens that depend more strongly on duration~Plack and Car-lyon, 1995; White and Plack, 1998!, can be ‘‘reset’’ by abrief silent interval~Plack and White, 2000a!, is affected bypreceding and following complexes regardless of theirF0~Micheyl and Carlyon, 1998!, and cannot be accurately compared to that of a simultaneously presented group of resoharmonics in a different frequency region~Carlyon andShackleton, 1994!. This conclusion has recently been bostered by evidence using a transfer-of-learning parad~Grimault et al., 2002!,8 and by Kaernbach and Bering’~2001! finding that a given listener’sF0 difference limen~‘‘ F0DL’’ ! for a group of unresolved harmonics is a gopredictor of that listener’sF0DL for another group of unre-solved harmonics, but not for a group of resolved harmonWhat our analysisdoes imply, however, is that models opitch perception which rely on higher-order intervals~Med-dis and Hewitt, 1991; Pattersonet al., 1995; Meddis andO’Mard, 1997! cannot account for the purely temporal forof pitch perception studied here. This is of course a problfor those models that rely on higher-order intervalsandclaim to be able to account for the pitches of both resolvand unresolved harmonics.

D. Summary and conclusions

This article describes two main findings, both of whicwere obtained using stimuli where place of excitation wunlikely to provide a cue, and both of which have implictions for theories of temporal pitch perception. First, tpitch of a pulse train in which the interpulse intervals altenate between 4 and 6 ms corresponds to a period of aboums, both in acoustic and electric hearing. Second, wheninharmonically related (ratio51.29) pulse trains, each owhich was perfectly periodic, were passed through the sabandpass filter and presented acoustically, the pitch cosponded roughly to that of the higher-rate pulse train.have argued that the results presented here and elsewheconsistent neither with Carlyon’s mean rate model~Carlyon,1995!, nor with ‘‘autocorrelogram’’ models that operate ohigher-order intervals~Meddis and Hewitt, 1991; Meddisand O’Mard, 1997!. Rather, these and other phenomenalating to ‘‘purely temporal’’ pitch perception can be captureby a model in which pitch is derived from a weighted sumthe first-order intervals present in the stimulus, and where


org

r-eom

rnin-up

in

na

Tto

in

z a

thve

gg,

b

ng

tesac

heeyoe

lo

al

osthed

ichetde,

is

ts

rvaalsdu

td.rth

rvaas

of, andcheson-ulseot

e fi-es.

owthe

also

thennicsse-

.,stic

oc.

-o-

xoc.

-

itch

R.udi-

ory

inn,

rsiol

form of this weighting favors longer intervals, at least finterval lengths up to 12 ms. The form of this weightinfunction is unlikely to result simply from refractory propeties of the AN; rather, it is likely to be applied by morcentral stages to the composite pattern of firing derived frthe combination of many AN fibers.

ACKNOWLEDGMENTS

We thank Peter Cariani for useful discussions conceing the limitations of using only acoustic pulse trains tovestigate temporal pitch perception. Author CJL was sported by a grant from the Leverhulme Trust to RPC.

1In one experiment, Thurlow and Small used a regular 100-Hz pulse trawhich an additional pulse had been addedx ms after every one of theoriginal pulses. In one condition they passed these stimuli through a ‘‘row’’ bandpass filter centered on 5 kHz, and observed that asx was in-creased from 0 to about 2 ms, there was a sharp increase in pitch.would have caused the longest first-order interval to decrease from 10ms, and so is consistent with thelongestintervals dominating pitch. How-ever, in a subsequent experiment they broadened the bandpass filter~by anunspecified amount! and used a stimulus in whichx gradually increasedfrom 0 to 10 ms; presumably, this was achieved by mixing two pulse traof slightly different rates. They reported that, at small values ofx, subjectsheard a very high pitch of 1000–2000 Hz, which then dropped to 200 Hx was increased to 5 ms, and then rose to 1000–2000 Hz asx was increasedfurther. The fact that these very high pitches were reported only forbroader filter bandwidth, for which place-of-excitation cues would habeen strongest, combined with the fact that intervals correspondin1/2000 Hz~0.5 ms! would have been obscured by auditory filter ringinadd weight to the conclusion that their data were strongly influenced‘‘place’’ cues.

2Many of these limitations were first pointed out by Peter Cariani in postito an online discussion group.

3Plack and White also studied conditions in which an additional silent inval was inserted just before the first delayed or advanced pulse. Thegap was inserted in the comparison stimulus. Generally, this produmuch smaller pitch shifts than in the conditions with no gap, a finding tattributed to the gap ‘‘resetting’’ the pitch integration mechanism. Thhave recently provided further evidence for this ‘‘resetting’’ phenomen~Plack and White, 2000a!, which we do not attempt to capture in thpresent model.

4A second complication arises because the distribution of intervals be1/F2 is continuous. In order to fit within the framework of Eq.~1!, we needto transform this into a discrete distribution by placing interpulse intervthat fall into a narrow range into a single bin. Because the functionf (pt)5pt

2 is expansive, the width of the these bins will influence the effect thintervals have on the predicted pitch estimate: Wider bins will increaseprobability within each bin, and will increase the effect of widely dispersintervals. When plotting Fig. 7 we chose a bin width of 0.1 ms, whcorresponds roughly to the change in period that normal listeners can din a perfectly periodic pulse train filtered in the same way as those useexperiment 2, and with anF0 of about 200 Hz; this is equal to a ratdifference of about 4 Hz~Krumbholz et al., 2000; Carlyon and Deeks2002!. However, it is recognized that the choice of bin widths usedsomewhat arbitrary, and that the deviation in predicted matches fromF2@the largest peak in the interspike histogram~‘‘ISIH’’ !# would change as afunction of bin width. It is also worth noting that the accuracy of the fiwill also depend on the exact form off (pt) at very low probabilities.

5The predicted mean rate was obtained by weighting the first-order intein the stimulus with this function, adding up the total number of intervremaining, and then dividing by the stimulus duration. The same procewas applied to the variable~matching! stimulus. TheF0’s of the regular-rate stimuli matched to each of theF1 andF2 mixtures used in experimen2, as predicted by the modified mean rate model, were then calculate

6McKinney and Delgutte~1999! reported that, for a 220-Hz tone, first-ordeneural intervals were equally frequent at one, two, and three timesperiod when the tone level was 52 dB SPL, but that a single intecorresponding to the period dominated the response when the level w


-

-

in

r-

his8

s

s

e

to

y

s

r-meedy

n

w

s

ee

ectin

ls

re

el67

dB SPL. This observation should be qualified by the fact that we knowno analogous measures for filtered pulse trains like those used herethat, to the best of our knowledge, no one has performed pitch matbetween pulse trains differing greatly in level. However, it seems reasable to assume that the responses of individual nerve fibers to filtered ptrains will be level dependent, but that the pitch of these stimuli will nvary dramatically across level.

7An additional advantage of combining responses across auditory-nervbers is that it could reduce the influence of ‘‘spontaneous’’ neural spikThis would occur if the postsynaptic neuron~e.g., in cochlear nucleus!required more than one input neuron to fire within a narrow time windbefore generating a spike. Summation is not, however, essential forinfluence of spontaneous spikes to be minimized, because this isachieved by the square-law form off (pt).

8Grimault et al. ~2001! trained subjects onF0 discrimination of either re-solved or unresolved harmonics filtered between 1375–1875 Hz, andtested discrimination performance for resolved and unresolved harmofiltered into different frequency regions. Compared to a pretraining baline, those trained on one type of complex~resolved or unresolved! im-proved more for complexes of a similar ‘‘resolvability.’’

ANSI ~1969!. ANSI S3.6-1969, ‘‘Specifications for audiometers’’~AmericanNational Standards Institute, New York!.

Blamey, P. J., Dowell, R. C., Tong, Y. C., Brown, A. M., Luscombe, S. Mand Clark, G. M.~1984!. ‘‘Speech processing strategies using an acoumodel of a multiple-channel cochlear implant,’’ J. Acoust. Soc. Am.76,104–110.

Burns, E. M., and Viemeister, N. F.~1976!. ‘‘Nonspectral pitch,’’ J. Acoust.Soc. Am.60, 863–869.

Burns, E. M., and Viemeister, N. F.~1981!. ‘‘Played again SAM: Furtherobservations on the pitch of amplitude-modulated noise,’’ J. Acoust. SAm. 70, 1655–1660.

Cariani, P. A., and Delgutte, B.~1996!. ‘‘Neural correlates of the pitch ofcomplex tones. I. Pitch and pitch salience,’’ J. Neurophysiol.76, 1698–1716.

Carlyon, R. P.~1995!. ‘‘Extracting the fundamental frequencies of two concurrent sounds,’’A Biological Framework for Speech Perception and Prduction ~Kyoto, Japan!.

Carlyon, R. P.~1996!. ‘‘Encoding the fundamental frequency of a completone in the presence of a spectrally overlapping masker,’’ J. Acoust. SAm. 99, 517–524.

Carlyon, R. P.~1997!. ‘‘The effects of two temporal cues on pitch judgments,’’ J. Acoust. Soc. Am.102, 1097–1105.

Carlyon, R. P.~1998!. ‘‘Comments on ‘A unitary model of pitch perception@J. Acoust. Soc. Am.102, 1811–1820~1997!#,’ ’’ J. Acoust. Soc. Am.104,1118–1121.

Carlyon, R. P., and Deeks, J. M.~2002!. ‘‘Limitations on rate discrimina-tion,’’ J. Acoust. Soc. Am.~in press!.

Carlyon, R. P., Demany, L., and Deeks, J.~2001!. ‘‘Temporal pitch percep-tion and the binaural system,’’ J. Acoust. Soc. Am.109, 686–700.

Carlyon, R. P., and Shackleton, T. M.~1994!. ‘‘Comparing the fundamentalfrequencies of resolved and unresolved harmonics: Evidence for two pmechanisms?,’’ J. Acoust. Soc. Am.95, 3541–3554.

Fitzpatrick, D. C., Kuwada, S., Kim, D. O., Parham, K., and Batra,~1999!. ‘‘Responses of neurons to click-pairs as simulated echoes: Atory nerve to auditory cortex,’’ J. Acoust. Soc. Am.106, 3460–3472.

Geurts, L., and Wouters, J.~2000!. ‘‘A concept for a research tool for ex-periments with cochlear implant users,’’ J. Acoust. Soc. Am.108, 2949–2956.

Grimault, N., Micheyl, C., Carlyon, R. P., Arthaud, P., and Collet, L.~2002!.‘‘Evidence for two pitch encoding mechanisms using a selective audittraining paradigm,’’ Percept. Psychophys.64, 189–197.

Guttman, N., and Pruzansky, S.~1962!. ‘‘Lower limits of pitch and musicalpitch,’’ J. Speech Hear. Res.5, 207–214.

Hoekstra, A.~1979!. ‘‘Frequency discrimination and frequency analysishearing,’’ Ph.D. Institute of Audiology, University Hospital, GroningeNetherlands.

Holton, T., and Weiss, T. F.~1983!. ‘‘Receptor potentials of lizard cochleahair cells with free-standing stereocilia in response to tones,’’ J. Phy345, 205–240.

Kaernbach, C., and Bering, C.~2001!. ‘‘Exploring the temporal mechanismsinvolved in the pitch of unresolved harmonics,’’ J. Acoust. Soc. Am.110,1039–1047.


tus

’s

m.

us

le

J.

LO

Th

tifi

’’ J

C

d

d a

-

plexSoc.

doc.

dis-

’’ J.

s-

f

Kaernbach, C., and Demany, L.~1998!. ‘‘Psychophysical evidence againsthe autocorrelation theory of auditory temporal processing,’’ J. AcoSoc. Am.104, 2298–2306.

Kiang, N. Y. S. ~1965!. Discharge Patterns of Single Fibers in the CatAuditory Nerve~M.I.T. Press, Cambridge, MA!.

Krumbholz, K., Patterson, R. D., and Pressnitzer, D.~2000!. ‘‘The lowerlimit of pitch as determined by rate discrimination,’’ J. Acoust. Soc. A108, 1170–1180.

Licklider, J. C. R.~1951!. ‘‘A duplex theory of pitch perception,’’ Experi-entia7, 128–133.

Long, C. J., Carlyon, R. P., McKay, C. M., and Vanat, Z.~2002!. ‘‘Temporalpitch perception: Examination of first-order intervals,’’ Int. J. Audiol.~inpress!.

McKay, C. M., and Carlyon, R. P.~1999!. ‘‘Dual temporal pitch perceptsfrom acoustic and electric amplitude-modulated pulse trains,’’ J. AcoSoc. Am.105, 347–357.

McKay, C. M., and McDermott, H. J.~1996!. ‘‘The perception of temporalpatterns for electrical stimulation presented at one or two intracochsites,’’ J. Acoust. Soc. Am.100, 1081–1092.

McKay, C. M., and McDermott, H. J.~1998!. ‘‘Loudness perception withpulsatile electrical stimulation: The effect of interpulse intervals,’’Acoust. Soc. Am.104, 1061–1074.

McKay, C. M., McDermott, H. J., and Carlyon, R. P.~2000!. ‘‘Place andtemporal cues in pitch perception: are they truly independent?,’’ AR~http://ojps.org/ARLO/top.html! 1, 25–30.

McKay, C. M., McDermott, H. J., and Clark, G. M.~1995!. ‘‘Pitch matchingof amplitude modulated current pulse trains by cochlear implantees:effect of modulation depth,’’ J. Acoust. Soc. Am.97, 1777–1785.

McKinney, M. F., and Delgutte, B.~1999!. ‘‘A possible neurophysiologicalbasis of the octave enlargement effect,’’ J. Acoust. Soc. Am.106, 2679–2692.

Meddis, R., and Hewitt, M.~1991!. ‘‘Virtual pitch and phase sensitivitystudied using a computer model of the auditory periphery: Pitch idencation,’’ J. Acoust. Soc. Am.89, 2866–2882.

Meddis, R., and O’Mard, L.~1997!. ‘‘A unitary model of pitch perception,’’J. Acoust. Soc. Am.102, 1811–1820.

Micheyl, C., and Carlyon, R. P.~1998!. ‘‘Effects of temporal fringes onfundamental-frequency discrimination,’’ J. Acoust. Soc. Am.104, 3006–3018.

Moore, B. C. J., Glasberg, B. R., and Peters, R. W.~1985!. ‘‘Relative domi-nance of individual partials in determining the pitch of complex tones,Acoust. Soc. Am.77, 1853–1860.

Patterson, R. D., Robinson, K., Holdsworth, J., McKeown, D., Zhang,


t.

t.

ar

e

-

.

.,

and Allerhand, M.~1991!. ‘‘Complex sounds and auditory images,’’ inAuditory Physiology and Perception, edited by Y. Cazals, L. Demany, anK. Horner ~Pergamon, Oxford!, pp. 429–446.

Patterson, R. D., Allerhand, M., and Gigue`re, C. ~1995!. ‘‘Time-domainmodelling of peripheral auditory processing: A modular architecture ansoftware platform,’’ J. Acoust. Soc. Am.98, 1890–1894.

Patterson, R. D., Nimmo-Smith, I., Holdsworth, J., and Rice, P.~1988!.‘‘Spiral Vos Final Report, Part A: The Auditory Filterbank.’’ Applied Psychology Unit, Cambridge, England, Contract Report: APU 2341.

Plack, C. J., and Carlyon, R. P.~1995!. ‘‘Differences in frequency modula-tion detection and fundamental frequency discrimination between comtones consisting of resolved and unresolved harmonics,’’ J. Acoust.Am. 98, 1355–1364.

Plack, C. J., and White, L. J.~2000a!. ‘‘Perceived continuity and pitchperception,’’ J. Acoust. Soc. Am.108, 1162–1169.

Plack, C. J., and White, L. J.~2000b!. ‘‘Pitch matches between unresolvecomplex tones differing by a single interpulse interval,’’ J. Acoust. SAm. 108, 696–705.

Plomp, R.~1967!. ‘‘Pitch of complex tones,’’ J. Acoust. Soc. Am.41, 1526–1533.

Shackleton, T. M., and Carlyon, R. P.~1994!. ‘‘The role of resolved andunresolved harmonics in pitch perception and frequency modulationcrimination,’’ J. Acoust. Soc. Am.95, 3529–3540.

Shannon, R. V.~1983!. ‘‘Multichannel electrical stimulation of the auditorynerve in man. I. Basic psychophysics,’’ Hear. Res.11, 157–189.

Small, Jr., A. M., and McClellan, M. E.~1963!. ‘‘Pitch associated with timedelay between two pulse trains,’’ J. Acoust. Soc. Am.35, 1246–1255.

Thurlow, W. R., and Small, A. M.~1955!. ‘‘Pitch perception for certainperiodic auditory stimuli,’’ J. Acoust. Soc. Am.27, 132–137.

Tong, Y. C., and Clark, G. M.~1985!. ‘‘Absolute identification of electricpulse rates and electrode positions by cochlear implant listeners,Acoust. Soc. Am.74, 73–80.

Townshend, B., Cotter, N., Compernolle, D. v., and White, R. L.~1987!.‘‘Pitch perception by cochlear implant subjects,’’ J. Acoust. Soc. Am.82,106–105.

van Wieringen, A., Carlyon, R. P., Long, C. J., and Wouters, J.~2002!.‘‘Pitch of amplitude-modulated irregular-rate stimuli in electric and acoutic hearing,’’ J. Acoust. Soc. Am.~submitted!.

van Wieringen, A., and Wouters, J.~1999!. ‘‘Natural vowel and consonantrecognition by Laura cochlear implantees,’’ Ear Hear.20, 89–103.

White, L. J., and Plack, C. J.~1998!. ‘‘Temporal processing of the pitch ocomplex tones,’’ J. Acoust. Soc. Am.103, 2051–2063.


Temporal pitch mechanisms in acoustic and electric hearing

Documents

Transcript of Temporal pitch mechanisms in acoustic and electric hearing