Efficient visual search by category: Specifying the features that ...

Copyright 2001 Psychonomic Society Inc 676

Perception amp Psychophysics2001 63 (4) 676-697

In most visual search experiments the subject attemptsto detect a target that is characterized on every trial by thepresence or absence of some single feature or conjunc-tion of features For example the task might be to detecta red X among a set of black Xs and red Os In generaltargets are almost always single objects or small sets ofobjects that share some basic characteristic Far fewerstudies have explored the complex features that charac-terize natural object categories In the experiments re-ported here subjects searched for targets specified onlyby their membership in a broad natural category and hid-den among a mixed set of distractors that also had incommon only membership in a different broad categorySpecifically in one block of trials subjects searched for1 of 32 animals in a mixed set of artifacts and in the otherthey searched for 1 of 32 artifacts among a mixed set ofanimals It is important to note that on any given trial thesubjects did not know the specific targetmdashthey only knewthat it would be some animal or some artifact Thus no

single instance of a given feature defined either the targetsor the distractors Instead whatever the visual features thatdistinguish these categories they were variously instanti-ated and not necessarily present in all the display items

We are interested in two issues First given evidencethat distinctions between artifacts and natural kinds laythe foundation for learning more specific categories (Keil1989) it is plausible that visual search findings may re-veal an efficient process for distinguishing these cate-gories perceptually Although the parameters of the taskconsidered in the context of current theories of visualsearch might lead one to predict a slow and diff icultsearch the foundational nature of these well-learned cat-egories may allow a much faster search than otherwisemight be expected Second we will attempt to specify atleast some of the features that affect search slopes in thistask Although no simple feature or property will con-sistently distinguish targets from distractors there are anumber of global perceptual differences between arti-facts and animals that could facilitate search if present ina given category exemplar This is particularly interestingin the context of an efficient search In such a case thefeatures driving the search should be available preatten-tively1 and by understanding them we can gain insightinto the degree of complexity available in preattentive vi-sion Although a long tradition of research has asked this

The first three experiments in this paper were included as part ofDTLrsquos doctoral dissertation submitted to Cornell University Thanks toGreg Zelinsky for reading and commenting on an earlier draft of thispaper Correspondence concerning this article should be addressed toD T Levin Department of Psychology Kent Hall Kent State Univer-sity Kent OH 44121-0001 (e-mail dlevinkentedu)

Efficient visual search by categorySpecifying the features that mark the difference

between artifacts and animals in preattentive vision

DANIEL T LEVIN and YUKARI TAKARAEKent State University Kent Ohio

ANDREW G MINERUniversity of Illinois at Urbana-Champaign Urbana Illinois

and

FRANK KEILYale University New Haven Connecticut

In this report we explored the features that support visual search for broadly inclusive natural cat-egories We used a paradigm in which subjects searched for a randomly selected target from one cate-gory (eg one of 32 line drawings of artifacts or animals in displays ranging from three to nine items)among a mixed set of distractors from the other We found that search was surprisingly fast Target-present slopes for animal targets among artifacts ranged from 108 to 160 msecitem and slopes for ar-tifact targets ranged from 55 to 62 msecitem Experiments 2ndash5 tested factors that affect both thespeed of the search and the search asymmetry favoring detection of artifacts among animals They con-verge on the conclusion that targetndashdistractor differences in global contour shape (eg rectilinearitycurvilinearity) and visual typicality of parts and form facilitate search by category We argue thatexisting theories are helpful in understanding these findings but that they need to be supplemented toaccount for the specific features that specify categories and to account for subjectsrsquo ability to quicklylocate targets representing heterogeneous and formally complex categories

VISUAL SEARCH BY CATEGORY 677

question for simple abstract stimuli no findings we knowof have tested what features inherent to natural object cat-egories affect search slopes The feature we explored firstwas the relatively rectilinear shape of artifact contoursas compared with the relatively curvilinear shape of an-imal contours In addition a number of other factors weretested for their ability to predict search slopes includingcomplexity ambiguity visual typicality and similaritybetween the target object and the distractor categories

Visual Search by CategoryResearch using a wide variety of paradigms has shown

that humans are capable of efficiently perceiving and re-membering natural objects Early experiments on naturalscene perception concentrated on the degree to which alarge number of individual scenes could be rememberedMost of these studies emphasized our apparently limit-less ability to discriminate familiar scenes from novel ones(Nickerson 1968 Shepard 1967 Standing Conezio ampHaber 1970) whereas later work focused on the amountof time necessary to encode a scene for both on-line iden-tification and later recognition Several studies foundthat brief 100- to 200-msec exposures were sufficient toidentify objects and scenes but not to remember them(Intraub 1981 Potter 1976 Potter amp Faulconer 1975)

Other research using a visual search paradigm hasshown that a specific object can be located relativelyquickly For example Biederman Blickle Teitelbaumand Klatsky (1988) found that subjects performed abovechance at detecting a named target in arrays presentedfor 100 msec However they also found that increases indisplay size reduced accuracy prompting them to suggestthat object search was not capacity free More recentlyZelinsky Rao Hayhoe and Ballard (1997) had subjectslocate a color image of a target object in a pseudorealis-tic setting (eg a stuffed animal among other toys in acrib) On each trial the target image was shown in isola-tion then the subjects indicated whether it was presenton the table An analysis of eye movements suggestedthat the first saccade after fixation was directed towardthe middle of the display (which contained no objectsbecause the stimuli were arranged in a semicircle) Thesecond saccade then proceeded in the general directionof the target followed by further saccades that were di-rected with increasing precision at the target Thereforealthough search reaction times (RTs) indicated a 28-msecitem search rate the subjectsrsquo eye movements revealedan ability to determine which part of the screen includedthe target without the benefit of a serial scan of each ob-ject in turn

Although this rapid target identification suggests sometype of preattentive coding the data reviewed above arebased on identifying particular targets and thereforecould plausibly be driven by simple perceptual cues Forexample the rapid detection in Zelinsky et alrsquos (1997)research could be based on strategic selection of distinc-tive perceptual features such as object color and basicaspects of shape Subjects asked to detect a screwdriver

might search for the particular shape and orientation ofits shaft thus doing the search on the basis of the kind ofsimple perceptual feature that is traditionally thought todrive parallel search The same could be said about thescene classification research which typically uses basic-level target descriptions if not actual images of the tar-get to be detected However in Intraubrsquos (1981) rapid iden-tification task subjects successfully responded on thebasis of the nonpresence of a category of object (egthey might be instructed to ldquolook for a picture that is notof house furnishings and decorationsrdquo p 608) This doessuggest a more sophisticated process because no partic-ular target was selected Indeed this sophistication doesfind support in a few studies that have assessed the degreeto which subjects can detect targets solely on the basis oftheir membership in a category that presumably is markedby no particular perceptual feature

The best-known example of this kind of visual searchis Egeth Jonides and Wall (1972) who showed that sub-jects could locate a number among letters or could do thereverse in parallel Jonides and Gleitman (1972) furthershowed that an ambiguous ldquoOrdquo stimulus (which could bethe number zero or the letter ldquoordquo) could be located in par-allel among both numbers and letters Thus they arguedthat their findings reflected a truly categorical visualsearch that could not be explained solely on the basis ofthe direct association between a given shape and a givenresponse However a number of attempts to replicate thiseffect have failed (eg Duncan 1983 Krueger 1984) soit must be considered to be tentative at best

Although the status of efficient alphanumeric categorysearch is questionable more recent data from experimentsusing broad natural categories reinforce the possibilityof efficient categorical visual search Thorpe Fize andMarlot (1996) tested subjectsrsquo ability to locate an animalin a series of natural scenes and found that RTs were asshort as 400 msec implying that the animals had actuallybeen perceived in less than 200 msec Evoked potentialsconfirmed this by showing a divergence in brain activityfor scenes with and without animals as soon as 150 msecafter stimulus onset (Thorpe et al 1996) If we assumethat their images contained a sufficient variety of animalsso that they were not all marked by the presence of a par-ticular part (such as legs) these data suggest that com-plex natural scenes can be rapidly parsed by category

This finding is particularly interesting because it mayreflect the foundational nature of Thorpe et alrsquos (1996)stimulus categories The contrast between artifacts andanimals may in fact represent one of the first ways thatchildren divide the world and may serve throughout adult-hood as a conceptual framework for more specific learn-ing (Keil 1989) For example Gutheil Vera and Keil(1998) taught young children a new property for a givenanimal and then asked them whether it would be presentin other objects The children readily attributed the newproperties to a wide variety of living things ranging fromdogs to worms yet refused to generalize it to nonlivingobjects In addition young children understand a num-

678 LEVIN TAKARAE MINER AND KEIL

ber of entailments on the basis of the animalartifact dis-tinction They know that living things grow as they agewhereas artifacts do not (Rosengren Gelman Kalish ampMcCormick 1991) and that animals can move by them-selves whereas artifacts are less likely to do so (Masseyamp Gelman 1988) Furthermore precursors of a distinctunderstanding of the living-kindartifact difference canbe seen in infancy For example Woodward (1998) foundthat 6-month-old infants distinguish between the goal-directed actions of a hand and the more automatic actionsof a stick and Bertenthal (1993) showed that infants dis-tinguish between biological and nonbiological motion

The assumption that the artifact animal distinctionserves a cognitive foundation is also reinforced by neuro-anatomical findings It appears that a subset of brain in-juries can selectively impair recognition of one categoryor another For example Farah McMullen and Meyer(1991) described two visual agnosics who had difficultyrecognizing living things as compared with artifacts adeficit that could not be explained on the basis of simpledifferences between the categories such as image com-plexity familiarity and so forth Sacchett and Hum-phreys (1992) reported the opposite deficitmdasha selectiveinability to name artifacts Although the specific inter-pretation of this deficit has been controversial (Funnellamp Sheridan 1992 Gaffan amp Heywood 1993 StewartParkin amp Hunkin 1992) most authors agree that thesedata reflect some kind of specialization by category or bybasic between-category processing differences and morerecently Caramazza and Shelton (1998) argued that theymay reflect a set of distinct visual subsystems that cor-respond closely to cognitive between-category distinc-tions If this is true efficient visual search for these cat-egories may involve detection of activity in one of thesesystems

On the whole then the research converges to suggestthat scenes can be rapidly parsed by category even bybroadly inclusive categories such as artifacts and ani-mals Both visual search and rapid serial visual presenta-tion tasks reveal efficient categorical parsing of scenes ata number of levels This efficiency may not be limited towell-defined or narrowly defined categories and may in-clude broad foundational categories such as artifacts andnatural kinds If an efficient search by category is possi-ble it is important to understand what specific featuresdrive the search Understanding the features that drive anefficient search will provide information about the rep-resentations available in preattentive vision Previous re-search testing object recognition and identification in nor-mal and brain-injured subjects suggests that a wide rangeof potential features could distinguish these categories

What Features DistinguishArtifacts and Animals

In the experiments reported here a visual search taskwas based on natural object categories that are associ-ated with a rich set of concepts and different inferentialpatterns The contrast between the categories is also as-

sociated with a variety of systematic perceptual differ-ences ranging from differences in the importance of partconfiguration to differences in basic contour shapes Anumber of authors have suggested that living things areprocessed by a system that makes more extensive use ofpart configuration as a differentiating cue Farah (1995)argued that face-specific deficits are the most dramaticexamples of this kind of configural processing deficit andthat those deficits for recognizing living things that oftenaccompany face-specific deficits are similar in this re-spect Artifacts and such objects as alphanumeric char-acters are processed by using more information aboutparts in isolation from their configuration

Another possibility is that visual information is glob-ally more central to identifying living things (Warringtonamp Shallice 1984) Warrington and her colleagues pro-posed that living things are mainly defined by their sen-sory features and that artifacts are mainly defined bytheir functional features This proposal implies that per-ceptual differences between animals and artifacts lie inhow we interact with them A number of studies supportthis view First findings in perception suggest that arti-facts are coded explicitly in terms of body-scaled actionaffordances For example Warren (1984) found that an es-timate of the degree to which a step is climbable is closelytuned to optimal energy efficiency for the subjectrsquos par-ticular size Second recent neuroimaging data have sug-gested that images of artifacts simultaneously activate pri-mary visual areas and an area associated with generatingaction terms whereas images of animals do not activate thissecond area (Martin Haxby Lalonde Wiggs amp Unger-leider 1995 Martin Wiggs Ungerleider amp Haxby 1996)

A more low-level perceptual distinction between arti-facts and natural kinds is that they have different contourshapes For example Zusne (1975) showed subjects ran-dom shapes that were either polygons with straight linesor similar closed forms with curved lines He found thatthe subjects generated more associations with living thingsfor the curved shapes More recently Kurbat (1997) mea-sured the average local curvature on the image contoursof artifact and natural kinds and found that artifacts wereon average characterized by straight edges and naturalkinds by more curved edges He also found that curvilin-earity was a significant predictor of whether an agnosicpatient would recognize a given object Other researchhas suggested that infants use contour shapes to differen-tiate categories Van De Walle and Hoerger (1996) testedthis hypothesis in infants using a stimulus set of toys thathad global part arrangements characteristic of one cate-gory but with contours characteristic of another categoryThus one stimulus might be a dog with parts suitablyarranged to represent a head a body and four legs How-ever each of these parts was made with straight edges andcorners giving the toy contours characteristic of an arti-fact Infants were shown a series of toys representing onecategory habituated to them and then were given a testtoy As in previous research (Mandler amp McDonough1993) the infants dishabituated to the new toy if it rep-


resented a different category from the toys on the habit-uation series In addition however Van De Walle and Hoer-ger found that the infants dishabituated to a test toy fromthe same category as the habituated toys so long as it hadcontours characteristic of the nonhabituated category

In summary it is possible that the visual system usesspecialized processes for dealing with artifacts and nat-ural kinds The question is what features does it use todistinguish these categories during the f irst stages ofprocessing Different proposals range from action affor-dances to contour shape Here we used a visual searchparadigm to test feature contrasts at two basic levels acomplex level that required relatively complete struc-tural descriptions of form versus a more simple percep-tual level based on contour shape (eg curvilinearityrectilinearity) In addition we tested the degree to whichthe visual typicality and similarity between the target ob-ject and the distracting category affected search slopesIt is important to point out that we are not committed to atop-down model of object perception or search wherebysemantic-level categorizations affect search slopes In-stead the effectiveness of a categorical contrast betweentargets and distractors is assumed to lie in the way thatlong-term learning (or genetics) has caused early and mid-level vision to reflect natural breakpoints in the structureof the visual world

The Present StudyWith the experiments reported in this paper we both

assessed the overall speed of visual search by categoryand in addition attempted to understand the featuresthat distinguish artifacts from natural kinds Althoughthese two goals are somewhat different they converge inthat a fast search would suggest a feature contrast that isdeeply embedded in preattentive vision Experiment 1was a basic visual search experiment in which subjectssearched for 1 of 32 targets of one category among a mixedset of distractors chosen from a set of 32 objects of theother category Experiment 2 tested the degree to whichcomplete structural descriptions drive the search by hav-ing subjects search for jumbled objects of one categoryamong jumbled objects of the other In Experiments 3 and4 a possible low-level perceptual feature was tested Ob-jects of both categories were assessed for the degree towhich their contours were rectilinear Search slopes foreach object were then collected and correlated with therectilinearity measure with the prediction that highlyrectilinear objects would be more quickly detected Fi-nally in Experiment 5 we completed a series of regres-sion analyses predicting the slopes observed in Experi-ments 3 and 4 using a variety of variables includingcomplexity ambiguity contour shape and typicality

EXPERIMENT 1

Experiment 1 was a basic visual search experiment inwhich the search target was defined by category Subjectstherefore searched for 1 of 32 different artifacts among amixed set of animal distractors or did the reverse search-

ing for 1 of 32 animals among a mixed set of artifactsThe basic issue of interest here was the efficiency of vi-sual search by category as reflected by search slopes

Before proceeding a note on the stimulus sets used inthis paper is in order We chose to test the contrast be-tween animals and artifacts as the search categories de-spite the fact that a more broad classification schememight subsume the animal category within all livingthings or even all natural kinds Despite the psycholog-ical salience of all of these levels of classification wechose to use animals because it simplified developmentof the stimuli (because it allowed us to avoid a numberof complexities such as the question of natural kindsthat are in some ways similar to artifacts such as fruitsand vegetables) In doing so we follow Caramazza andShelton (1998) Martin et al (1996) and Thorpe et al(1996) in assuming that animals represent a psychologi-cally salient and potentially dissociable category

MethodSubjects Eight Cornell University undergraduates participated

in the present experiment in exchange for course creditStimuli The stimulus set consisted of 64 realistic black-and-white

line drawings of artifacts and animals (32 of each Figures 1Andash1B)The drawings were shaded internally with a uniform light gray Someof the stimuli were from the original Snodgrass and Vanderwart(1980) set whereas others were culled from various collections ofclip art They were selected from a larger set pseudorandomly withthe constraint that they had to be reasonably unambiguous (this wasmost important for the artifact pictures) In addition some animalswithout legs protruding from the bottom were purposely selected(eg the butterfly the fish and the seahorse) to minimize the de-gree to which that particular feature was characteristic of the wholeset Once selected the two sets of 32 images were equalized for av-erage pixel area to prevent size from becoming a cue to reinforcethe search The size of the stimuli varied somewhat but averaged ap-proximately 100 (h) 3 80 (v) pixels (approximately 5 3 4 cm)

Apparatus Search trials were presented in 256-level gray scaleon Macintosh LC computers with 12-in monitors The subjects satin a comfortable unconstrained position with an average viewingdistance of approximately 60 cm and completed the experiment insmall groups ranging in size from 1 to 5 using different computersin the same room

Procedure The experiment consisted of two blocks of 144 tri-als each preceded by 12 practice trials In one block the subjectssearched for an animal among artifacts and in the other they searchedfor an artifact among animals Presentation order of blocks wascounterbalanced across subjects The subjects were instructed thatthey would be seeing a series of displays containing a number of ob-jects and that their job was to hit the ldquo1rdquo key if an object from thetarget category was present in the display and to hit the ldquo2rdquo key ifnone was present Thus for each trial no specific object was namedas a target Instead the subjects had to detect any of the animalsamong artifacts or do the reverse The subjects were instructed touse two fingers on their dominant hand to respond and to respondas quickly as possible without sacrificing accuracy The subjectswere also informed that they would receive feedback after each trialin the form of a ldquo+rdquo for correct responses and a ldquo2rdquo for incorrectresponses

On each trial the subjects saw a display containing three six ornine objects The objects were presented in randomly selected lo-cations within a 3 3 3 matrix on the screen The approximate visualangle subtended by each object was 47ordm 3 38ordm and the approxi-mate angle of the entire display was 155ordm 3 125ordm On half of thetrials an object from the target category was present and on half it


Figure 1 (A) Animal stimulus set (B) Artifact stimulus set

A

B


was absent There were 24 trials in each cell of the design for a totalof 288 trials (24 trials 3 2 tasks [animal target artifact target] 3 3display sizes [369] 3 2 target conditions [target-present target-absent] ) Each trial was response terminated and was immediatelyfollowed by feedback which remained on the screen for 350 msecTargets for each trial were chosen randomly from among the 32 mem-bers of each category and distractors were chosen randomly withreplacement 2

ResultsAll the analyses reported in this paper have focused

on target-present data although target-absent data are re-ported and graphed for the sake of completeness Al-though target-absent data are sometimes informative theytend to reflect decision stage processes and search ter-mination criteria (see Chun amp Wolfe 1996) that are notof central interest for the purposes of this report

Errors The average error rate was 195 The meanerror rate for target-present trials was 286 and the meanerror rate for target-absent trials was 104 The error ratesfor target-present trials were entered into a two-factor(task 3 display size) within-subjects analysis of variance(ANOVA) Neither the effects of these factors nor theinteraction between them was significant although theerror rate for artifact target trials was 365 and theerror rate for animal target trials was 208 [F(17) 54200 MSe 5 6975 p 5 0796] The error rates acrossdisplay sizes were 417 312 and 365 for three-six- and nine-item artifact target displays and 104104 and 417 for three- six- and nine-item animaltarget displays respectively

Search reaction times Prior to analysis the RT datafrom error trials were eliminated here and throughoutthis report The RT data were entered into a two-factor(task 3 display size) within-subjects ANOVA In additionaverage search slopes were computed for each condition

The analysis revealed two main findings First searchwas highly efficient The average target-present slope fordetecting an artifact among animals was 55 msecitem(target-absent slope 5 27 msecitem) whereas the target-present slope for animals among artifacts was 16 msecitem (target-absent slope 5 415 msecitem) Second anear-significant search asymmetry was revealed in thetask 3 display size interaction [F(214) 5 258 MSe 51776 p 5 1114] in which the search slope for artifactswas shallower than that for animals (see Figure 2)

One additional aspect of the data worth mentioning isthat there is a crossover in RTs for target-present three-item displays such that RTs were significantly faster whendetecting an animal among artifacts (578 msec) than whendetecting an artifact among animals (624 msec p lt 01Duncan test)

DiscussionOn the whole search slopes for target-present trials

were surprisingly shallow and the artifact target searchfell within the traditional cutoff of 10 msecitem for aputatively parallel search (Theeuwes 1993) A numberof factors would lead to the prediction of a slow ineffi-

cient search First no specific perceptual primitive appearsto have been sufficient to mark the target According tothe original feature integration theory (FIT Treisman ampGelade 1980) parallel search is possible only when asingle discriminable perceptual primitive distinguishestargets from distractors Although some perceptual fea-ture might at least partially have distinguished the cate-gories it would have been nothing like the primitivespreviously studied and in addition it would have beenunlikely to have reliably separated all the members of thetarget category from the nontarget category because ofthe heterogeneity of our stimuli Other variations and re-visions of FIT also seem unable to explain why the searchwas so fast For example Wolfersquos (Wolfe 1994 WolfeCave amp Franzel 1989) guided search model is more flex-ible in that it allows parallel searches for targets definedby a conjunction of features but again it is difficult topoint to a pair or a triplet of features that would have re-liably distinguished targets from distractors Furthermoreeven if some conjunction of primitives were to have dis-tinguished targets from distractors it would have been un-likely that the low-noise conditions specified in guidedsearch would have been met here (eg that activationson each feature map strongly distinguished target loca-tions from distractor locations)

The Duncan and Humphreys (1989) model of visualsearch also appears to predict a very slow search In par-ticular one of the major factors affecting search speedaccording to Duncan and Humphreys is the degree ofheterogeneity among distractors Displays with highly

Figure 2 Visual search reaction times (RTs) for animal targetsand artifact targets


varied distractors will be difficult to search whereas dis-plays with very similar distractors will be easy to searchThe distractors used in the present study were highly het-erogeneous yet the targets were located rapidly Thesesets of objects shared no diagnostic color orientationpart or part structure Although the distractors were prob-ably similar in some respects these respects were isolatedamong a large number of perceptually available featuresThus there was no a priori means of selecting a set offeatures on which to base an estimate of homogeneity Inthis context the notion of distractor homogeneity be-comes insufficient to explain why visual search mighthave been fast or slow in this case (However the modelmay provide some explanation for the direction of asym-metry Animal distractors appear more homogeneousthan artifacts because of common parts such as legswhich might facilitate search for artifact targets Thispossibility will be examined further in Experiments 3and 4) It is therefore critical to specify the features thatdistinguish the particular categories if we are to under-stand search behavior Experiments 2 and 3 were designedto test whether search is based on higher level featuresandor more simple features such as individual parts orcurvilinearrectilinear contour shape To examine the ef-fect of higher order information the targets and distrac-tors used in Experiment 2 were jumbled Jumbling wasintended to disrupt structural coding and therefore inter-fere with semantic processing

Before going on to Experiment 2 a note about target-absent search slopes is in order In Experiment 1 the ratioof target-present to target-absent slopes was 1491 forthe artifact targets and 1259 for the animal targets Thus

for the artifact targets target-absent search was quiteslow relative to target-present search It seems likely thatthis occurred because artifact search is easier than sub-jects think it should be making them more hesitant toterminate the search than they should be Certainly thismay represent an interesting effect perhaps indicatingthat this search task is considerably easier than subjectsexpect but in the interest of focus we will defer explo-ration of these differences for a later date and will continueto analyze only target-present slopes in the remainder ofthis paper

EXPERIMENT 2

If efficient visual search is being driven by a completestructural code jumbling the features of the objects shouldmarkedly slow the search Experiment 2 was thereforeidentical to Experiment 1 with the exception that all thestimuli were jumbled by trading locations and orienta-tions of most of the major parts within each object Thesubjects searched for a jumbled artifact among jumbledanimals or did the reverse searching for a jumbled ani-mal among jumbled artifacts

MethodSubjects Fourteen Cornell University undergraduates completed

the present experiment in exchange for course credit Of these1 was dropped from the analysis owing to a high error rate (16)

Stimuli The stimulus set consisted of jumbled versions of the64 images used in Experiment 1 Images were jumbled by using astandard image-processing program (Adobe Photoshop) to cut ro-tate and paste sections of each image In general most object partswere cut and pasted as wholes to avoid creating unnatural line ter-

Figure 3 Sample jumbled objects


minations or radical changes in contour direction (see Figure 3)However this constraint was not possible to satisfy in all cases

Apparatus and Procedure The apparatus and procedure wereidentical to those in Experiment 1

ResultsErrors The average error rate was 315 The mean

error rate for target-present trials was 512 and themean error rate for target-absent trials was 118 Theerror rates for target-present trials were entered into a two-factor (task 3 display size) within-subjects ANOVAThe task effect and the task 3 display size interactionswere nonsignificant (Fs lt 01) whereas the display sizeeffect was significant [F(224) 5 4500 MSe 5 40509p 5 0219] The error rates across display sizes were417 256 and 673 for three- six- and nine-itemartifact target displays and 352 417 and 962 forthree- six- and nine-item animal displays respectively

Search reaction times The data were entered into awithin-subjects two-factor task (artifact target or animaltarget) 3 display size (three six or nine) ANOVA Anadditional ANOVA included a between-subjects config-uration (normaljumbled) factor that allowed compari-son between Experiments 1 and 2

The primary finding in this experiment is plainly vis-ible in Figure 4 Jumbling features had little effect onsearch slopes which remained quite shallow The meanslope for artifact target trials was 123 msecitem as com-

pared with 55 msecitem in Experiment 1 (668 msecitem on target-absent trials) and the slope for animal tar-get trials was 171 msecitem as compared with 160 msecitem in Experiment 1 (751 msecitem on target-absenttrials) The search asymmetry was not significant (F lt 1)Although this represents a slowing of approximately7 msecitem in the artifact target trials as compared withExperiment 1 the between-experiments configuration 3display size interaction did not approach significanceeven when only RTs for artifact target trials were includedin the analysis (Fs lt 1)

Although search slopes were similar between Exper-iments 1 and 2 average RTs increased by 108 msec[F(119) 5 10934 MSe 5 31399 p 5 0037]

Finally mean target-present RTs in three-item displaysonce again crossed over with animals (691 msec) beingdetected more quickly than artifacts (702 msec) al-though the comparison was nonsignificant

DiscussionThe major finding in this experiment was that jumbling

features slowed the search only minimally Although theslowdown in the search slopes for artifacts suggests someinterference with the search-relevant features the smallsize of the effect points away from the possibility that thesearch is based on a completed structural code This is ofcourse compatible with many theories of visual searchand preattentive vision In particular recent work has sug-gested that efficient search cannot be based on a com-pleted structural description Wolfe and Bennett (1997)used a variety of configural features (ie features de-fined by the relationship between at least two parts) andshowed that in all cases search was relatively slow Assuch Wolfe and Bennett argued that objects in pre-attentive vision are ldquoshapeless bundlesrdquo of features Sim-ilarly Rensink (2000) has argued that preattentiverepresentations of objects are loosely organized and tran-sitory It is important to note however that these repre-sentations are unlikely to be limited to unprocessed prim-itives such as line segments and colors As has been notedby both Rensink (2000) and Wolfe and Bennett (1997)parallel search is possible on the basis of a number ofrelatively complex features including shape from shad-ing (Ramachandran 1988) and three-dimensional (3-D)orientation (Enns amp Rensink 1991) In addition Don-nelly Humphreys and Riddoch (1991) found that con-tour closure could support efficient search implying thatsome limited processing of form occurs prior to the arrivalof focal attention He and Nakayama (1992) went fartherand suggested not only that preattentive vision accountsfor depth relations but that visual search cannot accessthe prior feature-based representations at all

Clearly therefore Experiment 2 does not completelyeliminate the possibility that complex perceptual fea-tures drive visual search by category For example it ispossible that semantic information can be accessed onthe basis of individual parts It is important to note thateven if the features that distinguish these categories areexclusively perceptual the idea that they instantiate a

Figure 4 Combined results from target-present trials in Ex-periments 1 and 2 Boxes represent Experiment 1 (visual searchfor nonjumbled artifacts and animals) and circles represent Ex-periment 2 (visual search for jumbled artifacts and animals)


substantive category remains plausible given findingsof category-specific visual impairments reviewed aboveHowever given that Experiment 2 generally pointed awayfrom completed structural descriptions Experiments 3and 4 investigated more simple features by testing forcorrelations between rectilinearity and visual searchslopes whereas Experiment 5 tested the predictive powerof other factors that might be available on the basis of vi-sual information less complex than structural descrip-tions but more complex than primitives

EXPERIMENT 3

To test the hypothesis that artifact rectilinearity facil-itates visual search by category it was necessary to de-vise some way of measuring the global shape of the ob-ject contours We therefore measured rectilinearity byusing a standard edge detection algorithm that allows de-tection of pixel locations characterized by an image edgeand measures the orientation of that edge (Sobel methodsee Jain Kasturi amp Schunck 1995) Given edge orien-tations at each edge location it was possible to obtain ameasure of rectilinearity by simply computing the de-gree to which edges were collinear with each other Thismeasure was taken on each of the 64 artifact and animalimages Individual rectilinearity measures were then cor-related with behavioral data from a new experiment inwhich subjects completed a sufficient number of trialsto allow an estimate of search slopes for each stimulusThe search task was identical to that in Experiment 1

MethodSubjects Six Cornell undergraduate and graduate students com-

pleted the present experiment These included two of the authorsDTL and AGM

Apparatus The experiment was run on computers with 14-in(image size 269 cm horizontal 3 202 cm vertical) 256-level gray-scale displays set at a resolution of 640 3 480 The stimuli weretherefore somewhat larger than those in Experiments 1 and 2 andsubtended a maximum of 52ordm 3 43ordm whereas the entire displaysubtended approximately 201ordm 3 158ordm

Procedure The subjects completed a search experiment similarto that in Experiment 1 The major difference was that each stimu-lus used in Experiment 1 served as a target a total of 16 times8 times in a three-object display and 8 times in a nine-object displayThe subjects therefore completed a total of 2048 trials (64 stimuli3 2 display sizes 3 8 trials per display size 3 2 target conditions)To avoid fatigue the experiment was broken up into four sessionseach of which lasted approximately 20 min The 512 trials in eachsession included two blocks of 256 trials one for the animal targetcondition and one for the artifact target condition In addition 8practice trials preceded the 256 experimental trials in each blockThe subjects were given instruction sheets and were allowed to com-plete the four sessions at their convenience by going to the labora-tory unsupervised over a period of 2ndash3 days Many of the subjectscompleted two sessions in a single sitting but all were advised notto attempt more

Contour analysis Several measures of edge relatedness werecomputed to measure differences in contour shape between picturesof artifacts and animals All of them use an edge detection algo-rithm that assesses the degree to which each pixelrsquos surround sug-gests an edge In the present experiment this was done by measuringthe slope of change in luminance values over 5 3 5 pixel regions of

space The orientation of the edge was determined by taking deriv-atives in two orthogonal directions (different by 90ordm) and combin-ing them by taking the arcsine of their ratio (Sobel method see Jainet al 1995) Once all the pixels in the image were assigned an edgevalue (high if the region contained information suggesting an edge)and an orientation a threshold was applied to the edge valueswhich selected a subset of pixels to be classified as edges Orienta-tions of all pairs of neighboring edges were then subtracted to pro-duce a measure of the deviation between edge orientations Through-out this report this basic measure will be referred to as edge pairdeviation (EPD) The basic assumption is that pictures with a rela-tively high percentage of edge pairs that are either collinear or atright angles are rectilinear whereas pictures with few such pairs arecurvilinear

A key issue is whether to base computations on the entire edgemap or only on the map of external edges Following Kurbat (1997)and Tranel Logan Frank and Damasio (1997) we used only theexternal edges The primary benef it of using the whole edge map isthat it includes more information and edges that are likely to beused by the subjects to identify the object The major disadvantageof using the whole map is that it includes internal edges that are partsof textures and small surface markings in addition to major con-tours (a problem that is especially acute in the line drawings) Al-though texture and surface markings could plausibly be importantfactors in driving this search task it is extremely difficult to reliablydistinguish these from contours that might match those used by thevisual system (Sanocki Bowyer Heath amp Sarkar 1996) Externalcontours have no special status as markers of part boundaries butthey are at least continuous and also likely to be a fair match to asubset of contours that are actually used by the visual system toparse scenes and identify objects

Figure 5 summarizes the basic output of this analysis for a typi-cal animal and a typical artifact These figures show distributionsof EPD values across a range of linear distances from 2 to 40 pix-els in steps of two The horizontal rows in the graph represent dif-ferent distances whereas the columns represent different EPD val-ues (with bins ranging from 0ordmndash5ordm to 85ordmndash90ordm) The size of the squarein each cell of the chart represents the relative number of edge pairswithin a given range of EPD values and distances Cell values werecomputed by taking the ratio of the number of actual edge pairs tothe expected value based on the total number of edge pairs in theimage divided by the number of cells in the chart Thus observa-tions on each cell are normalized for the number of edge pairs in theimage but not for variance across rows or columns

As can be seen in Figure 5 the animal image is characterized bya gradual increase in the range of EPD values as edge-to-edge dis-tance increases showing that increased distance is associated withdecreased predictability of local contour orientation a finding thatis similar to the analysis reported by Olshausen and Field (1996)However the artifact image is quite different in that it appears to vi-olate this relationship Here increasing distance is associated withno regular increase in EPD values which reflects the preponderanceof straight lines in the image Also the artifact in Figure 5 is char-acterized by a relative increase in the number of edge pairs at a de-viation of 85ordmndash90ordm which is caused by the presence of right anglesin the image As edge pairs become more distant they are more likelyto include one edge on each side of a corner

On the basis of the different distributions for animals and arti-facts a direct measure of rectilinearity was computed for each stim-ulus based on the percentage of EPD values in the 0ordmndash5ordm 5ordmndash10ordmand 85ordmndash90ordm columns This particular selection was guided by theassumption that rectilinearity is reflected in the preponderance bothof aligned edges and of edges at right angles


error rate for target-present trials was 434 and the


mean error rate for target-absent trials was 156 Theerror rates for target-present trials were entered into atwo-factor (task 3 display size) within-subjects ANOVAThe task and display size effects were nearly significant[task effect F(15) 5 5254 MSe 5 1811 p 5 0705display size effect F(15) 5 6167 MSe 5 6064 p 50556] whereas the task 3 display size interaction wasnonsignificant [F(15) 5 1176] The error rates acrossdisplay sizes were 397 and 597 for three- and nine-item artifact target displays and 221 and 521 forthree- and nine-item animal displays respectively

Reaction times Target-present search slopes aver-aged over all stimuli were first entered into a one-factorwithin-subjects ANOVA with task (artifact targetanimaltarget) as the single factor This analysis confirmed thatthe overall results from the search task replicated thosefrom Experiment 1 The average search slope in the arti-fact target condition was 56 msecitem (target-absentslope 5 2575 msecitem) whereas the average in theanimal target condition was 137 msecitem (target-absentslope 5 310 msecitem) Target-present slopes were sig-

nificantly different replicating the search asymmetryobserved in Experiment 1 [F(15) 5 70185 MSe 5 2757p 5 0004] This asymmetry was also significant in anitem-based analysis [F(162) 5 35337 MSe 5 18316 p lt0001] As in Experiment 1 target-present RTs for three-item displays were faster for animal targets (531 msec)than for artifacts (549 msec) although this effect wasnonsignificant [F(15) 5 2697 MSe 5 3385 p 5 161]Target-present RTs for nine-item displays were signifi-cantly faster for artifact targets than for animal targets[F(15) 5 7939 MSe 5 3571 p 5 037]

An additional analysis assessed learning effects acrossthe four blocks of 512 trials Mean RTs were entered intoa three-factor task (artifact target or animal target) 3 dis-play size (three or nine) 3 block (one two three or four)within-subjects ANOVA RTs decreased across blocks(Block 1 630 msec Block 2 569 msec Block3 545 msecBlock 4 534 msec F(15) = 1587 MSe 5 2784 p lt0001] However the block 3 display size interaction didnot approach significance (F lt 1) indicating no change insearch slopes across blocks The task 3 block and task

Figure 5 Typical edge pair distributions for one animal and one artifact


3 display size 3 block interactions were nonsignificant(Fs lt 1)

In the second major analysis search slopes were com-puted for each stimulus in the experiment and were cor-related with rectilinearity This relationship was signifi-cant for artifact slopes (r 5 248 p lt 01) but not foranimals (r 5 2026 Figure 6) As can be seen in Figure 6one reason for the negative finding in animals appears tobe range restriction in that rectilinearity values variedonly slightly in this stimulus set

DiscussionExperiment 3 makes several things clear First it closely

replicates Experiment 1 both in terms of the search asym-metry favoring artifacts and in terms of actual searchslopes which were 55 and 137 msecitem for artifacttarget and animal target searches respectively as com-pared with 56 and 160 msecitem in Experiment 1 Sec-ond the experiment reveals a significant correlation be-tween search slopes for individual artifacts and theirobjectively measured rectilinearity This relationship isabsent for animals

The most important finding was the correlation be-tween search slopes and rectilinearity for artifact targetsThis suggests that rectilinearity could well serve as thefeature driving the search asymmetry favoring artifactsIn addition the pattern of means in Figure 6 suggests thatobjects with rectilinearity values of approximately 25 aresearched for at a normative rate of 12 msecitem irre-spective of category Artifacts that deviate from this nor-mative contour shape are the ones that drive the asymme-try because they are easier to locate although it remainsto be seen whether animals with rectilinear contours willalso be more easily located among curvilinear distractorsFurthermore it is possible that different contour popula-tions are represented by early vision in a spatiotopic mapof their own This kind of explanation fits well with re-

search on the connectivity patterns observed in area V1(Gilbert 1993)

Although rectilinearity has plausible physiologicalroots it might simply be interpreted as an autocorrela-tion function that is high when edges in one location pre-dict the orientation of edges in another and low when ori-entations are less correlated This alternative is howeververy similar to the rectilinearity concept The major cir-cumstance in which it is different (given that we have iso-lated a single external contour) is one in which there is alarge number of accidentally collinear edges that do nothappen to belong to a contour that continues uninterruptedin one direction A sawtooth edge is one example of thisbecause it is not straight but distant local edge orienta-tions are likely to be similar nonetheless At a minimumvisual inspection of the stimuli reveals no such edges inthe artifacts which on the whole do not have externalcontours that repeat any particular local variation (Eventhe handsawrsquos teeth are too small to be detected by the 5-pixel filters we used) Thus a preliminary analysis sug-gests that rectilinearity is a reasonable way of describingthe measure although other similar concepts such asautocorrelation might be adequate as well

Although rectilinearity predicts search slopes it plainlydoes not explain all of the variance It is possible that somedifferent measure of rectilinearity would be more validand successful but other factors may influence slopes aswell For example each category has characteristic partsThis is particularly true of animals many of which havea basically horizontal body and vertical legs As an inter-nal analysis we therefore selected 12 animals that seemedto violate the basic horizontal body + vertical legs for-mula (Animals 2 3 4 6 15 17 19 20 21 22 26 and 27)to test for differences in search slopes and intercepts Bothsearch slopes and intercepts for the violators were sig-nificantly slower (slope 140 msec intercept 511 msec)than those for the other animal stimuli [slope 106 msec

Figure 6 Correlation between search slopes and rectilinearity in Experiment 3


intercept 488 msec slope F(130) 5 674 MSe 5 13313p 5 0145 intercept F(130) 5 4371 MSe 5 898 p 50450] Therefore a typical set of parts also seems to fa-cilitate search at least in animals This suggests that acollection of features is used to discriminate categoriesSince there is no single feature present consistently in an-imal or artifact categories and animals and artifacts arepossibly differentiated along many dimensions it wouldbe a poor strategy to try to locate a target on the basis of asingle feature The implication that multiple diagnosticfeatures have for theories of visual search will be furtherexamined in Experiment 5 and in the General Discussionsection

However before concluding that visual search by cat-egory can be efficient and that rectilinearity facilitatesvisual search we looked more closely at the possibilitythat stimulus factors biased these experiments The stim-uli used in Experiments 1ndash3 were simple line drawingsof artifacts and animals Although the drawings were re-alistic they still emphasized contours and were mediatedby an artistrsquos hand At a minimum these factors may in-troduce unnatural contours that unnaturally facilitate thesearch This is particularly relevant for the external con-tours which are drawn with heavy lines that may over-emphasize form primitives that are not present in naturalobjects In addition it is impossible to know what distor-tions might be introduced in the process of reproducingthe images For example it is possible that artists tend tooveremphasize the straightness of artifact contours andthe curvedness of animal contours which may accentu-ate a difference that only minimally marks the differencebetween actual categories Another potential interpretiveissue is the lack of color or texture information in the draw-ings especially given that color and texture informationmight be more central to the classification of animals ascompared with artifacts (Keil 1994) For these reasons andmore generally to replicate the basic findings of Exper-iment 3 we developed a set of full-color stimuli of photo-graphic origin and again tested for efficient search and acorrelation between contour shape and search slopes

EXPERIMENT 4

In Experiment 4 we created a set of 64 color photo-graphic stimuli (32 animals and 32 artifacts) on the basisof photo collections and scanned images from booksmagazines and catalogs As in Experiment 3 6 subjectssearched for each of the stimuli in heterogeneous collec-tions of distractors representing the other category

MethodSubjects Six participants completed the present experiment in-

cluding DTL and 5 graduate and undergraduate students One ofthe graduate students had also completed Experiment 3

Stimuli A new set of stimuli was created by assembling a set of59 artifact images and 72 animals from a variety of sources Thestimuli were chosen from stock photo collections and were scannedin from books magazines and catalogs During the initial stimuluscollection an effort was made to locate images that represented awide variety of shapes and more specific categories For the animals

a special effort was made to reduce the prevalence of canonicalbody+legs-at-bottom images Otherwise the initial stimulus col-lection process was based on the availability of suitable images thathad fully visible external contours and avoided highly unusualviews Once the initial set of stimuli was collected the f inal set of32 for each category was randomly selected with the constraint thata wide variety of categories be represented Therefore when ran-dom selection caused overrepresentation of one very narrow cate-gory or the removal of too many distinctive shapes the object wasreplaced and another object was chosen For these reasons randomchoices were overruled for four animals In addition one artifactwas replaced because it had a picture of an animal on it In severalinstances two members of the same basic-level category were al-lowed into the stimulus set The Appendix includes the names of allthe stimuli in the set

Once selected the images were separated from their backgroundsSome had been photographed on neutral backgrounds so this pro-cess could be done automatically Others however had to be sepa-rated manually using Photoshop clipping paths Although this doesrepresent some human intrusion in determining external contourshape only pictures with easily visible figureground contours wereselected for inclusion and manual error was most likely 1 pixel orless given that clipping paths were created using high-resolution ver-sions of the stimuli which were reduced to approximately one fourthof their original size for presentation The animals from one of thephoto collections were provided with predefined clipping paths

Apparatus The apparatus was similar to that used in Experi-ment 3 with the exception that all the monitors displayed stimuli in256-level color and with a screen resolution of 600 3 800 pixels on15-in monitors Individual stimuli subtended approximately 49ordm3 46ordm and the entire array subtended approximately 184ordm 3 149ordm

Procedure The procedure was identical to that used in Experi-ment 3 Again the subjects completed the experiment in four ses-sions at their convenience The contour analysis procedure was iden-tical to that used in Experiment 3


error rate for target-present trials was 455 and themean error rate for target-absent trials was 236 Theerror rates for target-present trials were entered into atwo-factor (task 3 display size) within-subjects ANOVAThe task and display size effects were significant [taskeffect F(15) 5 11279 MSe 5 4771 p 5 0201 displaysize effect F(15) 5 40690 MSe 5 7921 p 5 0374]whereas the task 3 display size interaction was nearlysignificant [F(15) 5 3525 MSe 5 1414 p 5 1193]The error rates across display sizes were 430 and781 for three- and nine-item artifact target displaysand 221 and 391 for three- and nine-item animaldisplays respectively

Reaction times As in previous analyses only target-present trials were analyzed Mean search slopes and in-tercepts for each condition closely replicated those ob-served in Experiments 1 and 3 (see Figure 7) Search forartifact targets was significantly faster (62 msecitemas compared with 56 msecitem in Experiment 3) thansearch for animals [108 msecitem as compared with137 msecitem F(15) 5 68544 MSe 5 0898 p 50004] An item-based analysis comparing the 32 artifacttarget slopes with the 32 animal target slopes was similarlysignificant [F(162) 5 9191 MSe 5 3424 p 5 0035]Target-absent slopes were 168 msecitem in the artifact


target condition and 221 msecitem in the animal targetcondition Mean RTs for three-item displays in the target-present trials were significantly faster for animal targets(512 msec) than for artifact targets [534 msec F(15) 57890 p 5 038] There was no significant difference be-tween RTs in large displays (572 msec for animal targetsand 577 msec for artifact targets F lt 1)

The analysis of RTs over the four blocks of 512 trialsrevealed no learning effects RTs did not decrease overblocks nor was there any interaction between displaysize and block

As is evident from Figure 8 rectilinearity was corre-lated with search slopes (r 5 238 p lt 05) and inter-cepts (r 5 255 p lt 002) for artifacts For animals therectilinearityndashslope and rectilinearityndashintercept correla-tions were nonsignificant (r 5 28 p 5 11 and r 5 16p 5 38 respectively)

DiscussionExperiment 4 closely replicates Experiments 1 and 3

by revealing highly efficient search by category in colorimages of photographic origin Thus the present resultsgeneralize well across stimulus sets that are perceptuallyquite different In addition the same search asymmetrynearly the same slopes and the same rectilinearityndashsearchcorrelations were observed The latter finding is partic-ularly important because it confirms that global contourshape affects search by category in stimuli with minimalmediation via an artistrsquos hand

Before proceeding to Experiment 5 it is important todiscuss the difference in search slopes between the tasksIn Experiments 1 3 and 4 a search asymmetry was ob-served in which artifacts were located more quickly thananimals In Experiment 4 although the correlation is non-significant the direction of the relationship suggests thatrelatively rectilinear animals are more difficult to detectamong artifacts the reverse of the relationship betweenrectilinearity and artifact slopes This suggests that dif-ferences in contour shape between targets and distractorsgenerally facilitate search Curvilinear animals are rela-tively easy to detect among rectilinear artifacts and recti-linear artifacts are relatively easy to detect among curvi-linear animals This combined with the fact that animaldistractors are generally more homogeneous in contourshape might be sufficient to explain the search asym-metry without appeal to asymmetrically defined category-specifying features In both of the stimulus sets the stan-dard deviation of rectilinearity values for animals wasless than that for artifacts (Experiment 3 animal SD 5

Figure 7 Summary of the search data for Experiments 1 3 and 4

Figure 8 Correlation between search slopes and rectilinearity in Experiment 4 Note thatthe line of best fit for animal targets is nonsignificant even though it departs from zero prob-ably owing to range restriction in the rectilinearity measurements

Sea

rch

Slo

pe

Sea

rch

Slo

pe


032 artifact SD 5 097 Experiment 4 animal SD 5

059 artifact SD 5 245) Thus the animals had to be lo-cated among the heterogeneous artifacts whereas arti-facts could be located among the relatively homogeneousanimals According to Duncan and Humphreys (1989)heterogeneous distractors make targets difficult to de-tect Thus differences in the variability of contour shapemay have generally facilitated detection of artifacts Ifthis is added to the assumption that targetndashdistractor sim-ilarity with respect to contour shape also affects searchslopes we can explain both the asymmetry and the re-sponsiveness of both tasks to the rectilinearity of targets

EXPERIMENT 5

Although it seems that artifacts and animals differ interms of their external contours there are other between-category differences that might account for the results ofExperiments 3 and 4 First at a more general level therectilinearity measure could reflect general visual com-plexity If an objectrsquos shape is simple perhaps its con-tours do not change direction often which would resultin a high rectilinearity measure Thus facilitated detec-tion of artifacts would be based on their simplicity Sec-ond each object has parts that might be characteristic ofeither its own category or the other For example partssuch as handles and heads are characteristic of artifactsand animals respectively In addition some objects haveparts characteristic of the contrasting category For ex-ample the jaws of the pliers (Object 32 in Figure 1B) aresimilar in appearance to the jaws on an animal and theantennae on the butterfly (Object 3 in Figure 1A) mightbe similar to the antenna on a TV As is suggested by theinternal analysis of Experiment 3 the typicality of theobject parts may contribute to search efficiency in addi-tion to the rectilinearity of their contours

For Experiment 5 we completed a series of regressionanalyses predicting search slopes for all the stimuli usedin Experiments 3 and 4 In addition to rectilinearity pre-dictor variables included ratings of complexity ambigu-ity global typicality and a set of five more specific andfocused ratings of typicality Our goal was twofold Firstwe wanted to determine whether computed rectilinearitywould make a unique contribution to slope predictionsindependent of other commonly measured variables Sec-ond we wanted to determine whether other variableswould predict the slopes in a given task If rectilinearityand only rectilinearity predicts search slopes perhaps thetask is more similar to simple feature searches than wehad originally thought

MethodSubjects The analysis included three sets of ratings completed

by different groups of subjects The f irst two sets of ratings weredone on the black-and-white drawings in Experiment 3 and thecolor images in Experiment 4 by 11 and 14 subjects respectivelyand were completed as part of the stimulus development processThese subjects were undergraduate and graduate psychology stu-dents at Cornell University For the additional focused ratings oftypicality 7 subjects (graduate students in psychology at Kent State

University) rated all 128 objects from Experiments 3 and 4 In allcases the subjects who completed the ratings were different fromthose who completed the search experiments

Materials The first two sets of ratings used in this experimentwere originally collected during the process of generating the orig-inal stimulus sets for Experiments 3 and 4 These included ratingsof visual complexity typicality and ambiguity The subjects com-pleted the ratings by using paper packets depicting the stimuli(printed in color for the color set) They were instructed to rate com-plexity and ambiguity on the basis of the visual appearance of theobjects as pictured but were told to rate typicality more broadly andto include visual and nonvisual characteristics All ratings used a7-point scale with a rating of 7 being given to highly typical com-plex and ambiguous objects respectively

Another set of subjects completed f ive additional focused ratingsof typicality for each of the 128 objects in Experiments 3 and 4 Forthe first rating ( part-typicality ) the subjects indicated the degree towhich each object had parts that were typical of its category and theinstructions emphasized the need to rate typicality relative to ldquotheappearance of the objectrdquo The second rating (configuration typi-cality) was similar except that it emphasized the configuration ofparts instead of the parts themselves asking the subjects ldquoHow welldo the relative locations of the parts match your understanding ofthe typical arrangement for each kindrdquo Ratings 1 and 2 used scalesranging from very typical (rating of 1) to very atypical (rating of 7)The third rating (view typicality ) asked subjects ldquoHow well doesthe specific view in this specific photograph reveal the parts and con-figuration of parts as you know them for this object categoryrdquo andused a scale ranging from very characteristic (1) to not very char-acteristic (7) The fourth and fifth ratings were similar to the firstand second except that they focused on the similarity of the partsand configuration of each object to those from the other categoryFor the fourth rating (other part) the subjects were asked ldquoDoesthis object have parts that look like they might come from the otherkindrdquo and for the fifth (other whole) the subjects were asked ldquoDoesthis object as a whole look like an object from the other kindrdquo An-chor points for both scales were characteristic of own kind (ratingof 1) and characteristic of other kind (rating of 7) Although it wouldhave been possible to make the fifth rating parallel the second moreclosely (by referring directly to the relative locations of parts) wechose to refer to the ldquowholerdquo because we considered it unlikely thatthe subjects would rate many objects as specifically sharing a partconfiguration with the other category

The subjects were given rating packets that included a numberedprintout of the stimuli an instruction sheet containing the ratingscales and a set of scantron sheets on which to record their re-sponses All of the subjects were given a verbal summary of thescales then completed them on their own The entire task took ap-proximately 15 h

ResultsItem reliability Alpha scores were computed for each

item across subjects For the ambiguity complexity andtypicality ratings reliabilities were 88 95 and 76 re-spectively for the drawings and 79 88 and 66 re-spectively for the color images Reliabilities for the fo-cused typicality ratings were 52 40 71 66 and 73 forpart-typicality configuration-typicality view-typicalityother-part and other-whole ratings respectively A closerlook at the less reliable part- and configuration-typicalityratings shows that artifacts were particularly difficult torate reliabilities for artifact part- and configuration-typicality ratings were 39 and 17 respectively Individ-ual typicality ratings were therefore not included in theartifact analysis Animal ratings were better (reliabilities


of 58 and 52) but still not very reliable so individualratersrsquo responses were factor analyzed to determinewhether a subset of raters with higher agreement could beisolated Separate analyses of the animal part-typicalityand configuration-typicality ratings each yielded two pri-mary factors (collectively accounting for 605 of vari-ance in part-typicality ratings and 52 of variance inconfiguration-typicality ratings) The first (and strongest)factor in both cases included the same group of 3 ratersso new reliabilities were computed using only these ratersReliability for the first-factor part-typicality ratings was86 and the reliability of the configuration-typicality rat-ing was 79 The mean from these raters will constitute thetypicality ratings for the remainder of the analysis (How-ever significant typicality effects were rerun with themean of all raters and were not qualitatively different3)

Because the part-typicality and configuration-typicalityratings were associated with relatively low reliabilitiesand owing to the high part-typicalityconfiguration-typicality and other-partother-whole correlations (743and 732 respectively) a summary measure was com-puted that included the mean response to all four ratingsThis measure was created by averaging the mean of allfour ratings for the 3 first-factor subjects discussed abovewith the mean of the more reliable other-part and other-whole ratings for all 7 raters (Another set of regressionswas run that included all 7 ratersrsquo data on all four scalesand this produced almost identical results) The alpha re-liability for this composite rating was 81 and it will bereferred to as the focused typicality rating This score ishigh for objects that are atypical of their own category orsimilar to the other category

General strategy for regression analysis Analysespredicting search slopes were completed using two blocksof variables For all regressions search intercepts wereentered in an initial block to examine the effect of searchtermination independent of other variables In an initialregression all the remaining variables were entered in thesecond block This will be referred to as the full modelA second regression was run that focused on maximiz-ing multiple r values with the most parsimonious modelwhile keeping colinearity to a minimum (basic model)This was done by avoiding regressions that contained pre-dictors with correlations greater than 60 A correlationmatrix including all seven predictors was created for eachcategory For artifacts one correlation (21 possible) wasabove 60 (the correlation between the basic typicalityrating and complexity was 619) and for animals nonewas Finally these analyses were confirmed with stepwiseregressions using all variables done after a first blockforcing intercepts into the equation (stepwise model)

In addition departures from normality were correctedtransformations that were applied to all variables exceptthe focused typicality rating and search slopes Cube roottransformations were used in all cases except for searchintercepts which were adjusted with a reciprocal trans-formation

Animal slope predictions The full animal model ac-counted for 467 of the variance in slopes (see Table 1A)

Intercepts when entered alone in an initial block werenonsignificant predictors of slopes (b 5 19 p 5 13)but were significant when combined with the other vari-ables in the full regression (b 5 50 p 5 0001) Othersignificant predictors were complexity rectilinearityand the focused typicality rating The ambiguity and globaltypicality measures were marginally nonsignificant andthe view typicality measure did not approach significance(see Table 1A) A more parsimonious model that did notinclude the marginal predictors was also run It accountedfor 429 of the variance in search slopes (R 5 655)and included intercepts complexity rectilinearity andfocused typicality as predictors In addition a stepwiseregression was run to verify the predictiveness of variablesin the basic model In this regression intercepts were againentered in a first block then the other six variables wereentered stepwise in the second block The criteria werep 5 05 for inclusion and p 5 10 for exclusion This anal-ysis largely confirmed the initial analysis by producinga model that included intercepts rectilinearity and thefocused typicality measure (see Table 1A)

Variations of the basic model were tested by substi-tuting the individual typicality ratings for the summaryrating These showed that the other-wholeconfiguration-typicality ratings were stronger predictors than the otherpart part-typicality ratings and that the part-typicalityconfiguration-typicality ratings were stronger predictorsthan the other-partother-whole ratings In particular themodel with the configuration-typicality and other-wholeratings explained 56 of the variance in slopes whereasthe model with the part-typicality and other-part ratingsexplained 38 of the variance in slopes In the part-

Table 1ARegressions Predicting Search Slopes for Animal Targets

Variable B SE B b T Sig T

Full Model (Multiple R 5 684 R2 5 467)INTT 21660640 5098476 504383 4248 0001AMBT 5665797 4036385 165408 1404 1659CMPLT 10510682 4287268 267231 2452 0174RLINT 19390005 5364974 384114 3614 0006TYPT 28546618 5125400 2187687 21668 1010FocTyp 3798596 767565 552792 4949 0000ViewTypT 1348397 4526186 033590 298 7669(Constant) 266726922 16274271 24100 0001

Basic Model (Multiple R 5 655 R2 5 429)INTT 23073850 4871097 537290 4737 0000CMPLT 7566668 4039317 192380 1873 0660RLINT 18751544 5379376 371466 3486 0009FocTyp 3926384 752464 571389 5218 0000(Constant) 267857371 14063832 24825 0000

Stepwise Model (Multiple R 5 628 R2 5 395)INTT 22018171 4938517 512708 4458 0000RLINT 17931437 5472488 355220 3277 0017FocTyp 4205522 752826 612010 5586 0000(Constant) 254961835 12517628 24391 0000

NotemdashINTT transformed search intercept AMBT transformed am-biguity rating CMPLT transformed complexity rating RLINT trans-formed rectilinearity measure TYPT transformed global typicality rat-ing FocTyp focused typicality rating ViewTypT transformed viewtypicality rating


typicalityother-part regression the part-typicality ratingwas associated with a b of 587 ( p 5 0001) whereas theother-part rating was associated with a b of 210 ( p 5431) In the configuration-typicalityother-whole regres-sion the typicality rating was associated with a b of 572( p lt 0001) whereas the other-whole rating was associ-ated with a b of 18 (p 5 118)

Artifact slope predictions In the full model inter-cepts rectilinearity and the focused typicality ratingsemerged as significant predictors (see Table 1B) in amodel that explained 328 of slope variance (R 5 573)Again intercepts were nonsignificant predictors whentested alone in an initial block (b 5 2013) In the moreparsimonious basic model (Table 1B) artifact search slopeswere significantly predicted by intercepts rectilinearityand the focused typicality ratings and were marginallypredicted by ambiguity ratings Collectively these vari-ables predicted 312 of the variance (R 5 558) A step-wise analysis the same as that run for animal targets againconfirmed the predictiveness of variables in the basicmodel It included intercepts rectilinearity and the fo-cused typicality measure (see Table 1B)

Regressions substituting individual focused typicalityratings for the summary rating were run for artifacts us-ing only the other-part and other-whole ratings becausethe reliability coefficients for the part- and configuration-typicality ratings were very low for artifacts Two variantsof the basic model were run one using intercepts recti-linearity ambiguity and the other-part rating and onethat substituted the other-whole rating for the other-partrating The two regressions were very similar with the

whole rating producing a slightly better fit The modelwith the other-whole rating explained 30 of the variancein slopes whereas the model with the other-part ratingexplained 28 of the variance in slopes The other-partpredictor narrowly missed conventional levels of signifi-cance (b 5 231 p 5 0740) and the other-whole ratingwas significant (b 5 289 p 5 0281)

DiscussionExperiment 5 reveals two primary findings First rec-

tilinearity predicts search slopes for both artifacts andanimals Second in addition to rectilinearity the focusedtypicality ratings predict search slopes for both categor-ies The primary differences between the regressions foreach category are that the signs on the rectilinearity co-efficients are opposite and that the relative effectivenessof the two focused typicality ratings switch The formerdifference reflects the finding that rectilinear artifacts areeasy to locate among curvilinear animals and converselycurvilinear animals are easy to locate among rectilinearartifacts

At this point it is interesting to consider the relationshipbetween rectilinearityrsquos successful prediction of searchslopes and previous findings that curvature can act as aldquobasicrdquo or primitive feature in visual search (Treisman ampGormican 1988 Wolfe Yee amp Friedman-Hill 1992)In these previous findings curved line segments wereeasily located among straight line segments whereasstraight lines were more difficult to locate among curvedsegments Thus curvature would appear to be a featurewhereas straightness is not because it is easier to locate afeature-positive item among feature-negative items thanthe reverse (Treisman amp Gormican 1988) This seemsto be in contrast with our finding in which curvilinearityand rectilinearity appear to constitute different featurevalues on a single dimension As was discussed abovethe search asymmetry we observed can be explained byassuming that artifact slopes are generally shallower be-cause animal distractors are more homogeneous in con-tour shape (and probably parts as well) Also althoughrectilinear artifacts are more easily located among curvi-linear animals the reverse is also true Curvilinear ani-mals are more easily located among the rectilinear arti-facts Therefore there is no real feature coding asymmetryin the present case whereas there is one favoring curvi-linear line segments This finding is reminiscent of earlywork on serial visual search in which subjects searcheddown columns of letters and were able to scan morequickly when looking for curved letters (such as Q) thanfor rectilinear letters (Z) among rectilinear distractorswhereas the reverse was true when distractors were curvi-linear (Neisser 1963) although it is not clear whetherthese stimuli would produce a parallel search in a simul-taneously scanned object array One possibility is thatcurvature is a primitive in line segments whereas the en-tire continuum from rectilinearity to curvilinearity hasequal status in objects This qualification points out thedifficulty of extrapolating from the primitives defined

Table 1BRegressions Predicting Search Slopes for Artifact Targets



Basic Model (Multiple R 5 558 R2 5 312)INTT 11947608 4974639 336855 2402 0195AMBT 6778213 3575112 222232 1896 0629RLINT 219069971 5551178 2470472 23435 0011FocTyp 2627636 1075148 319888 2444 0175(Constant) 218814889 10151024 21853 0688




by simple stimuli such as line segments to more complexobjects and reinforces the suggestion that visual searchoften operates more on completed forms or surfaces thanon primitive features (eg He amp Nakayama 1992 Su-zuki amp Cavanagh 1995)

The switch in effectiveness of the part-typicality ratingand the other-part rating reflects the difficulty in ratingthe visual typicality of artifacts Both the part-typicalityand the configuration-typicality ratings were unreliablefor artifacts suggesting that the subjects were unable toconceive of what the typical artifact ought to look like Foranimals on the other hand these ratings were much morereliable and proved to be successful in predicting searchslopes As was mentioned above a number of authorshave suggested that distinguishing among animals re-quires more extensive visual processing because theytend to share one of a limited number of parts and con-figurations (eg Farah 1995 Warrington amp Shallice1984) For this reason it was probably considerably eas-ier for judges to rate each animal with reference to oneor more representations of their typical appearance Con-versely it was also possible for judges to reliably rate thedegree to which artifacts were visually similar to animalsTherefore it remains an open question as to whether a dif-ferent typicality rating would have been more successfulfor artifacts For example judges might rate how typicaleach artifact was with reference to its basic-level categoryIt is also important to note that the focused typicality rat-ings were successful where the more global rating wasnot Most likely this was due to the visual emphasis in-herent in the former The global typicality ratings did notspecify visual typicality per se and may have been influ-enced by nonvisual factors such as membership in a typ-ical species or knowledge of typical behaviors Generallyhowever one finding is clear from Experiment 5 Bothrectilinearity and some measure of visual typicality orsimilarity to the contrasting category predicted searchslopes for artifacts and animals

Another finding should be noted because it conflictsto a degree with the findings of Experiment 2 in whichjumbling slowed search only minimally If the noneffectof jumbling means that configural and form informationare completely ineffective in facilitating search why didthe configuration-typicality and other-whole ratings pre-dict search slopes so well Although it is not possible tobe certain at least two factors may be relevant for produc-ing this finding First the part-typicality and configuration-typicality ratings were correlated (r 5 641 for artifactsand r 5 862 for animals) as were the other-part and other-whole ratings (r 5 806 for artifacts and r 5 588 for an-imals) Therefore both ratings may have tapped a generalimpression of visual typicality that depended on partsand configuration It is also possible that the raters tookthe configurationwhole ratings to mean that they shouldrate the entire object inclusive of all parts and perhapsfocused on only a single part for the part ratings Finallythe stimuli in Experiment 2 may have disrupted the rela-tions among some parts while leaving other aspects of

the objectrsquos general form intact For example in Figure 3the basic horizontal body shape is still present in some ofthe animals and in addition some part whole relation-ships remain intact (eg legs and heads still protrude fromthe body) Given the research cited earlier that shows thatsome basic aspects of form may be available preatten-tively it is possible that the global shape affected both theratings and the search slopes If we assume that animalsare more likely to be associated with a specific globalshape this explanation fits well with the fact that wholeratings improved animal slope predictions the most

We interpret these findings to suggest that visual searchby category is a complex process that depends on multi-ple levels of simultaneous analysis Consistent with cur-rent theory the faster search (that for the artifact target)is relatively more dependent on contour rectilinearitywhich is a plausible perceptual primitive and in the sloweranimal search is more dependent on the typicality mea-sure (although this search is also fast with many targetsbeing located in less than 10 msecitem) However incontrast to established theory we find no sharp distinc-tion between tasks that depend on a primitive versus amore conceptual variable in that both tasks depend to adegree on both kinds of property Indeed a recent reviewby Wolfe (1998b) confirms blurring of the distinctionbetween parallel primitive-based search and a serial searchbased on focal attention and goes on to suggest that thisfinding implies the need for a revision in our conceptu-alization of the visual search task as a whole In the Gen-eral Discussion section we summarize our findings andargue that they suggest the need for an added layer of ex-planation to account for the efficiency with which peoplesearch for members of broad natural object categories

GENERAL DISCUSSION

The results presented here provide evidence for an ef-ficient process that can distinguish categories and pointsto specific features that underlie this process Experi-ments 1 3 and 4 converge to show that search for an un-specified artifact in a heterogeneous field of animals pro-ceeds at 55ndash62 msecitem These experiments also reveala search asymmetry search slopes for animal targetsamong artifacts were 108ndash160 msecitem In both casesshallow slopes were accompanied by relatively low in-tercepts (512ndash624 msec) suggesting that the subjectswere not withholding responses to three-item displays(and thereby producing artificially low slopes) Experi-ment 2 tested the hypothesis that the efficient search ob-served in Experiment 1 was based on high-level featuresand nonperceptual information by jumbling objects inorder to disrupt completion of structural descriptionsSearch slopes were not significantly slower for part-jumbled targets although intercepts were approximately100 msec longer Experiments 3 and 4 tested the possi-bility that between-category differences in contour shapefacilitated the search We found that targetndashdistractordifferences in rectilinearity combined with homogeneity


of contour statistics facilitate search and in the case ofrectilinear artifacts among relatively homogeneous ani-mals the slope approaches 0 msecitem Thus visualsearch by category is efficient and can be facilitated by thepresence of natural nondefining features Finally Ex-periment 5 tested for other properties of targets that mightexplain search slopes In a multiple regression analysisnot only did rectilinearity predict search slopes but vi-sual typicality (for animals) and similarity to the distract-ing category (for artifacts) did so as well

Efficient Search by Category andTheories of Visual Search

The distinction between serial and parallel search iscentral to many models of visual search For exampleFIT suggests that parallel searches are observed onlywhere a single feature distinguishes targets from distrac-tors In addition that feature must be one among a smallset of perceptual primitives coded early in the visual sys-tem A target that is a different color from distractors orone that has a fundamentally different shape from dis-tractors will ldquopop outrdquo and be located without the neces-sity of a serial search Other theories such as guided search(Wolfe et al 1989) are similar in that they emphasize therole of unique features or unique conjunctions of featuresin allowing a parallel search Duncan and Humphreys(1989) described a more general model relying on thesimilarity between targets and distractors and variationsin distractor similarity to explain visual search All of thesetheories were developed and tested using artificial stimulicarefully controlled to reflect simple feature differencesthat are almost entirely diagnostic of the targetndashdistractordifference The question is can these theories serve as use-ful frameworks to understand visual search for naturalcategories as well

We think existing models are helpful in understandingthe speed of the search and the search asymmetry It ap-pears as though parallel search was possible at least partlyon the basis of features very similar to the perceptualprimitives discussed by Treisman and Wolfe Rectilin-earity might be seen as a form primitive that is indeedbased on early visual codes As was mentioned previ-ously rectilinear and curvilinear contours may be repre-sented in functionally different early visual maps In ad-dition the variable nature of the rectilinearity feature isclearly not problematic in that both FIT and guided searchinclude variability in the amount of signal each featurerepresents as measured against the noise represented bythe distractors Somewhat rectilinear features induce asmaller amount of signal in the relevant feature map andhighly rectilinear targets induce a stronger signal As wassuggested in the discussion of Experiment 4 the Duncanand Humphreys (1989) model fits our data as well Sim-ilarity between targets and distractors and similarityamong distractors both appear to affect search in waysthat this model can explain

Although current models seem to provide a helpfulpoint of departure for understanding they do not appear

to offer a complete explanation In particular it appearsthat no specific primitive feature or conjunction of fea-tures can distinguish all targets from distractors As wasmentioned above rectilinearity might be a primitive butby itself it is not sufficient for the task and the predic-tive power of visual typicality for animal targets and sim-ilarity to the distractor category for artifacts points awayfrom an exclusively primitive-based explanation In ad-dition artifacts that were not different from animals inrectilinearity were still located quickly (10ndash12 msecitem)Of course one could argue that we simply have notfound the right primitive and that if we were to searchfurther perhaps some simple feature could be found thatdistinguishes all of our targets from all of our distractorsand that in addition explains the variance captured byboth rectilinearity and typicality Although a careful lookthrough our stimuli by ourselves and others has yieldedno workable candidates we admit that we cannot provethat none exists However proving this kind of negativehypothesis is inherently impossible

Instead we should consider more carefully the funda-mental advantage of referring to feature primitives in thefirst place The benefit of primitives to any theory of vi-sion is to propose a limited set of features that serve asbuilding blocks for more complex representations There-fore these features should be few in number simple andcombine productively to produce a wide variety of rep-resentations However as Nakayama and Joseph (1997)point out we might need to question current models ofpreattentive vision in part because these requirementsare apparently not being met For example they arguethat quite a number of primitives are being cataloged fortheir ability to drive efficient search and in addition thatthese primitives are not particularly simple and often de-pend on completed descriptions of surfaces depth rela-tions and 3-D structure Given this critique primitive-based theories will have difficulty predicting a priori whena search will be efficient on the basis of known stimulusproperties In a sense Wolfe (1998a) points out this dif-ficulty by suggesting that the list of primitives will bedifferent when measured in different ways Here it ap-pears that the list will also be different for different kindsof stimuli within the search task This problem can bemost clearly seen in that curvature and rectilinearity haveequal status as feature values on a given dimensionwhereas previous research suggested that curvature is afeature against the norm of straightness As was men-tioned above the difference between the present case andprevious findings is that curvature is a characteristic ofclosed object contours here whereas in previous researchcurvature has been a property of line segments Thereforeit appears that combining segment primitives into con-tour primitives is not straightforward This problem maybe particularly acute in complex natural stimuli becausein these situations it seems necessary to posit more andmore complex primitives Therefore we should worryabout being stuck inventing new primitives for each cat-egory contrast we test If one allows for complex primi-


tives that change in different sets of stimuli especiallyones that closely conform to the perceptual boundariesbetween specific natural categories they cease to be prim-itives that can be combined productively Instead theymight be better described as the perceptual descriptionof the categories themselves

If the features that contrast specific categories can guideefficient visual search we should consider the need toexplain how these features combine and guide a categorysearch on any given trial One possibility is that a knowncategory is described by a search template (eg Duncanamp Humphreys 1989) that specifies features at a varietyof levels of complexity ranging from simple aspects ofform to a list of typical parts In the case of animals thismight involve a fairly straightforward analysis of the de-gree to which potential targets have typical animal partssuch as eyes a head legs and a body As was mentionedabove artifacts are more difficult to associate with aglobally typical set of parts or a configuration and ourratings clearly reflect this Therefore the predictivenessof the similar-to-other ratings in artifacts may reflect abetween-category difference in the usability of typicalvisual information or it may reflect a difference in thelevel of generality in the analysis between the categoriesIn either case detection based on both a simple percep-tual feature and the relationship between each object andits category also suggests that processing prior to focalattention proceeds at multiple levels of analysis simulta-neously In part this analysis might reflect segregatedactivations in parts of the visual system specialized forprocessing each category especially if one assumes thatvisually typical objects produce more activation in theirrespective parts of the visual system

At this point it is helpful to mention an important lim-itation to the data reported here We have tested only onebroad categorical distinction and have assumed that theefficiency associated with it is caused by its familiarityor foundational nature However although we consider itunlikely it remains possible that other similar nonfoun-dational contrasts would also be efficiently processedeven in cases in which no single feature distinguishes thecategories Therefore it might be worthwhile to attemptexperiments in which the same heterogeneous stimulusset was divided into both foundational categories andmore arbitrary (but equally heterogeneous) nonfounda-tional categories For example it might be possible tohave subjects search a carefully chosen set of animals andartifacts using either the animalartifact distinction oranother distinction such as that between objects typicallyfound in America versus those found in Africa or that be-tween objects larger than a breadbox versus those smallerthan a breadbox Although it might be difficult to arguethat inefficiently distinguished nonfoundational con-trasts are as separable in form as the efficiently distin-guished foundational contrast the experiment seems worthattempting because it would further support the hypoth-

esis that real object categories can escape the effects oftheir formal complexity

According to the above argument understanding vi-sual search in natural categories does not require aban-doning current theory Rather it requires an added levelof explanation of how features cohere into known bun-dles or categories This explanation will probably includeboth a general description of the limits of feature com-bination (a good example is Wolfe amp Bennett 1997) anda description of the specific categories that search stimulirepresent The concept literature provides a good startingpoint in suggesting that knowledge is organized arounda few foundational categories (Carey 1985 Keil 1989)that may have similar organizing effects in the visual sys-tem Therefore category-specific explanations need notproliferate uncontrollably and might limit themselves tobroadly inclusive categories One area of research thatseems to confirm the tractability of this problem is neu-ropsychological findings that brain injuries can selec-tively reduce visual performance for one broadly inclu-sive category or another

Neuropsychological FindingsRecent neuropsychological evidence suggests that

both semantic and object recognition systems may be or-ganized by categories and levels of abstraction of fea-tures that are different in each stage Caramazza and Shel-ton (1998) described a patient EW with a specificdeficit in recognizing animals but not other living or non-living things EW performed poorly in naming and realunreal object judgment tasks for animals Overall the pat-tern of deficit suggested that her impairment was spe-cific to the animal kind rather than to a particular fea-ture (eg sensory or functional features) Caramazzaand Shelton suggested that the semantic system is orga-nized by category and that the neural substrate for rec-ognizing animals and plants may be distinct from that forother objects because recognizing animals and plants ac-curately gives a large evolutionary advantage

If in fact semantic storage is organized by categoryit is possible that perceptual processing is similarly or-ganized In fact a number of patients show a category-specific impairment in the realunreal object-decision taskwhich indicates a category-specific impairment in thestructural analysis of objects (Mauri Daum SartoriRiesch amp Birbaumer 1994 Sartori amp Job 1988 SartoriJob amp Colheart 1992) Furthermore a category-specificimpairment in the structural analysis stage can be inde-pendent of semantic deficits Sheriden and Humphreys(1993) described a patient with difficulty in recognizinganimals and food who performed adequately in realnonreal object judgment for animals and food Some ev-idence suggests that a selective impairment in processinga lower level feature can also cause an impairment in rec-ognizing a particular class of objects Kosslyn Hamil-ton and Bernstein (1995) reported a study of a prosopag-


nosia patient whose difficulty in recognizing faces seemedto derive from his difficulty in processing curvature

The neuropsychological data therefore suggest thatprocessing systems may be segregated by category inlate vision and furthermore that different early visualsystems may connect differentially to these later areasGiven that the search asymmetry in the present experi-ment could be explained by differences in the hetero-geneity of the two stimulus categories it is possible thatcurvilinear and rectilinear contours are processed by sys-tems that are independent at some point We might there-fore expect at some point to observe a patient who has dif-ficulty integrating rectilinear contours to contrast withpatients who seem to have a curvilinearity deficit

Summary and ConclusionsThe experiments reported here suggest that an effi-

cient visual search for targets in a broad category dependson both basic perceptual features such as rectilinearityand a more sophisticated analysis of part typicality Thusthe surprisingly efficient visual search finding seems torepresent a case in which early visual processes prior tothe arrival of focal attention reflect the natural categor-ical organization of the visual world Although this find-ing would not necessarily have been predicted on thebasis of current theories of visual search it does not in-validate them Rather it suggests that added levels of ex-planation are necessary to model real-world visual searchsuccessfully In particular it seems necessary to explainhow different perceptual features cohere to form de-scriptions of targets and distractors that can be used asthe basis of efficient visual search

REFERENCES

Bertenthal B I (1993) Infantsrsquo perception of biomechanical mo-tions Intrinsic image and knowledge based constraints In C Granrud(Ed) Visual perception and cognition in infancy (pp 175-214) Hills-dale NJ Erlbaum

Biederman I Blickle T W Teitelbaum R C amp Klatsky G J(1988) Object search in nonscene displays Journal of ExperimentalPsychology Learning Memory amp Cognition 14 456-467

Caramazza A amp Shelton J R (1998) Domain-specific knowledgesystems in the brain The animatendashinanimate distinction Journal ofCognitive Neuroscience 10 1-34

Carey S (1985) Conceptual change in childhood Cambridge MAMIT Press

Chun M M amp Wolfe J M (1996) Just say no How are visualsearches terminated when there is no target present Cognitive Psy-chology 30 39-78

Donnelly D Humphreys G W amp Riddoch M J (1991) Parallelcomputation of primitive shape descriptions Journal of Experimen-tal Psychology Human Perception amp Performance 17 561-570

Duncan J (1983) Category effects in visual search A failure to repli-cate the ldquoohndashzerordquo phenomenon Perception amp Psychophysics 34 221-232

Duncan J amp Humphreys G W (1989) Visual search and stimulussimilarity Psychological Review 96 433-458

Egeth H Jonides J amp Wall S (1972) Parallel processing in multi-element displays Cognitive Psychology 3 674-698

Enns J amp Rensink R A (1991) Preattentive recovery of three-dimensional orientation from line drawings Psychological Review98 101-118

Farah M J (1995) Dissociable systems for recognition A cognitiveneuropsychology approach In S M Kosslyn amp D N Osherson (Eds)Visual cognition An invitation to cognitive science (pp 101-120)Cambridge MA MIT Press

Farah M J McMullen P A amp Meyer M M (1991) Can recog-nition of living things be selectively impaired Neuropsychologia 29185-193

Funnell E amp Sheridan J (1992) Categories of knowledge Unfa-miliar aspects of living and nonliving things Cognitive Neuropsy-chology 9 135-153

Gaffan D amp Heywood C A (1993) A spurious category-specificvisual agnosia for living things in normal human and nonhuman pri-mates Journal of Cognitive Neuroscience 5 118-128

Gilbert C D (1993) Circuitry architecture and functional dynam-ics of visual cortex Cerebral Cortex 3 373-386

Gutheil G Vera A amp Keil F C (1998) Do houseflies think Pat-terns of induction and biological beliefs in development Cognition66 33-49

He Z J amp Nakayama K (1992) Surface versus features in visualsearch Nature 359 231-233

Intraub H (1981) Rapid conceptual identification of sequentiallypresented pictures Journal of Experimental Psychology HumanPerception amp Performance 7 604-610

Jain R Kasturi R amp Schunck B G (1995) Machine vision NewYork McGraw-Hill

Jonides J amp Gleitman H (1972) A conceptual category effect in vi-sual search O as letter or as digit Perception amp Psychophysics 12457-460

Keil F C (1989) Concepts kinds and cognitive development Cam-bridge MA MIT Press

Keil F C (1994) Explanation association and the acquisition of wordmeaning Lingua 92 169-196

Kosslyn S M Hamilton S E amp Bernstein J H (1995) The per-ception of curvature can be selectively disrupted in prosopagnosiaBrain amp Cognition 27 36-58

Krueger L E (1984) The category effect in visual search depends onphysical rather than conceptual differences Perception amp Psycho-physics 35 558-564

Kurbat M A (1997) Can the recognition of living things really be se-lectively impaired Neuropsychologia 35 813-827

Mack A amp Rock I (1998) Inattentional blindness Cambridge MAMIT Press

Mandler J M amp McDonough L (1993) Concept formation in in-fancy Cognitive Development 8 291-318

Martin A Haxby J Lalonde F M Wiggs C L amp Ungerlei-der L G (1995) Discrete cortical regions associated with knowl-edge of color and knowledge of action Science 270 102-105

Martin A Wiggs C L Ungerleider L G amp Haxby J V (1996)Neural correlates of category-specif ic knowledge Nature 379649-652

Massey C M amp Gelman R (1988) Preschoolersrsquo ability to decidewhether a photographed unfamiliar object can move itself Develop-mental Psychology 24 307-317

Mauri A Daum I Sartori G Riesch G amp Birbaumer N(1994) Category-specific semantic impairment in Alzheimerrsquos dis-ease and temporal lobe dysfunction A comparative study Journal ofClinical amp Experimental Psychology 16 689-701

Nakayama K amp Joseph J (1997) Attention pattern recognition andpopout in visual search In R Parasuraman (Ed) The attentive brain(pp 279-298) Cambridge MA MIT Press

Neisser U (1963) Decision-time without reaction-time Experimentsin visual scanning American Journal of Psychology 76 376-385

Nickerson R S (1968) A note on long-term recognition memory forpictorial material Psychonomic Science 11 58

Olshausen B A amp Field D J (1996) Relations between the ldquoas-sociation fieldrdquo and the statistics of natural scenes [Abstract] Inves-tigative Ophthalmology amp Visual Science 37 2370

Potter M C (1976) Short-term conceptual memory for picturesJournal of Experimental Psychology Human Language amp Memory2 509-522


Potter M C amp Faulconer B A (1975) Time to understand picturesand words Nature 253 437-438

Ramachandran V S (1988 Month) Perceiving shape from shadingScientific American 259 76-83

Rensink R (2000) The dynamic representation of scenes Visual Cog-nition 7 17-42

Rosengren K S Gelman S A Kalish C W amp McCormick M(1991) As time goes by Childrenrsquos early understanding of growth inanimals Child Development 62 1302-1320

Sacchett C amp Humphreys G W (1992) Calling a squirrel a squirrelbut a canoe a wigwam A category-specific deficit for artifactual ob-jects and body parts Cognitive Neuropsychology 9 73-86

Sanocki T Bowyer K Heath M amp Sarkar S (1996 May) Areedges sufficient for object recognition Paper presented at the ObjectPerception and Memory Conference Chicago

Sartori G amp Job R (1988) The oyster with four legs A neuropsycho-logical study on the interaction of visual and semantic informationCognitive Neuropsychology 5 105-132

Sartori G Job R amp Colheart M (1992) The organization of ob-ject knowledge Evidence from neuropsychology In D E Meyer ampS Kornblum (Eds) Attention and performance XIV Synergies in ex-perimental psychology artificial intelligence and cognitive neuro-science (pp 451-465) Cambridge MA MIT Press

Shepard R N (1967) Recognition memory for words sentences andpictures Journal of Verbal Learning amp Verbal Behavior 6 156-163

Sheridan J amp Humphreys G W (1993) A verbal-semantic category-specific recognition impairment Cognitive Neuropsychology 10143-184

Snodgrass J G amp Vanderwart M (1980) A standardized set of260 pictures Norms for name agreement image agreement familiar-ity and visual complexity Journal of Experimental PsychologyHuman Learning amp Memory 6 174-215

Standing L Conezio J amp Haber R N (1970) Perception andmemory for pictures Single-trial learning of 2500 visual stimuliPsychonomic Science 19 73-74

Stewart F Parkin A J amp Hunkin N M (1992) Naming impair-ments following recovery from herpes simplex encephalitis Category-specific Quarterly Journal of Experimental Psychology 44A 261-284

Suzuki S amp Cavanagh P (1995) Facial organization blocks accessto low-level features An object-inferiority effect Journal of Experi-mental Psychology Human Perception amp Performance 21 901-913

Theeuwes J (1993) Visual selective attention A theoretical analysisActa Psychologica 83 93-154

Thorpe S Fize D amp Marlot C (1996) Speed of processing in thehuman visual system Nature 381 520-522

Tranel D Logan C G Frank R J amp Damasio A R (1997) Ex-plaining category-related effects in the retrieval of conceptual and lex-ical knowledge for concrete entities Operationalization and analysisof factors Neuropsychologia 35 1329-1339

Treisman A amp Gelade G (1980) A feature integration theory of at-tention Cognitive Psychology 12 97-136

Treisman A amp Gormican S (1988) Feature analysis in early visionEvidence from search asymmetries Psychological Review 95 15-48

Van de Walle G A amp Hoerger M (1996) The perceptual founda-tions of categorization in infancy [Abstract] Infant Behavior amp De-velopment 19 794

Warren W H (1984) Perceiving affordances Visual guidance of stairclimbing Journal of Experimental Psychology Human Perceptionamp Performance 10 683-703

Warrington E K amp Shallice T (1984) Category specific seman-tic impairments Brain 107 829-854

Wolfe J M (1994) Visual search in continuous naturalistic stimuliVision Research 34 1187-1195

Wolfe J M (1998a) Visual search In H Pashler (Ed) Attention(pp 13-73) Hove UK Psychology Press

Wolfe J M (1998b) What can 1 million trials tell us about visualsearch Psychological Science 9 33-39

Wolfe J M amp Bennett S C (1997) Preattentive object files Shape-less bundles of basic features Vision Research 37 25-43

Wolfe J M Cave K R amp Franzel S L (1989) Guided searchAn alternative to the feature integration model for visual searchJournal of Experimental Psychology Human Perception amp Perfor-mance 15 419-433

Wolfe J M Yee A amp Friedman-Hill S R (1992) Curvature is abasic feature for visual search tasks Perception 21 465-480

Woodward A L (1998) Infants selectively encode the goal object ofan actorrsquos reach Cognition 69 1-34

Zelinsky G J Rao R P N Hayhoe M M amp Ballard D H(1997) Eye movements reveal the spatiotemporal dynamics of visualsearch Psychological Science 8 448-453

Zusne L (1975) Curved contours and the associative response Per-ceptual amp Motor Skills 40 203-208

NOTES

1 The concept of preattentive vision has come under attack recentlygiven findings that little information seems to be registered under con-ditions of inattention (eg Mack amp Rock 1998) Thus we refer to pre-attentive vision in the less literal sense of visual processing that occursprior to the arrival of focal attention on a specif ic object or part of anobject while leaving open the possibility that prior to the arrival of focalattention the scene as a whole is processed by a more broad applicationof attention

2 Choosing distractors with replacement eliminates item-to-item de-pendencies that may allow subjects to focus on a subset of features oncea given distractor has been rejected For example if one particular dis-tractor is known to be difficult to reject subjects may be able to rejectit once then assume it will not recur and focus on ldquoeasierrdquo featuresChoosing distractors with replacement does of course have the draw-back of causing some displays to have multiple copies of an object amongthe distractors and may be better suited to a slower search than ours inwhich a serial rejection of distractors (or groups of distractors) morestrongly affects search slopes In the present case it is possible thatslopes were artificially reduced if one assumes that duplications amongdistractors make search easier because those duplications are more likelyin more numerous displays (eg 132 three-item target-present displays1032 six-item target-present displays and 2832 nine-item target-presentdisplays will have at least one pair of identical distractors) However ifthis were the case one would expect steeper slopes when comparingthree-item displays with six-item displays than when comparing six-item displays with nine-item displays because the likelihood of dupli-cates increases more between the six- and the nine-item displays A lookat the data from Experiments 1 and 2 do not support this hypothesis andthe average slopes based on the three- and six-item displays (131 msecitem based on the combined results of all target-present data in Exper-iments 1 and 2) are almost identical to those computed for the six- andnine-item displays (135 msecitem)

3 Regressions with significant part- and whole-typicality ratingswere rerun with the mean rating from all 7 raters For the part-typicalityother-part model predicting artifact slopes the colinearity measure andthe part-typicality rating were again significant (part typicality b 5 436p 5 0018 colinearity b 5 330 p 5 0065) whereas the complexity andother-part ratings were nonsignificant (complexity b 5 209 p 5 0712other part b 5 2011 p 5 931) The multiple R for the regression asa whole was 568 For the whole-typicalityother-whole model predict-ing animal slopes the colinearity measure and the whole-typicality rat-ing were again significant (whole typicality b 5 474 p 5 0003 col-inearity b 5 429 p 5 0001) and the complexity and other-part ratingswere also significant (complexity b 5 212 p 5 0293 other part b 5249 p 5 0384) The multiple R for the regression as a whole was 715


APPENDIXTarget Names for the Color Image Set

Animals Artifacts

badger ashtraybear bookbobcat hairbrushlion calculatorcheetah calculatorchipmunk video cameracrab chairdolphin clamptropical fish clocktropical fish diskeagle CD ROM drivekangaroo dollymale lion headphoneslizard helicopterorangutan allen wrench setostrich shoeporcupine ice scraperpanther circuit boardfrog binderrabbit stereo amplifiermouse penseal needle nose pliersshark plierssnake podiumsnake carspider scaletarantula screwdriverstingray flour siftergoat cartiger staple removerzebra briefcaseotter sunglasses

(Manuscript received March 2 2000revision accepted for publication September 8 2000)


question for simple abstract stimuli no findings we knowof have tested what features inherent to natural object cat-egories affect search slopes The feature we explored firstwas the relatively rectilinear shape of artifact contoursas compared with the relatively curvilinear shape of an-imal contours In addition a number of other factors weretested for their ability to predict search slopes includingcomplexity ambiguity visual typicality and similaritybetween the target object and the distractor categories

Visual Search by CategoryResearch using a wide variety of paradigms has shown

that humans are capable of efficiently perceiving and re-membering natural objects Early experiments on naturalscene perception concentrated on the degree to which alarge number of individual scenes could be rememberedMost of these studies emphasized our apparently limit-less ability to discriminate familiar scenes from novel ones(Nickerson 1968 Shepard 1967 Standing Conezio ampHaber 1970) whereas later work focused on the amountof time necessary to encode a scene for both on-line iden-tification and later recognition Several studies foundthat brief 100- to 200-msec exposures were sufficient toidentify objects and scenes but not to remember them(Intraub 1981 Potter 1976 Potter amp Faulconer 1975)

Other research using a visual search paradigm hasshown that a specific object can be located relativelyquickly For example Biederman Blickle Teitelbaumand Klatsky (1988) found that subjects performed abovechance at detecting a named target in arrays presentedfor 100 msec However they also found that increases indisplay size reduced accuracy prompting them to suggestthat object search was not capacity free More recentlyZelinsky Rao Hayhoe and Ballard (1997) had subjectslocate a color image of a target object in a pseudorealis-tic setting (eg a stuffed animal among other toys in acrib) On each trial the target image was shown in isola-tion then the subjects indicated whether it was presenton the table An analysis of eye movements suggestedthat the first saccade after fixation was directed towardthe middle of the display (which contained no objectsbecause the stimuli were arranged in a semicircle) Thesecond saccade then proceeded in the general directionof the target followed by further saccades that were di-rected with increasing precision at the target Thereforealthough search reaction times (RTs) indicated a 28-msecitem search rate the subjectsrsquo eye movements revealedan ability to determine which part of the screen includedthe target without the benefit of a serial scan of each ob-ject in turn

Although this rapid target identification suggests sometype of preattentive coding the data reviewed above arebased on identifying particular targets and thereforecould plausibly be driven by simple perceptual cues Forexample the rapid detection in Zelinsky et alrsquos (1997)research could be based on strategic selection of distinc-tive perceptual features such as object color and basicaspects of shape Subjects asked to detect a screwdriver

might search for the particular shape and orientation ofits shaft thus doing the search on the basis of the kind ofsimple perceptual feature that is traditionally thought todrive parallel search The same could be said about thescene classification research which typically uses basic-level target descriptions if not actual images of the tar-get to be detected However in Intraubrsquos (1981) rapid iden-tification task subjects successfully responded on thebasis of the nonpresence of a category of object (egthey might be instructed to ldquolook for a picture that is notof house furnishings and decorationsrdquo p 608) This doessuggest a more sophisticated process because no partic-ular target was selected Indeed this sophistication doesfind support in a few studies that have assessed the degreeto which subjects can detect targets solely on the basis oftheir membership in a category that presumably is markedby no particular perceptual feature

The best-known example of this kind of visual searchis Egeth Jonides and Wall (1972) who showed that sub-jects could locate a number among letters or could do thereverse in parallel Jonides and Gleitman (1972) furthershowed that an ambiguous ldquoOrdquo stimulus (which could bethe number zero or the letter ldquoordquo) could be located in par-allel among both numbers and letters Thus they arguedthat their findings reflected a truly categorical visualsearch that could not be explained solely on the basis ofthe direct association between a given shape and a givenresponse However a number of attempts to replicate thiseffect have failed (eg Duncan 1983 Krueger 1984) soit must be considered to be tentative at best

Although the status of efficient alphanumeric categorysearch is questionable more recent data from experimentsusing broad natural categories reinforce the possibilityof efficient categorical visual search Thorpe Fize andMarlot (1996) tested subjectsrsquo ability to locate an animalin a series of natural scenes and found that RTs were asshort as 400 msec implying that the animals had actuallybeen perceived in less than 200 msec Evoked potentialsconfirmed this by showing a divergence in brain activityfor scenes with and without animals as soon as 150 msecafter stimulus onset (Thorpe et al 1996) If we assumethat their images contained a sufficient variety of animalsso that they were not all marked by the presence of a par-ticular part (such as legs) these data suggest that com-plex natural scenes can be rapidly parsed by category

This finding is particularly interesting because it mayreflect the foundational nature of Thorpe et alrsquos (1996)stimulus categories The contrast between artifacts andanimals may in fact represent one of the first ways thatchildren divide the world and may serve throughout adult-hood as a conceptual framework for more specific learn-ing (Keil 1989) For example Gutheil Vera and Keil(1998) taught young children a new property for a givenanimal and then asked them whether it would be presentin other objects The children readily attributed the newproperties to a wide variety of living things ranging fromdogs to worms yet refused to generalize it to nonlivingobjects In addition young children understand a num-















EXPERIMENT 1












A

B


















EXPERIMENT 2






















EXPERIMENT 3

































EXPERIMENT 4





















Sea

rch

Slo

pe

Sea

rch

Slo

pe




EXPERIMENT 5













































GENERAL DISCUSSION
























REFERENCES





































































NOTES






Animals Artifacts

















EXPERIMENT 1












A

B


















EXPERIMENT 2






















EXPERIMENT 3

































EXPERIMENT 4





















Sea

rch

Slo

pe

Sea

rch

Slo

pe




EXPERIMENT 5













































GENERAL DISCUSSION
























REFERENCES





































































NOTES






Animals Artifacts





A

B


















EXPERIMENT 2






















EXPERIMENT 3

































EXPERIMENT 4





















Sea

rch

Slo

pe

Sea

rch

Slo

pe




EXPERIMENT 5













































GENERAL DISCUSSION
























REFERENCES





































































NOTES






Animals Artifacts




















EXPERIMENT 2






















EXPERIMENT 3

































EXPERIMENT 4





















Sea

rch

Slo

pe

Sea

rch

Slo

pe




EXPERIMENT 5













































GENERAL DISCUSSION
























REFERENCES





































































NOTES






Animals Artifacts



















EXPERIMENT 3

































EXPERIMENT 4





















Sea

rch

Slo

pe

Sea

rch

Slo

pe




EXPERIMENT 5













































GENERAL DISCUSSION
























REFERENCES





































































NOTES






Animals Artifacts






















EXPERIMENT 4





















Sea

rch

Slo

pe

Sea

rch

Slo

pe




EXPERIMENT 5













































GENERAL DISCUSSION
























REFERENCES





































































NOTES






Animals Artifacts












Sea

rch

Slo

pe

Sea

rch

Slo

pe




EXPERIMENT 5













































GENERAL DISCUSSION
























REFERENCES





































































NOTES






Animals Artifacts






EXPERIMENT 5













































GENERAL DISCUSSION
























REFERENCES





































































NOTES






Animals Artifacts





































GENERAL DISCUSSION
























REFERENCES





































































NOTES






Animals Artifacts

























REFERENCES





































































NOTES






Animals Artifacts



































NOTES






Animals Artifacts





Animals Artifacts



Efficient visual search by category: Specifying the features that ...

Documents

Transcript of Efficient visual search by category: Specifying the features that ...