Maria Korman*

Occupational Therapy Department

The Faculty of Social Welfare and

Health Sciences

University of Haifa

Mount Carmel, Haifa, Israel, 31905

Kinneret Teodorescu

Adi Cohen

Miriam Reiner

Daniel Gopher

Technion

Israel Institute of Technology, Israel

Effects of Order and SensoryModality in Stiffness Perception

Abstract

The stiffness properties of an environment are perceived during active manual manipu-

lation primarily by processing force cues and position-based tactile, kinesthetic, and vis-

ual information. Using a two alternative forced choice (2AFC) stiffness discrimination

task, we tested how the perceiver integrates stiffness-related information based on

sensory feedback from one or two modalities and the origins of within-session shifts in

stiffness discrimination ability. Two factors were investigated: practice and the amount

of available sensory information. Subjects discriminated between the stiffness of two

targets that were presented either haptically or visuohaptically in two subsequent

blocks. Our results show that prior experience in a unisensory haptic stiffness discrimi-

nation block greatly improved performance when visual feedback was subsequently

provided along with haptic feedback. This improvement could not be attributed to

effects induced by practice or multisensory stimulus presentation. Our findings suggest

that optimization integration theories of multisensory perception need to account for

past sensory experience that may affect current perception of the task even within a

single session.

1 Introduction

Stiffness sensitivity is crucial for perception and discrimination of objects

and is essential for many complex tasks (Lederman & Klatzky, 2009). Surgeons,

for example, rely on their ability to discriminate between tissue types, locate

tumors embedded within tissues, and identify tissue abnormalities (Howell,

Conatser, Williams, Burns, & Eland, 2008). In teleoperation as well, stiffness

information is essential to facilitate optimal performance (Sherman, Cavusoglu,

& Tendick, 2000). During the examination of an object, the haptic system

acquires tactile information as well as information about arm displacement in

conjunction with signals of applied force (i.e., kinesthetic information; Clark &

Horch, 1986). Yet, information regarding the displacement or deformation of

an object may also be obtained from the visual system, which adds valuable in-

formation over time if afforded (Lederman & Klatzky, 2009). In virtual envi-

ronments (VEs), perception of haptic stiffness was shown to be influenced by

the visual display of object deformation (Srinivasan, Beauregard, & Brock,

1996). The human somatosensory system thus integrates both tactile and kines-

thetic signals, along with visual information, to generate the perception of an

object’s stiffness (Kuschel, Di Luca, Buss, & Klatzky, 2010). However, in

many real-life tasks, there is an inherent uncertainty about the amount of visual

J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I

Presence, Vol. 21, No. 3, Summer 2012, 295–304

ª 2012 by the Massachusetts Institute of Technology *Correspondence to korman.maria@gmail.com.

Korman et al. 295

information that will be available; for example surgery

procedures with possible occlusion of the visual field.

How inputs from the haptic and visual modalities

unite to form an integrated sense of an object is still an

open issue. Past theories have frequently regarded the

haptic modality as inferior to vision in terms of percep-

tual accuracy (e.g., the concept of visual dominance; see

Posner, Nissen, & Klein, 1976; Srinivasan et al., 1996).

However, recent theories of sensory integration suggest

that acquiring information from multiple sensory modal-

ities can produce better performance than from a single

modality (Shams & Seitz, 2008). It has been proposed

by Ernst and Banks (2002) that when redundant multi-

modal information is obtained, the unimodal estimates

are integrated in a statistically optimal fashion. Accord-

ingly, the combined percept is a weighted average of the

unimodal estimates, where weights are inversely propor-

tional to the noise squared of the unisensory estimates.

Therefore, the degree of dominance is determined by

the statistical reliability of the available sensory informa-

tion and the modality with the highest reliability contrib-

utes most to the combined percept. This model implies

that in situations where one source of information (for

example, visual input) is unreliable during multisensory

task performance, perception is based more on the other

modality. Furthermore, Yechiam and Gopher (2008)

showed that the reliance on visual information can be

shaped and changed by training whenever a nonvisual

strategy is advantageous in the long run.

Haptic research is increasingly performed in VE,

wherein manipulation and perception of objects are

commonly mediated by the use of a tool. When using a

probe to explore an object, forces are applied through

the probe. Upon contact with the virtual object, haptic

signals are generated to provide information about the

object’s purely haptic physical properties, such as stiff-

ness, texture, and shape, that may be combined with ar-

bitrary or matched visual and auditory properties. Several

factors that may affect the perception of stiffness were

recently addressed in the literature: contact mode

(Friedman, Hester, Green, & LaMotte, 2008), visual

and force delays (Di Luca, Knorlein, Ernst, & Harders,

2011; Nisky, Baraduc, & Karniel, 2010), angle of view

(Widmer & Hu, 2007), and sensory nature of stimuli

(Kuschel et al., 2010; Hecht & Reiner, 2009). In VE,

perception of stiffness is mediated via a rigid probe; an

experiment by LaMotte (2000) examined the abilities of

humans to discriminate the stiffness of real rubber

objects with a probe with different types of sensory infor-

mation available and methods of palpation. The results

indicated that for a given type of active palpation (tap-

ping or pressing), differences in stiffness were equally

perceived either when a stylus was used or when direct

contact with a finger pad was applied. It was concluded

that subjects discriminate just as well by contacting com-

pliant objects with a tool as with a finger pad.

The role of multimodal stimuli presentation and the

possibilities of enhancing and training stiffness percep-

tion are the focus of several studies, suggesting that

alterations in visual, auditory, and haptic feedback dur-

ing haptic task performance can influence perceived stiff-

ness (Stein & Meredith, 1993; Klatzky, Lederman, &

Matula, 1993; Wu, Basdogan, & Srinivasan, 1999;

Harris, Arabzadeh, Moore, & Clifford, 2007; Stein &

Meredith, 1993; Tan, Durlach, Beauregard, & Srinivasan,

1995). For example, Wu and colleagues showed that

when only haptic feedback was provided, subjects who

manually explored a displayed slot with a PHANToM

device judged compliant objects that were placed further

away as being softer than those presented closer. The

addition of visual-perspective feedback reduced this bias,

showing the advantage to represent virtual stimuli bimo-

dally. Similarly, in the visual-auditory perceptual domain,

Shams and Seitz (2008) argue that multisensory training

protocols (in comparison with unisensory) are more

effective for perceptual learning, as evidenced in recent

audiovisual studies. Other studies have shown that mul-

tisensory experiences influence current unisensory pro-

cessing and memory (Lehmann & Murray, 2005; Mur-

ray et al., 2004); for example, repeated images are better

discriminated if initially presented as audiovisual pairs,

rather than only visually. While these studies focused on

audiovisual multisensory integration, they raise the pos-

sibility of memory interactions in other combinations of

sensory stimuli. However, we are unaware of previous

studies concerning practice-induced changes in the con-

text of crossmodal interaction in the domain of stiffness

perception. In the current study, we examined the inter-

J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I

296 PRESENCE: VOLUME 21, NUMBER 3

action between modality effects and practice effects using

a stiffness discrimination task in VE, which enables

manipulating sensory feedback easily and accurately.

In a recent project (Bouchigny, Hoffmann, Megard,

Gosselin, & Korman, 2012), we developed a surgical

training platform for maxillofacial surgery (MFS). This

surgery requires highly controlled tool interactions with

tissues of different stiffness, primarily the jaw bones, with

a major reliance on haptics over visual information.

These tool–tissue interactions are of a complex multisen-

sory nature, combining tactile, kinesthetic, and visual

input: Changes in bone structure should be rapidly eval-

uated to detect the transition between bone layers fea-

turing different stiffness. Current research aimed to

resolve, for a specific range of stiffness values (captured

during cadaver head surgery, not reported here) that

need to be discriminated during MFS, how the perceiver

integrates stiffness-related information based on prior

experience; and more specifically, to understand how ini-

tial unimodal or bimodal experience affects subsequent

performance in similar or different sensory conditions

within a given session.

2 Method

2.1 Participants and Procedure

This project was reviewed and approved by the

Institutional Review Board for research involving human

subjects of the Technion–Israel Institute of Technology.

A total of 48 Technion students participated in the

study. Prior to the commencement of testing, all partici-

pants were provided with an information sheet and a

consent form was signed. Subjects were screened for

handedness and general health using two questionnaires:

the Edinburgh Handedness Inventory (Oldfield, 1971)

and the General Health & Habits Questionnaire, to

exclude left-handers and people with medical conditions

that could affect motor and sensory perception. Medical

students and other professionals specifically trained to

palpate objects were excluded from this study. All sub-

jects were paid for participation, and additional bonuses

were granted to the subjects who received the highest

20% scores, in order to maintain high motivation

through the experimental session. Participants were

asked to read the experimental instructions, followed by

additional verbal directions by the experimenter. A short

pre-experimental session composed of eight practice tri-

als was provided at the beginning, in order to promote

good understanding of both the experimental task and

the operation of the setup. The entire experimental ses-

sion lasted approximately 1 hr per subject.

2.2 Setup

The experiments were conducted using a VE

touch-enabled computer interface capable of providing

users with visual and haptic stimuli differing in the stiff-

ness-intensities presented (see F1Figure 1). The apparatus

included a computer, monitor, 3D eyeglasses, mouse,

and the PHANToM Desktop–a penlike stylus arm

gripped and moved as in handwriting or drawing.1

2.3 Task and Experimental Design

The experimental task was a two alternative forced

choice (2AFC) discrimination task (Gescheider, 1997).

The right, dominant hand held the stylus for probing

the targets, while the left, nondominant hand was

placed on the mouse to indicate the subject’s answer to

the task. The targets were programmed using Open

Haptic and Open GL software, utilizing a static haptic

J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I

1. Full technical descriptions of this virtual haptic system are avail-

able at http://www.reachin.se and http://www.sensable.com

Figure 1. Experimental setup. The computer screen with two targets,

PHANToM, 3D glasses, and a mouse.

Korman et al. 297

model (i.e., manipulating the stiffness parameter of a vir-

tual spring beneath the solid nondeformable target

square and providing the force feedback and the visual

feedback as a change in the size of the target linearly

proportional to the force applied). For each trial, two

targets were presented on the screen in the form of red

square plates. Across all conditions, participants had to

probe the targets with the stylus and determine which

was softer. The force applied by the subject in the per-

pendicular direction to the target surface, shown from a

side view, was used to calculate the force feedback and

visual change in target size (i.e., the z direction in the

setup coordinate system), according to Newton’s Third

Law. The target shift (dz) was proportional to the force

applied by the subject, dz ¼ F/k, where k is stiffness.

The visual change in target size (in the [x,y] plane)

depended on this shift according to the laws of projec-

tive geometry of the VE. The visual and haptic move-

ments were aligned and both were perpendicular to the

screen plane.

The psychophysical function for discriminating com-

pliance was assessed by the method of constant stimuli.

Each discrimination pair was composed of one standard

stiffness value of 0.25 N/mm, and one comparison value

out of 11 possible. The range of values was chosen in a

preliminary field study (not reported here), to reflect the

real stiffness values that should be discriminated by sur-

geons during jaw surgery (Bouchigny et al., 2012). The

comparison values were chosen to create a range of dis-

crimination scores, starting from roughly chance level

and up to near-perfect discrimination. The values were

as follows: 0.256; 0.263; 0.270; 0.278; 0.286; 0.294;

0.312; 0.333; 0.357; 0.384; 0.417 N/mm, correspond-

ing to fractional stiffness differences (FSD)—the size of

the difference in stimuli intensity related to baseline

stimulus intensity—of 0.03, 0.05, 0.08, 0.11, 0.14,

0.18, 0.25, 0.33, 0.43, 0.54, and 0.67. These values

were chosen such that the intervals between the compar-

ison values close to the baseline will be smaller than

intervals for the far comparison values. The reason why

the standard stimuli were not centered in the middle of

the comparison stimuli is that, due to the large number

of comparison stimuli, we assumed that the sigmoid

functions fitted to the data were symmetric. The partici-

pants were unaware that the same standard stimulus was

presented in each trial. The location of standard and

comparison targets (left or right) was randomized and

counterbalanced.

Two types of sensory stimuli were used in the study:

Haptic (H), and visuohaptic (VH). For the haptic stim-

uli, during target probing, only the haptic property of

stiffness was presented (with a fixed, nonmatched visual

component, in which the target square did not change

its size and the stylus disappeared after contact with the

target square to exclude visual feedback from stylus

movement). For the visuohaptic stimuli, during target

probing, matched (congruent in terms of visual target

size and force-feedback) haptic and visual changes

appeared for each of the targets under the application of

force. All subjects performed the discrimination task in

two consecutive blocks, separated by 5 min of rest. Each

block included 11 paired comparisons with the standard

stiffness value; for each comparison, 10 repetitions were

performed, altogether 110 trials per block and 220 trials

per subject. An experimental session lasted approxi-

mately 45 min in total.

To address the question of how the primary unimodal

experience affects later stiffness discrimination in the

bimodal condition, we tested subjects first in a haptic-

only block that was followed by a visuohaptic second test

block (experimental group H1-VH2). Pure practice

effects were assessed in control group H1-H2 (haptic

first, haptic second), that experienced the same sensory

condition in both blocks. Twenty-four participants were

tested in each group, one excluded (see below). In the

complementary experimental groups, the effect of pri-

mary bi-modal experience in later stiffness discrimination

in either unimodal or bimodal condition was assessed:

subjects were tested first in a visual-haptic block that was

followed by either a haptic-only block (group VH1-H2)

or a visual-haptic block (group VH1-VH2), that experi-

enced the same sensory condition in both blocks. Twelve

participants were tested in each group.

2.4 Analysis

The results are first shown as a function of the pro-

portion of correctly discriminated pairs for each compari-

J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I

298 PRESENCE: VOLUME 21, NUMBER 3

son pair. The results are averaged across subjects with

respect to conditions. For statistical analysis, individual

psychometric functions were derived from the responses

of the subjects for each block. We derived the psycho-

metric function, relating a subject’s performance to an

independent variable, quantifying the physical property

of a stimulus, by estimating the subject’s probability to

answer ‘‘stimulus is softer’’ as a function of the actual dif-

ference DK ¼ Kstim � Kstd, where Kstim is the stiffness

of the stimulus target, Kstd is the stiffness of the refer-

ence target, N(DK) is total number of trials per compari-

son stiffness, and A(n) is a binary representation of the

answer.

PðDK Þ ¼PN ðDKÞ

n¼1 AðnÞN ðDK Þ AðnÞ 1; stimulus is softer;

0; standard is softer:

�

Psychophysical functions were fitted with a cumulative

Gaussian function using ‘‘psignifit’’ software, version

2.5.6 for MATLAB (Wichmann & Hill, 2001), and

l values (mean perceptual estimates at 75%) were

calculated.

One subject in the H1-VH2 group was excluded from

the analysis due to a l value that was higher than two SD

of the group mean and abnormal psychophysical func-

tion shape. Repeated-measures analysis of variance

(ANOVA) followed by post hoc tests on the l values

(4 � 2 design; group, block) was performed. The null

hypothesis was rejected at the level of 0.05.

3 Results

Discriminating between two stimulus pairs was

found to produce psychophysical functions of the

expected shape. The raw behavioral data are depicted in

F2 Figure 2 in terms of performance measures of mean pro-

portion of correctly discriminated pairs (PC) and stand-

ard error (SE) of the means calculated for each compari-

son value over all subjects.

It is apparent that the pattern of performance in the

first and the second test blocks is qualitatively different

in the group H1-VH2 as compared to the other groups,

suggesting that prior training in a unisensory haptic stiff-

ness discrimination task boosted the second block per-

formance when visual feedback was subsequently pro-

vided, both in terms of the mean PC and the variance of

the responses. In order to quantitatively evaluate this hy-

pothesis, we used the calculated l values to test whether

there was a shift to the left in the psychometric curves in

the second performance block, a sign of a between-block

improvement in discrimination ability, and whether

there are special significant interactions for the group

H1-VH2.

The psychometric curves fitted to the performance

of subjects revealed a quantitative difference in the dis-

crimination ability at the second performance block for

the group H1-VH2 (for individual examples from each

group, see F3Figure 3). A repeated-measures ANOVA

was performed on the mean perceptual estimates

(threshold at 75%); the l values, obtained from the

individual fit of subjects’ responses, revealed a signifi-

cant block effect, F(1, 67) ¼ 7.76, p < .01, suggesting

that there was a significant change in the discrimination

ability from Block 1 to Block 2 (see F4Figure 4[a]). No

significant differences in the l values at the first block

were found, suggesting that there was no initial

advantage for bimodal performance. Post hoc compari-

sons further clarified that the only group significantly

contributing to the main block effect is the H1-VH2

group (post hoc, p ¼ .015, Block 1 to Block 2).

Individual differences in thresholds (Dl) between

Block 2 and Block 1 were calculated for each experimen-

tal group and tested using a single sample t-test (see

Figure 4[b]). The only group that was found to have

mean Dl significantly different from zero was again the

H1-VH2 group (t ¼ –5.35, p < .01). Control groups

VH1-VH2 and H1-H2 showed a trend for psychometric

curve shift to the left, but did not reach significance due

to the high between-subject and within-subject variance

in performance.

Altogether, the analysis of psychometric curves sug-

gests that sensory conditions and their order of presenta-

tion had robust selective effect on the discrimination

ability at the second block: second block performance in

the group H1-VH2 was boosted as compared to second

block performance in all three other groups, VH1-VH2,

VH1-VH2, and H1-H2, as reflected in the significant

shift of the psychometric curve to the left.

J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I

Korman et al. 299

J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I

Figure 2. Upper panels: Group-mean percent-correct discrimination rates in first and second performance blocks—effects of practice

and sensory condition. Lower panels: SE of the mean in first and second performance blocks. Mean percent correct value 6 SE in Block 1

versus Block 2 are shown. Abscissa points correspond to 11 levels of comparisons stiffness values, from the most difficult to the easiest.

(a) Cross-modal group H1-VH2 and (b) control group H1-H2 (n ¼ 24 per group), (c) Cross-modal group VH1-H2 and (d) control group

VH1-VH2 (n ¼ 12 per group).

300 PRESENCE: VOLUME 21, NUMBER 3

4 Discussion

Our study addressed the question, how can the

within-session change in the affordance of visual infor-

mation affect the perception of stiffness? We found that

control groups with the same sensory experience during

Block 1 and Block 2 (VH-VH, H-H) showed a trend to

improvement in performance as reflected in a decrease in

the perceptual thresholds across the two blocks. How-

ever, this effect did not reach significance. Surprisingly,

no deterioration in discrimination ability was obtained

for switching from bimodal experience to unimodal ex-

perience in the group VH1-H2, although the amount of

sensory information in the stimuli was reduced across

the blocks. Possible interpretation of this finding is that

participants of this group were actually not perceptually

affected by the presence of visual information in Block 1,

causing no significant crossmodal effects on the Block 2

performance. This finding is in line with recent studies

by Kuschel et al. (2010), which showed that basic haptic

and visuohaptic stiffness discrimination estimates had

nearly identical psychophysical curves and variability.

However, it challenges recent notions on multisensory

training benefits on later unisensory performance,

reported for the visuoauditory domain (Shams & Seitz,

2008).

The most intriguing finding of our study was the

profound enhancement in discrimination ability when

J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I

Figure 3. Psychometric curves of typical subjects: The curves are fitted to trials of the first and the second block: (a) H1-VH2, (b) H1-H2,

(c) VH1-H2, and (d) VH1-VH2. Black line, Block 1; dotted line, Block 2.

Korman et al. 301

participants switched from unihaptic performance to

bimodal performance (group H1-VH2). The gain of this

group could not be attributed to practice or modality

effects, as it did not reach significance in the control

groups. We interpret the boosted performance in the

VH2 condition as a within-session order effect, reflecting

interaction between unimodal experience at the first

block and the subsequent bimodal experience. Thus, the

amount and type of sensory information, which are the

traditional focus of multimodal integration studies, are

not the only critical factors affecting stiffness discrimina-

tion ability, as even a within-session manipulation of

order of experience may have a robust effect on the way

that visual and haptic sensory cues fuse. How current

integration theories of multisensory perception (e.g.,

Ernst & Banks, 2002; Kuschel et al., 2010) should

account for differential task representation, as a function

of order of experience, is a novel and challenging ques-

tion. Standard psychometric function analysis that relates

response probability to stimulus intensity assumes that

responses are dependent only on the intensity of a pre-

sented stimulus and not on other factors such as stimulus

sequence, duration of the experiment, or the responses

on previous trials. Factors such as learning contradict this

basic assumption (for recent discussion on this topic see

Frund, Haenel, & Wichmann, 2011). Thus, credible

intervals for parameters of the psychometric function can

be underestimated or overestimated, depending on the

changes in the variance of the observers’ responses with

the learning process. This limitation may lead to a

reduced ability to capture the within-session learning

qualitatively observable in the original data, e.g., practice

effects in the control groups VH-VH and H-H.

How does unisensory experience benefit later multi-

sensory performance? An answer may involve an altera-

tion in the weights of the perceptual estimates during

the course of a single session experience, in other words,

in the representation of the task. Perceptual brain plastic-

ity that underlies perceptual learning is based on the abil-

ity of primary and high sensory cortices to be function-

ally rewired upon repetitive experience to develop new

representations. The interaction found in our study

implies that memory processes may play a significant role

in developing these representations. One such process is

priming, a memory phenomenon that increases the effi-

ciency of processing repeated stimuli (Tulving &

Schacter, 1990) and is thought to be caused by spread-

ing activation neuronal mechanisms (Reisberg, 2010).

The direct connections between primary sensory areas

that are activated during the tactile and visual task sub-

serve the phenomenon of cross-modal priming (Falchier,

Clavagnier, Barone, & Kennedy, 2002). For instance, an

imaging study by James and colleagues (2002) has

shown that previous haptic experience with three-

dimensional haptic objects enhanced activation in visual

areas when these same objects were subsequently viewed.

J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I

Figure 4. Group-mean threshold l 6 SE in (a) first (triangles) and second (circles) performance blocks and (b) group-mean Dl 6 SE

in four experimental groups: VH1-H2, H1-VH2, VH1-VH2, and H1-H2.

**p < .05.

302 PRESENCE: VOLUME 21, NUMBER 3

Another study of 3D object haptic recognition with

blindfolded participants who named everyday objects

presented haptically (Craddock & Lawson, 2008)

showed that haptic priming was greater than visual pri-

ming and crossmodal priming. In line with these studies

and recent notions in cognitive and neural correlates of

tactile memory (Gallace & Spence, 2009), our findings

suggest that early presentation of unisensory haptic in-

formation may help to retrieve the visual properties of

the objects presented visuohaptically. In future studies

it would be interesting to test whether incongruent

univisual stiffness discrimination (with constant haptic

feedback) will also be differentially affected by preceding

experience, either unimodal or bimodal. Current inter-

pretation would imply that haptic experience, either

unimodal or bimodal, in the first block would negatively

affect the subsequent unimodal visual stiffness discrimi-

nation, but not vice versa.

Understanding the effect of order of experience on

perception is expected to promote surgery training and

rehabilitation protocols in VE, where possibilities of

combinations between modalities and order of presenta-

tion in a simulated task can be easily manipulated. The

fact that the order of probing along with the modalities

that are involved (with or without vision) can signifi-

cantly affect the perception of stiffness must be consid-

ered in building the successful training platforms for

tasks relying on stiffness discrimination. In a recent pa-

per, Tsuda, Scott, Doyle, and Ones (2009) asserted that

training complex manual skills such as surgery in VE sim-

ulators implies that simulator-based training needs to be

carefully structured and that sensory feedback is one of

the key factors throughout the development of a skill.

Sensory feedback can be provided at different points in

the learning process and in a variety of ways in order for

trainees to achieve maximum benefit. The current study

adds to this assertion by refining the particular condi-

tions that may support such improvement with respect

to the use of haptic and visual modalities in stiffness-

discrimination–based tasks throughout the course of

practice. More specifically, our results suggest that in

order to benefit from the addition of visual cues in stiff-

ness-discrimination–related tasks, it is important to first

become familiar with the purely haptic aspect of the task.

Acknowledgments

The authors would like to thank Tami Gelfeld and Michael

Sambur for software development.

This research was supported by the European Commission

Integrated Project IP-SKILLS-35005.

References

Bouchigny, S., Hoffmann, P., Megard, C., Gosselin, F., &

Korman, M. (2012, in press). Designing a VR training plat-

form for surgeons: Theoretical framework, technological sol-

utions and results. In M. Bergamasco, B. G. Bardy, & D.

Gopher (Eds.), Skills training in multimodal virtual envi-

ronments. Oxford, UK: Taylor & Francis.

Clark, F. J., & Horch, K. W. (1986). Kinesthesia. In K. Boff,

L. Kaufman, & J. Thomas (Eds.), Handbook of perception

and human performance. New York: Wiley.

Craddock, M., & Lawson, R. (2008). Repetition priming and

the haptic recognition of familiar and unfamiliar objects.

Perception & Psychophysics, 70(7), 1350–1365.

Di Luca, M., Knorlein, B., Ernst, M. O., & Harders, M.

(2011). Effects of visual-haptic asynchronies and loading-

unloading movements on compliance perception. Brain

Research Bulletin, 85(5), 245–259.

Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual

and haptic information in a statistically optimal fashion.

Nature, 415, 429–433.

Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H.

(2002). Anatomical evidence of multimodal integration in

primate striate cortex. Journal of Neuroscience, 22(13),

5749–5759.

Friedman, R. M., Hester, K. D., Green, B. G., & LaMotte, R.

H. (2008). Magnitude estimation of softness. Experimental

Brain Research, 191(2), 133–142.

Frund, I., Haenel, N. V., & Wichmann, F. A. (2011).

Inference for psychometric functions in the presence of

nonstationary behavior. Journal of Vision, 11(6).

doi:10.1167/11.6.16

Gallace, A., & Spence, C. (2009). The cognitive and neural

correlates of tactile memory. Psychological Bulletin, 135(3),

380–406.

Gescheider, G. A. (1997). Psychophysics: The fundamentals (3rd

ed.). Mahwah, NJ: Lawrence Erlbaum Associates.

Harris, J. A., Arabzadeh, E., Moore, C. A., & Clifford, C. W.

(2007). Non-informative vision causes adaptive changes in

J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I

Korman et al. 303

tactile sensitivity. Journal of Neuroscience, 27(27), 7136–

7140.

Hecht, D., & Reiner, M. (2009). Sensory dominance in com-

binations of audio, visual and haptic stimuli. Experimental

Brain Research, 193(2), 307–314.

Howell, J. N., Conatser, R. R., Williams, R. L., Burns, J. M., &

Eland, D. C. (2008). Palpatory diagnosis training on the vir-

tual haptic back: Performance improvement and user evalua-

tions. Journal of the American Osteopathic Association,

108(1), 29–36.

James, T. W., Humphrey, G. K., Gati, J. S., Servos, P., Menon,

R. S., & Goodale, M. A. (2002). Haptic study of three-

dimensional objects activates extrastriate visual areas. Neuro-

psychologia, 40(10), 1706–1714.

Klatzky, R. L., Lederman, S. J., & Matula, D. E. (1993). Hap-

tic exploration in the presence of vision. Journal of Experi-

mental Psychology: Human Perception and Performance,

19(4), 726–743.

Kuschel, M., Di Luca, M., Buss, M., & Klatzky, R. L. (2010).

Combination and integration in the perception of visual-

haptic compliance information. IEEE Transactions on Hap-

tics, 3(4), 234–244.

LaMotte, R. H. (2000). Softness discrimination with a tool.

Journal of Neurophysiology, 83(4), 1777–1786.

Lederman, S. J., & Klatzky, R. L. (2009). Haptic perception:

A tutorial. Attention, Perception, & Psychophysics, 71(7),

1439–1459.

Lehmann, S., & Murray, M. M. (2005). The role of multisen-

sory memories in unisensory object discrimination. Cognitive

Brain Research, 24(2), 326–334.

Murray, M. M., Michel, C. M., Grave de Peralta, R., Ortigue,

S., Brunet, D., Andino, S. G., et al. (2004). Rapid discrimi-

nation of visual and multisensory memories revealed by elec-

trical neuroimaging. NeuroImage, 21(1), 125–135.

Nisky, I., Baraduc, P., & Karniel, A. (2010). Proximodistal gra-

dient in the perception of delayed stiffness. Journal of Neuro-

physiology, 103(6), 3017–3026.

Oldfield, R. C. (1971). The assessment and analysis of handed-

ness: The Edinburgh Inventory. Neuropsychologia, 9(1),

97–113.

Posner, M. I., Nissen, M. J., & Klein, R. M. (1976). Visual

dominance: An information-processing account of its origins

and significance. Psychological Review, 83(2), 157–171.

Reisberg, D. (2010). Cognition: Exploring the science of the

mind (4th ed., pp. 255–517). New York: Norton.

Schacter, D. L. (1995). Implicit memory: A new frontier for

cognitive neuroscience. In M. Gazzaniga (Ed.), The

cognitive neurosciences (pp. 815–824). Cambridge, MA:

MIT Press.

Shams, L., & Seitz, A. R. (2008). Benefits of multisensory

learning. Trends in Cognitive Sciences, 12(11), 411–417.

Sherman, A., Cavusoglu, M. C., & Tendick, F. (2000). Com-

parison of teleoperator control architectures for palpation

task. Proceedings of the ASME Dynamic Systems and Control

Division (Haptic Symposium), 69(2), 1261–1268.

Stein, B. E., & Meredith, M. A. (1993). The merging of the

senses. Cambridge, MA: MIT Press.

Srinivasan, M. A., Beauregard, G. L., & Brock, D. L. (1996).

The impact of visual information on the haptic perception of

stiffness in virtual environments. Proceedings of the ASME

Dynamic Systems and Control Division, 58, 555–559.

Tan, H. Z., Durlach, N. I., Beauregard, G. L., & Srinivasan,

M. A. (1995). Manual discrimination of compliance using

active pinch grasp: The roles of force and work cues. Percep-

tion and Psychophysics, 57(4), 495–510.

Tsuda, S., Scott, D., Doyle, J., & Ones, D. B. (2009). Surgical

skills training and simulation. Current Problems in Surgery,

46(4), 271–370.

Tulving, E., & Schacter, D. L. (1990). Priming and human

memory systems. Science, 155, 301–310.

Widmer, A., & Hu, Y. (2007). The role of viewing angle in

integrating the senses of vision and touch for perception of

object softness. Canadian Journal of Electrical and Com-

puter Engineering, 32(4), 193–198.

Wichmann, F., & Hill, N. (2001). The psychometric function:

I. Fitting, sampling, and goodness of fit. Perception and Psy-

chophysics, 63(8), 1293–1313.

Wu, W. C., Basdogan, C., & Srinivasan, M. (1999). Visual,

haptic and bimodal perception of size and stiffness in virtual

environments. Proceedings of the ASME Dynamic Systems and

Control Division, 67, 19–26.

Yechiam, E., & Gopher, D. (2008). Deliberate suppression of

vision as a training tool for multimodal skills. In C. A. Aviz-

zano (Ed.), SKILLS: Beyond movement: The history and

future of gesture analysis (pp. 80–103). Copenhagen, Den-

mark: Alinea Publishing House.

J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I

304 PRESENCE: VOLUME 21, NUMBER 3