Maria Korman*
Occupational Therapy Department
The Faculty of Social Welfare and
Health Sciences
University of Haifa
Mount Carmel, Haifa, Israel, 31905
Kinneret Teodorescu
Adi Cohen
Miriam Reiner
Daniel Gopher
Technion
Israel Institute of Technology, Israel
Effects of Order and SensoryModality in Stiffness Perception
Abstract
The stiffness properties of an environment are perceived during active manual manipu-
lation primarily by processing force cues and position-based tactile, kinesthetic, and vis-
ual information. Using a two alternative forced choice (2AFC) stiffness discrimination
task, we tested how the perceiver integrates stiffness-related information based on
sensory feedback from one or two modalities and the origins of within-session shifts in
stiffness discrimination ability. Two factors were investigated: practice and the amount
of available sensory information. Subjects discriminated between the stiffness of two
targets that were presented either haptically or visuohaptically in two subsequent
blocks. Our results show that prior experience in a unisensory haptic stiffness discrimi-
nation block greatly improved performance when visual feedback was subsequently
provided along with haptic feedback. This improvement could not be attributed to
effects induced by practice or multisensory stimulus presentation. Our findings suggest
that optimization integration theories of multisensory perception need to account for
past sensory experience that may affect current perception of the task even within a
single session.
1 Introduction
Stiffness sensitivity is crucial for perception and discrimination of objects
and is essential for many complex tasks (Lederman & Klatzky, 2009). Surgeons,
for example, rely on their ability to discriminate between tissue types, locate
tumors embedded within tissues, and identify tissue abnormalities (Howell,
Conatser, Williams, Burns, & Eland, 2008). In teleoperation as well, stiffness
information is essential to facilitate optimal performance (Sherman, Cavusoglu,
& Tendick, 2000). During the examination of an object, the haptic system
acquires tactile information as well as information about arm displacement in
conjunction with signals of applied force (i.e., kinesthetic information; Clark &
Horch, 1986). Yet, information regarding the displacement or deformation of
an object may also be obtained from the visual system, which adds valuable in-
formation over time if afforded (Lederman & Klatzky, 2009). In virtual envi-
ronments (VEs), perception of haptic stiffness was shown to be influenced by
the visual display of object deformation (Srinivasan, Beauregard, & Brock,
1996). The human somatosensory system thus integrates both tactile and kines-
thetic signals, along with visual information, to generate the perception of an
object’s stiffness (Kuschel, Di Luca, Buss, & Klatzky, 2010). However, in
many real-life tasks, there is an inherent uncertainty about the amount of visual
J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I
Presence, Vol. 21, No. 3, Summer 2012, 295–304
ª 2012 by the Massachusetts Institute of Technology *Correspondence to [email protected].
Korman et al. 295
information that will be available; for example surgery
procedures with possible occlusion of the visual field.
How inputs from the haptic and visual modalities
unite to form an integrated sense of an object is still an
open issue. Past theories have frequently regarded the
haptic modality as inferior to vision in terms of percep-
tual accuracy (e.g., the concept of visual dominance; see
Posner, Nissen, & Klein, 1976; Srinivasan et al., 1996).
However, recent theories of sensory integration suggest
that acquiring information from multiple sensory modal-
ities can produce better performance than from a single
modality (Shams & Seitz, 2008). It has been proposed
by Ernst and Banks (2002) that when redundant multi-
modal information is obtained, the unimodal estimates
are integrated in a statistically optimal fashion. Accord-
ingly, the combined percept is a weighted average of the
unimodal estimates, where weights are inversely propor-
tional to the noise squared of the unisensory estimates.
Therefore, the degree of dominance is determined by
the statistical reliability of the available sensory informa-
tion and the modality with the highest reliability contrib-
utes most to the combined percept. This model implies
that in situations where one source of information (for
example, visual input) is unreliable during multisensory
task performance, perception is based more on the other
modality. Furthermore, Yechiam and Gopher (2008)
showed that the reliance on visual information can be
shaped and changed by training whenever a nonvisual
strategy is advantageous in the long run.
Haptic research is increasingly performed in VE,
wherein manipulation and perception of objects are
commonly mediated by the use of a tool. When using a
probe to explore an object, forces are applied through
the probe. Upon contact with the virtual object, haptic
signals are generated to provide information about the
object’s purely haptic physical properties, such as stiff-
ness, texture, and shape, that may be combined with ar-
bitrary or matched visual and auditory properties. Several
factors that may affect the perception of stiffness were
recently addressed in the literature: contact mode
(Friedman, Hester, Green, & LaMotte, 2008), visual
and force delays (Di Luca, Knorlein, Ernst, & Harders,
2011; Nisky, Baraduc, & Karniel, 2010), angle of view
(Widmer & Hu, 2007), and sensory nature of stimuli
(Kuschel et al., 2010; Hecht & Reiner, 2009). In VE,
perception of stiffness is mediated via a rigid probe; an
experiment by LaMotte (2000) examined the abilities of
humans to discriminate the stiffness of real rubber
objects with a probe with different types of sensory infor-
mation available and methods of palpation. The results
indicated that for a given type of active palpation (tap-
ping or pressing), differences in stiffness were equally
perceived either when a stylus was used or when direct
contact with a finger pad was applied. It was concluded
that subjects discriminate just as well by contacting com-
pliant objects with a tool as with a finger pad.
The role of multimodal stimuli presentation and the
possibilities of enhancing and training stiffness percep-
tion are the focus of several studies, suggesting that
alterations in visual, auditory, and haptic feedback dur-
ing haptic task performance can influence perceived stiff-
ness (Stein & Meredith, 1993; Klatzky, Lederman, &
Matula, 1993; Wu, Basdogan, & Srinivasan, 1999;
Harris, Arabzadeh, Moore, & Clifford, 2007; Stein &
Meredith, 1993; Tan, Durlach, Beauregard, & Srinivasan,
1995). For example, Wu and colleagues showed that
when only haptic feedback was provided, subjects who
manually explored a displayed slot with a PHANToM
device judged compliant objects that were placed further
away as being softer than those presented closer. The
addition of visual-perspective feedback reduced this bias,
showing the advantage to represent virtual stimuli bimo-
dally. Similarly, in the visual-auditory perceptual domain,
Shams and Seitz (2008) argue that multisensory training
protocols (in comparison with unisensory) are more
effective for perceptual learning, as evidenced in recent
audiovisual studies. Other studies have shown that mul-
tisensory experiences influence current unisensory pro-
cessing and memory (Lehmann & Murray, 2005; Mur-
ray et al., 2004); for example, repeated images are better
discriminated if initially presented as audiovisual pairs,
rather than only visually. While these studies focused on
audiovisual multisensory integration, they raise the pos-
sibility of memory interactions in other combinations of
sensory stimuli. However, we are unaware of previous
studies concerning practice-induced changes in the con-
text of crossmodal interaction in the domain of stiffness
perception. In the current study, we examined the inter-
J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I
296 PRESENCE: VOLUME 21, NUMBER 3
action between modality effects and practice effects using
a stiffness discrimination task in VE, which enables
manipulating sensory feedback easily and accurately.
In a recent project (Bouchigny, Hoffmann, Megard,
Gosselin, & Korman, 2012), we developed a surgical
training platform for maxillofacial surgery (MFS). This
surgery requires highly controlled tool interactions with
tissues of different stiffness, primarily the jaw bones, with
a major reliance on haptics over visual information.
These tool–tissue interactions are of a complex multisen-
sory nature, combining tactile, kinesthetic, and visual
input: Changes in bone structure should be rapidly eval-
uated to detect the transition between bone layers fea-
turing different stiffness. Current research aimed to
resolve, for a specific range of stiffness values (captured
during cadaver head surgery, not reported here) that
need to be discriminated during MFS, how the perceiver
integrates stiffness-related information based on prior
experience; and more specifically, to understand how ini-
tial unimodal or bimodal experience affects subsequent
performance in similar or different sensory conditions
within a given session.
2 Method
2.1 Participants and Procedure
This project was reviewed and approved by the
Institutional Review Board for research involving human
subjects of the Technion–Israel Institute of Technology.
A total of 48 Technion students participated in the
study. Prior to the commencement of testing, all partici-
pants were provided with an information sheet and a
consent form was signed. Subjects were screened for
handedness and general health using two questionnaires:
the Edinburgh Handedness Inventory (Oldfield, 1971)
and the General Health & Habits Questionnaire, to
exclude left-handers and people with medical conditions
that could affect motor and sensory perception. Medical
students and other professionals specifically trained to
palpate objects were excluded from this study. All sub-
jects were paid for participation, and additional bonuses
were granted to the subjects who received the highest
20% scores, in order to maintain high motivation
through the experimental session. Participants were
asked to read the experimental instructions, followed by
additional verbal directions by the experimenter. A short
pre-experimental session composed of eight practice tri-
als was provided at the beginning, in order to promote
good understanding of both the experimental task and
the operation of the setup. The entire experimental ses-
sion lasted approximately 1 hr per subject.
2.2 Setup
The experiments were conducted using a VE
touch-enabled computer interface capable of providing
users with visual and haptic stimuli differing in the stiff-
ness-intensities presented (see F1Figure 1). The apparatus
included a computer, monitor, 3D eyeglasses, mouse,
and the PHANToM Desktop–a penlike stylus arm
gripped and moved as in handwriting or drawing.1
2.3 Task and Experimental Design
The experimental task was a two alternative forced
choice (2AFC) discrimination task (Gescheider, 1997).
The right, dominant hand held the stylus for probing
the targets, while the left, nondominant hand was
placed on the mouse to indicate the subject’s answer to
the task. The targets were programmed using Open
Haptic and Open GL software, utilizing a static haptic
J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I
1. Full technical descriptions of this virtual haptic system are avail-
able at http://www.reachin.se and http://www.sensable.com
Figure 1. Experimental setup. The computer screen with two targets,
PHANToM, 3D glasses, and a mouse.
Korman et al. 297
model (i.e., manipulating the stiffness parameter of a vir-
tual spring beneath the solid nondeformable target
square and providing the force feedback and the visual
feedback as a change in the size of the target linearly
proportional to the force applied). For each trial, two
targets were presented on the screen in the form of red
square plates. Across all conditions, participants had to
probe the targets with the stylus and determine which
was softer. The force applied by the subject in the per-
pendicular direction to the target surface, shown from a
side view, was used to calculate the force feedback and
visual change in target size (i.e., the z direction in the
setup coordinate system), according to Newton’s Third
Law. The target shift (dz) was proportional to the force
applied by the subject, dz ¼ F/k, where k is stiffness.
The visual change in target size (in the [x,y] plane)
depended on this shift according to the laws of projec-
tive geometry of the VE. The visual and haptic move-
ments were aligned and both were perpendicular to the
screen plane.
The psychophysical function for discriminating com-
pliance was assessed by the method of constant stimuli.
Each discrimination pair was composed of one standard
stiffness value of 0.25 N/mm, and one comparison value
out of 11 possible. The range of values was chosen in a
preliminary field study (not reported here), to reflect the
real stiffness values that should be discriminated by sur-
geons during jaw surgery (Bouchigny et al., 2012). The
comparison values were chosen to create a range of dis-
crimination scores, starting from roughly chance level
and up to near-perfect discrimination. The values were
as follows: 0.256; 0.263; 0.270; 0.278; 0.286; 0.294;
0.312; 0.333; 0.357; 0.384; 0.417 N/mm, correspond-
ing to fractional stiffness differences (FSD)—the size of
the difference in stimuli intensity related to baseline
stimulus intensity—of 0.03, 0.05, 0.08, 0.11, 0.14,
0.18, 0.25, 0.33, 0.43, 0.54, and 0.67. These values
were chosen such that the intervals between the compar-
ison values close to the baseline will be smaller than
intervals for the far comparison values. The reason why
the standard stimuli were not centered in the middle of
the comparison stimuli is that, due to the large number
of comparison stimuli, we assumed that the sigmoid
functions fitted to the data were symmetric. The partici-
pants were unaware that the same standard stimulus was
presented in each trial. The location of standard and
comparison targets (left or right) was randomized and
counterbalanced.
Two types of sensory stimuli were used in the study:
Haptic (H), and visuohaptic (VH). For the haptic stim-
uli, during target probing, only the haptic property of
stiffness was presented (with a fixed, nonmatched visual
component, in which the target square did not change
its size and the stylus disappeared after contact with the
target square to exclude visual feedback from stylus
movement). For the visuohaptic stimuli, during target
probing, matched (congruent in terms of visual target
size and force-feedback) haptic and visual changes
appeared for each of the targets under the application of
force. All subjects performed the discrimination task in
two consecutive blocks, separated by 5 min of rest. Each
block included 11 paired comparisons with the standard
stiffness value; for each comparison, 10 repetitions were
performed, altogether 110 trials per block and 220 trials
per subject. An experimental session lasted approxi-
mately 45 min in total.
To address the question of how the primary unimodal
experience affects later stiffness discrimination in the
bimodal condition, we tested subjects first in a haptic-
only block that was followed by a visuohaptic second test
block (experimental group H1-VH2). Pure practice
effects were assessed in control group H1-H2 (haptic
first, haptic second), that experienced the same sensory
condition in both blocks. Twenty-four participants were
tested in each group, one excluded (see below). In the
complementary experimental groups, the effect of pri-
mary bi-modal experience in later stiffness discrimination
in either unimodal or bimodal condition was assessed:
subjects were tested first in a visual-haptic block that was
followed by either a haptic-only block (group VH1-H2)
or a visual-haptic block (group VH1-VH2), that experi-
enced the same sensory condition in both blocks. Twelve
participants were tested in each group.
2.4 Analysis
The results are first shown as a function of the pro-
portion of correctly discriminated pairs for each compari-
J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I
298 PRESENCE: VOLUME 21, NUMBER 3
son pair. The results are averaged across subjects with
respect to conditions. For statistical analysis, individual
psychometric functions were derived from the responses
of the subjects for each block. We derived the psycho-
metric function, relating a subject’s performance to an
independent variable, quantifying the physical property
of a stimulus, by estimating the subject’s probability to
answer ‘‘stimulus is softer’’ as a function of the actual dif-
ference DK ¼ Kstim � Kstd, where Kstim is the stiffness
of the stimulus target, Kstd is the stiffness of the refer-
ence target, N(DK) is total number of trials per compari-
son stiffness, and A(n) is a binary representation of the
answer.
PðDK Þ ¼PN ðDKÞ
n¼1 AðnÞN ðDK Þ AðnÞ 1; stimulus is softer;
0; standard is softer:
�
Psychophysical functions were fitted with a cumulative
Gaussian function using ‘‘psignifit’’ software, version
2.5.6 for MATLAB (Wichmann & Hill, 2001), and
l values (mean perceptual estimates at 75%) were
calculated.
One subject in the H1-VH2 group was excluded from
the analysis due to a l value that was higher than two SD
of the group mean and abnormal psychophysical func-
tion shape. Repeated-measures analysis of variance
(ANOVA) followed by post hoc tests on the l values
(4 � 2 design; group, block) was performed. The null
hypothesis was rejected at the level of 0.05.
3 Results
Discriminating between two stimulus pairs was
found to produce psychophysical functions of the
expected shape. The raw behavioral data are depicted in
F2 Figure 2 in terms of performance measures of mean pro-
portion of correctly discriminated pairs (PC) and stand-
ard error (SE) of the means calculated for each compari-
son value over all subjects.
It is apparent that the pattern of performance in the
first and the second test blocks is qualitatively different
in the group H1-VH2 as compared to the other groups,
suggesting that prior training in a unisensory haptic stiff-
ness discrimination task boosted the second block per-
formance when visual feedback was subsequently pro-
vided, both in terms of the mean PC and the variance of
the responses. In order to quantitatively evaluate this hy-
pothesis, we used the calculated l values to test whether
there was a shift to the left in the psychometric curves in
the second performance block, a sign of a between-block
improvement in discrimination ability, and whether
there are special significant interactions for the group
H1-VH2.
The psychometric curves fitted to the performance
of subjects revealed a quantitative difference in the dis-
crimination ability at the second performance block for
the group H1-VH2 (for individual examples from each
group, see F3Figure 3). A repeated-measures ANOVA
was performed on the mean perceptual estimates
(threshold at 75%); the l values, obtained from the
individual fit of subjects’ responses, revealed a signifi-
cant block effect, F(1, 67) ¼ 7.76, p < .01, suggesting
that there was a significant change in the discrimination
ability from Block 1 to Block 2 (see F4Figure 4[a]). No
significant differences in the l values at the first block
were found, suggesting that there was no initial
advantage for bimodal performance. Post hoc compari-
sons further clarified that the only group significantly
contributing to the main block effect is the H1-VH2
group (post hoc, p ¼ .015, Block 1 to Block 2).
Individual differences in thresholds (Dl) between
Block 2 and Block 1 were calculated for each experimen-
tal group and tested using a single sample t-test (see
Figure 4[b]). The only group that was found to have
mean Dl significantly different from zero was again the
H1-VH2 group (t ¼ –5.35, p < .01). Control groups
VH1-VH2 and H1-H2 showed a trend for psychometric
curve shift to the left, but did not reach significance due
to the high between-subject and within-subject variance
in performance.
Altogether, the analysis of psychometric curves sug-
gests that sensory conditions and their order of presenta-
tion had robust selective effect on the discrimination
ability at the second block: second block performance in
the group H1-VH2 was boosted as compared to second
block performance in all three other groups, VH1-VH2,
VH1-VH2, and H1-H2, as reflected in the significant
shift of the psychometric curve to the left.
J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I
Korman et al. 299
J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I
Figure 2. Upper panels: Group-mean percent-correct discrimination rates in first and second performance blocks—effects of practice
and sensory condition. Lower panels: SE of the mean in first and second performance blocks. Mean percent correct value 6 SE in Block 1
versus Block 2 are shown. Abscissa points correspond to 11 levels of comparisons stiffness values, from the most difficult to the easiest.
(a) Cross-modal group H1-VH2 and (b) control group H1-H2 (n ¼ 24 per group), (c) Cross-modal group VH1-H2 and (d) control group
VH1-VH2 (n ¼ 12 per group).
300 PRESENCE: VOLUME 21, NUMBER 3
4 Discussion
Our study addressed the question, how can the
within-session change in the affordance of visual infor-
mation affect the perception of stiffness? We found that
control groups with the same sensory experience during
Block 1 and Block 2 (VH-VH, H-H) showed a trend to
improvement in performance as reflected in a decrease in
the perceptual thresholds across the two blocks. How-
ever, this effect did not reach significance. Surprisingly,
no deterioration in discrimination ability was obtained
for switching from bimodal experience to unimodal ex-
perience in the group VH1-H2, although the amount of
sensory information in the stimuli was reduced across
the blocks. Possible interpretation of this finding is that
participants of this group were actually not perceptually
affected by the presence of visual information in Block 1,
causing no significant crossmodal effects on the Block 2
performance. This finding is in line with recent studies
by Kuschel et al. (2010), which showed that basic haptic
and visuohaptic stiffness discrimination estimates had
nearly identical psychophysical curves and variability.
However, it challenges recent notions on multisensory
training benefits on later unisensory performance,
reported for the visuoauditory domain (Shams & Seitz,
2008).
The most intriguing finding of our study was the
profound enhancement in discrimination ability when
J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I
Figure 3. Psychometric curves of typical subjects: The curves are fitted to trials of the first and the second block: (a) H1-VH2, (b) H1-H2,
(c) VH1-H2, and (d) VH1-VH2. Black line, Block 1; dotted line, Block 2.
Korman et al. 301
participants switched from unihaptic performance to
bimodal performance (group H1-VH2). The gain of this
group could not be attributed to practice or modality
effects, as it did not reach significance in the control
groups. We interpret the boosted performance in the
VH2 condition as a within-session order effect, reflecting
interaction between unimodal experience at the first
block and the subsequent bimodal experience. Thus, the
amount and type of sensory information, which are the
traditional focus of multimodal integration studies, are
not the only critical factors affecting stiffness discrimina-
tion ability, as even a within-session manipulation of
order of experience may have a robust effect on the way
that visual and haptic sensory cues fuse. How current
integration theories of multisensory perception (e.g.,
Ernst & Banks, 2002; Kuschel et al., 2010) should
account for differential task representation, as a function
of order of experience, is a novel and challenging ques-
tion. Standard psychometric function analysis that relates
response probability to stimulus intensity assumes that
responses are dependent only on the intensity of a pre-
sented stimulus and not on other factors such as stimulus
sequence, duration of the experiment, or the responses
on previous trials. Factors such as learning contradict this
basic assumption (for recent discussion on this topic see
Frund, Haenel, & Wichmann, 2011). Thus, credible
intervals for parameters of the psychometric function can
be underestimated or overestimated, depending on the
changes in the variance of the observers’ responses with
the learning process. This limitation may lead to a
reduced ability to capture the within-session learning
qualitatively observable in the original data, e.g., practice
effects in the control groups VH-VH and H-H.
How does unisensory experience benefit later multi-
sensory performance? An answer may involve an altera-
tion in the weights of the perceptual estimates during
the course of a single session experience, in other words,
in the representation of the task. Perceptual brain plastic-
ity that underlies perceptual learning is based on the abil-
ity of primary and high sensory cortices to be function-
ally rewired upon repetitive experience to develop new
representations. The interaction found in our study
implies that memory processes may play a significant role
in developing these representations. One such process is
priming, a memory phenomenon that increases the effi-
ciency of processing repeated stimuli (Tulving &
Schacter, 1990) and is thought to be caused by spread-
ing activation neuronal mechanisms (Reisberg, 2010).
The direct connections between primary sensory areas
that are activated during the tactile and visual task sub-
serve the phenomenon of cross-modal priming (Falchier,
Clavagnier, Barone, & Kennedy, 2002). For instance, an
imaging study by James and colleagues (2002) has
shown that previous haptic experience with three-
dimensional haptic objects enhanced activation in visual
areas when these same objects were subsequently viewed.
J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I
Figure 4. Group-mean threshold l 6 SE in (a) first (triangles) and second (circles) performance blocks and (b) group-mean Dl 6 SE
in four experimental groups: VH1-H2, H1-VH2, VH1-VH2, and H1-H2.
**p < .05.
302 PRESENCE: VOLUME 21, NUMBER 3
Another study of 3D object haptic recognition with
blindfolded participants who named everyday objects
presented haptically (Craddock & Lawson, 2008)
showed that haptic priming was greater than visual pri-
ming and crossmodal priming. In line with these studies
and recent notions in cognitive and neural correlates of
tactile memory (Gallace & Spence, 2009), our findings
suggest that early presentation of unisensory haptic in-
formation may help to retrieve the visual properties of
the objects presented visuohaptically. In future studies
it would be interesting to test whether incongruent
univisual stiffness discrimination (with constant haptic
feedback) will also be differentially affected by preceding
experience, either unimodal or bimodal. Current inter-
pretation would imply that haptic experience, either
unimodal or bimodal, in the first block would negatively
affect the subsequent unimodal visual stiffness discrimi-
nation, but not vice versa.
Understanding the effect of order of experience on
perception is expected to promote surgery training and
rehabilitation protocols in VE, where possibilities of
combinations between modalities and order of presenta-
tion in a simulated task can be easily manipulated. The
fact that the order of probing along with the modalities
that are involved (with or without vision) can signifi-
cantly affect the perception of stiffness must be consid-
ered in building the successful training platforms for
tasks relying on stiffness discrimination. In a recent pa-
per, Tsuda, Scott, Doyle, and Ones (2009) asserted that
training complex manual skills such as surgery in VE sim-
ulators implies that simulator-based training needs to be
carefully structured and that sensory feedback is one of
the key factors throughout the development of a skill.
Sensory feedback can be provided at different points in
the learning process and in a variety of ways in order for
trainees to achieve maximum benefit. The current study
adds to this assertion by refining the particular condi-
tions that may support such improvement with respect
to the use of haptic and visual modalities in stiffness-
discrimination–based tasks throughout the course of
practice. More specifically, our results suggest that in
order to benefit from the addition of visual cues in stiff-
ness-discrimination–related tasks, it is important to first
become familiar with the purely haptic aspect of the task.
Acknowledgments
The authors would like to thank Tami Gelfeld and Michael
Sambur for software development.
This research was supported by the European Commission
Integrated Project IP-SKILLS-35005.
References
Bouchigny, S., Hoffmann, P., Megard, C., Gosselin, F., &
Korman, M. (2012, in press). Designing a VR training plat-
form for surgeons: Theoretical framework, technological sol-
utions and results. In M. Bergamasco, B. G. Bardy, & D.
Gopher (Eds.), Skills training in multimodal virtual envi-
ronments. Oxford, UK: Taylor & Francis.
Clark, F. J., & Horch, K. W. (1986). Kinesthesia. In K. Boff,
L. Kaufman, & J. Thomas (Eds.), Handbook of perception
and human performance. New York: Wiley.
Craddock, M., & Lawson, R. (2008). Repetition priming and
the haptic recognition of familiar and unfamiliar objects.
Perception & Psychophysics, 70(7), 1350–1365.
Di Luca, M., Knorlein, B., Ernst, M. O., & Harders, M.
(2011). Effects of visual-haptic asynchronies and loading-
unloading movements on compliance perception. Brain
Research Bulletin, 85(5), 245–259.
Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual
and haptic information in a statistically optimal fashion.
Nature, 415, 429–433.
Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H.
(2002). Anatomical evidence of multimodal integration in
primate striate cortex. Journal of Neuroscience, 22(13),
5749–5759.
Friedman, R. M., Hester, K. D., Green, B. G., & LaMotte, R.
H. (2008). Magnitude estimation of softness. Experimental
Brain Research, 191(2), 133–142.
Frund, I., Haenel, N. V., & Wichmann, F. A. (2011).
Inference for psychometric functions in the presence of
nonstationary behavior. Journal of Vision, 11(6).
doi:10.1167/11.6.16
Gallace, A., & Spence, C. (2009). The cognitive and neural
correlates of tactile memory. Psychological Bulletin, 135(3),
380–406.
Gescheider, G. A. (1997). Psychophysics: The fundamentals (3rd
ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
Harris, J. A., Arabzadeh, E., Moore, C. A., & Clifford, C. W.
(2007). Non-informative vision causes adaptive changes in
J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I
Korman et al. 303
tactile sensitivity. Journal of Neuroscience, 27(27), 7136–
7140.
Hecht, D., & Reiner, M. (2009). Sensory dominance in com-
binations of audio, visual and haptic stimuli. Experimental
Brain Research, 193(2), 307–314.
Howell, J. N., Conatser, R. R., Williams, R. L., Burns, J. M., &
Eland, D. C. (2008). Palpatory diagnosis training on the vir-
tual haptic back: Performance improvement and user evalua-
tions. Journal of the American Osteopathic Association,
108(1), 29–36.
James, T. W., Humphrey, G. K., Gati, J. S., Servos, P., Menon,
R. S., & Goodale, M. A. (2002). Haptic study of three-
dimensional objects activates extrastriate visual areas. Neuro-
psychologia, 40(10), 1706–1714.
Klatzky, R. L., Lederman, S. J., & Matula, D. E. (1993). Hap-
tic exploration in the presence of vision. Journal of Experi-
mental Psychology: Human Perception and Performance,
19(4), 726–743.
Kuschel, M., Di Luca, M., Buss, M., & Klatzky, R. L. (2010).
Combination and integration in the perception of visual-
haptic compliance information. IEEE Transactions on Hap-
tics, 3(4), 234–244.
LaMotte, R. H. (2000). Softness discrimination with a tool.
Journal of Neurophysiology, 83(4), 1777–1786.
Lederman, S. J., & Klatzky, R. L. (2009). Haptic perception:
A tutorial. Attention, Perception, & Psychophysics, 71(7),
1439–1459.
Lehmann, S., & Murray, M. M. (2005). The role of multisen-
sory memories in unisensory object discrimination. Cognitive
Brain Research, 24(2), 326–334.
Murray, M. M., Michel, C. M., Grave de Peralta, R., Ortigue,
S., Brunet, D., Andino, S. G., et al. (2004). Rapid discrimi-
nation of visual and multisensory memories revealed by elec-
trical neuroimaging. NeuroImage, 21(1), 125–135.
Nisky, I., Baraduc, P., & Karniel, A. (2010). Proximodistal gra-
dient in the perception of delayed stiffness. Journal of Neuro-
physiology, 103(6), 3017–3026.
Oldfield, R. C. (1971). The assessment and analysis of handed-
ness: The Edinburgh Inventory. Neuropsychologia, 9(1),
97–113.
Posner, M. I., Nissen, M. J., & Klein, R. M. (1976). Visual
dominance: An information-processing account of its origins
and significance. Psychological Review, 83(2), 157–171.
Reisberg, D. (2010). Cognition: Exploring the science of the
mind (4th ed., pp. 255–517). New York: Norton.
Schacter, D. L. (1995). Implicit memory: A new frontier for
cognitive neuroscience. In M. Gazzaniga (Ed.), The
cognitive neurosciences (pp. 815–824). Cambridge, MA:
MIT Press.
Shams, L., & Seitz, A. R. (2008). Benefits of multisensory
learning. Trends in Cognitive Sciences, 12(11), 411–417.
Sherman, A., Cavusoglu, M. C., & Tendick, F. (2000). Com-
parison of teleoperator control architectures for palpation
task. Proceedings of the ASME Dynamic Systems and Control
Division (Haptic Symposium), 69(2), 1261–1268.
Stein, B. E., & Meredith, M. A. (1993). The merging of the
senses. Cambridge, MA: MIT Press.
Srinivasan, M. A., Beauregard, G. L., & Brock, D. L. (1996).
The impact of visual information on the haptic perception of
stiffness in virtual environments. Proceedings of the ASME
Dynamic Systems and Control Division, 58, 555–559.
Tan, H. Z., Durlach, N. I., Beauregard, G. L., & Srinivasan,
M. A. (1995). Manual discrimination of compliance using
active pinch grasp: The roles of force and work cues. Percep-
tion and Psychophysics, 57(4), 495–510.
Tsuda, S., Scott, D., Doyle, J., & Ones, D. B. (2009). Surgical
skills training and simulation. Current Problems in Surgery,
46(4), 271–370.
Tulving, E., & Schacter, D. L. (1990). Priming and human
memory systems. Science, 155, 301–310.
Widmer, A., & Hu, Y. (2007). The role of viewing angle in
integrating the senses of vision and touch for perception of
object softness. Canadian Journal of Electrical and Com-
puter Engineering, 32(4), 193–198.
Wichmann, F., & Hill, N. (2001). The psychometric function:
I. Fitting, sampling, and goodness of fit. Perception and Psy-
chophysics, 63(8), 1293–1313.
Wu, W. C., Basdogan, C., & Srinivasan, M. (1999). Visual,
haptic and bimodal perception of size and stiffness in virtual
environments. Proceedings of the ASME Dynamic Systems and
Control Division, 67, 19–26.
Yechiam, E., & Gopher, D. (2008). Deliberate suppression of
vision as a training tool for multimodal skills. In C. A. Aviz-
zano (Ed.), SKILLS: Beyond movement: The history and
future of gesture analysis (pp. 80–103). Copenhagen, Den-
mark: Alinea Publishing House.
J_ID: Z92 Customer A_ID: PSEN Cadmus Art: 00114 Date: 18-AUGUST-12 Stage: I
304 PRESENCE: VOLUME 21, NUMBER 3
Top Related