A Mechatronic Platform for

Early Diagnosis of Neurodevelopmental Disorders

Domenico Campolo1 Cecilia Laschi2 Flavio Keller3 Eugenio Guglielmelli1

1Biomedical Robotics & EMC Lab, Universita Campus Bio-Medico, 00155 Roma - Italy

2Advanced Robotics Technology and Systems (ARTS) Lab, Scuola Superiore Sant’Anna, 56127 Pisa - Italy

3Developmental Neuroscience and Neural Plasticity Lab, Universita Campus Bio-Medico, 00155 Roma - Italy

Abstract

Neurodevelopmental Engineering is a new interdisciplinary research area at the intersection of

developmental neuroscience and bioengineering, mainly concerned with quantitative analysis and

modeling of human behavior during neuro-development. Along such a line, our research focuses on

early diagnosis of neurodevelopmental disorders such as Autism, currently diagnosed after 3 years

of age. We propose a novel approach to the assessment of basic patterns of goal-directed actions in

normally developing babies, in the 0-24 months of age range, both under laboratory and naturalistic

conditions. In particular we propose novel mechatronic devices, either wearable or embedded into

toys, for ecological, unobtrusive assessment of infants behavior. Guidelines for the design of a

particular toy embedding both kinematics and force sensing capabilities as well as of experimental

scenarios where similar instrumented toys can be deployed are provided. Preliminary experimental

data carried out in ecological conditions are also presented.

keywords: Neurodevelopmental Engineering, Ecological Behavioral Assessment, Inertial/Magnetic Mo-

tion Tracking, Force Sensing

1 INTRODUCTION

Neurodevelopmental Engineering is a new interdisciplinary research area at the intersection of develop-

mental neuroscience and bioengineering aiming at providing new methods and tools for: i) understanding

neuron-biological mechanisms of human brain development; ii) quantitative analysis and modeling of

human behavior during neuro-development; iii) assessment of neuro-developmental milestones achieved

by humans from birth onwards; iv) studying neuro-developmental disorders; v) conceiving new telematic,

mechatronic and robotic components and systems for applications on infants and toddlers, which can

be used also in ecological conditions for long periods of time; vi) investigating ethical, epistemological

and social implications related to this area.

Main application fields of Neurodevelopmental Engineering are:

1

- New clinical protocols and standards for early diagnosis, functional evaluation and therapeutic

treatments of neuro-developmental disorders;

- New generations of educational, interactive toys which can provide adequate stimuli and guidance

for supporting the physiological neuro-development process

This technology is expected to be also useful in the long term for developing new tools, e.g. toys, which

can sustain, in ecological scenarios, the regular development of motor and cognitive abilities of the child,

based on a rigorous scientific approach.

In line with the philosophy of Neuro-Developmental Engineering, we propose novel methods and

devices to evaluate basic patterns of goal-directed actions in normally developing babies, both under

laboratory and naturalistic conditions, in order to establish standards against which development of

infants at risk for neuro-developmental disorders, particularly Autism Spectrum Disorders (ASD), can

be measured, with the aim of detecting early signs of disturbed development.

In particular, the proposed methods and devices shall improve our ability to assess:

- basic sensorimotor integration / patterns of gaze;

- expression of emotions;

- social communication;

in infants.

1.1 Scientific Background on Neurodevelopmental Disorders

Autism is a behavioral disorder, with onset in childhood, which is characterized by deficits in three

basic domains: social interaction, language and communication, and pattern of interests. There is no

doubt that autism has a strong genetic component, and that biological disease mechanisms leading

to autism are already active during foetal development and/or infancy, as demonstrated, for example,

by the abnormal pattern of brain growth during late foetal and early postnatal life (see [1], for a

review). Autism is usually diagnosed at the age of 3 years, in many cases after a period of seemingly

normal neurological and behavioral development. The diagnosis of autism is purely clinical, there are

no laboratory tests to confirm or disprove the diagnosis. It has been recognized that, although typical

autism is not associated with major neurological deficits, autism has characteristic manifestations in the

perceptual and motor domains.

Motor impairments in autism include deficits in postural reflexes [2, 3, 4], repetitive, stereotyped

movements and awkward patterns of object manipulation, lack of purposeful exploratory movements

(see e.g. [5], gaze abnormalities [6], unusual gait pattern [7], and alterations of movement planning and

execution, which express themselves as “hyper-dexterity” [8, 9]. Motor abnormalities may be observed

retrospectively in infants who later develop the autistic syndrome, on the basis of home-made movies

made during the first year of life [10, 11]. These clinical observations are consistent with a large body

2

of evidence of subtle structural and functional abnormalities of cortical and subcortical neural systems

involved in movement planning and execution, such as the prefrontal cortex, the basal ganglia and the

cerebellum [1].

1.2 The Proposed Platform

The purpose of our research is to develop technological platforms and methods to extract more infor-

mation on perceptual and intersubjective capacities of human infants than is currently possible; this

information could be later used for early diagnosis of developmental disorders. In particular, diagnosis

of ASD is currently made at 3 years of age; ADHD is always considered as an alternative diagnosis of

“high functioning” autism; Tourette syndrome is diagnosed at age 7 or later. ASD is therefore a natural

candidate for demonstrating the validity of the proposed approach. Given the affinity of neurodevel-

opmental disorders, in terms of their neural origin and behavioral manifestation, results achieved for

autism would be extendable to other neurodevelopmental disorders, and vice versa.

The most advanced technological set-ups available in research labs on autism/developmental disor-

ders may currently include some sophisticated systems for movement analysis, such as:

- Stereophotogrammetric systems for movement analysis: these are rather sophisticated and costly

technologies. They require highly structured environments, generally cameras have to be carefully

positioned and calibrated, leading to field-of-view problems, i.e. markers placed on the subjects

body have to be always spotted by a given number of cameras in order to reconstruct its position

in 3D space. Highly trained personnel is required to operated them;

- Gaze-tracking devices thus far in use, including commercially available ones, can be applied in

specific contexts. Generally, the subject is required to face specific directions, e.g. they are

particularly suitable when the subject looks at a TV monitor. Gaze tracking devices become

unsuitable in the case of infants;

- Force platforms, often combined with photogrammetric systems, are used to measure Ground

Reaction Forces (GRFs) as the subject walks on the platform. A major limitation is that measures

are provided only when the subject steps on the platform, i.e. they are not continual. Data

extracted from a force platform are very useful when the walking pattern is somehow cyclic, i.e.

the time the subjects stays on the platform is representative of the whole walking dynamics. In

the case of infants and toddlers, clearly the time spent on the platform is no longer significant for

the whole walking;

- Data gloves: only recently some research projects are facing the problem of developing data gloves

for the child, which can be potentially combined with customized virtual reality environments;

A limitation common to all previously mentioned technologies is that, despite providing very accurate

measurements and for the extracted data to make sense, tasks need to be performed in well-controlled

3

and highly structured environments as well in accurately known and repeatable conditions. This is

rarely possible when infants are involved and often impossible in poorly structured environments such

as clinical centres and houses. Deployment of state of the art technologies, e.g. interactive mobile

robots, already proved very beneficial for research on autism.

Scassellati developed the ESRA robot, able to show a small set of facial expressions, with the

objective of developing objective and quantitative methods and tools for autism diagnosis, of exploring

the possibility of using robots as therapeutic aids, and of speculating on how the use of social robots in

autism research might lead to a greater understanding of the disorder [12]. He performed clinical trials

with 13 subjects, mean age 3.4 years, including autistic children, and found an increased engagement of

all children with the robot. In the case of autistic children such positive proto-social behaviors had been

rarely seen in a naturalistic context. Dautenhahn and Billard [13, 14] used the Robota robot dolls, in the

framework of the EU-funded AuRoRa Project, to study if and how robots can have an educational or

therapeutic role for children with autism. Robota is a 45-cm high humanoid robotic doll. The main body

of the doll contains the electronic boards and the motors that drive the arms, legs and head giving 1 DOF

to each. Robota can engage children with autism in coordinated and synchronized interactions with

the environment thus helping them to develop and increase their communication and social interaction

skills. The Keepon robot [15] is a small puppet-like robot, conveying basic communicative actions, i.e.

so-called attentive action and emotive action, which can be easily understood by children interacting

with it. These actions are given by the motion of 4 DOF: nodding (tilt) within +/-40 degrees, shaking

(pan) within +/-180 degrees, rocking (side) within +/-25 degrees, and bobbing (shrink) with a 15-mm

stroke. The head part of the robot has two wide-lens (120 horizontally) color cameras and a microphone.

With attentive actions Keepon focuses a target; with emotive actions Keepon shows pleasure/excitement

by shaking its body up-down and/or side to side. Keepon was tested in a day-care center for children

with developmental delays and/or disorders, with over 500 children in the range of 2-4 years of age, in

unstructured tests. In longitudinal observations, the children showed active interaction with Keepon

and in some cases they showed vivid facial expressions that even their parents had not seen before. They

also showed prosocial actions like trying to feed Keepon, putting on a cap on its head, and kissing it. The

observation that children with difficulty in inter-personal communication (especially, those with PDD

and autism) showed curiosity and security in approaching Keepon suggests that this is probably due to

Keepon shape, seeming to be neither a complex human nor a simple toy. Some children extended their

dyadic interaction with Keepon into triadic inter-personal interaction, trying to share their pleasure and

surprise in interacting with Keepon with others, like their caregivers and nursing staff. The Roball is

a very simple robot in the shape of ball, capable of intentional self-propelled movements and various

interplay actions, by using motion, messages, sounds, illuminated parts, and other sensors. In trials

with 2-to-4-year-old children it also showed to attract increased interest and engagement [16]. However,

the design criteria for what makes individuals with autism likely to respond to these devices are not

understood. The robots used in these studies include an expressive face, anthropomorphic robotic dolls,

an expressive snowman-like device, a spherical robot ball. These robots show diverse anthropomorphic

4

characteristics, behavioral repertoires, aesthetics, and sensory and interactive capabilities. While there

are many studies of the effects of these interaction variables on typical adults, very little is known about

how individuals with autism respond to these design dimensions. Each of these studies has demonstrated

that robots generate a high degree of motivation and engagement in subjects, including subjects who

are unlikely or unwilling to interact socially with human therapists. However, mechatronic and ICT

technologies have been used so far to provide new tools for therapy rather than for diagnosis. Moreover,

such attempts do not apply to infants.

As a matter of fact, the only attempts at an early diagnosis of autism have been made by rating

of home videotapes of behavior from very young children later diagnosed with autism [10, 11]. This

qualitative approach proved very useful in laying down the bases for research in this field but at the

same time urges for novel quantitative approaches and enabling technologies.

Technologies for the domain of Neuro-Developmental Engineering shall comply with strict functional

and operative requirements, not fully met by current state-of-the-art. We aim at developing conceptually

new devices for neuro-developmental monitoring which respond to the following main requirements:

Non-obtrusive technology: the new devices shall be designed with the final goal of continual

monitoring but without being distressful to the child. In fact, the child should not perceive the

presence of such instruments at all (e.g. wearable microphones, cameras, etc...), or like to play

with them (e.g. instrumented toys). This clearly sets constraints on the kind of technology to be

used. In particular, small size, light weight, wireless, and portable will be the key features to keep

in mind during technical design of such devices.

Minimally structured operating environments: current tools for behavioral analysis, e.g.

photogrammetric devices for motion analysis or force platforms for gait analysis or state-of-the-art

gaze tracking devices, are only suitable to controlled and highly structured environments such as

research laboratories. Screening of a large number of children for diagnostic purposes is therefore

not feasible due to high costs and limited availability of the equipment. We propose development

of relatively low-cost devices requiring minimally structured environments. Possible settings range

from totally unstructured home-like situations, e.g. the child plays with interactive toys while a

caregiver steers the game along predefined play protocols and tasks, to situations with an increasing

degree of structure.

Given the previous considerations, our main focus is conceiving and developing new integrated sensor

technologies, having in mind two main typologies of applications relevant to the new domain of Neuro-

Developmental Engineering:

- Wearable devices for movement analysis and video/audio cues extraction.

- Toys and objects of ordinary use in naturalistic scenarios, embedding miniaturized sensor and

communication technology;

5

Whether wearable or embedded in regular toys, miniaturized technology for motion analysis is a key

ingredient. In what follows, devices will be presented that are going to be used as technological platforms

for monitoring infants behavior.

2 Instrumented Toys for Ecological Behavioral Analysis

Our approach is based on a closed-loop dialogue between biomedical and bioengineering researchers, with

a common and rigorous methodological strategy based on: the definition of novel scientific hypotheses

on ASD early diagnosis; the customized design of new mechatronic technologies enabling the validation

of such new hypotheses; and the execution of multisite experimental tests.

Results from ongoing experiments are expected to continuously provide a flow of new knowledge

and hypotheses to be incorporated into the design of a new generation of technologies, e.g. toys and

wearable devices, which in turn will allow the design of new scientific experiments. We will follow a

step-by-step approach going from more basic to more complex smart toy devices.

Toys are age-specific but, among many possible choices, cubes, balls and rattles are the ones that

could be used equally well with all children in the 0-24 months range of age. For the first generation of

devices, used to derive preliminary data, commercially available toys have been equipped with sensors.

This allowed to have prototypes and therefore run experiments immediately and, therefore, we focused

on developing a miniaturized sensing unit which is independent of the specific shape of the toy and

that can then be embedded in other toys to allow use in various experimental scenarios (including

experiments with older children).

Toys such as cubes, balls and rattles can in fact be used in a very broad set of experimental scenarios

but attention was particularly paid to define operating scenarios which would allow studying specific

biomechanical aspects of the movements of infants during tasks with a minimum set of involved “free”

variables, especially considering that only few data of this kind are available in literature at least as

long very young children are concerned.

A key element in choosing a toy is its affordance, i.e. its perceived usage inferred from its appearance

and look [17]. The shape of the toy suggests the use to be done with the toy itself and can, in part,

guide the child to perform certain movements instead of others1 (i.e. shaking in air vs. random banging

on objects around the child). After careful examination of different commercial rattles, we opted for a

maracas-like shape since it suggests shaking rather than banging (as an hammer-like object would). In

order to reduce the number of “free” variables (i.e. the child’s choice of orientations, grasping patterns

etc. . . when playing with toy) a maracas-like rattle offers a pen-like handle, encouraging thus only a

limited number of grasping hand-configurations.

Differently from a rattle, balls and cubes pose less problems as long as the outer shape is concerned:

they should be small enough to be handled with a single hand also by a 6 month old child.

Despite differences in the external look, all devices should allow monitoring kinematic patterns arising1The caregiver, of course, will also guide the game, imitation is a key element to be exploited in designing games.

6

during the playful activity, allowing for example monitoring imitative and rhythmic skills of the child

as well as monitoring the grasping force/pressure.

Musical feedback is considered important not only for motivating the child to keep playing with the

toy but also as a programmable/adaptive way of guiding the game. Attractive sounds/melodies may

be associated with specific movements, inviting thus the child to repetitively perform such movements.

Appropriate melodies can be created on-line by directly mapping movements kinematics (from movement

sensors embedded in the toys) to sound.

2.1 Proposed Experimental Scenarios

In this section, two possible experimental scenarios are provided as examples for designing games that

stimulate the child to perform clinically relevant motor tasks. In the first one, the child is sitting on a

orthopedic high chair constrained so that only the upper limbs are free to move (torso is also fixed to

the chair) only for 6 months old children. For 12 and 18 months: belts applied only to lower extremities.

Belts should be used for safety purposes. A caregiver - possibly mother - is sitting next to the child.

A toy is placed on a support along the midline at a predetermined distance, according to the length of

upper limb, on the desk which is shaped as in Fig. 1-a.

top view front view

desk

toy

chairout

in

outer trajectory

innertrajectory

SensorizedBALLsensors

Plexiglas

a) b)

Figure 1: Experimental scenarios.

The caregiver (sitting next to the desk) first shakes the toy to draw the attention of the child and

then places the toy on a support (neutral color) fixed on the table, used to keep the toy in place.

The child is expected to perform the following actions: i) reach for the toy; ii) grasp it; iii) lift it

up; iv) freely shake it (for a while).

A second experimental scenario is shown in Figure 1-b. A sensorized ball can be inserted only in

the top-right hole and, after a fixed inner trajectory, exits always from the bottom-left hole. During the

inner trajectory the child can only watch the ball falling down (Plexiglas transparent material). As the

ball exits, the child can reach for it, grasp it and re-insert it via the top-right hole. The outer trajectory

is totally up to the child. The task starts with the caregiver showing the child how to play with the

7

system. The game is meant to be repetitive, i.e. the inner trajectory never changes and although the

outer trajectory is up to the child, the start and the end points are fixed. The child should be able to

repeat the game several times (at least 5).

Measurements of interest are: the ball trajectory, at least the outer one; movement accuracy; pres-

ence/frequency of shaking; the force/pressure exerted on the ball during grasping; the child’s gaze2

during inner trajectory (the Plexiglas allows watching but not touching). Every toy shall be provided

with both kinematics and force sensing units.

2.2 Functional and Technical Specifications

In order to derive functional and technical specifications for the instrumented toys, preliminary studies

have been conducted with infants at a local kindergarten in order to assess acceptance of toys with

different shapes, materials, size and weight.

Preliminary quantitative experiments have been performed using a commercial IMU (Inertial Mea-

surement Unit, model MTi from Xsens, containing inertial/magnetic sensors) embedded into a soft cube

large enough to contain it (each side was 10 cm). Several children were let to play with it while kinematic

data (i.e. raw data from the 3-axis accelerometers, 3-axis magnetometers and 3-axis gyroscopes inside

the MTi) were being downloaded on a PC (connected via a USB cable to the sensorized cube) and while

the playing session was being filmed on camera.

Such experiments showed that:

i) children were often distracted by the wire, so a wireless connection would be highly preferred;

ii) the cube was too large so that children would rather adopt a bimanual manipulation, or use the

wire to lift up the toy;

iii) a ball, because of its symmetry, would be preferable to a cube in order to reduce the number of

possible grasping patterns;

Previous considerations lead us to devise a wireless toy shaped as a ball, capable of sensing kinematics

(i.e. orientation, and linear accelerations), grasping forces during manipulation and small enough to be

handled with a single hand by a child as young as 6 months old. This can be quantified in a ball’s

diameter less or equal to 5 cm.

The wireless constraint also means that the device should be autonomous in terms of energy, i.e. it

should include batteries and should be able to run for a sufficient time. Experiments involving children,

in particular very young ones, usually last a few tens of minutes, in any case they never exceed one hour.

This shall be considered when selecting the proper battery, i.e. with enough capacity to guarantee one

hour operation before battery is replaced or recharged.2Sensors on the side represent commercial gaze trackers which typically track the gaze of a child as he/she is watching

a TV screen, the proposed game is purposely designed to be 2-dimensional without distorting the 3D physical feeling.

8

2.2.1 Measuring Kinematics

Measuring movements can be rightfully considered part of motion tracking. Motion tracking can count

on a host of different technological solutions, operating on entirely different physical principles, with

different performance characteristics and designed for different purposes. As shown in [18] there is not a

single technology that can fit all needs. Each application defines the best technology to be implemented.

The most appealing technology, from an ecological perspective, is represented by the sourceless

inertial/magnetic sensing for orientation tracking3. Such a technology is based on accelerometers and

magnetometers used to sense respectively the gravitational and the geomagnetic field as well as on

gyroscopes used to enhance performance at higher frequencies. Gravitational and geomagnetic fields

can in fact be used to estimate rotations of a rigid body relatively to an earth-fixed coordinate frame,

as shown in [19, 20].

Complementary filters and Kalman filters have traditionally been used to derive 3D orientation

(attitude) from inertial and magnetic measurements [21, 22, 23]. With reference to Fig. 2 whose details

can be found in [20] and [18], inertial and magnetic sensors embedded in an object (e.g. a toy) provide

measurements of “vector” quantities (e.g. accelerations, angular velocities and magnetic field) in a frame

of coordinates relative to the object itself (moving frame). On the other hand, certain quantities can be

easily expressed only in a local frame of coordinates. For example, the gravity vector G can be expressed

as G = [0 0 − g]T (where g = 9.81m/s2) only in the local frame, where gravity also defines important

concepts such as “vertical” and “horizontal”. Furthermore, in order to derive position from acceleration

the following two steps are required:

i) the gravity vector G should be subtracted from the sensed acceleration, since accelerometers are

sensitive to both linear acceleration and to gravity, as shown in Fig. 2;

ii) linear acceleration should be double integrated (this step only makes sense in a local, i.e. fixed,

frame of coordinates) and this typically leads to large drift errors just after a few seconds of

integration time [18].

Human movements, especially in the case infants, are characterized by accelerations much smaller

if compared with the magnitude of the gravitational field (g). Only during high speed impacts (e.g.

banging) large accelerations can be generated but impact dynamics is out of the scope of this research.

The full scale range of a sensor, given the full scale ranges of a set of available sensors, is selected as

the minimum one containing the all signal excursions. A smaller range may in fact lead to saturation

when certain operative conditions occur while a larger range would reduce sensitivity.

As long as accelerometers are concerned, it is safe to consider that an infant would never exceed

the ± 20m/s2 ≈ ±2g range of accelerations. An accelerometer was vigorously shaken by an adult and

accelerations larger than ±2g were hardly generated.3Commonly available motion tracking systems typically provide position of points (markers) in space, orientation is

consequently derived from the position of a group of non-collinear points. As dimensions of objects (or body parts)

decrease, also accuracy in determining orientation decreases.

9

magnetometers

gyroscopes

accelerometers

integration+

compare

orientation

change ofcoordinates -

linear acceleration

gravity [0 0 -g]T

moving frame coordinates local frame coordinates

complementary filter

Figure 2: Complementary filter approach to orientation tracking.

As for the magnetometers, since only the geomagnetic field (about 0.6 gauss) needs to be measured,

the full measurement range should be in the order of ±1 gauss.

Measurements for gyroscopes, as shown in Fig. 2, after a correction are directly fed into the “inte-

gration” block. It can be shown [24] that gyroscopes are the main responsible for tracking orientation

when fast rotations are involved. Movements such as wrist rotations, considering the application of

interest (hand-held toys), can be quite fast. Saturation of gyroscopes would result in a problematic loss

of tracking and therefore, given several commercially available gyroscopes with full scale ranging from

±150 deg/s to ±1200 deg/s, the maximum scale range (±1200 deg/s) was selected.

2.2.2 Bandwidth Considerations

Although ecological devices should be primarily designed to work in unstructured environments, they

could also be used in research laboratories (e.g. laboratories investigating developmental disorders). It

is not uncommon for such laboratories to be equipped with traditional marker-based motion analysis

systems such as photogrammetric ones (ELITE, VICON, QUALYSIS, to name a few). Such systems

consist of a set of cameras (often infrared ones) which are used to film the experimental session from

different view-points. Such cameras, at least as long infants movements are concerned, are typically

used to film the scene at 100 frames/s, although some system may also allow higher frame rates.

In line with such specifications, since our new devices could be used in conjunction with photogram-

metric systems, a 100 Hz sampling frequency was considered to be the appropriate one. This translates

into an anti-aliasing filter cut-off frequency at 50Hz.

Higher sampling frequencies would not be a technological problem per se but since signals of interest

(movements of humans and in particular of infants) are confined well within a few tens of Hz, a larger

bandwidth would only increase the noise level resulting thus in a loss of resolution.

10

2.2.3 Force Sensing

To the authors’ knowledge, only few quantitative data exist in literature about forces at the fingertips

exerted by children during manipulation tasks, especially as long as infants are concerned. Available

data [25, 26] refer to older children (older than 4 years old) performing manipulative tasks. Exerted

forces are reported to be in the range of 20 − 50 N .

Force measurement is an indirect measurement since essentially reduces to sensing the displacement

or the strain induced by a force in an instrumented deformable mechanical structure [27]. A wide variety

of configurations can be deployed for force and tactile sensing, depending on the transduction principle:

mechano-electric, mechano-magnetic, mechano-optic. In order to assess manipulative skills of children,

the force sensors need to be placed on the outer part of the toy, where the grasping is most likely to

occur. Typically, the outer shape of toys is curved, as in the ball case. This poses strict constraints on

the mechanical assembly. In particular, a sensor technology is preferred which can be assembled on thin

films, e.g. kapton films, whose compliance allows both sensors and electrical connections to conform to

curved surfaces.

Our final choice is for Quantum Tunneling Composites (QTCs), a recent technology consisting of a

flexible polymer, commercially available in form of pills as small as 3.6mm×3.6mm×1mm. Such small

dimensions have two main advantages for our application: first, they can easily be mounted on curved

surfaces with a curvature radius as in the 5 cm ball; second, considering the average dimensions of the

fingers of an infant, they allow a very high spacial resolution. Force sensing layers, consisting of QTC

pills mechanically and electrically bonded to flexible kapton films for data acquisition, are currently

under development which should cover the outer surface of toys.

As for the force range of interest, another advantage of the QTCs is that their electrical resistance is

inversely proportional to the applied force, i.e. they behave as insulator (electrical resistance ≈ 1012Ω)

when no force is applied but their resistance drops down to a few Ohms as increasing forces are applied.

In particular, their sensitivity is maximum for very small forces, which is the case of interest.

2.2.4 Design of a Sensorized Ball

In this section, following the functional and technical specifications given in the previous section, the

design of a sensorized ball is briefly described. Such a device is currently under development. As shown

schematically in Fig. 3, the sensorized ball is shaped as a sphere with a 5 cm diameter and comprises: a

kinematics sensing unit; a force sensing unit; a data acquisition and communication unit; and a power

unit.

Kinematics sensing unit: a commercial devices (MAG3 from MemSense Corporation) was chosen

which embeds a triaxial accelerometer (±2g range), a triaxial magnetometer and a triaxial gyroscope

(±1200 deg/s) in a highly compact case (17.8 mm× 17.8 mm× 10.2 mm). Furthermore, each channel is

filtered with a 50 Hz low-pass filter.

As for calibration, the MAG3 device as well as other commercial accelerometers and magnetometers

11

Li-Ion batteries

main PCB:- inertial/magnetic sensors- connectors for Kapton films- acquisition electronics

communication PCB

force sensors onKapton flexible film

outer shell A

outer shell B

slots

Figure 3: Exploded view of the sensorized ball (5 cm diameter).

have special pins that can be used to produce a known excitation against which the output signal can

be calibrated. Especially in the case of magnetometer (where this extra pins are used to generate a

magnetic field), large currents are required which is unsuitable with the application of interest. Novel

calibration procedures have been set up which do not require any additional electronics and that can

be performed in unstructured environments by non trained personnel, details can be found in [28].

Force sensing unit: Kapton films, on which QTC pills are bonded, cover the outer structure (shell

A and B in Fig. 3) of the ball and contact electrical connectors on the main PCB via several slots

purposely designed along the borders of the two outer shells. Signals from QTC pills are conditioned

via voltage dividers consisting of a QTC pill and a regular resistor in series. The signal is then buffered

and filtered. All the electronics is on the main PCB.

Data acquisition and communication: Data from kinematics and force sensing units are nu-

merically converted via several multi-channel A/D converters which, after conversion, transmit acquired

data over an I2C serial bus. A central microcontroller acts as the master of the I2C bus and sequen-

tially requests data from the different A/D converters (which work in slave modality and are uniquely

identified a specific bus address). The central microcontroller (an rfPIC from Microchip Corporation) is

physically located on the communication PCB (see Fig.3) and is capable of transmitting data wireless

to a nearby station. Every 10 ms (corresponding to a 100 Hz sampling rate) a packet is sent which

consists of the following sequence: 1 byte preamble (the start of a new message), 1 byte ID (to iden-

tify the sensorized device since multiple devices may operate in parallel), 2 bytes time-stamp (used for

synchronization), several bytes of data, 1 byte checksum (to detect possible errors during transmission).

Power unit: as in any portable device, power consumption is one of the main issues. The most

critical components, as long as the energetic budget is concerned, are the MAG3 device, which works at

5V and requires 57 mA maximum current, and the wireless transmission which is reported not to exceed

12

11 mA current. Among battery technologies, the Lithium-Ion (Li-Ion) cells exhibit the highest energy

density. A single cell only provides 3.6V , therefore two cells electrically in series are required to supply

a 5V operating voltage. The smallest batteries which would guarantee 1 hour operation and supply

continuously 75 mA (a safe overestimate of the total required current), were identified4 in the Lir2477

Lithium-Ion Rechargeable coin-cells from Powerstream Corporation, whose dimensions are suitable with

the mechanical assembly as shown in Fig. 3.

3 System Integration and Preliminary Results

Instrumented devices, such as the previously described sensorized toy, are indeed novel tools which

have great potential for studies on infants behavior. Nevertheless such devices, designed to be able to

operate in generally unstructured conditions, can be best exploited if used for multi-modal analysis in

combination with traditional tools such as video cameras and microphones. In Fig. 4, a platform for

early diagnosis of neurodevelopmental disorders is shown which consists of a minimum set of several

sensorized toys for kinematics/force sensing, one video camera and one microphone. Nevertheless, the

proposed platform can be easily extended with other available commercial devices by simply following

the synchronization method as explained below.D

AQ

MICROCONTROLLER

RF

IMU

DAQ

POWER

DA

Q

MICROCONTROLLER

RF

IMU

DAQ

POWER

RF

VIDEOAUDIO-LAUDIO-R

CAMERA

MICROPHONE

TOY #1

TOY #2LOCAL STATION

VIDEO-AUDIOCAPTURE

WIRELESSCOMM.

WIRELESSCOMM.

SYNC

Figure 4: Complete platform for multi-modal assessment of infants behavior.

Several sensorized toys can be used, e.g. one for the child and one for the caregiver for assessment

of imitation skills. A central PC (local station) is used to acquire data which are transmitted (wireless)

by the sensorized toys. Data are thus stored on a file for later analysis. A fundamental issue in multi-4Energy calculations took into account that not more than 80% of the total capacity should be extracted from a cell

and that for large discharge rates only half of the available energy can be extracted.

13

modal analysis is that it should always be possible to synchronize data coming from different toys, video

cameras, microphones etc...

Synchronization can be achieved thanks to the time-stamps that each sensorized toy transmits with

the data in each packet. The time-stamp is simply a counter that increases each time a packet is sent

(frame rate is 100Hz). We deploy a 16 bits counter, which means that the counter resets approximately

every 11 minutes (10 × 216 ms). As shown in Fig. 4, after receiving wireless data and before sending

such data to the local station, the time-stamp relative each device is read and one audio line is marked

(just like a modem communicates over phone lines, the ID and its relative time-stamp are also written

over one audio channel).

A commercial device is used that captures audio (stereo, i.e. two channels) and video analog data

and sends it to the local station which stores them on a single audio-video file (on which audio and

video are synchronized). Since one audio channel is marked with time-stamps from the sensorized toys

(multiple toys can be used), the audio-video file can be synchronized with the data file relative to each

sensorized toy.

time

Figure 5: Synchronized superposition of the video sequence and of the orientation sequence of a toy as

this is being manipulated by a 1 year old child. The orientation is directly derived from kinematic data

as sensed by the kinematics sensing unit embedded in the toy.

Fig. 5 shows an example of synchronization between video data and a first prototype of sensorized

toy. The experiment was carried out in a kindergarten. The whole set up consisted of a laptop PC,

a video camera and a soft cube embedding commercial MTi device, as described in section 2.2. After

the experimental session was completed, raw data (9 channels) from accelerometers, magnetometers and

14

gyroscopes embedded in the MTi device were fed into the complementary filter shown in Fig. 2, resulting

thus in a sequence of matrices (R, i.e. the filter output) describing the orientation of the cube. Such an

orientation was superimposed on the video frame sequence, the result is shown in Fig. 5. Video images

are captured at 25 frames/s (standard rate for regular video cameras) while the MTi was running at

100 samples/s, i.e. 4 samples of kinematic data for each video frame. For sake of clarity, only two

kinematic samples are shown for each video frame.

4 CONCLUSION

Recent studies have demonstrated that the use of robotic technology is beneficial when applied to

children who are unlikely or unwilling to socially interact with human therapists. However, mechatronic

and ICT technologies have been used so far to provide new tools for therapy rather than for diagnosis.

Moreover, such attempts do not apply to infants.

In this work a novel approach to early diagnosis of neurodevelopmental disorders is presented. In

particular, a platform for the ecological assessment of infants behavior is proposed which is based on

non-obtrusive technology and purposely designed to operate in non-structured environments, a funda-

mental requirement when screening of a large number of children for diagnostic purposes is pursued.

Despite such novel and distinctive features, the proposed technology can also operate in conjunction

with technological settings typically present in infants laboratories.

Design guidelines of an instrumented toy for ecological behavioral analysis of infants are provided

together with concept experimental scenarios where such a toy could be used. Preliminary tests of early

prototypes in a kindergarten with real end users (infants) demonstrated full acceptability of the proposed

concepts as well as the great potential of the proposed platform for research on infants behavior.

Next steps of the research are aimed at releasing the final prototypes of the proposed platform and

subsequently at starting experimental trials in three infants clinical laboratories located in Europe.

Acknowledgement

This work was supported by a grant from the European Union, FP6-NEST/ADVENTURE program,

contract no. 015636.

REFERENCES

[1] F. Keller, A. M. Persico, “The neurobiological context of autism”, Mol. Neurobiol. 28:1-22, 2003.

[2] N.J. Minshew, K. Sung, B. L. Jones, J. M. Furman, “Underdevelopment of the postural control

system in autism”, Neurology 63:2056-2061, 2004.

15

[3] C. Schmitz, J. Martineau, C. Barthelemy, C. Assaiante, “Motor control and children with autism:

deficit of anticipatory function?”, Neurosci. Lett. 348:17-20, 2003.

[4] C.A. Molloy, K.N. Dietrich, A. Bhattacharya, “Postural stability in children with autism spectrum

disorder”, J Autism Dev Disord 33:643-652, 2003.

[5] K. Pierce, E. Courchesne, “ Evidence for a cerebellar role in reduced exploration and stereotyped

behavior in autism”, Biol Psychiatry 49:655-664, 2001.

[6] J.A. Sweeney, Y. Takarae, C. Macmillan, B. Luna, N. J. Minshew, “Eye movements in neurodevel-

opmental disorders”, Curr Opin Neurol. 17:37-42, 2004.

[7] M. Hallett, M. K. Lebiedowska, S. L. Thomas, S. J. Stanhope, M. B. Denckla, J. Rumsey, “Loco-

motion of autistic adults”. Arch Neurol 50:1304-1308, 1993.

[8] N. J. Rinehart, J. L. Bradshaw, A. V. Brereton, B. J. Tonge, “Movement preparation in high-

functioning autism and Asperger disorder: a serial choice reaction time task involving motor repro-

gramming”, J Autism Dev Disord 31:79-88, 2001.

[9] M. Mari, U. Castiello, D. Marks, C. Marraffa, M. Prior, “The reach-to-grasp movement in children

with autism spectrum disorder”, Philos Trans R Soc Lond B Biol Sci. 358:393-403, 2003.

[10] P. Teitelbaum, O. Teitelbaum, J. Nye, J. Fryman, and R. G. Maurer, “Movement analysis in

infancy may be useful for early diagnosis of autism”, Proc Natl Acad Sci USA 95: 13982-13987,

1998.

[11] O. Teitelbaum, T. Benton, P. K. Shah, A. Prince, J. L. Kelly, P. Teitelbaum, “Eshkol-Wachman

movement notation in diagnosis: the early detection of Asperger’s syndrome”, Proc Natl Acad Sci

US A 101:11909-11914, 2004.

[12] B. Scassellati, “How social robots will help us to diagnose, treat, and understand autism”, 12th

International Symposium on Robotics Research (ISRR). San Francisco, CA. Oct. 2005.

[13] I. P. Werry, K. Dautenhahn, “Applying Mobile Robot Technology to the Rehabilitation of Autistic

Children”, in Proceedings of SIRS 99, Symposium on Intelligent Robotics Systems, 20-23 July 1999.

[14] A. Billard, B. Robins, K. Dautenhahn and J. Nadel “Building Robota, a Mini-Humanoid Robot for

the Rehabilitation of Children with Autism”. the RESNA Assistive Technology Journal, in Press,

2006.

[15] H. Kozima, C. Nakagawa, Y. Yasuda, “Interactive robots for communication-care: A case-study in

autism therapy”, IEEE International Workshop on Robot and Human Interactive Communication

(ROMAN-2005) Nashville, TN, USA, pp.341-346, 2005.

16

[16] F. Michaud, C. Thberge-Turmel “Mobile robotic toys and autism”, Socially Intelligent Agents - Cre-

ating Relationships with Computers and Robots, Kerstin Dautenhahn, Alan Bond, Lola Canamero,

Bruce Edmonds (editors), Kluwer Academic Publishers, pages 125-132, 2002.

[17] Norman, DA, “The design of everyday things”, New York: Doubleday, 1990.

[18] G. Welch and E. Foxlin, “Motion Tracking: No Silver Bullet, but a Respectable Arsenal”, IEEE

Computer Graphics and Applications, Vol. 22, No. 6, pp. 24-38, 2002.

[19] D. Campolo, M. Molteni, E. Guglielmelli, F. Keller, C. Laschi, P. Dario “Towards Development

of Biomechatronic Tools for Early Diagnosis of Neurodevelopmental Disorders”, 28th Conf. of the

IEEE Engineering in Medicine and Biology Society (EMBC), New York, USA, Aug.30 - Sept.3,

2006.

[20] D. Campolo, F. Keller, E. Guglielmelli, “Inertial/Magnetic Sensors Based Orientation Tracking on

the Group of Rigid Body Rotations with Application to Wearable Devices”, IEEE/RSJ Int. Conf.

on Intelligent Robots and Systems (IROS), Beijing, P.R. China, October 9-14, 2006.

[21] B. Kemp, A. J. M. W. Janssen, B. van der Kamp, “Body position can be monitored in 3D using

miniature accelerometers and earth-magnetic field sensors”, Electroencephalography and Clinical

Neurophysiology, Vol. 109, pp. 484-488, 1998.

[22] E. R. Bachmann, X. Yun, D. McKinney, R. B. McGhee, M. J. Zyda, “Design and Implementation

of MARG Sensors for 3-DOF Orientation Measurement of Rigid Bodies”, Proc. of the 2003 IEEE

Intl. Conf. on Robotics and Automation (ICRA 2003), Taipei, Taiwan, September 14-19, 2003.

[23] R. Zhu and Z. Y. Zhou, “A Real-Time Articulated Human Motion Tracking Using Tri-Axis Iner-

tial/Magnetic Sensors Package”, IEEE Trans. on Neural Systems and Rehabilitation Engineering,

Vol. 12, No. 2, 2004.

[24] R. G. Brown , P. Y. C. Hwang, “Introduction to random signals and applied Kalman filtering”,

New York: J.Wiley, 1992.

[25] A. M. Gordon, S. V. Duff, “Fingertip forces during object manipulation in children with hemiplegic

cerebral palsy. I: anticipatory scaling”, Developmental Medicine and Child Neurology, 41(3):166-75,

1999.

[26] D.J. Serrien, P. Kaluzny, U. Wicki, M. Wiesendanger, “Grip force adjustments induced by pre-

dictable load perturbations during a manipulative task”, Experimental Brain Research, Vol. 124,

No. 1, pp. 100-106, 1999.

[27] P. Dario, C. Laschi, S. Micera, F. Vecchi, M. Zecca, A. Menciassi, B. Mazzolai, M.C. Carrozza,

“Biologically-Inspired Microfabricated Force and Position Mechano-Sensors”, in Proc. of MPATHE

2000, Il Ciocco, Italy, June 11-16, 2000.

17

[28] D. Campolo, M. Fabris, G. Cavallo, D. Accoto, F. Keller, E. Guglielmelli, “A Novel Procedure for

In-field Calibration of Sourceless Inertial/Magnetic Orientation Tracking Wearable Devices”, the

first IEEE / RAS-EMBS Intl Conf. on Biomedical Robotics and Biomechatronics (BIOROB), Pisa,

Italy, February 20-22, 2006.

18