* erica.nocerino@uniparthenope.it; phone +39 81 5476632; fax +39 81 547 6501; http://ltf.uniparthenope.it

Low-cost human motion capture system for

postural analysis onboard ships

Erica Nocerino*a,b

, Sebastiano Ackermanna, Silvio Del Pizzo

a, Fabio Menna

a,c, Salvatore Troisi

a

aLTF - Dept. of Applied Sciences, “Parthenope” University of Naples (Italy),

http://ltf.uniparthenope.it bDept. of Naval Engineering, “Federico II” University of Naples (Italy)

c3DOM Research Group, FBK - Fondazione Bruno Kessler, Trento (Italy),

http://3dom.fbk.eu

ABSTRACT

The study of human equilibrium, also known as postural stability, concerns different research sectors (medicine,

kinesiology, biomechanics, robotics, sport) and is usually performed employing motion analysis techniques for recording

human movements and posture. A wide range of techniques and methodologies has been developed, but the choice of

instrumentations and sensors depends on the requirement of the specific application. Postural stability is a topic of great

interest for the maritime community, since ship motions can make demanding and difficult the maintenance of the

upright stance with hazardous consequences for the safety of people onboard. The need of capturing the motion of an

individual standing on a ship during its daily service does not permit to employ optical systems commonly used for

human motion analysis. These sensors are not designed for operating in disadvantageous environmental conditions

(water, wetness, saltiness) and with not optimal lighting. The solution proposed in this study consists in a motion

acquisition system that could be easily usable onboard ships. It makes use of two different methodologies: (I) motion

capture with videogrammetry and (II) motion measurement with Inertial Measurement Unit (IMU). The developed

image-based motion capture system, made up of three low-cost, light and compact video cameras, was validated against

a commercial optical system and then used for testing the reliability of the inertial sensors. In this paper, the whole

process of planning, designing, calibrating, and assessing the accuracy of the motion capture system is reported and

discussed. Results from the laboratory tests and preliminary campaigns in the field are presented.

Keywords: Videogrammetry, motion capture, IMU, low-cost instruments, human postural stability onboard ships

1. INTRODUCTION

Human postural stability is a multidisciplinary topic of great interest for several sciences and disciplines, since it has a

profound impact on several aspects of everyday life. How individuals maintain balance, employ correct and efficient

postural strategies, are able to prevent falls and occurrence of severe injuries are aspects that relate to medicine and

neurophysiology, biomechanics and robotics, as well as rehabilitation surgery, occupational safety, ergonomics and sport

applications.

The study of postural stability and human movements in general is performed by employing different measurement

methodologies that differ according to the sensors used. These techniques can be roughly classified in direct

measurement methods, based on mechanical, electromagnetic, acoustic, inertial trackers, and indirect optical (imaged-

based) methods1,2

. In the past, direct measurement techniques made use of goniometers, electrical potentiometers

attached to body’s limbs to measure joint angles3. In the last decade, inertial sensors, often integrated with magnetic

sensors or GPS receivers, have become quite common for motion capturing and measuring2,4

. Image-based sensors are

nowadays very popular in both human body modelling and movement analysis1,2,5,6

. Among the wide range of existing

methodologies and technologies, the choice of instrumentations and sensors depends on the requirements of the specific

experimentation. For example, high accuracy is needed for medical applications, as in gait analysis of cerebral palsy

patients3. In sport applications, high data acquisition rate is necessary for better understanding athlete movements and

improving performances4. Character animation in film and game industries, virtual reality, video surveillance are further

sectors where motion capture is required, but in these cases photorealism of the final 3D model is the principal aspect1,2

.

Postural stability is a critical matter as far as the naval and commercial ship sector is concerned, since ship motions

heavily influence human performance and can negatively affect safety of personnel working on deck. In any motion

environment, losing balance is a likely event and potentially its occurrence increases with worsening sea conditions.

Several models were proposed for predicting loosing of balance events, named Motion Induced Interruptions (MIIs) in

naval jargon, or simulating the postural behaviour of a crewmember onboard sailing vessels7. The commonly used model

assumes the individual to react as a rigid body (a dummy) and underestimates the human capacity to counteract external

disturbances (i.e. ship motions). More complex postural models were proposed but necessitate to be verified7. Validating

theoretical models requires many experiments to be conducted, in order to obtain relevant (statistically) data and infer

significant results. A general procedure should require having one or more ships completely available for executing a set

of trials in different environmental conditions, testing many individuals involved in several tasks. This would be an

“optimum” for obtaining data in a real environment, without the limitation on motion magnitude imposed by motion

simulators, but would result in unbearable costs. In this sector, studying human stability and postural movements in the

field means to conduct experiments in locations that differs substantially from any conventional ashore laboratory. A

ship is not only a movable platform, but it is a place where people perform daily working activities and also spend leisure

and resting time. The choice and setting of a motion capture and measurement system cannot disregard these

peculiarities. The need of capturing the motion of an individual standing on a ship during its daily service does not

permit the use of optical systems commonly employed for human motion analysis in indoor, motionless laboratories.

This paper presents an own-developed procedure for measuring human motion onboard vessels. The proposed motion

capture system represents the core of a wider test equipment where different techniques (videogrammetry, inertial

measurement, global positioning system) and instruments (consumer-grade digital cameras, low-cost inertial and

positioning system) were integrated with the ultimate goal of acquiring human motions in a moving environment (ship).

The whole process of planning, designing, calibrating, and assessing the accuracy of the motion capture system is

reported and discussed in the following sections. Results from laboratory tests and preliminary campaigns in the field are

presented.

2. REQUIREMENTS FOR AN OPTICAL-BASED SYSTEM ONBOARD SHIPS

Motion capture systems (or MoCap) involving optical sensors are based on photogrammetric algorithms that allow to

obtain 3D trajectory of one or more signalized points on a moving human body with high precision6. Such systems imply

the use of 2 or more digital recording devices that acquire the scene: the images can be processed in real time (in this

case video devices usually store only image coordinates and no single frames) or in post processing with

photogrammetric algorithms and dedicated software. This measurement technique is today most popular for human

movement analysis in biomechanics, medicine, virtual reality, game industry, thanks to the achievable high accuracy but

also to its flexibility and reduced time for data processing. Commercial optical-based motion capture systems (e.g.,

Motion AnalysisTM

, ViconTM

, QualisysTM

) do not require the human body being equipped with sensors but make use of

both retro-reflective adhesive passive and active markers located in crucial position for capturing the movements of

interest5. Thanks to the absence of wires and circuitry on body, by using passive markers the measured subject can easily

move without any mechanical or physical constraints. Conversely, active marker systems allow better accuracies and

sampling rate5. Moreover, commercial systems need to be employed in appropriate locations or dedicated laboratories,

due to their technical requirements such as power-line, dedicated PC and DAQ board, light controlled environment, etc.

Nowadays, low-cost optical sensors, mainly developed for gaming and entertainment experience, allow retrieving 3D

models of human body and tracking people movements. The growing interest for such “recreational” devices is

witnessed by the development of numerous software (often shared on public websites), applications, and publication of

scientific studies aiming to verify their metric capabilities8.

In order to encompass the strong practical limitations related to the execution of postural stability onboard ships, an ad-

hoc motion capture system was realised. The unavoidable requirement to be fulfilled by the system can be formulated as

followings: “the motion acquisition system must be usable onboard ships while accomplishing their daily mission”.

Furthermore, it should be handled even by only one person in order to minimise the encumbrance onboard. Bearing these

leading principles in mind, the characteristics of the proposed motion measurement system are: (I) affordable cost for

operating in potentially “dangerous” environment (water, wetness, saltiness), (II) ease of use and calibration even in

disadvantageous conditions, as on a ship’s deck; (III) speed in setting up in different configurations and removing in

order to not hinder the normal execution of onboard tasks; (IV) flexibility for being employed on a movable platform, in

cramped spaces, in unfavourable lighting conditions (also in the dark); (V) taking up a minimal amount of space; (VI) to

be portable on different platforms (different kind of ships), without interfering with onboard work. The realised

videogrammetric system was also designed for operating together with and serving as “reference” for inertial sensors for

both human and ship motion measurement. Capturing and analysing human stability on moving vessels have to be

necessarily related to the motion of the platform itself. With the advent of new technologies, flexible and low-cost

instrumentations for measuring the motion of objects are now available. Advances of Micro-Electro-Mechanical System

(MEMS) technology have permitted the development of low-cost inertial sensors (composed of a cluster of

accelerometers and gyroscopes), that can be used for evaluating both ship’s angular motions and linear accelerations7.

Lightweight and portable inertial units can be successfully used for biomechanics, rehabilitation, and also sport

application9,10

. As widely discussed in literature2,10

there are important limitations in the current motion measurement

systems based on MEMS Inertial Measurement Units (IMU). These arise mainly because the inherent drift of the

orientation and position estimates limits long-term stability. The presented optical-based system was also employed for

testing the reliability of the inertial sensors for the application of interest.

3. THE PROPOSED VIDEOGRAMMETRIC SYSTEM

The proposed low-cost videogrammetric system does not require a computer controller (no cables are necessary); each

video camera operates autonomously storing data on an internal memory. The system is designed not to operate in real-

time.

3.1 Hardware overview

The system is composed of three commercial video cameras SonyTM

HDR CX106E with a HD (High Definition) sensor

resolution. The camcorders have a 1920×1080 pixels resolution CMOS (Complementary Metal-Oxide Semiconductor)

sensor and an acquisition frequency of 50fps interlaced mode (50i) for an effective 25fps output video.

Interlaced video is a technique of doubling the perceived frame rate of a video signal: the recording system acquires odd

and even lines (also called upper and lower fields) in two consecutive moments (for PAL standard one field is acquired

in 1/50th

of a second). In order to obtain one full frame in 1/25th

of a second, the two fields are combined into a single

frame. That results in a slight shift effect when the object or the camera is moving while recording (figure 1-left). Several

deinterlacing procedures for solving this misalignment effect are available. The “line doubling” deinterlacing method

was applied to the videos acquired by the employed cameras. This method takes the lines of each interlaced field and

interpolates the missing lines preserving the 50fps acquisition frequency. The result of the deinterlacing procedure is

shown in figure1-right.

Figure 1. Image frame before (left) and after (right) the deinterlacing process.

3.2 System calibration

Commercial motion capture systems and computer-machine vision community refer to system calibration as the

procedure for the simultaneous determination of interior and exterior orientation parameters of the cameras1,11

. For the

proposed system, camera calibration and orientation are performed separately: (I) camera calibration is performed in

laboratory through self-calibration, (II) camera orientation is computed every time the system is set up.

The precision in determining calibration and orientation parameters for the whole set of cameras influences the precision

in measuring 3D coordinates. However, when dynamic events are observed, the synchronization of the cameras also

plays a fundamental role.

- Camera Calibration

For each of the three camcorders, the Interior Orientation parameters (camera constant and principal point position) and

additional parameters12

(radial and decentering distortions) were computed with an off-line13

self-calibration approach14

.

A 3D test-field with circular coded targets (figure 2) was used in order to achieve a sub-pixel accuracy with the LSM

(Least Squares Matching) technique6. The entire procedure was performed with Photomodeler 6 software by EOS

systems Inc.TM

which implements the Brown camera model12

. Camera calibrations were checked in different epochs

verifying their temporal stability for the demanded accuracy15

.

Figure 2. Left and centre: test field used for self-calibration. Right: resulting bundled camera station network.

- System synchronization

A fundamental requirement of a videogrammetric system is the synchronization among the cameras. Professional motion

capture systems adopt dedicated hardware (frame grabber) to which cameras are connected and related management

software which ensure the camera synchronization. For synchronizing the proposed low-cost optical-based system, a

LED (Light Emitting Diode) was used. The LED lighting time (≈106

seconds) can be considered instantaneous,

compared to the camera frame rate. The LED is switched on (off) within the Field of View (FOV) of all the recording

cameras: the first frame in which it changes its state (from off to on or vice-versa) is taken as the synchronization event.

The misalignment error between the three video tracks can be one frame at most.

This systematic synchronization error affects the accuracy of point coordinates as much as the recorded object or the

recording system moves with high speed.

- Camera orientation

The low-cost motion capture system was designed for being flexible: the camera configuration (relative positions) can

differ significantly according to the setting where the system is required to operate. Therefore, the camera orientation has

to be performed every time the system is set up. Since for multi-camera configuration the standard procedure for exterior

orientation by means of pre-signalized targets is not always convenient or feasible, a specific procedure is employed. A

white high power LED is used as an active 3D target: it is moved in all directions within the cameras’ FOV in order to

cover the whole measurement volume.

- Own-developed tracking algorithm

The image observations are collected with an own developed automated tracking algorithm. The tracking software

isolates the pixels covered by the LED, which are characterized by a high brightness in the scene, and tracks their

centroid. The process needs to start with approximate image coordinates of the LED on the first frame. The following

steps are then used to process the subsequent images: (I) a ROI (Region Of Interest) is centred on the centroid

coordinates marked on the previous frame; (II) the image area within the ROI is binarized; (III) a segmentation of the

ROI is performed, the best segment (for shape and dimension) is selected and the image coordinates of its centroid are

computed. Image coordinates of the LED for each frame are used as observation for a bundle adjustment automatically

performed with PhotomodelerTM

via Windows standard DDE (Dynamic Data Exchange) interface. The photogrammetric

model is scaled by using a reference calibrated frame or bar located within the measuring volume. The tracking software

is also able to automatically track, one by one, multiple LEDs or well-distinguishable targets (e.g., circular retro-

reflective markers) moving in the scene.

4. VIDEOGRAMMETRIC LOW-COST SYSTEM VALIDATION

In order to evaluate the performances (accuracy and reliability) of the proposed videogrammetric system, a comparison

with a commercial motion capture system, based on high-speed cameras, was performed. With the two optical system a

reference bar was tracked simultaneously in order to verify that the relative distance between the bar’s ends throughout

the measurement volume was preserved. Table 1 shows the main technical characteristics of the employed systems.

Table 1. Technical specifications of the acquisition instruments.

Qualisys MoCap Low-cost videogrammetric system

Camera Model: QualisysTM

Proreflex MCU 240 Sony HDR CX106E

Sensor Type: CCD CMOS

Resolution: 658×496 pixels 1920×1080 pixels

Pixel size N/A 1.5 μm

Focal Length (nominal) N/A 3.2 mm

N° used cameras 6 3

Frames per second 100 50i (Interlaced)

4.1. Systems set-up for the comparison

The commercial motion capture system used for testing the proposed videogrammetric system is installed in a teaching

hospital for studying human movement kinematics. It is composed of six QualisysTM

Proreflex MCU 240 CCD cameras

centrally controlled by a PC unit that guarantees the system synchronization. Each camera integrates an illumination

system based on infrared (IR) flashing LEDs circularly displaced around the camera lens (Figure 3) that enhances the

brightness of the passive retro-reflective targets used for human movement tests. An internal CPU elaborates the frames

in real time; the image coordinates of the targets identified within the measuring volume are transferred to the PC

without storing the image sequence.

Figure 3. Left: IR-LED illumination for QualisysTM Proreflex camera. Right: low-cost camera illumination system.

The low-cost system was set up in a typical configuration used onboard ships: the three cameras were arranged in

convergent axis configuration with a maximum baseline of 2.5 meters and the object to be tracked located at a maximum

distance of 3 meters. With this setting, an average intersecting angle of 45 degrees and a theoretical spatial precision of

1.7 mm were expected.

Figure 4. Left: 3D retro-reflective target (15 mm diameter) used for the comparison between the commercial and low-cost

videogrammetric systems. Centre and right: effect on centroid determination caused by bad and correct illumination.

A common coordinate system was used for the two videogrammetric systems; it was defined on a steal reference frame

positioned in the measurement volume with the X axis approximately along the largest baseline of the low-cost cameras,

the Z axis upward along the vertical, and the Y axis to complete a left-handed coordinate system.

To validate the low-cost videogrammetric system, targets have to be tracked simultaneously by the two optical systems.

Since the illumination system of the QualisysTM

emits infrared light, a set of three lamps (one per camera) was used for

enhancing the brightness of the retro-reflective targets also within the visible spectrum. The lamps were positioned as

closer as possible to the cameras (Figure 3) in order to avoid that an eccentricity between the light source and the camera

produced a partial illumination of the sphere. This condition could produce an error in computing the target centroid

(Figure 4). The temporal synchronization between the two systems was performed by switching on two LEDs

simultaneously, one in the visible spectrum (visible by the low-cost cameras) and the second one in the IR spectrum

(visible by the ProReflex cameras).

4.2. Performed tests

In order to estimate the accuracy of a videogrammetric system, different kinds of trials, usually classified into two

groups, are suggested in literature5. The first group of tests aim at revealing the deviation of the inter-target distance

observed in each epoch from a reference measured distance, which is invariant. With the second type of tests, the

validation is usually performed by comparing the measured trajectory of a single target (or of a set of targets having an

invariant relative position) with its expected path. The reference trajectory could be carried out by using targets fixed on

a rotating platform manoeuvred with a servo motor16,17

, or by conducting gravity tests18

.

Table 2. Summary of the performed validation tests.

Test Description Calibrated bar (wand) movement

Full Volume5,18

A solid bar mounting two spherical targets at each end

is vertically aligned approximately with the Z (vertical)

axis of the reference coordinate system and then moved

parallel to each axis, throughout the entire

measurement survey volume. The speed of the rod

movement is kept constant and the two targets move at

similar speed.

Pendulum5,18

A rigid pendulum mounting two markers in known

relative positions is suspended to swing in two

orthogonal planes (XZ and YZ planes). The 3D

coordinates of the markers are recorded for at least two

complete oscillations. In this test, the top target is hold

almost steady, while the bottom target swings.

Rotation

The bar is rotated around Y axis with respect to its

centre. The bar angular speed is kept almost constant

and the two targets move at similar angular speed.

In this paper, the results from the first type of tests are presented, since they are of major interest for the conducted study.

As discussed in the following sections, the meaningful measures to be retrieved are angles, that are evaluated on the

basis of inter-marker distances.

For the validation test, two spherical retro-reflective targets located at the ends of a calibrated aluminium bar were

tracked simultaneously by the two optical systems. The inter-marker distance was measured while the bar was shifted

and rotated in order to evaluate the deviation from the reference measure. The bar was previously measured by

photogrammetry; the computed inter-target distance of 932.798 ±0.048 mm was used as reference measurement.

Three different validation tests, summarized in Table 2, were performed: Full Volume5,18

, Pendulum5,18

, and Rotation.

The full volume trial was performed by shifting the targeted bar at almost constant speed in all the directions. A

maximum speed of ≈0.4 m/sec was registered. Figures from 5 to 7 are related to the abovementioned test; a maximum

difference of 3.1 mm from the reference distance is evident (Table 3). The pendulum and rotation trials (Figure 8 and 9)

show that the low-cost system is less stable than the commercial system mainly due to the camera synchronization errors

and the blur effect for the camera exposure time. In particular, for swing oscillations in XZ plane, the maximum

difference between observed and reference distance occurred at a maximum speed of about 1.8 m/sec of the moving

target. Statistical parameters from the validation tests are reported in Table 3; the precision of the low-cost system for

low-dynamic motion is in agreement with the expected performance, satisfying the requirements for human postural

stability analysis onboard. For high-dynamic motion, the results are clearly conditioned by the synchronization error and

exposure time.

Figure 5. “Full Volume” test: bar shifting along X direction.

Figure 6. “Full Volume” test: bar shifting along Y direction.

Figure 7. “Full Volume” test: bar shifting along Z direction.

Figure 8. Left: “Pendulum” test oscillations in YZ plane: Right: “Pendulum” test oscillations in XZ plane.

Figure 9. “Rotation” test comparison: the bar was rotate around Y reference axis.

Table 3. Test Statistics in mm.

X-Shift Y-Shift Z-Shift

Pendulum

YZ-plane

Pendulum

XZ-plane Rotation

QualisysTM MoCap

MEAN measured distance 932.309 932.290 932.289 932.226 932.370 932.619

MEAN – Ref. distance* -0.489 -0.508 -0.509 -0.572 -0.428 -0.179

RMS Error** 0.375 0.272 0.306 0.347 0.471 0.503

MAX Error 1.567 1.424 1.386 1.544 2.588 1.832

Low-Cost System

MEAN measured distance 933.826 933.071 933.659 934.047 934.557 932.019

MEAN – Ref. distance* 1.028 0.273 0.861 1.249 1.759 -0.779

RMS Error** 1.158 0.845 0.567 1.019 4.797 2.403

MAX Error 3.002 1.816 3.068 4.592 9.855 7.580

*Reference distance = 932.798 mm ** RMS Error = Root Mean Square of the difference between the measured and reference distance

5. VIDEOGRAMMETRIC SYSTEM FOR IMU STABILITY CHECKING

Once validated the proposed motion capture system against the commercial QualisysTM

System, several experiments

were performed with the aim of testing the reliability of low-cost inertial units. MEMS sensors were used for measuring

both ship and human movements and form, together with the videogrammetric system, the motion acquisition system for

executing postural stability tests onboard a moving platform.

The inertial sensors employed are two miniature units consisting of three accelerometers, three gyros, three

magnetometers and static pressure (barometer) produced by XSens Technologies (www.xsens.com). For measuring ship

motions, the Motion Tracker MTi-G, integrating a GPS antenna with the inertial unit, was chosen. A lightweight portable

system, made up of a master (Xbus Master) and a Motion Tracker (MTx), was used for measuring the human motion.

Both the sensors provide three linear accelerations, rates of turn and orientation data (Euler angles) referred to an inertial

reference frame. When the GPS signal is available, the MTi-G also supplies position and velocity data.

Two different set of tests were conducted. The videogrammetric system was arranged into an indoor laboratory for

executing the experiments with the MTx (Figure 10), the IMU (Inertial Measurement Unit) for measuring human motion.

In the case of the MTi-G employed for recording the ship motion, the experiments were carried on outside, in order to

test the integration with the GPS.

Figure 10. Left: Indoor laboratory setting for dynamic tests with inertial sensor. Right : The non-magnetic base with photogrammetric

targets for the inertial unit.

A suitable base of support was built of non-magnetic rigid material. The IMUs were fastened on the top of the plate,

where a recognisable path with four coded targets was drawn. The targets were: (I) used for defining on the plate a

reference frame parallel to the IMU-fixed coordinate system; (II) tracked for retrieving both the linear and angular

movements of the plate and, consequently, the sensor motion.

In order to synchronise videos and IMU data, a suitable system was assembled. A mechanic switch simultaneously hits

the inertial sensor and turns on a LED fixed to the IMU housing. The inertial sensor measures the hit as a peak in the

value of the acceleration along the Z axis and, at same time, the videogrammetric system captures the LED switching on:

this point is selected as the synchronization moment for the two different measurement systems.

During the tests, the plate was manually moved by an operator, gradually increasing the dynamics of movement. The

motion of the plate was recorded by the three camcorders and, simultaneously, calibrated data from the inertial sensors

were acquired and sampled at 100 Hz using the factory software. Each trial lasted about 90 seconds and was composed

of three phases: (I) an initial phase in which the plate remained still (with the Z axis in the vertical direction); (II) a

central phase where the dynamics was gradually increasing; (III) a final phase in which the plate was placed again in the

original position.

Figure 11 shows a representative comparison between the Euler angles measured by the MTx (in red) and calculated

with videogrammetry (in blue). The roll and pitch angles from the inertial sensor do not show a significant drift

compared to the videogrammetric output: in the last part of the test, when the plate is placed again in the original

stationary position, they go back to the initial values. The maximum difference between the two measurement techniques

(less than 6 degrees) is registered in the measurement of the pitch angle when the plate moved around the Y axis at a turn

rate of about 4 degrees/sec. The yaw angle provided by the MTx is not sufficiently reliable, since a non-negligible drift is

visible at the end of the test. The reason is that the magnetometers, that provide the measurement of the yaw angle, are

no longer able to compensate the local magnetic field. On the contrary, the yaw angle provided by the MTi-G was

proved to be more stable thanks to the integration with the GPS.

Figure 11. Comparison between the Euler angles measured with videogrammetry and inertial sensor (MTx).

6. HUMAN CENTRE OF MASS CALCULATION

The study of human postural stability requires the measurement of people posture during the task of interest in different

environments, characterized by various movements of the platform. The word posture expresses the orientation of the

body relative to the gravity; as defined by Winter3, it is an angular measure from the vertical. Usually, human postural

stability is analyzed considering the body motion into two perpendicular planes, as shown in Figure 12 left: (I) the

sagittal plane (in red) which bisects the body into left and right halves; (II) the coronal plane (in blue) which divides the

body into antero/posterior or front/back parts. The simplest biomechanical model that describes the postural behavior of

a person standing upright is a single-segment inverted pendulum, having a length equal to the person’s Centre of Mass

(COM) height. The inverted pendulum is a suitable model for embodying the intrinsic instability of human posture,

characterized by continuous oscillation of the COM around the equilibrium position. Representing the human body as an

inverted pendulum is common in literature for modelling the postural behaviour in the sagittal plane7 (Figure 12 centre,

in red). Usually, in the coronal plane more complex models are employed; however, Winter et al.19

proposed to adopt the

inverted pendulum simplification also in this plane (Figure 12 centre, in blue).

Figure 12. Left: anatomical planes in a human: the sagittal plane is shown in red, the coronal in blue. Centre: biomechanical models

for describing human postural stability in the sagittal (red) and coronal (blue) planes. Right: Leaning angle in the antero/posterior

direction (sagittal plane, in red) and in the lateral direction (coronal plane, in blue).

Figure 13. Location of 21 markers in the 14 segment 3D model to estimate total body COM3.

Accurate determination of the COM position requires a fully body kinematics analysis20

, which is usually performed, for

example, in clinical studies. For the present study, evaluating the COM’s height was necessary in order to obtain an

approximated location for placing the inertial sensor and videogrammetric targets. This estimation is important for the

trials onboard ships, setting that is characterised by cramped spaces and where to perform a fully body kinematics

analysis is not easily feasible (sometimes not possible). For this purpose, the model proposed by Winter3 has been used:

the human body is divided into 14 segments, identified by 21 markers (Figure 13). Measuring the 3D coordinates of the

21 markers with a videogrammetric system, the COM position in 3D is determined according to the general formula for

barycentre calculation3,7

.

In a suitable space (indoor laboratory), the proposed capture system was set up and calibrated. A set of 21 adhesive

bandage markers were prepared for each individual to be measured. The markers were drawn as black circles on a white

background, in order to ensure high contrast; this made easier the point measurement phase with the photogrammetric

software PhotomodelerTM

.

Once calculated the COM’s height, each subject was also instrumented with the IMU placed on the reference plate; a

high-power LED was attached to the sensor housing in order to synchronise the inertial sensor with the videogrammetric

system (see Section 5).

Figure 14. Left: 2D displacement of the COM for an 80-second record in quiet standing. Right: out-of-plane COM component from

videogrammetry.

Assuming that the person body is rigid, the COM motion can be tracked recording the displacement of the

photogrammetric reference plate positioned at the measured COM height. Figure 14 left reports the two-dimensional

displacement of the COM for an 80-second record in quiet standing. The trend shows that the main motion component is

in the anterior/posterior direction, in agreement with the hypothesis of modelling a person as an inverted pendulum. In

Figure 14 right, the motion component outside of the coronal plane is illustrated; the trend gives information about the

validity of the “inverted pendulum” hypothesis. If the magnitude of the out-of-plane component increased significantly,

the “inverted pendulum” model would be no longer suitable for describing the individual postural stability. The final part

of the displayed trace shows that the out-of plane COM component is increasing: this corresponds to the moment where

the recorded subject is moving at the end of the postural stability test.

Figure 15. Trend of the leaning angle in the antero/posterior direction: comparison between inertial sensor and videogrammetry.

A measure for quantifying the postural stability of a person standing on a stationary (quiet standing) or moving

(perturbed or dynamic standing) is provided by the sway or leaning angles of the person’s COM from the ankle joint. In

Figure 12 right, the angles in the sagittal and coronal plane are shown in the case of quiet standing. The angles are

labelled and respectively.

These quantities were measured simultaneously by the two motion capture techniques: the inertial sensor and optical-

based system. Figure 15 shows the comparison between the leaning angles obtained by the two measurement

techniques. The trend corresponds to voluntary movements of the tested subject in the antero/posterior direction. The two

outputs slightly differ: the angle from the MTx is systematically larger, in the order of dynamic accuracy (2°) declared by

the manufacturer.

7. POSTURAL STABILITY TEST ONBOARD SHIPS

Onboard trials were conducted to test the proposed motion capture system in the real environment, characterized by

challenging conditions (cramped spaces, adverse lighting conditions, vibrations, etc.). The ship employed for the tests is

a 16.30 meter long wooden fishing vessel named “San Gaetano”, whose activity is concentred within the Bay of Naples

(Italy).

In order to not interfere with the normal activities of the crew members, the trials were performed inside a cabin located

behind the pilot bridge. The camcorders were secured to the ship structures using suitable clamps. Two video cameras

were placed for viewing simultaneously an individual facing forward. The third camera was arranged in order to form a

base with one of the other two devices and record an individual facing laterally. The two baselines between the cameras

were about 1.3 meters; with the object to be tracked at an average distance of 1.25 meters, an average intersecting angle

of 50 degrees and a spatial precision in 3D coordinates of 0.6 mm were expected.

The tested subjects (Figure 16 right) were instrumented with the inertial sensor MTx positioned at the COM height

estimated during preliminary laboratory tests (Section 6). The photogrammetric plate to which the MTx was secured

(Section 5) was modified for improving and making the tracking performed with the own-developed algorithm (Section

3) more robust. Four coloured high-power LEDs were fixed at the corners of the plate and were used as active “color-

coded” 3D markers (Figure 16 left and centre). The MTi-G for ship motions measurement was positioned on the cabin

deck close to the vertical projection of the individual’s COM (Figure 16 right) and the GPS antenna was located on the

top of the pilot bridge.

The two separate inertial sensors were connected to the same PC and the data from both the MTi-G and MTx were

collected via the factory software at a 120 Hz acquisition rate. This allowed to reduce the hardware required onboard and,

at the same time, to synchronise the data from the two inertial units. The method explained in Section 5 was applied to

synchronize the inertial sensors with the videogrammetric system. Several trials were executed: the postural motion of

the tested subjects was measured while maintaining the upright stance facing forward and laterally.

Figure 16. Left and centre: improved videogrammetric plate for the MTx sensor equipped with coloured LEDs for onboard tests.

Right: videogrammetric plate with inertial unit (MTx) at the subject’s COM (red rectangle) and inertial sensor (MTi-G) for ship

motion measurement (cyan rectangle).

The inertial sensors provide the orientation of human body and ship relative to an inertial frame, while the optical-based

system measures the movement of the observed individual in the ship-fixed frame. Figure 17 shows the angles measured

by the three devices for a person facing laterally: (I) and are the ship roll and pitch angles from the MTi-G; (II) and

are the human sway angles in the antero/posterior and lateral directions relative to the vertical (inertial frame) from the

MTx; (III) and are the human sway angles in the antero/posterior and lateral directions relative to the normal to ship

deck (ship-fixed frame) from videogrammetry.

The orientation output for the MTx were opportunely transformed (rotated) and compared with the leaning angles from

videogrammetry. Figures 18 and 19 show the postural sway measured both by the inertial device and with

videogrammetry for an individual facing laterally. The deck roll and pitch angles are also illustrated. The RMS errors

between inertial sensors and videogrammetry resulted to be 2.2 degrees and 0.7 degrees for the and respectively.

Figure 17. Human postural sway on a moving ship deck: the person is facing laterally. Left: in the sagittal plane the human stability is

influenced by the deck roll angle ; the postural sway angle in the antero/posterior direction is relative to the vertical and to the

normal to ship deck. Right: in the coronal plane the human stability is influenced by the deck pitch angle ; the postural sway angle in

the lateral direction is relative to the vertical and to the normal to the deck.

Figure 18. Postural sway angle in the antero/posterior direction relative to the normal to ship deck) from videogrammetry (red) and

inertial sensor (blue); ship roll angle in green. The person is facing laterally.

Figure 19. Postural sway angle in the lateral direction relative to the normal to ship deck) from videogrammetry (red) and inertial

sensor (blue); ship pitch angle in green. The person is facing laterally.

8. CONCLUSIONS

This contribution presented a low-cost multi-camera system for human motion capture onboard ships. The proposed

videogrammetric system was designed in order to be flexible, light and manageable: it is made up of three consumer-

grade video cameras that do not require any central controller or additional cables and can be installed in different

configurations and settings (by means of suitable tripods and clamps). Because of the technical characteristics of the

optical-based system, the acquired data, stored in the internal memory of each camera, are post processed. The paper

covered the main aspects of system calibration, describing the own-developed procedure for synchronizing the three

cameras and algorithms that allow the automatic tracking of multiple well-distinguishable targets (high-intensity LEDs,

circular retro-reflective markers).

The proposed system was tested and compared with a commercial MoCap system in order to verify its accuracy and

robustness in surveying objects moving at different speeds. The performed tests showed an accuracy of about 1:1000 in

measuring distances when the captured object moves slowly, but this sensibly decreased for faster movements, mainly

due to the combination of synchronization error and shutter time. For low movements, the comparison showed that

employing low-cost optical systems is an acceptable compromise between cost, flexibly and achievable accuracy.

The videogrammetric system was also designed for operating together with inertial sensors for both human and ship

motion measurements. Laboratory tests were conducted for testing the reliability of low-cost inertial units by using the

videogrammetric system as reference. For 120 second last tests at most, the inertial data provided satisfactory results, in

terms of stability and accuracy. Several trials were conducted onboard a fishing vessel to test the proposed motion

capture system in the real environment. Despite the practical difficulties related to the specific setting (cramped spaces,

adverse lighting conditions, vibrations, etc.) the motion acquisition system provided satisfactory results. The inertial

sensors are able to provide all the parameters for studying human postural stability onboard. Nevertheless, the integration

of videogrammetry guarantees redundancy and control (reliability) in their estimation and allows a deeper understanding

and reconstruction of the phenomena.

ACKNOWLEDGEMENTS

The authors would like to express their grateful to: M.Sc. Alberto Greco, Dept. of Applied Sciences, for the valued

technical support; M.Sc. Dario Avella, M.Sc. Rosaria Rucco and Ph.D. Valeria Agosti, staff members of the

Neuromechanics Laboratory of “Parthenope” University of Naples (Italy), headed by Prof. Giuseppe Sorrentino, for the

cooperation in the execution of tests with QualisysTM

motion capture system; the crew members of M/T “S. Gaetano” for

the assistance during the onboard surveys.

REFERENCES

[1] D'Apuzzo, N., "Surface Measurement and Tracking of Human Body Part from Multi Station Video Sequences",

Ph.D. Thesis, ETH - Swiss Federal Institute of Technology, Zurich, Switzerland, (2003).

[2] Roetenberg, D., "Inertial and Magnetic Sensing of Human Motion", Ph.D. Thesis, University of Twente, Enschede,

The Netherland, (2006).

[3] Winter, D. A., [Biomechanics and Motor Control of Human Movement], John Wiley and Sons Inc., (2005).

[4] Brodie, M. A. D., "Development of Fusion Motion Capture for Optimisation of Performance in Alpine Ski Racing",

Ph.D. Thesis, Massey University, Wellington, New Zealand, (2009).

[5] Chiari, L., Croce, U. D., Leardini, A., Cappozzo, A., "Human movement analysis using stereophotogrammetry: Part

2: Instrumental errors", Gait & posture 21(2), 197-211, (2005).

[6] Remondino, F., "Videogrammetry for human movement analysis", Ninth international symposium on 3D analysis

of human movement (2006).

[7] Nocerino, E., "Human postural stability onboard ship as seakeeping criterion. Stance control model and procedure

for validating it: a proposal", Ph.D. Thesis, "Federico II" University of Naples, Naples, Italy, (2011).

[8] Menna, F., Remondino, F., Battisti, R., Nocerino, E., "Geometric investigation of a gaming active device", Proc.

SPIE 8085, (2011), (in press).

[9] Ferrari, A., Cutti, A., Garofalo, P., Raggi, M., Heijboer, M., Cappello, A., Davalli, A., "First in vivo assessment of

“Outwalk”: a novel protocol for clinical gait analysis based on inertial and magnetic sensors", Medical and

Biological Engineering and Computing 48(1), 1-15, (2010).

[10] Chang, H., Xue, L., Qin, W., Yuan, G., Yuan, W., "An Integrated MEMS Gyroscope Array with Higher Accuracy

Output", Sensors 8(4), 2886-2899, (2008).

[11] Gruen, A., "Fundamentals of videogrammetry -- A review", Human Movement Science 16(2-3), 155-187, (1997).

[12] Brown, D. C., "Close-Range Camera Calibration", Photogrammetric Engineering & Remote Sensing 37(8), 855-

866, (1971).

[13] Luhmann, T., "Close range photogrammetry for industrial applications", ISPRS Journal of Photogrammetry and

Remote Sensing 65(6), 558-569, (2010).

[14] Fraser, C. S., "Digital camera self-calibration", ISPRS Journal of Photogrammetry and Remote Sensing 52(4), 149-

159, (1997).

[15] Wackrow, R., Chandler, J. H., Bryan, P., "Geometric consistency and stability of consumer-grade digital cameras

for accurate spatial measurement", The Photogrammetric Record 22(118), 121-134, (2007).

[16] Cappello, A., Leardini, A., Benedetti, M. G., Liguori, R., Bertani, A., "Application of stereophotogrammetry to

total body three-dimensional analysis of human tremor", IEEE Transactions on Rehabilitation Engineering 5(4),

388-393, (1997).

[17] Richards, J. G., "The measurement of human motion: A comparison of commercially available systems", Human

Movement Science 18(5), 589-602, (1999).

[18] Cappozzo, A., Della Croce, U., Catani, F., Leardini, A., Fioretti, S., Maurizi, M., et al., "Stereometric system

accuracy tests", In: Measurement and data processing methodology in clinical movement analysis-preliminary,

CAMRAC II Internal Report, (1993).

[19] Winter, D. A., Patla, A. E., Prince, F., Ishac, M., Gielo-Perczak, K., "Stiffness control of balance in quiet standing",

Journal of Neurophysiology 80(3), 1211-1221, (1998).

[20] Eames, M. H. A., Cosgrove, A., Baker, R., "Comparing methods of estimating the total body centre of mass in

three-dimensions in normal and pathological gaits", Human Movement Science 18(5), 637-646, (1999).