AN ACTION-TUNED NEURAL NETWORK ARCHITECTUREFOR HAND POSE ESTIMATION

Giovanni Tessitore, Francesco Donnarumma and Roberto PreveteDepartment of Physical Sciences, University of Naples Federico II, Naples, Italy

Keywords: Neural networks, Grasping action, Hand pose estimation, Mixture density networks.

Abstract: There is a growing interest in developing computational models of grasping action recognition. This interestis increasingly motivated by a wide range of applications in robotics, neuroscience, HCI, motion capture andother research areas. In many cases, a vision-based approach to grasping action recognition appears to be morepromising. For example, in HCI and robotic applications, such an approach often allows for simpler and morenatural interaction. However, a vision-based approach to grasping action recognition is a challenging problemdue to the large number of hand self-occlusions which make the mapping from hand visual appearance tothe hand pose an inverse ill-posed problem. The approach proposed here builds on the work of Santelloand co-workers which demonstrate a reduction in hand variability within a given class of grasping actions.The proposed neural network architecture introduces specialized modules for each class of grasping actionsand viewpoints, allowing for a more robust hand pose estimation. A quantitative analysis of the proposedarchitecture obtained by working on a synthetic data set is presented and discussed as a basis for further work.

1 INTRODUCTION

Over the last few years, there has been a keen in-terest in developing computational models for actionrecognition. Notably, grasping actions are of par-ticular interest for various research areas includingrobotics, neuroscience, motion capture, telemanipu-lation, and human–computer interaction (HCI). Sev-eral works have been proposed in literature which ad-dress the problem of recognizing of grasping actions(Palm et al., 2009; Ju et al., 2008; Aleotti and Caselli,2006). More specifically, as the direct use of the handas input source is an attractive method, most of theseworks make use of wired gloves in order to performinput computation. Moreover, the only technologythat currently satisfies the advanced requirements ofhand-based input for HCI is glove-based sensing. Forexample, recognition can be based on wired glovekinematic information (Ju et al., 2008), or hybrid ap-proaches in which glove information is put togetherwith tactile sensors information (Palm et al., 2009;Keni et al., 2003; Aleotti and Caselli, 2006).

This technology has several drawbacks includingthe fact that it hinders the natural user interactionswith the computer-controlled environment. Further-more, it requires time-consuming calibration and se-

tup procedures.In contrast with this, vision-based approaches

have the potential to provide more natural, non-contact solutions, allowing for simpler and more natu-ral interactions between user and computer-controlledenvironment in HCI, as well as in robotics, wheregrasping actions recognition is mainly framed in thecontext of programming by demonstration (PbD).

There are approaches which make use of ad hocsolutions, like markers, in order to simplify this task(Chang et al., 2007). In general, as reported in (Wein-land et al., 2010; Poppe, 2007), markless vision-basedaction recognition is acknowledged to be a challeng-ing task. In particular, a major problem with graspingactions is the occlusion problem: hand-pose estima-tion from an acquired image can be extremely hardbecause of possible occlusions among fingers or be-tween fingers and object being grasped.

For this reason, a body model approach seemsmore appropriate in this context. A body model ap-proach usually consists of two steps: in a first stepone estimates a 3D model of the human body (in thecase of grasping actions this step coincides with theestimation of hand pose), in a second step recognitionis made on the basis of joint trajectories.

Vision-based hand pose estimation is itself a chal-

358Tessitore G., Donnarumma F. and Prevete R. (2010).AN ACTION-TUNED NEURAL NETWORK ARCHITECTURE FOR HAND POSE ESTIMATION.In Proceedings of the International Conference on Fuzzy Computation and 2nd International Conference on Neural Computation, pages 358-363DOI: 10.5220/0003086403580363Copyright c© SciTePress

lenging problem (see (Erol et al., 2007) for a review).More specifically, addressing such a problem withoutany constraints on the hand pose makes the mappingfrom visual hand appearance to hand configurationvery difficult to estimate. In this work, we presenta neural network architecture composed of a series ofspecialized modules, each one implementing a map-ping from visual hand appearance to hand configura-tion only when the hand belongs to a particular grasp-ing action and is observed from a specific viewpoint.

The paper is organized as follows: in Section 2,we present the functional architecture and its actualimplementation by means of Mixture Density Net-works. In Section 3, we present the experimental setup in which the tests of Section 4 are carried out. Sec-tion 5 is devoted to conclusions and future work.

2 MODEL ARCHITECTURE ANDIMPLEMENTATION

The main idea behind the present approach is to de-velop a set of specialized functional mappings tunedto predefined classes of grasping actions, and to usethese specialized mappings to estimate hand poses ofvisually presented grasping actions. This approach isbased on the assumption that grasping actions can besubdivided into different classes of grasping actions,and that coordinated movements of hand fingers re-sult, during grasping actions, in a reduced number ofphysically possible hand shapes (see (Santello et al.,2002; Prevete et al., 2008)).

Moreover, view-independent recognition was ob-tained by developing view-dependent functional map-pings, and by combining them in an appropriate way.Thus, the system is basically based on a set of special-ized functional mappings, and a selection mechanism.The specialized mapping functions perform a map-ping from visual image features to likely hand posecandidates represented in terms of joint angles. Eachfunctional mapping is tuned to a predefined viewpointand a predefined class of grasping actions. The selec-tion mechanism selects an element of the set of can-didate hand poses. In the next two subsections wewill first provide an overall functional description ofthe system, followed by a detailed description of thesystem neural network structure.

2.1 Functional Model Architecture

The system proposed here can functionally be subdi-vided into three different modules: Feature Extraction(FE) module, Candidate Hand Configuration (CHC)module, and Hand Configuration Selection (HCS)

module. The whole functional architecture of the sys-tem is shown in Figure 1. The functional roles of eachmodule can be described as follows:

FE module. This module receives anR×Sgray-level image as input. The output of the module is a1×D vectorx. TheFE module implements a PCAlinear dimensional reduction.

CHC module. The CHC module receives the out-put x of FE as input. This module is composed of abank ofN×M sub-modulesCHCi j , with i = 1,2, ...,Nand j = 1,2, ...,M. N is the number of predefinedclasses of grasping actions, andM is the number ofpredefined viewpoints. Given the inputx, each sub-moduleCHCi j provides the most likely hand config-uration, in terms of a 1×Ndo f vectort i j . Moreover,eacht i j is associated with an estimation errorerri j ,assuming that the inputx is obtained during the ob-servation of a grasping action belonging to thei-thclass from thej-th viewpoint. Thus each sub-moduleCHCi j , for thei-th grasping action andj-th viewpoint,performs a specialized mapping from visual imagefeatures to hand poses. The basic functional unit ofeach sub-moduleCHCi j is an inverse-forward modelpair. The inverse model extracts the most likely handconfigurationt i j , given x, while the forward modelgives as output an image feature vectorxi j , givent i j .The errorerri j is computed on the basis ofx andxi j .

HCS module. TheHCSmodule receives the out-put ofCHC as input. It extracts a hand pose estima-tion on the basis of estimation errorserri j , by select-ing the hand pose associated with the minimum errorvalue.

Action NView-point M

Action 1View-point M

Action NView-point 1

Action 1View-point 1

t11

Extract bestcandidates

tN1

tNM

Inverse Model

Forward Model

p11(t|x)

Error

t11

x

x11

err11

x

FeaturesExtraction

(FE module)

z

HandConfiguration

Selection

(HCS module)

errN1

errNM

t

Condidate Hand

Configuration

(CHC module)

t1M

err1M

Figure 1: The system is functionally composed of three dif-ferent modules: Feature Extraction (FE) module, Candi-date Hand Configuration (CHC) module, and Hand Config-uration Selection (HCS) module.

2.2 Neural Network Implementation

The FE module implements a PCA linear dimen-sional reduction by means of an autoassociative neu-ral network (Bishop, 1995). This is a multilayer per-ceptron composed ofR× S input nodes,D hiddennodes, andR×Soutput nodes. Both hidden nodes and

AN ACTION-TUNED NEURAL NETWORK ARCHITECTURE FOR HAND POSE ESTIMATION

359

output nodes have linear activation and output func-tion. The input vectors are obtained by linearizingthe gray-level input images of sizeR×S into singlevectors of size 1×R·S. The network is trained toassociate input vectors with themselves by a standardback-propagation algorithm. Once trained, the vec-torsx are obtained as the output of the hidden units.

As theCHCmodule is solely composed byCHCi jmodules, we focus now on a genericCHCi j module,and describe how it can be developed by means of aneural network. As described above, aCHCi j mod-ule is composed of an inverse-forward model pair.Let us firstly consider the inverse-model. This pro-vides a hand configurationt, given a visual imagefeature vectorx. A major mathematical problem aris-ing in this context concerns the ill-posed character ofthe required transformation fromx to hand configu-rations insofar as the same visually presented handcan be associated with various hand configurations.Therefore the mapping fromx to hand configura-tions t is not a functional mapping, and assumes theform of an inverse ill-posed problem (Friston, 2005;Kilner et al., 2007). According to (Bishop, 1995),one can cope with this problem by estimatingp(t|x)in terms of a Mixture Density Network (MDN) ap-proach:p(t|x) = ∑K

i=1ci(x)φi(t|x). Theφi(x) are ker-nel functions, which are usually Gaussian functions

of the form: φi(t|x) = 1(2π)D/2σD

i (x)exp

{

−‖t−µi(x)‖

2

2σ2i (x)

}

.

The parametersci(x) can be regarded as prior proba-bilities of t to be generated from thei-th componentof the mixture. The coefficients of the mixture,ci(x),and the parameters of the kernel functions,φi(t,x)(µi(x) and σi(x) for a Gaussian kernel), depend onthe sensory inputsx. A two-layer, feed-forward neu-ral network can be used to model the relationshipbetween visual inputsx and corresponding mixtureparametersci(x), µi(x) andσi(x). Accordingly, theproblem of estimating the conditional probability dis-tribution p(t|x) can be approached in terms of neuralnetworks by combining a multi-layer perceptron and aRadial Basis Function (RBF) like network. The RBFnetwork will be composed ofNdo f input, K hiddennodes and one output node. The form of basis func-tion is the same as the Gaussian functions expressedabove, with the Gaussian parameters of the first layerset toµi(x) andσi(x), and the second layer weightsset toci(x). Thus, given a previously unseen hand vi-sual descriptionx, one can obtain an estimate of handconfigurationt as the central value of the more prob-able branch ofp(t|x).

The network was trained using a dataset composedof visual feature vectorsxn and hand configurationstn

collected during the observation of grasping actionsbelonging to theCi-th class from thej-th viewpoint,

and by usingE = −∑n{∑ j c j(xn)φ(tn|xn)} as errorfunction. Once trained, the network output, i.e., thecandidate hand configurationt i j , is obtained as thevector µh(x) associated to the highestch(x) value.This operation is achieved by the moduleExtract bestcandidate.

The design of the forward-model, involves amulti-layer perceptron composed ofNdo f input andD output units. The neural network receives a handconfigurationt as input, and computes an expectedimage feature vectorxi j as output. The vectorxi j isthe expected image feature vector corresponding tothe hand configurationt when the hand is observedduring a grasping action belonging to theCi-th classfrom the j-th viewpoint. The network was trained us-ing theRProplearning algorithm by exploiting againa dataset composed of hand configurations and visualfeature vectors, collected during the observation ofgrasping actions belonging to theCi-th class from thej-th viewpoint.

The errorerri j associated with the candidate handconfigurationt i j is computed as the sum-of-square er-ror between the vectorsx andxi j . Finally, given theset of candidate hand configurationsxi j and associ-ated errorserri j , with i = 1,2, ..,N and j = 1,2, ...,M,the candidate hand configuration with the lowest as-sociated error is identified as the output of the wholesystem.

3 EXPERIMENTAL SETTING

We performed two main types of experiments to con-trol the performance of the proposed architecture. Inthe first type of experiments we used different actionclasses (DA-TEST); and in the second type we useddifferent viewpoints (DV-TEST).

A major problem in testing the performance of ahand pose estimation system is to obtain a groundtruth. In fact, in the case of a quantitative analysis,one needs to know the actual hand configuration cor-responding to the hand picture which is fed as input tothe system. This is difficult to achieve with real data.For this reason, we decided to work with a syntheticdataset constructed by means of a dataglove and a 3Drendering software. The dataglove used for these ex-periments is the HumanGlove (HumanGlove, Human-ware S.r.l., Pontedera (Pisa), Italy) endowed with 16sensors. This dataglove feeds data into the 3D ren-dering software which reads sensor values and con-stantly updates a 3D human hand model. Thus, thisexperimental setting enables us to collecthand jointsconfiguration - hand imagepairs.

In the DA-TEST two different types of grasps

ICFC 2010 - International Conference on Fuzzy Computation

360

were used in accordance with the treatment in(Napier, 1956): precision-grasp(PG) andpower-grasp, the latter being also known as whole handgrasp (WH). In performing a power grasp, the ob-ject is held in a clamp formed by fingers and palm;in performing a precision grasp, the object is pinchedbetween the flexor aspects of the fingers and the op-posing thumb. Two different objects were used: atennis ball was used for the WH actions, and a pencup for the PG actions. We collected 20 actions foreach class of actions. In the DV-TEST we renderedthe 3D hand model of the PG grasping actions data,from 9 different viewpoints as reported in Table 2. Inboth DA-TEST and DV-TEST, the capability of theproposed architecture in recovering a hand pose wasmeasured in terms of Euclidean distance between theactual hand poset and the estimated hand poset̂, thatis ENORM =‖ t − t̂ ‖. This measure was reported inother works such as (Romero et al., 2009). However,due to the differences in the experimental settings itis difficult to make a clear comparison between re-sults. For this reason, we decided to compute a fur-ther term, that is, the Root-Mean-Square (RMS) er-ror, in order to obtain a more complete interpretationof our results. The RMS error over a set of actual handposestn and estimated hand posest̂n is computed as:

ERMS=∑N

n=1‖t̂n−tn‖2

∑Ni=1‖tn−t̄‖2 . Heret̄ is defined as the average

of actual hand pose vectors, that is:t̄ = 1N ∑N

n=1 tn.In this way the RMS error approaches 1 when themodel predicts the mean value of the test data, andapproaches to 0 value when the model’s predictioncaptures the actual hand poses. Thus, we expect tohave a good performance when the RMS error for oursystem is very close to zero.

Moreover, we have computed the “selection error”(ESEL) which measures how many times the input im-age belonging to thei-th action class andj-th view-point does not result in the lowest error for the moduleCHCi j . TheESEL error is expressed as percentage offrames (belonging to the action classi and the view-point j) which do not give rise the lowest error for themoduleCHCi j .

The architecture of the model used for these testsis composed of a number ofCHCi j modules depend-ing on the number of different action classes andviewpoints. For eachCHCi j module the MDN com-ponent, implementing the inverse model, was trainedusing different values ofH hidden units andK ker-nels. The feedforward neural network (FNN), imple-menting the forward model, is instead trained withdifferent values of the hidden unit numberL. For bothMDN and FNN only the configuration was consid-ered which gives rise to the highest likelihood, for the

MDN, and to the lowest error for the FNN, computedon a validation set.

Table 1 summarizes the parameters used for bothDA-TEST and DV-TEST.

Table 1: Parameters used for both DA-TEST and DV-TEST.

DA-TEST DV-TESTviewpoints (N) 1 9

Action class (M) 2 1Hidden Nodes (H) From 5 to 10 at step 2

Kernels (K) From 3 to 10 at step 1Hidden nodes (L) From 5 to 10 at step 2Num inputs (D) 30

Input Image dim (R,S) 135×98

Table 2: Sample input images from the 9 different view-points used for DV-TEST.

−80o −60o −40o

−20o 0o 20o

40o 60o 80o

4 DA-TEST RESULTS

In this test we used 20 PG grasping actions and20 WH grasping actions. The architecture consistsof two CHC modules. Each of these modules wastrained on 5 of the corresponding actions (PG or WH)whereas 5 actions have been used for validation, andthe remaining 10 actions for testing.

In Table 3 the mean and the standard deviation oftheENORM error over all hand poses contained in the10 test actions is reported. Moreover the RMS Erroris reported there, insofar as it provides more meaning-ful error information. Finally, we reported the error ofselectionESEL.

As one can see from theESEL error, the systemis almost always able to retrieve the correct moduleCHC for processing the current input image. More-over, the system gives a good hand pose estimationsince the RMS Error is close to zero for both PG ac-tions and WH actions. TheENORM error is also re-ported for comparison with other works. In Table 4sample input images together with their hand estima-tions1 and correspondingENORM errors are reported.One can see that if theENORM error is close to mean

1The picture of the estimated hand configuration is ob-tained by means of the 3D simulator.

AN ACTION-TUNED NEURAL NETWORK ARCHITECTURE FOR HAND POSE ESTIMATION

361

error (reported in Table 3), then the input and esti-mated hand images are very similar.

Table 3: RMS error and the Mean and standard deviation oftheENORM error for test actions belonging to class PG andWH. Moreover theESEL error is reported.

PG WHRMS 0.24 0.12

ENORM (µ±σ) 20.1±27 11.7±19ESEL 3% 9%

Table 4: Sample input images together with the correspond-ing hand estimated images drawn from both PG and WHtest actions.

Actual Estimated Error

PG 118.3

PG 19.7

WH 74.9

WH 12.3

5 DV-TEST RESULTS

The DV-TEST is divided into two phases. In a firstphase, we test the system on the same viewpointwhich it was trained. Note that although test view-points are the same as training viewpoints, when animage frame of a test action is fed as input, the systemdoes not “know” what is the viewpoint correspond-ing to that action, and must retrieve such informationfrom the input. In a second phase, we test the sys-tem on viewpoints that are different with respect tothe ones it has been trained with.

In the first phase, we used 20 PG grasping actionsrendered from 9 different viewpoints. The model ar-chitecture consists of 9CHCi j modules, one for eachviewpoint, with i = 1 and j = 1, . . . ,9. The forwardand inverse models of theCHCi j module were trainedon data related to thej-th viewpoint only. In partic-ular, 5 actions were used for training and 5 other ac-tions for validation. Once trained, the remaining 10actions for each viewpoint were fed as input to thesystem.

Table 5 shows the selection errorESEL together

with the two estimation errors,ENORM and RMS forDV-TEST. One can see that the system is able to re-trieve the right viewpoint for the hand input image(ESEL almost 0 for all viewpoints) and is able to givea reasonable hand pos estimation as confirmed by theRMS error which is close enough to zero.

Table 5: Selection errorESEL, estimation errorENORM, andRMS error for test viewpoints used in DV-TEST first phase.

viewpoint 0o 20o 40o

RMSError 0.02 0.04 0.2ENORM (µ±σ) 9.1±9 12.4±15 25.2±38

ESEL 0% 0% 1%viewpoint 60o 80o −20o

RMSError 0.08 0.13 0.06ENORM (µ±σ) 15.9±24 21.2±30 14.9±21

ESEL 0% 1% 0%viewpoint −40o −60o −80o

RMSError 0.19 0.08 0.03ENORM (µ±σ) 23.6±37 16.5±23 12.1±14

ESEL 0% 0% 0%

In the second phase of the DV-TEST we used fiveCHC modules only, corresponding to the viewpointsat 0, 40, 80,−40, and−80 degrees. The system wastested on the remaining viewpoints at 20, 60,−20,and−60 degrees.

Table 6 reports errors in recovering hand posefrom viewpoints that the system was not trained on:only for viewpoint corresponding to−20 degrees theerror is acceptable, in the other cases, the error is high.

Table 6: Selection errorESEL together with estimation errorENORM and RMS error for all test viewpoints used in DV-TEST second phase.

viewpoint 20o 60o

RMSError 1.69 2.22ENORM (µ±σ) 119±58 137.5±65

viewpoint −20o −60o

RMSError 0.42 1.47ENORM (µ±σ) 59.8±28 114.5±46

6 CONCLUSIONS

The neural architecture described in this paper wasdeployed to address the problem of vision-based handpose estimation during the execution of grasping ac-tions observed from different viewpoints. As statedin the introduction, vision-based hand pose estima-tion is, in general, a challenging problem due tothe large amount of self-occlusions between fingerswhich make this problem an inverse ill-posed prob-lem. Even though the number of degrees of free-dom is quite large, it has been showed (Santello et al.,

ICFC 2010 - International Conference on Fuzzy Computation

362

2002) that a hand, during grasping action, can effec-tively assume a reduced number of hand shapes. Forthis reason it is reasonable to conjecture that vision-based hand pose estimation becomes simpler if one“knows” which kind of action is going to be executed.This is the main rationale behind our system. The re-sults of the DA-TEST show that this system is ableto give a good estimation of hand pose in the case ofdifferent grasping actions.

In the first phase of the DV-TEST comparable re-sults with respect to the DA-TEST have been ob-tained. It must be emphasized, moreover, that al-though the system has been tested on the same view-points it was trained on, the system does not know inadvance which viewpoint a frame drawn from a testaction belongs to.

In the second phase of the DV-TEST an accept-able error was obtained from one viewpoint only. Thisnegative outcome is likely to depend on excessivelyhigh differences in degrees between two consecutivetraining viewpoints. Thus a more precise investiga-tion must be performed with a more comprehensiveset of viewpoints. A linear combination of the out-puts of the CHC modules, on the basis of the producederrors, can be investigated too. Furthermore, the FEmodule can be replaced with more sophisticated mod-ules, in order to extract more significant features suchas Histograms of Oriented Gradients (HOGs) (Dalaland Triggs, 2005). A comparison with other ap-proaches must be performed. In this regard, however,the lack of some benchmark datasets make meaning-ful comparisons between different systems difficult toproduce. Finally, an extension of this model mightprofitably take into account graspable object proper-ties (Prevete et al., 2010) in addition to hand visualfeatures.

ACKNOWLEDGEMENTS

This work was partly supported by the project Dex-mart (contract n. ICT-216293) funded by the EC un-der the VII Framework Programme, from Italian Min-istry of University (MIUR), grant n. 2007MNH7K2003, and from the projectAction Representationsand their Impairment(2010-2012) funded by Fon-dazione San Paolo (Torino) under the NeuroscienceProgramme.

REFERENCES

Aleotti, J. and Caselli, S. (2006). Grasp recognition in vir-tual reality for robot pregrasp planning by demonstra-

tion. In ICRA 2006, pages 2801–2806.

Bishop, C. M. (1995).Neural Networks for Pattern Recog-nition. Oxford University Press.

Chang, L. Y., Pollard, N., Mitchell, T., and Xing, E. P.(2007). Feature selection for grasp recognition fromoptical markers. InIROS 2007, pages 2944 – 2950.

Dalal, N. and Triggs, B. (2005). Histograms of oriented gra-dients for human detection. InCVPR’05 - Volume 1,pages 886–893, Washington, DC, USA. IEEE Com-puter Society.

Erol, A., Bebis, G., Nicolescu, M., Boyle, R. D., andTwombly, X. (2007). Vision-based hand pose estima-tion: A review. Computer Vision and Image Under-standing, 108(1-2):52–73.

Friston, K. (2005). A theory of cortical responses.PhilosTrans R Soc Lond B Biol Sci, 360(1456):815–836.

Ju, Z., Liu, H., Zhu, X., and Xiong, Y. (2008). Dynamicgrasp recognition using time clustering, gaussian mix-ture models and hidden markov models. InICIRA ’08,pages 669–678, Berlin, Heidelberg. Springer-Verlag.

Keni, B., Koichi, O., Katsushi, I., and Ruediger, D. (2003).A hidden markov model based sensor fusion ap-proach for recognizing continuous human graspingsequences. InThird IEEE Int. Conf. on HumanoidRobots.

Kilner, J., James, Friston, K., Karl, Frith, C., and Chris(2007). Predictive coding: an account of the mirrorneuron system.Cognitive Processing, 8(3):159–166.

Napier, J. R. (1956). The prehensile movements of the hu-man hand. The Journal of Bone and Joint Surgery,38B:902–913.

Palm, R., Iliev, B., and Kadmiry, B. (2009). Recognition ofhuman grasps by time-clustering and fuzzy modeling.Robot. Auton. Syst., 57(5):484–495.

Poppe, R. (2007). Vision-based human motion analysis:An overview. Computer Vision and Image Under-standing, 108(1-2):4 – 18. Special Issue on Visionfor Human-Computer Interaction.

Prevete, R., Tessitore, G., Catanzariti, E., and Tamburrini,G. (2010). Perceiving affordances: a computationalinvestigation of grasping affordances.Accepted forpublication in Cognitive System Research.

Prevete, R., Tessitore, G., Santoro, M., and Catanzariti,E. (2008). A connectionist architecture for view-independent grip-aperture computation.Brain Re-search, 1225:133–145.

Romero, J., Kjellstrom, H., and Kragic, D. (2009). Monoc-ular real-time 3d articulated hand pose estimation .In IEEE-RAS International Conference on HumanoidRobots (Humanoids09).

Santello, M., Flanders, M., and Soechting, J. F. (2002). Pat-terns of hand motion during grasping and the influ-ence of sensory guidance.Journal of Neuroscience,22(4):1426–1235.

Weinland, D., Ronfard, R., and Boyer, E. (2010). A Surveyof Vision-Based Methods for Action Representation,Segmentation and Recognition. Technical report, IN-RIA.

AN ACTION-TUNED NEURAL NETWORK ARCHITECTURE FOR HAND POSE ESTIMATION

363