!"#$%&"'(#')**&+,-($"+'()$,*-'(-+',-$"-$,*-(.,$/',-'0.(/,-!'!%1,-'2%#,)'

Frank Desmet, Marc Leman and Micheline Lesaffre

ABSTRACT

This paper presents the results of a study of the movement velocities of a guqin player captured using 4 infrared cameras at 11 different joints for 3 different songs. The study is part of a larger experiment involving participants to move along with the recorded music. The aim of this research is the definition of appropriate statistical strategies as a suitable method to process movement data related to a music performer’s gestures. A combination of statistical techniques is used, with a focus on uncovering measures that reflect coordinated action and intentionality. First, the resulting velocity data of the player was classified using Principal Components Analysis (PCA). Then, anticipation and delay were quantified using Dynamic Time Warping (DTW). Finally, Cladogram Analysis (CA) enabled a qualitative representation of the intentionality of the player. It was found that the measured joints could be classified in subgroups according to the technique of guqin playing. The head of the player is the precursor of the other movements, which are sequentially delayed with respect to the movement of the head. The intentionality could be represented in a phylogenetic tree depicting the distal pattern of the movement of the player. The combination of PCA, DTW and CA enables to classify the movements and identify the relation between the movement velocities at different joints.

Keywords guqin, movement, coordinated action, gesture, PCA, DTW, phylogeny

3 ,-$&*+%)$,*-'

Playing a music instrument is a sophisticated activity, which requires years of training and daily practice. The coordination of different body parts is thereby an essential aspect of the actions that generate and control sound in view of musical expression and communication. Examples of coordinated body activity are: plucking a string of a guitar with the right hand and shortening the length of the string with the left hand, or lip pressure and attack and key control on the trumpet.

The coordination of body parts during music playing can be approached from the viewpoint of body movement, that is, changes in physical position. However, these movements form part of an action-oriented ontology of the musician, that is, a set of patterns that are linked up with frames of reference (body schemata) for motor effectuators, and with representations (body images) of actions, posture, and body, in relation to the musical task (Gallagher, 2005; Henbing & Leman, 2007; Leman, 2008). Seen from that perspective, coordination in music playing is purposeful, and therefore, related to action sequences that aim at having an expressive character, in one way or another (Rolf et al., 2009). The goal-directed nature of such movements, and the fact that they may be accessible as action chunks (‘out of time’) to the human mind makes such movements peculiar. Therefore, it is straightforward to call such a body movements: gestures (Godøy & Leman, 2010).

The study of how humans coordinate their movements has been investigated in view of different tasks (Cordo et al., 1994; Stergiou et al., 2001), including musical tasks (Large, 2000; Li et al., 1999; Wanderley et al., 2005). Thereby, attention has been focused on technical playing (Baader et al., 2005; Shan & Visentin, 2003), on the understanding of spontaneous movement in response to music (De Bruyn et al., 2008; De Bruyn et al., 2009; Desmet et al., 2009; Toiviainen et al., 2009), and on the gross coordination within a broader framework of gesture-based musical communication, in which the sharing of expression is a key element (Naveda & Leman, in press). In addition, musical expression has been studied from the viewpoint of purposive actions (Schogler et al., 2008). However, despite the recent interest in music and body movement, less attention has been devoted to coordination of body parts.

4 5()6!&*%-+'

2.1 Embodied music cognition The present study is carried out within the research framework of embodied music cognition, in which coordination of body parts is considered within a framework of mediations between the mental world of intended actions, experiences, feelings, emotions, expressions, and the world of moving body parts, or energetic patterns (as sound, visual information) (Leman, 2008). A general communicative framework for music is adopted, in which the player is assumed to encode expressive patterns into sound, which the listener decodes on the basis of an embodied engagement with sound patterns.

The focus is on the coordinated movements of a single musician, namely a guqin player, which we study from the viewpoint of its intended action and the communication of expressiveness. We thereby start from kinematic data (velocity) that is obtained from guqin playing by motion caption (i.e. infra red cameras). The

Chinese guqin music has been the topic of a couple of studies that focused on playing gestures (Henbing & Leman, 2007), guqin sound synthesis (Penttinen et al., 2006), and shared expression (Leman et al., 2009).

Accordingly, the following hypotheses are considered: (1) the movements of the joints of a music playing body are driven by a small number of action strategies, (2) the movements of the joints are inter-correlated by lag and delay patterns depending on the musical excerpt, (3) the movement of the head reflects the intentionality of the player and is the precursor of the consecutive movements of the other distal joints. The latter hypothesis implies that the head is the mediator for the translation of the brain activity to the other motor actions of the performer.

The coordination of movements is explored with three techniques, namely, PCA, DTW, and phylogenetic trees. The guiding principle for combining these strategies is that the model should be adequate to answer the research questions being posed. The primary use of PCA is to obtain a non-redundant set of variables for a compact description of certain processes or phenomena (dimensionality reduction) (Daffertshofer et al., 2004). The primary use of DTW is to account for small effects of anticipation and/or delay in the movement (Leman et al., 2009). The use of phylogenetic trees is here explored as a possible way of representing complex movement patterns into a single representational schema (Guastavino et al., 2008).

This paper is further organized as follows: in section 2 the choice for a study involving the guqin is motivated, in section 3 data collection is described, section 4 is devoted to the analysis process, principles and techniques, and in section 5 the results of the study are presented followed by a discussion and conclusion.

2.2 Guqin music The guqin (pronounced ku-chin in English) is interesting for our purpose. The instrument belongs to the family of the zither, a plucked stringed instrument, which consists of a long and narrow hollow wooden box that functions as a sound box and whose upper part functions as a fretless fingerboard on top of which there are 7 strings attached, each about 110 cm long (Penttinen et al., 2006). The way in which guqin music is played, namely by plucking the string with the right hand and moving the finger of the left hand over the string, makes it suitable for a detailed study of the body movement in relation to sound and the musical communicative context. Indeed, the guqin has no bow, and fingering is directly related to sound, as there are no frets to interfere within the sliding. Although the playing technique is rather complex, the sliding-tones in guqin music can be conceived as sound patterns that reflect aspects of the playing movement, without much intermediate technology. Guqin music is thus of particular interest to embodied music cognition research because the encoding of playing gestures into sound patterns proceeds in a relatively direct way, which facilitates the study of possible relationships between movement and sound.

Figure 1. Construction of the Guqin a) top view b) front view

The seven strings are tuned as a pentatonic scale. The basic tuning of the open strings is C2;D2; F2;G2;A2;C3, and D3 from the lowest string (no. 1) to the highest (no. 7). Zhang Jianhua in Beijing constructed the guqin used in this experiment in 1999. The boards are made of fir, and the roughcast is deer horn powder and raw lacquer. Shangy steel-nylon strings are used. For a more detailed description of the construction of the instrument see (Henbing & Leman, 2007).

Nowadays, the Guqin is played on a table and is placed on anti-slip mats. The neck of the instrument is positioned at the right side. The right hand plucks the strings (between bridge and first mark) whilst the left hand is used to press the strings against the top plate of the body and to produce smooth sliding tones on the fretless instruments. The most frequently used compound glides are based on a variation upwards or downwards from the targeted note but there are some less used techniques that have glides on both sides of this note. Finally, there are a number of vibrati, i.e. small variations either side of the main note. In the glides and vibrati great attention is paid to the speed, rhythm and size of the movements. This does not

imply that one of them may be fast or slow or big or small. It may start fast and slow down or vice versa; the acceleration or deceleration must be in the right place and there may be variations in the attack at the beginning and firmness of the close of the technique. A vibrato, for example, may start with a quick, fairly large movement, which gradually slows down and becomes smaller. For more details, see (Henbing & Leman, 2007).

Figure 2. Henbing Li playing the instrument (a) and experimental setup (b)

7 +($('

The dataset was obtained in the context of an experiment reported in (Leman et al., 2009).

3.1 Musical stimuli Three musical fragments (P1, P2 and P3) of a traditional Chinese piece, called “Missing an Old Friend”, were played on the guqin by Henbing Li, an experienced guqin player educated at the Central Conservatory of music in Beijing. Each fragment was about 30 seconds in length. Of the chosen musical fragments, P1 and P2 have a rather fluent and clearly structured melodic line. In contrast, P3 has a more narrative character with a less fluent melodic line.

Figure 3. Spectrograms of the 3 musical fragments

Comparing the spectrograms shows differences of the audio fragments (figure 3). The vertical lines correspond with the plucking on the instrument, followed by the effect of the gliding. In the case of piece 3 there are less plucking onsets than in the other pieces enabling longer gliding. The audio and video of this performance was recorded on hard disk.

3.2 Motion capture A motion capturing system with infrared cameras was used to monitor the movements of 11 different markers attached to different joints of the musician. The system used is Qualisys Motion Capture System, Sweden1, using a sampling rate of 100 Hz.

1 www.qualisys.com

a b

Figure 4. Experimental setup: Position and labeling of the joints

3.3 Raw data The raw data delineate a 3-dimensional vector function of time. As the aim of this study is to investigate and compare movement patterns, bias toward particular directions should be removed and the vector function should be reduced to a single scalar function of time. For each joint/piece combination the magnitude of velocity was calculated over a 250 ms interval and the corresponding sequence plots were generated for visual inspection of the time series.

Figure 5. Example of velocity magnitude time series (P1, Joint 7)

Due to the nature of the measure (magnitude of the velocity) normality cannot be assumed (KS test, p < 0.05), and it is even quite common in datasets of human movement on music (Desmet et al., 2009). Moreover, as the sample size (109) is small, there is a degradation of the correlations between poorly distributed variables. Therefore, variable transformation is a common method to overcome these problems and enhance the results of statistical techniques (Berg et al., 2006). In this study, a square root transformation of the variables was applied to overcome the problem. Consequently, the transformed data allow the assumptions of normality (KS, p > 0.05) and homogeneity of variance (Levene, p > 0.05) to be met. Grubb’s test was used to check for outliers and extreme values. The H0 hypothesis (no outliers present) could be accepted for the transformed data. The results enable the use of statistical methods where the assumptions of normality, homogeneity of variance and absence of outliers are prerequisites.

Figure 6. Example of histograms before (left) and after (right) transformation (Head - Piece 1)

8 (-(./#,#'

In this section we describe the analysis process, principles and techniques for testing the hypotheses. Firstly the use of PCA for classification of movement velocities is reported. Secondly the anticipation and delay analysis based on DTW is considered. Thirdly the application of CA for the exploration of intentionality is worked out.

4.1 Classification of movement velocities A detailed visual inspection of the recorded videos was carried out, using an EyesWeb2 patch that extracted the quantity of motion on a frame-to-frame basis. This tool made visual inspection more straightforward and enabled a qualitative difference-based analysis in which several interesting features of the movements of the player were observed.

First of all, it was observed that the head moves somewhat before the movements of the other joints. There is a focus of the head towards either the left or right hand in combination with an up and down bending. The shoulders move slightly up and down while the elbows move away from and back to the body. The difference between the left and right lower joints (wrist and fingers) is obvious. The left hand is used to produce the glides whilst the right hand plucks the strings. Other videos of guqin playing (taken from YouTube) were also inspected and confirm these observations of the video recordings.

Figure 7. Difference based video analysis revealing the amount of movement of the player body. Frame differencing

shows the parts of the body were movement occurs (right figure).

The qualitative findings enable to distinguish three types of movement, carried out by three distinguished body parts, namely, gliding (with the left hand and arm), plucking (with the right hand and arm) and corporal expression (which is an intention-related activity, basically visible in the movements of the head).

Principal Components Analysis (PCA) was then used to investigate the first assumption that the proposed action strategies can indeed be extracted from the registered movement velocities. PCA is often used to describe the relationships among variables in terms of underlying, but unobservable, quantities called factors (Johnson & Wichern, 2002). In the present context, PCA is used to test the hypothesis that the movement velocity patterns can be divided in a small number of action strategies.

The appropriate tests were used to evaluate the validation of this type of analysis. The tests for criteria for minimum amount of values, absence of biased variables, normality and a sufficient heterogeneity of the data show that PCA can be used. The PCA analysis was done for each piece separately (n = 109, 11 variables). No missing values were present in the dataset. A cutoff of eigenvalues >1 was used as the criterion for component extraction and Varimax rotation with Kaiser normalization was selected. Kaiser-Meyer-Olkin (KMO) and Bartlett’s tests were used to check for sampling adequacy and sphericity. For all 3 pieces, the KMO values are larger than 0.5 indicating a sufficient high proportion of variance caused by underlying factors. The Bartlett’s test results were significant in all cases proving that the correlation matrix is not an identity matrix and that the strength of the relationships is high (p < 0.05). For all 3 pieces, 3 principal components are found with an eigenvalue higher than 1. The explained total variance table shows that 76.25 % (Piece 1), 75.83 % (Piece 2) and 77.61 % (Piece 3) of the variance is indeed explained by the extracted components and that each component contributes with comparable (20 - 30%) proportions to the total variance after rotation (table 1).

2 http://musart.dist.unige.it/EywMain.html

Table 1. Total variance explained before and after rotation (PCA)

Table 2. Rotated component matrices

The rotated factor matrices enable to define the components. The output is made easier for interpretation by removing the clutter of low correlations that are probably not meaningful anyway (< 0.3) and by rearranging the joints and swapping component 1 and 2 of Piece 3 (table 2). The gliding component can be attributed to Factor 1 (Piece 1 and 2) and Factor 2 (Piece 3), the plucking component to Factor 2 (Piece 1 and 2) and Factor 1 (Piece 3) and Factor 3 to the expressive corporal component. The connection between action strategy (gliding, plucking, expressing) and body part (left arm, right arm, head/shoulders) suggests that for Piece 1 and 3, the movement of the right elbow (joint 8) is a combination of the plucking and corporal component. The shoulder movements of Piece 1 and Piece 3 are not related with the gliding or plucking components, while for Piece 2 they are related to the plucking component. The movement of the head is different over the pieces. Piece 1 reveals that the movement of the head is a combination of the corporal expressive and the plucking component. For Piece 3 the movement of the head is only the result of the expressive corporal component. In this context Piece 2 is quite interesting. The movement of the head in this case is only defined by the plucking component (this is also reflected in the shoulder movements).

Figure 8. Scree plot, Component plots and predicted versus observed correlations for the 3 pieces.

A scatter plot of the calculated versus predicted correlations shows that the principal components can be accepted as good predictors for the correlations between the joints. The dashed line represents the ideal maximum correlation (slope = 1). The solid line represents the calculated regression of the predicted versus observed data. The results of the linear regression for each piece are summarized in table 3. The linear model can be accepted (high and significant F-values) and the small deviation of the significant values of the slopes (high and significant t-values) from 1 quantifies the validity of the data reduction into the proposed components.

Table 3. Regression analysis of Predicted versus Observed Correlations

In short, PCA confirms the hypothesis that the movement velocity patterns of the joints are driven by a small number of action strategies for the three different pieces and that action strategies are associated with body parts. PCA enables to reduce the variables into 3 new variables, namely, Corporal/Intentional (Head, Shoulders), Gliding (Left arm) and Plucking (Right arm). The statistical finding corresponds with what is seen in the video recordings. In Piece 2 the player is indeed more focused on the right arm than on the left arm when compared to Piece 1 and 3, and the head is moving nearly all the time back and forward. In the other two pieces the head moves not only back and forward but follows the gliding of the left hand by rotating as well. These findings suggest refinement of the hypothesis in the sense that the intentionality is not to be considered as an action strategy on its own, but rather something that goes along with action strategies (gliding, plucking). ANOVAs of the velocities were used to look for significant differences of the components for the 3 pieces. The movement velocities of the corporal component are the same for all pieces. Head and shoulders were also analyzed separately to inspect possible differences but there were no differences found. The gliding component shows a significant higher difference between Piece 3 and the other two pieces. For the plucking component there is a significant difference between all pieces. Piece 2 has a significant higher value and Piece 3 has a significant lower value when compared to Piece 1.

Figure 9. Mean plots and 95% confidence intervals of the components for the 3 pieces (ANOVA).

4.2 Anticipation and delay analysis From a study on the kinematics of human movements of the arms and hands it is known that large-scale movements are carried out from the proximal joints of the upper limb (shoulder and elbow), whereas small-scale movements are carried out by the most distal joints (elbows wrist and fingers) (Stergiou et al., 2001).

Using the components of the PCA in addition to the corresponding video fragment, the different velocity patterns of the components can be explained in accordance with the way the guqin is played. The results reveal also the presence of lag and delay with respect to the corporal component.

Figure 10. Component velocities time series plots.

The head – shoulders component moves towards left hand while plucking with right hand initializing the tone. The right hand moves toward new position (glide up). The corresponding velocity pattern shows an increase while moving to the new position, when the new position is nearly reached the movement is slowed down to get precisely to the new position. From this position, the speed of the left hand is again increased and decreased to start the compound sequence of the glides (moving around the new position). Before

starting the compound glide the head is moved towards the left arm followed by a left hand movement. The compound glide is followed by the glide down preparing the next glide up. During the glide down movement (left arm) the head moves towards the right hand (this is most clear in Piece 2), passing the energy to the shoulders and the right arm, which is moved away from the instrument.

A more detailed study of anticipation and delay can further access the above description. Dynamic Time Warping (DTW) with correlation optimization (Tomasi et al., 2004) was used to investigate the presence of anticipation and/or delay between the extracted principal components (Keogh & Ratanamahatana, 2005). DTW aligns time series

U = u1,...,um and

V = v1,...,vn by finding the minimum cost path

W =W1,...,Wk , where each

Wk is an ordered pair

(ik, jk ), such that

(i, j)"W means that the points

ui and

v j are aligned. The alignment is assessed with respect to a local cost function

d( i, j ), usually represented as an

m " n matrix, which assigns a match cost for aligning each pair

(ui,v j ) . The path cost is the sum of the local match costs along the path. There are several constraints on

W , namely, (i) bounded by the ends of both sequences, (ii) monotonic, (iii) continuous. The minimum cost path can be calculated in quadratic time by dynamic programming, using the recursion formula:

where

D(i, j ) is the cost of the minimum cost path from

1,1( ) to

i, j( ) , and

D(1,1) = d(1,1) . The path itself is obtained by tracing the recursion backwards from

D(m,n ) . A Sakoe-Chuba window of 5% (± 2.5%) was used to increase calculation speed. All joints were warped against the reference time series (Joint 1, movement of the head) for each piece using the head data as reference. The DTW plots reveal the presence of anticipation and delay in the original series.

Figure 11. Example of DTW path plot (Piece 2). The dashed lines represent the Sakoe-Chuba window. The calculated

warped path reveals deviations from the diagonal.

Figure 12. Example of Anticipation and Delay (Piece 1, 15-19 sec interval)

DTW enables the alignment of anticipation and delay by means of a cost function. The warped path is likely

to deviate more from the diagonal as the dissimilarity between the time series increases. Consequently the closed area formed around the diagonal of the grid is a measure of dissimilarity. The amount of anticipation is related to the fraction of positive values of the cost function whilst the negative portion is a measure for delay. Calculation of the area below and above the diagonal is then used to estimate anticipation and delay. The delay cost can be defined as a measure of the translation of the brain activity in the resulting motor task towards the distal joints. The anticipation cost is then a measure of distal joint feedback to the brain (Hommel et al., 2002; Viviani, 2002).

Figure 13. Amount of Anticipation and Delay with Corporal component as reference

First of all DTW was applied on the 3 components from the PCA analysis using the corporal component as reference. It can be seen that the overall delay cost is larger than the anticipation cost (Figure 12) and that there is a difference between the pieces. In the case of Piece 3 there is little anticipation, Piece 2 shows a higher anticipation for the gliding component whilst in the case of Piece 2 there is more anticipation for both components.

Figure 14. Amount of Anticipation and Delay with Head as reference

From the PCA analysis it was found that in the case of Piece 2 the head and shoulders contribute differently to the other components for the 3 pieces. Therefore DTW was also applied with the head movement as a reference and adding the shoulders as a separate component. The results (Figure 13) was found that for all 3 pieces, the delay is largest for the gliding component, associated with the left arm, which is the most complex of the motor task (glide up, compound glide, glide down). The corresponding values are also comparable with the results of the DTW analysis with the corporal component as reference. The delay for the plucking component shows a difference. When compared to the head the delay of the plucking component is smaller than when compared with the corporal component. Moreover there is a difference over the Pieces: Piece 2 has a lower value of the delay than Piece 1 and 2. Piece 2 shows a difference with the other pieces for the shoulder movement. The anticipation is higher in this case. There is more focus from the head to the right side. Switching the focus between the plucking and gliding components results in more feedback, there may be more need for small corrections of the player in this case.

4.3 Expressive Intention From several studies (Gaser & Schlaug, 2003; Peretz & Coltheart, 2003) it is known that the expressive intention starts in the brain cortex. This activity is then neurologically transmitted to the other parts of the body resulting in actual movement. If there is an expressive intention starting from the brain, then the head movement can be defined as the precursor (‘ancestor’) of the movements of shoulders and arms or other more distant joints. In other words, one can assume that the intention of the player starts with brain activity (energy), which is then transported to consecutive parts of the body (joints) decoding the brain energy into movement.

This complex process can be modeled using a technique called Phylogenetic analysis, which is based on methods in biochemical sequence analysis.3 This technique has been used in de study of behavioral sequences (Robillard et al., 2006) and for the analysis of similarities in African (Toussaint, 2003) and Flamenco rhythmic patterns (Guastavino et al., 2009). Phylogenetic analysis is here used to inspect the consecutive velocities of the joints in relation to the head movement.

Many cladograms are possible for any given set of taxa, but one is chosen based on the principle of parsimony: the most compact arrangement, that is, with the fewest character state changes (synapomorphies), is the relationship accepted in this analysis (Wiley, 1981).

The open source program Phylip was used to generate the phylogenetic trees. In order to perform a phylogenetic analysis the player vector data need to be recoded into a language. A 3-letter alphabet language based on the velocity range was used. Values below 25 % of the range were labeled as W (weak), between 25-75 % as N (normal) and above 75% as H (high). A text file of the resulting alphabet vectors for input in Phylip was generated. In all cases the head is the first vector as it is the supposed precursor for the resulting voluntary movement of the other parts of the body.

Figure 15. Cladograms of the 3 pieces

The branch length (length of the lines) represents the ‘evolutionary’ relationship between the joints. The shorter the branch distances the closer they are related. The length, or number of steps, is the total number of character state changes necessary to support the relationship of the taxa in the tree. The resulting cladograms depend on the musical excerpt (Figure. 12). In the case of Piece 1 both shoulder movements are closest related to the movement of the head. The next 4 taxa correspond with the plucking component (right arm) whilst the left arm (gliding) is found at the end of the cladogram. Piece 2 is more complex. First of all there is a larger distance from the head to the closest related joint (left shoulder) when compared with Piece 1 and Piece 3. The left arm and the right arm form the next branch. Remarkable is the finding that the right shoulder is at the end of the cladogram. This corresponds with the results from the PCA and the video analysis. Piece 3 shows a quite straightforward cladogram showing a close relationship of the shoulders and a clear and distinct separation of left and right arm. These findings suggest that the player uses different strategies for different pieces of music. When inspecting the recordings it is found that in the case of piece 2 the player moves the head from the left arm to the right arm, which is not the case for piece 1 and 3 where

3 This technique is known as cladogram or phylogenetic analysis and is for example used in DNA and protein sequence comparisons and evolution research. Cladistic (evolutionary) thinking was first formulated by Charles Darwin (Notebook B: [Transmutation of species (1837-1838)]. CUL-DAR121. - Transcribed by Kees Rookmaaker. (Darwin Online, http://darwin-online.org.uk/)).

the player concentrates only to the gliding.

Figure 16. Movement of the head from left to right occurring in Piece 2

9 &"#%.$#'(-+'+,#)%##,*-'

The results provide evidence for the stated hypothesis, namely, that (i) the movements of the joints of a music playing body are driven by a small number of action strategies, (ii) the movements of the joints are inter-correlated by lag and delay patterns depending on the musical excerpt, (iii) the movement of the head reflects the intentionality of the player and is the precursor of the consecutive movements of the other distal joints.

Using PCA, it was shown that the movements of the joints can be reduced to 3 components, which correspond with the technique of playing the Guqin, namely, Corporal/Intentional (Head, Shoulders), Gliding (Left arm), Plucking (Right arm). The extracted components reveal differences between the 3 pieces. It was found that the plucking component contributes the most to different action strategies.

Using DTW, the movements of the extracted components reveal lag and delay patterns depending on the musical excerpt. The gliding component is the most complex movement and has the highest amount of delay with respect to the head movement .

Finally, using Cladogram analysis, it was possible to show that the movement of the head reflects the purposive action of the player. Moreover, it was possible to show that the head is the precursor of the consecutive movements of the other distal joints. The latter finding suggests that the head movement mediates the translation of the brain activity to the other motor actions of the performer.

Combining PCA, DTW and Cladogram analysis, detailed information of the movement of a musician can be obtained. Seen from that perspective, coordination in music playing is related to the notion of gesture, that is, movement, but likely also intended (i.e. goal-directed) movement, which is related to actions that aim at generating sounds that are expressive in one way or another. This analytical strategy opens perspectives for further research. It would be interesting to investigate performers with different skill levels and to analyze other instruments. This can also be promising technique in order to define new instruments or the visualize gesture induced audio based on audience movement capture. An on-line DTW algorithm for a visual representation of musical expression based on the performer movements seems to be an interesting approach for further research.

: !"-"&(.'+,#)%##,*-'

In the research framework of embodied music cognition (Leman, 2008) the human body is seen as a mediator between physical environment (where musical actions are seen as body movements) and subjective experience (where musical actions are seen in terms of values, intentions, expressions, feelings). Associated with this is a musical communication model in which the player encodes expressions through corporeal articulations, and in which the listener decodes the transmitted information through corporeal imitations (in an overt or covert way) (Leman et al., 2009; Overy & Molnar-Szakacs, 2009). Although recent studies have contributed to a more detailed picture of the human body as mediator in music playing (e.g. (Palmer et al., 2009), few studies have actually focused on the coordination of different body parts during music playing. As music playing is based on purposive gestures, involving time-dependent musical targets related to the expressive domain, it is generally assumed that the coordination of different body parts plays an essential role (see e.g. (Williamon, 2004)). The viewpoint of the human body as mediator between physical environment and subjective experience provides a framework that may clarify the role of the coordination of different body parts, in particularly in relation to purposive gestures.

However, the present study is limited by the fact that gestures are considered from the viewpoint of a continuous movement. Hence, the resulting observations are related to general action patterns, such as gliding and plucking during a (rather long) melodic phrase. A more fine-grained approach could consider

the segmentation of the continuous movement into gestural units. As shown in (Henbing & Leman, 2007), the most elementary gestures pertain to the movement of a finger from point A to point B on the string. Typically, a tone in guqin music can be conceived as a concatenation of such elementary gestures, and a motive can be conceived as concatenations of tone-related gestures. Seen from that perspective, the elementary gestures, as studied in (Henbing & Leman, 2007), and the general gestures, as studied in the present study, provide two different scopes on gestures, namely, a fine grained one, and a global one. Clearly, further work is needed on the intermediate level, where gestures pertain to expressive action units, such as whole tones or sequences of tones (called motives or figures). In that perspective, (Godøy & Leman, 2010) has suggested that gestures have a hierarchical and nested nature, with target points. This viewpoint complies with our analysis, suggesting that aspects of anticipation and delay, as well as the coordinated action between head and hands could be taken into consideration as indicators of target points. To further determine target points, it may be of interest to consider head movements in relation to the coordination of body movements in more detail, using physiological data (e.g. electromyography).

Although coordinated body movements in musical playing can be considered within a broader theoretical framework of action perception couplings e.g. (Friston, 2010; Schogler et al., 2008), many studies have reported on specific brain activities, such as the motor control of a single part of the body (Fuchs et al., 2000; Haueisen & Knösche, 2001; Watson, 2006), or the brain activity of during a limited non-musical task (Chen et al., 2008; Magescas et al., 2009; Zatorre et al., 2007). Recent studies aim at the continuous monitoring of auditory-motor interactions (feed forward or feedback) during playing (Lindenberger et al., 2009). The ability of this fine-grain correction of individual movements is unique to music and may be controlled by hierarchical auditory-motor frames of reference that guide actions (Feldman & Levin, ).

Important for our study is that specific regions in the brain correlate strongly with movement velocity, independent of movement direction and mode of coordination (Gross et al., 2002; Kelso et al., 1998). Consequently the analysis of movement velocity can provide information of brain activities involved in the process of music performance. Furthermore, understanding the anticipation-delay mechanism is of importance in musical education (learning) in order develop skills to control delay and anticipation and enhance performance. Also in the domain of medical treatment and rehabilitation, such as improving the recovery of patients with movement disorders due to illness (Parkinson) or trauma (stroke patients), knowledge about this mechanism is an important issue.

; )*-).%#,*-'

In studying coordinated action in music playing, it is possible to define statistical strategies to process movement data in relation to a music performer’s gestures. In the present paper, a combination of statistical techniques is used, with a focus on uncovering measures that reflect coordinated movement of body parts in relation to intentionality. The methods involve Principal Components Analysis (PCA) as data-reduction method, then, Dynamic Time Warping (DTW) as a method to quantify anticipation and delay, and finally, Cladogram Analysis (CA) as a means to obtain a qualitative representation of the intentionality of the player. It was found that the head of the player is the precursor of the other movements, which are sequentially delayed with respect to the movement of the head. The intentionality could be represented in a phylogenetic tree depicting the distal pattern of the movement of the player. The combination of PCA, DTW and CA enables to classify the movements and identify the relation between the movement velocities at different joints.

< (6-*=."+!"2"-$#'

REFERENCES

Baader, A. P., Kazennikov, O., and Wiesendanger, M., Coordination of bowing and fingering in violin playing, Cognitive brain research, Vol. 23, No. 2-3, p. 436--443, 2005, Amsterdam: Elsevier Science Publishers, c1992-c2005. Chen, J. L., Penhune, V. B., and Zatorre, R. J., Listening to musical rhythms recruits motor regions of the brain, Cerebral Cortex,, 2008, Oxford Univ Press Cordo, P., Carlton, L., Bevan, L., Carlton, M., and Kerr, G., Proprioceptive coordination of movement sequences: role of velocity and position information, Journal of neurophysiology, Vol. 71, No. 5, p. 1848, 1994, Am Physiological Soc Daffertshofer, A., Lamoth, C. J. C., Meijer, O. G., and Beek, P. J., PCA in studying coordination and variability: a tutorial, Clinical Biomechanics, Vol. 19, No. 4, p. 415--428, 2004, Elsevier De Bruyn, L., Leman, M., and Moelants, D., Quantifying children’s embodiment of musical rhythm in individual and group settings, in Proceedings of the 10th International Conference on Music Perception and Cognition. Sapporo, Japan, 2008 De Bruyn, L., Leman, M., Moelants, D., and Demey, M., Does Social Interaction Activate Music Listeners?, Computer Music Modeling and Retrieval. Genesis of Meaning in Sound and Music, p. 93--106, 2009, Springer Desmet, F., Leman, M., Lesaffre, M., and De Bruyn, L., Statistical analysis of human body movement and group interactions in response to music, Studies in classification, data analysis and knowledge organization. Berlin, Heidelberg: Springer-Verlag,, 2009, Springer Feldman, A. G. and Levin, M. F., The Equilibrium-Point Hypothesis--Past, Present and Future, Progress in Motor Control, p. 699--726, Springer Friston, K., The free-energy principle: a unified brain theory?, Nature Reviews Neuroscience, Vol. 11, No. 2, p. 127--138, 2010, Nature Publishing Group Fuchs, A., Jirsa, V. K., and Kelso, J. A. S., Theory of the relation between human brain activity (MEG) and hand movements, Neuroimage, Vol. 11, No. 5, p. 359--369, 2000, Elsevier Gallagher, S., How the body shapes the mind, 2005, Oxford University Press, USA Gaser, C. and Schlaug, G., Brain structures differ between musicians and non-musicians, Journal of Neuroscience, Vol. 23, No. 27, p. 9240, 2003, Soc Neuroscience Godøy, R. I. and Leman, M., Musical Gestures: Sound, Movement, and Meaning, 2010, Routledge Gross, J., Timmermann, L., Kujala, J., Dirks, M., Schmitz, F., Salmelin, R., and Schnitzler, A., The neural basis of intermittent motor control in humans, Proceedings of the National Academy of Sciences of the United States of America, Vol. 99, No. 4, p. 2299, 2002, National Acad Sciences Guastavino, C., Gomez, F., Toussaint, G., Marandola, F., and Gomez, E., Measuring Similarity between Flamenco Rhythmic Patterns, Journal of New Music Research, Vol. 38, No. 2, p. 129--138, 2009, Routledge Guastavino, C., Toussaint, G., Gómez, F., Marandola, F., and Absar, R., Rhythmic similarity in Flamenco music: Comparing psychological and mathematical measures, Proceedings of the 4th Conference on Interdisciplinary Musicology (CIM'08), 2008 Haueisen, J. and Knösche, T. R., Involuntary motor activity in pianists evoked by music perception, Journal of Cognitive Neuroscience, Vol. 13, No. 6, p. 786--792, 2001, MIT Press Henbing, L. and Leman, M., A gesture-based typology of sliding-tones in guqin music, Journal of New Music Research, Vol. 36, No. 2, p. 61--82, 2007, Routledge Hommel, B., Müsseler, J., Aschersleben, G., and Prinz, W., The theory of event coding (TEC): A framework for perception and action planning, Behavioral and Brain Sciences, Vol. 24, No. 05, p. 849--878, 2002, Cambridge Univ Press Johnson, R. and Wichern, D., Applied multivariate statistical analysis, 2002, Prentice-Hall: New Jersey, Inc. Kelso, J., Fuchs, A., Lancaster, R., Holroyd, T., Cheyne, D., and Weinberg, H., Dynamic cortical activity in the human brain reveals motor equivalence, Nature, Vol. 392, No. 6678, p. 814--817, 1998, [London: Macmillan Journals], 1869- Keogh, E. and Ratanamahatana, C. A., Exact indexing of dynamic time warping, Knowledge and Information Systems, Vol. 7, No. 3, p. 358--386, 2005, Springer Large, E. W., On synchronizing movements to music, Human Movement Science, Vol. 19, No. 4, p. 527--566, 2000, Elsevier Leman, M., Embodied music cognition and mediation technology, 2008, The MIT Press Leman, M., Desmet, F., Styns, F., Van Noorden, L., and Moelants, D., Sharing musical expression through embodied listening: a case study based on Chinese guqin music, Music Perception, Vol. 26, No. 3, p. 263--278, 2009, Univ

California Press Li, L., van den Bogert, E. C. H., Caldwell, G. E., van Emmerik, R. E. A., and Hamill, J., Coordination patterns of walking and running at similar speed and stride frequency, Human Movement Science, Vol. 18, No. 1, p. 67--85, 1999, Elsevier Lindenberger, U., Li, S. C., Gruber, W., and M{\\"u}ller, V., Brains swinging in concert: cortical phase synchronization while playing guitar, BMC neuroscience, Vol. 10, No. 1, p. 22, 2009, BioMed Central Ltd Magescas, F., Urquizar, C., and Prablanc, C., Two modes of error processing in reaching, Experimental Brain Research, Vol. 193, No. 3, p. 337--350, 2009, Springer Naveda, L. and Leman, M., The representation of spatiotemporal music gestures, using Topological Gesture Analysis (TGA), in press Overy, K. and Molnar-Szakacs, I., Being together in time: Musical experience and the mirror neuron system, Music Perception, Vol. 26, No. 5, p. 489--504, 2009, Univ California Press Palmer, C., Koopmans, E., Loehr, J. D., and Carter, C., Movement-Related Feedback and Temporal Accuracy in Clarinet Performance, Music Perception, Vol. 26, No. 5, p. 439--449, 2009, Univ California Press Penttinen, H., Pakarinen, J., Välimäki, V., Laurson, M., Li, H., and Leman, M., Model-based sound synthesis of the guqin, The Journal of the Acoustical Society of America, Vol. 120, p. 4052, 2006 Peretz, I. and Coltheart, M., Modularity of music processing, Nature Neuroscience, Vol. 6, No. 7, p. 688--691, 2003 Robillard, T., Legendre, F., Desutter-Grandcolas, L., and Grandcolas, P., Phylogenetic analysis and alignment of behavioral sequences by direct optimization, Cladistics, Vol. 22, No. 6, p. 602--633, 2006, Blackwell Publishing Rolf, I. G., Godøy, R. I., and Leman, M., Musical Gestures: Sound, Movement, and Meaning, 2009, Routledge Schogler, B., Pepping, G. J., and Lee, D. N., TauG-guidance of transients in expressive musical performance, Experimental Brain Research, Vol. 189, No. 3, p. 361--372, 2008, Springer Shan, G. and Visentin, P., A quantitative three-dimensional analysis of arm kinematics in violin performance, Medical problems of performing artists, Vol. 18, No. 1, p. 3--10, 2003, Hanley & Belfus, inc. Stergiou, N., Jensen, J. L., Bates, B. T., Scholten, S. D., and Tzetzis, G., A dynamical systems investigation of lower extremity coordination during running over obstacles, Clinical Biomechanics, Vol. 16, No. 3, p. 213--221, 2001, Elsevier Toiviainen, P., Luck, G., and Thompson, M., Embodied metre: hierarchical eigenmodes in spontaneous movement to music, Cognitive Processing, Vol. 10, p. 325--327, 2009, Springer Tomasi, G., van den Berg, F., and Andersson, C., Correlation optimized warping and dynamic time warping as preprocessing methods for chromatographic data, Journal of Chemometrics, Vol. 18, No. 5, p. 231--241, 2004, Chichester, Sussex, England: Wiley, c1987- Toussaint, G. T., Classification and phylogenetic analysis of African ternary rhythm timelines, Proceedings of BRIDGES: Mathematical Connections in Art, Music and Science, 2003p. , 25--36, Citeseer Viviani, P., Motor competence in the perception of dynamic events: A tutorial, Common mechanisms in perception and action: Attention and performance XIX, p. 406--442, 2002 Wanderley, M. M., Vines, B. W., Middleton, N., McKay, C., and Hatch, W., The musical significance of clarinetists' ancillary gestures: An exploration of the field, Journal of New Music Research, Vol. 34, No. 1, p. 97--113, 2005 Watson, A. H. D., What can studying musicians tell us about motor control of the hand?, Journal of Anatomy, Vol. 208, No. 4, p. 527, 2006, Blackwell Publishing Wiley, E., Phylogenetics: the theory and practice of phylogenetic systematics, 1981, New York Williamon, A., Musical excellence: Strategies and techniques to enhance performance, 2004, Oxford University Press Zatorre, R. J., Chen, J. L., and Penhune, V. B., When the brain plays music: auditory-motor interactions in music perception and production, Nature Reviews Neuroscience, Vol. 8, No. 7, p. 547--558, 2007, London: Nature Pub. Group van den Berg, R. A., Hoefsloot, H. C. J., Westerhuis, J. A., Smilde, A. K., and van der Werf, M. J., Centering, scaling, and transformations: improving the biological information content of metabolomics data, BMC genomics, Vol. 7, No. 142, p. 1471--2164, 2006, BioMed Central Ltd.