Manipulating the Syncopation in Rhythmic Performances

Syncopalooza: Manipulating the Syncopation in Rhythmic Performances

George Sioros1, Marius Miron2, Diogo Cocharro2 Carlos Guedes1 and Fabien Gouyon2

1 Faculdade de Engenharia da Universidade do Porto

2 INESC-TEC Porto [email protected]

Abstract. In this paper we present a novel method that can effectively manipulate the syncopation in rhythmic performances. The core of the method consists of a set of formalized transformations that can effectively remove and create new syncopations in a binary pattern. One can obtain a multitude of rhythmic patterns with different degrees of syncopation by successively applying the transformations to a single input pattern. The “Syncopalooza” software application implements the method as two Max4Live devices: an audio device that can transcribe drum performances in real time and a MIDI device that manipulates the syncopation of the transcribed patterns. In addition, we evaluate the syncopation transformations and show that they perform as they are intended.

Keywords: rhythm, syncopation, transformations, meter, automatic rhythm generation, real time.

1 Introduction

Syncopation is an essential musical phenomenon found in several and diverse styles of western music, as well as in certain non-western music. It is commonly related to rhythmic complexity and music tension [1, 2 p. 310]. Definitions of syncopation often relate syncopation to the musical meter, describing it, for example, as a contradiction to the prevailing meter [3].

Syncopation is a matter of magnitude and the subject of comparison between rhythmic patterns: patterns are considered to be more or less syncopated than others. This idea led to numerous models oriented towards the analysis and measurement of syncopation [1, 4–6]. However, only few algorithms have been proposed to generate syncopation, such as the one by Sioros and Guedes [7]. To our knowledge, no formalized algorithm has been proposed that allows for the manipulation of syncopation, e.g. a computer algorithm that is able to remove or introduce syncopation in a certain rhythmic pattern in a controlled way.

In this paper we present a method and a software tool for musicians which can manipulate the syncopation of rhythmic performances. In the core of the software lies a set of novel transformations that systematically remove existing syncopations or introduce new ones. The software is developed as two Max4Live devices and is intended to be used as a tool to explore syncopation both in live performance and in off-line composition.

Proc. of the 10th International Symposium on Computer Music Multidisciplinary Research, Marseille, France, October 15-18, 2013

CMMR2013 - 454

2 George Sioros et al. The first Max4Live device is an audio device that takes a real time audio stream of

a drum performance and transcribes it into three streams of event onsets, effectively separating the kick-drum, snare-drum and cymbals.

The second Max4Live device is a MIDI device. It takes either one of the stream of onsets produced by the first device or a user defined pattern and 1) removes all syncopation, and 2) introduces new syncopations, different from the ones in the original stream. The de-syncopation and re-syncopation is achieved by shifting onsets to stronger or weaker metrical positions based on a certain metrical template. Finally, the transformed streams are converted into MIDI note events which can be fed to any standard MIDI synthesizer, generating new rhythmic performances in real time that are always in synchrony with the original one. In order to enhance the metrical and the syncopation feel, we assign dynamic accents to the onsets, either on the beat (thus enhancing the metrical feel) or off the beat (thus enhancing the syncopation feel). The transformations are applied in such a way that the syncopation is removed and re-introduced gradually, resulting in a smooth transition from original pattern, to the de-syncopated one, to the re-syncopated one.

In section 2, we provide a brief overview of the definitions and models of syncopation that are the most relevant to the syncopation transformations introduced in this paper. In section 3, we present the syncopation transformations. In section 4, we evaluate the syncopation transformations. In section 5, we describe the Max4Live software devices. In section 6, we close with the conclusions and future work.

2 Background

Longuet-Higgins and Lee presented a syncopation model that identifies the syncopation in the pairs of notes and the rests or tied notes that follow them [4]. Accordingly, a rest or tied note in a strong metrical position preceded by an event in a weaker metrical position constitutes a syncopation. David Huron, in his study of the American popular music [6], used a similar definition using the term “Lacuna” to describe syncopation. Behind both of the definitions lies the same expectation principle: an event in a weak metrical position is bound to an event in the following strong metrical position and when the expected strong note does not occur the weak one is left hanging [2 p. 295].

Longuet-Higgins and Lee defined syncopation with the aid of metrical weights. The metrical weights correspond directly to the metrical level that each metrical position in a bar initiates (see section 3.1 for a more detailed description). The slower the metrical level the higher the metrical weight. In Fig. 1, the metrical levels of a 4/4 meter are numbered from 0 to 4. The metrical weights are the negative of those indexes (i.e. from 0 to -4). Syncopations are found when the metrical position of a rest or tied note has a higher weight than the position of the preceding note onset.

W. T. Fitch and A. J. Rosenfeld derived a syncopation measure [5] (hereafter LHL) from the Longuet-Higgins and Lee definition. The LHL measure attributes to each syncopation a score that is the difference between the two metrical weights described above. The total syncopation in a rhythmic pattern is the sum of all the syncopation scores. In this paper, when a rhythmic pattern consists of more than one


CMMR2013 - 455

Syncopalooza: Manipulating Syncopation 3

bar, we divide the sum by the number of bars in order for all the results to have comparable scales.

David Temperley developed a model for exploring the uses of syncopation in rock [8], which bares strong resemblance to both, the LHL definition of syncopation and Huron’s expectation principle. In his model, syncopation is defined as the displacement of events from metrically strong positions to preceding weaker ones. The syncopation transformations that we introduce in this paper are inspired by this model.

3 Syncopation Transformations

In order to remove the syncopation or introduce new syncopations in a stream of onsets, we constructed a metrical template that describes the meter in which the rhythmic patterns are heard. The template consists of pulses with different metrical strength values that represent the alternating strong and weak beats commonly found in a musical meter (Fig. 1, top). The pulses constitute a metrical grid which quantizes the time positions of the onsets in a rhythmic input. The pulses belong to the various metrical levels according to the time signature, e.g. the quarter notes or the sixteenth notes found in a 4/4 bar. This way, a continuous stream of onsets is converted into a binary string consisting of 1s and 0s.

Two kinds of transformations are then applied by shifting the onsets (1s) to empty pulses (0s) similarly to the shifts described by Temperley [8]. First, the onsets are shifted forward to strong metrical positions to remove the existing syncopations. Each shift of an event results in a new pattern with decreased syncopation. When all events have been shifted to their non-syncopating positions the resulting de-syncopated pattern has no syncopation. Second, new syncopations are introduced. To that end, the onsets found in strong metrical positions are anticipated to weaker positions. This time, each shift of an event results in a new pattern with increased syncopation. When all “strong” onsets have been moved to weaker positions the resulting re-syncopated pattern has maximum syncopation.

Pulses that belong to slower metrical levels are considered to be stronger, so that the de-syncopation process moves events from fast metrical levels to slower ones. Conversely, the re-syncopation process “pushes” events to the faster metrical levels. Although the initial and final patterns of each transformation, i.e. the original, the de-syncopated, and the re-syncopated, are uniquely defined, the path that leads from one to the other depends on the order in which the events are shifted.

Finally, dynamic accents are applied to the onsets. Dynamic accents can enhance the perception of meter for the onsets that do not syncopate and at the same time increase the feeling of syncopation for those onsets that are anticipated. The accents are applied to the de-syncopated pattern but once an onset receives an accent it carries it to any position it is shifted to, in the original and the re-syncopated version as well as in any step in between. In that way, accents can be introduced both on the strong beats as well as in weaker pulses when syncopating.

At a secondary level, “extra” syncopation can also be introduced without the need to shift onsets. Instead, onsets that are found in weak metrical positions (in the fast metrical levels) could be accented, while at the same time, onsets in the following


CMMR2013 - 456

4 George Sioros et al.

strong metrical positions (in the slower metrical levels) would be attenuated. This secondary mechanism for generating syncopation is especially useful in cases where moving onsets is either impossible or ineffective due to the specific characteristics of a rhythmic pattern. Such patterns have usually a high density of events per bar, like drum rolls where all metrical subdivisions, even the fastest one, are fully occupied by onsets. In that case, one can reveal the underlying meter and syncopation only by accenting certain events.

We begin the detailed description of the transformations, in section 3.1, with the construction of the metrical template needed for the transformations. We continue, in section 3.2, with the algorithm for removing the syncopation in binary patterns. In section 3.3, we present the algorithm for generating syncopation by shifting the onsets. Finally, in section 3.4, we describe how dynamic accents can be used to enhance the metrical feel and generate syncopation.

3.1 Metrical Template.

The metrical template is constructed automatically for each meter and tempo that the rhythmic patterns are heard in. The meter determines a hierarchical structure like the one found in the Generative Theory of Tonal Music by Lerdahl and Jackendoff [9]. Such a structure can be constructed by successively subdividing the bar into faster metrical levels. For example, the 4/4 meter can be subdivided first into two half notes, then each half note into two quarter notes, each quarter note into two eight notes and so on, until the fastest metrical subdivision is reached. The metrical levels are indexed by numbers, referred to as metrical indexes for easy reference, starting with the number 0 for the slower one and increasing as one goes to faster levels. The process of successively subdividing the bar results in alternating weak and strong pulses, forming a pattern characteristic of each time signature. The stronger a pulse is, the

Fig. 1 Top: Example of the construction of a metrical template for a 4/4 meter. The meter is successively subdivided generating the 5 metrical levels. The metrical strength (grey rectangles) of each metrical position corresponds to the metrical levels it belongs to. Bottom: A continuous stream of onsets is quantized to the fastest subdivision of the metrical template. A binary pattern is created containing 0s (empty boxes) and 1s (filled boxes) according to the position of the onsets (vertical lines).


CMMR2013 - 457


slower the metrical level it belongs to (lower index), so that weak pulses belong to faster metrical levels (higher indexes). In Fig. 1 an example of such a metrical template is given. A detailed description of an automatic way of generating such a metrical template for any given time signature can be found in [7].

The duration of each metrical subdivision depends on the tempo, e.g. the quarter note at 100bpm has duration of 600ms. The lower threshold for the duration of a metrical subdivision has been estimated in several studies to be roughly around 100ms [10–12]. The fastest metrical subdivision that we included in the metrical template is the fastest subdivision above that threshold.

We intend to use this template as the basis for the transformations that will remove or create syncopation in binary patterns. The definition of syncopation, as originally formulated by Longuet-Higgins and Lee [4] or Huron [2 p. 295] and also adopted here, attributes syncopation in the pair of an onset on a weak pulse with the following silent strong pulse. One can, therefore, talk about and define the duration of the syncopation as the duration between those two pulses. The duration of the syncopation depends, on one hand, on the metrical levels of the corresponding pulses, and on the other hand, on the tempo. One question that arises is: how does the duration of the syncopation affect the feeling of syncopation?

Think of the example of Fig. 2, where a quarter note is tight to the next half note. Although, according to the definition of syncopation, this is a clear case of syncopation, the actual syncopation felt by the listener depends on the tempo at which this pattern is performed. For example, at 100 bpm (q.n. = 600ms) the pattern is not felt as syncopated since all events fall on the beat. Nevertheless, if the pattern is played at double speed (q.n. = 300ms), it is felt as strongly syncopated. The actual beat level would now be that of the half note, i.e. on the 1st and 3rd quarter notes, so that the 2nd and 4rth would be off-beat. The metrical hierarchy in both cases is the same, what changes is the salience of each metrical level.

Fig. 2: Syncopation at slow metrical levels. Top: Metrical template for a 4/4 meter. The most salient pulse is notated i) at 100bpm with black rectangles and ii) at 200bpm with grey rectangles. Bottom: A syncopated rhythmic pattern. Syncopation is depicted with the curved arrow line.

Fig. 3: Example of how tempo affects the metrical template. Very fast metrical levels (below 100ms duration) and very slow ones (above 1s duration) are disregarded.


CMMR2013 - 458

6 George Sioros et al. This aspect of syncopation and its relation to pulse salience has not been

thoroughly studied. However, based on our experience, we understand that syncopation involving only slower metrical levels is not felt as strong. Metrical levels have a peak salience in the region between 500ms – 1s [11]. Syncopations with duration longer than that of the most salient level seem to be felt significantly less strong. The LHL syncopation measure [4] takes this effect into account indirectly, by giving to the syncopation that involves adjacent metrical levels a relatively small weight. Other syncopation measures, such as the weight-note-to-beat distance (WNBD) [13], relate syncopation directly to the beat level, ignoring all slower metrical levels.

In order to take the above effect of tempo into consideration in the construction of the template, we employ a similar approach to the WNBD by essentially “chopping off” the slow metrical levels. We chose the level that falls in the range between 500ms and 1s as the slowest metrical level represented in our structure. For example, in the case of a 4/4 meter at 160bpm (q.n. = 375ms), the metrical template of Fig. 1 will become as in Fig. 3. At this tempo, only 3 metrical levels survive with corresponding durations of 750ms (0), 375ms (1) and 187.5ms (2).

3.2 De-syncopation.

The de-syncopation algorithm is a direct consequence of the operational definition of syncopation followed in this article [2 p. 295, 4]: syncopation in the binary patterns is found in those events that are placed in a relatively weak metrical position (faster metrical levels, higher metrical indexes) and are not followed by an event in the following stronger position (slow metrical level, low metrical index). In order to de-syncopate such an event, we shift its onset to a position where it no longer syncopates, i.e. to the following strong metrical position that was initially empty, thus evoking the feeling of syncopation.

An onset is not shifted if one or more other onsets block its way. This rule ensures that the order that the events are performed is preserved. In such cases, the onset that is blocking the shifting is also syncopating and should be de-syncopated first. For that reason, the de-syncopation process is a recursive process that stops when all syncopating events have been moved to positions where they no longer syncopate.

The above rule might seem at first glance as having no purpose. After all, the onsets in the binary string are indistinguishable from one another, so that the order in which they are played back is unimportant. Nevertheless, we will later attribute to the onsets dynamic accents. Each one will receive a specific amplitude value and it will carry it in all the generated variations. Thus, onsets will be eventually distinct and therefore their order must be preserved. Moreover, this way the process is more general and can be expanded to include other types of phenomenal accents, such as pitch or timbre changes, or other properties of the events that can make them distinct.

Fig. 4 illustrates the process through an example. In the example the pattern is scanned from right to left and each event is examined and shifted when appropriate. However, the order in which the onsets are examined and shifted can be, in principle, freely chosen. The first event found is denoted by ①. It belongs to the fastest metrical level in pulse 2. Since pulse 3 belongs to a slower metrical level and it is not occupied by another event, ① is pushed forward to pulse 3. The next event is found


CMMR2013 - 459


in pulse 7. The following pulse belonging to a slower metrical level is pulse 9 but it is already occupied by an event, so that event ② in not to be shifted as it is not syncopating. The 3rd event found in pulse 9 already belongs to the slowest metrical so that it has no pulse to be shifted to. The process re-starts from the beginning. Event ① is now found in pulse 3 and is pushed to pulse 5. This last shift ends the process since there is no longer any event in the pattern that syncopates. The following pseudo code illustrates the basic steps in the de-syncopation algorithm:

Repeat

For each onset N

P = position of N

Find next stronger position R

if positions P to R are empty

Shift onset N to position R

Output Pattern

Until no onsets can be shifted

The described process guarantees that the resulting pattern will not syncopate. Something that is not true for other similar transformations. In some cases, one could de-syncopate off-beat events by shifting them to the preceding on-beat position instead of the following one. However, the two directions are not equivalent. Our operational definition of syncopation attributes the syncopation feel to the lack of an event in the following strong position. The forward shifting of events always guarantees the resulting pattern will not syncopate, as the “empty” strong pulse that was causing previously the syncopation will now receive an onset.

3.3 Re-syncopation.

The re-syncopation process takes a binary pattern and creates syncopations by anticipating the events found in strong metrical positions. It is essentially the reverse process of de-syncopation. Onsets in pulses that belong to strong metrical positions (slow metrical levels, low metrical indexes) are shifted to preceding pulses belonging to weaker metrical positions (faster metrical levels, higher level indexes). Again, as in

Fig. 4: Illustration of a de-syncopation process. The metrical template shown corresponds to a 4/4 meter at 100bpm (q.n. = 600ms). Each onset of the binary pattern is numbered in a circle. The de-syncopated process is shown as arrows.


CMMR2013 - 460


the de-syncopation process, the order in which events are heard should be preserved, so that, an onset cannot be shifted when it is blocked by some other onset(s) found “on the way” of the shift (in between the initial and destination pulse, or at the destination pulse).

Contrary to the de-syncopation process, in the re-syncopation there might be more than one option for where to shift the onsets in question. In other words, there might be more than one way to syncopate at a certain metrical position. A strong pulse might be preceded by more than one weak pulse, e.g. a pulse that belongs to the quarter note level is preceded by a pulse at the eighth note level and another one at the sixteenth note level, both belonging in faster metrical levels than the initial one (Fig. 5 A). The decision of where such an onset should be shifted to is a “stylistic” one, as all the options result in syncopating patterns but each one syncopates in a different way.

Several “stylistic” rules can be invented in order to take such decisions in an automatic way. Stylistic rules can be “learned”, e.g. by analyzing a collection of rhythms in a certain music style. The de-syncopation process can serve as an analysis that associates metrical positions with particular syncopation shifts. In that way, one can apply the particular syncopation of a music style in any rhythmic pattern.

Stylistic rules can also be created by a performer, even during the performance. In that case the rules must be simpler and should have a generic form that can be easily controlled.

We decided to include in our algorithm one such rule that we believe can work well in a variety of music styles and can be controlled in real time through a single parameter. Our stylistic rule allows shifts only between certain metrical levels according to the difference of the metrical indexes. For example, if the difference is set to 1, onsets would be shifted to the immediately faster metrical subdivision, while if it is to 2, onsets would be shifted two metrical subdivisions faster. An onset found at the quarter note level, in the first case, would be shifted to the preceding eight note metrical position. In the second case, it would be shifted to the preceding sixteenth note position. In cases when the level difference results to a metrical level faster than the fastest metrical subdivision included in the metrical template, then this fastest metrical subdivision is used disregarding the rule. For example, if the level difference is set to 2, an onset at the eight note level should be shifted to the preceding 32nd note. If the template only goes as fast as the sixteenth note level, then the onset should be shifted to the preceding sixteenth note, disregarding the rule.

The re-syncopation process comprises two main stages: 1) the pattern is scanned and the events that are not blocked and can be shifted are marked alongside with their possible shifts, and 2) the actual shifts are applied. In Fig. 5 B we present an example illustrative of the process. In the example, the pattern is scanned from right to left.

Fig. 5 A: Example of multiple choices of syncopation shifts for a single onset. B: Illustration of a re-syncopation process. Each onset of the binary pattern is numbered in a circle. The re-syncopated process is shown as arrows.


CMMR2013 - 461


However, the order in which the onsets are examined and shifted can be, in principle, freely chosen. The stylistic rule is set so that the metrical level difference should always be equal to 1. In the first stage, the pattern is scanned without making any transformations. First, event ③ is examined. It belongs to metrical level 0 and therefore should be shifted to pulse 6 that belongs to level 1 (difference = 1). Since pulse 6 is already occupied, event ③ cannot be shifted. Second, event ② is examined. It can be moved from pulse 6 (level 1) to pulse 5 (level 2). The shift is stored but not applied until all events have been examined. Event ① is examined. It can be moved from pulse 4 (level 0) to pulse 2 (level 1). It can be further moved, in a second syncopation step, from pulse 4 to pulse 1 (level 2). The examination of event ① ends the first stage of the process. In the second stage, the transformations are applied. First, event ② is shifted to pulse 5. Then, event ① is shifted to pulse 2. Finally, event ① is moved to pulse 1.

The two separate stages are used to ensure that syncopations created by shifting one onset are not “cancelled” by shifting another onset to the silent pulse of the syncopation. For example, if event ③ was shifted to pulse 6 in the final pattern, then the syncopation of event ② would be weakened.

The following pseudo code illustrates the basic steps in the re-syncopation algorithm:

T = Array of Transformation

For each onset N

P = position of N

Repeat

Find previous weaker position R

if positions P to R are empty

Store transformation [P R] into T

Set P = R

Until P does NOT change

end

For each element [P R] in array T

Apply transformation [P R]

Output Pattern

end

3.4 Dynamic Accents.

We introduce dynamic accents to the binary patterns for two main reasons. First, the metrical feel is enhanced by accenting the onsets that exist on strong pulses. Second, we can syncopate onsets that could not be shifted during the re-syncopation process by moving the accents to off-beat onsets. Each onset is assigned a specific dynamic accent in the range between 0 (meaning a silent onset) and 1 (loudest possible). In that way, the binary string that is created based on the drum transcription becomes now a


CMMR2013 - 462


sequence of amplitudes, which can take values other than just 0s and 1s. In the paragraphs that follow we will describe first, how the onsets are assigned amplitudes that enhance the metrical feel, and second, the generation of syncopation by manipulating the amplitudes of the onsets.

Enhancing the Metrical Feel. The metrical feel is enhanced when phenomenal accents coincide with the metrical accents, i.e. when changes in the loudness, pitch etc. coincide with the alternation of strong and weak pulses characteristic of the meter. Based on this principle, each onset is assigned an amplitude value relative to the metrical strength of the pulse that the onset belongs to. However, the same onset can be found in different pulses in the different versions of the patterns, i.e. the original, the de-syncopated or the re-syncopated versions. Syncopation is described as a contradiction to the underlying meter [10], which means that the version of our binary pattern that represents best the meter would be the fully de-syncopated one.

We assign the amplitudes to the de-syncopated positions of the onsets. The amplitudes are assigned to the onsets themselves and not to the pulses, so that when the onsets are shifted to different positions in the various syncopated versions, they still carry their accents from the de-syncopated version. For example, an onset found at the beat or quarter note level in the de-syncopated version will still be a loud onset with a high amplitude value even when it is shifted to a faster metrical level. In that way, the feeling of syncopation is intensified. The syncopating event is distinct from other events that are found in fast metrical levels but that do not syncopate. Those non-syncopating events receive lower amplitudes since they belong in fast metrical levels in the de-syncopated version.

The following mathematical formula is used to calculate the amplitudes of onsets:

𝐴(𝑖) = 𝐶 ( ) (1)

where A(i) (range: 0 to 1) is the amplitude of event i, C is a parameter that controls the contrast between strong and weak metrical positions, pi is the pulse that event i belongs to in the de-syncopated version and L(pi) is the metrical level of that position. The parameter C ranges between 0 (only the slowest metrical level survives) and 1 (all metrical levels receive the same amplitude).

Generating Syncopation. Dynamic accents can also be employed to create syncopation when events cannot be shifted to faster metrical levels. Imagine the extreme case where all pulses, even at the fastest metrical subdivision, have an onset. That would be, for example, the case of drum rolls. In such cases, onsets cannot be shifted since there are no empty pulses available and the only way to reveal the meter or contradict it is through accents. In order to introduce syncopation to such a pattern, we stress certain off the beat onsets while attenuating the ones that follow them on the beat. For those onsets, the phenomenal accents would not coincide with the metrical accents, and that mismatch of phenomenal to metrical accents would be felt as syncopation.

Looking closer at the metrical template and the way it is constructed, one can see that strong and weak pulses are always alternating. Strong pulses, which belong to slow metrical levels, are always followed by weak pulses, which belong to faster metrical levels. Noticing that and in order to syncopate certain onsets, we shift the


CMMR2013 - 463


metrical template to the left by one pulse before assigning them amplitudes. In effect, the dynamic accents get anticipated. In that way, the events occurring off the beat would be accented relative to the events on the beat.

The generation of syncopation based on dynamic accents is a secondary mechanism, needed only when the onsets themselves cannot be shifted. When the onsets can be shifted, they carry with them the dynamic accents, so that onsets and accents are anticipated together.

The example in Fig. 6 illustrates how dynamic accents are used. Metrical accents are calculated according to the metrical template (A). They are, then, assigned to the onsets of a de-syncopated pattern (B and C). Finally, syncopation is generated in two ways (D): 1) by shifting the onsets which also carry their accents, and 2) by shifting the metrical accents instead of the onsets themselves.

4 Evaluation of Syncopation Transformations

In this section we evaluate the syncopation transformations described in 3.2 and 3.3 in practice, i.e. with actual musical material and not just theoretical binary patterns. We applied the transformations on a collection of MIDI drum loops. Then, we used the LHL syncopation metric described in section 2 to measure the syncopation in the original loops and in three more versions that have undergone the syncopation transformations: 1) the de-syncopated version, 2) a re-syncopated version with 30% of the total transformations applied, and 3) a re-syncopated version with 70% of the total transformations applied. The onsets that were shifted in the re-syncopated versions were chosen randomly from the total of onsets that could be shifted in each pattern.

The MIDI collection was taken from [14] and consists of 160 drum loops of different music styles. The MIDI files were segregated into three separate MIDI streams according to the MIDI note numbers that correspond to the kick drums, snare drums and hi-hats. All 480 loops (3 x 160) were in a 4/4 meter and quantized to the 16th note grid and they were converted to binary patterns before applying any transformation. In all cases we used a metrical template that corresponds to a 4/4

Fig. 6: Example illustrating the process of assigning dynamic accents. A: Metrical accents calculated based on the metrical template. B: A binary, de-syncopated pattern. C: Dynamic accents on the events in their de-syncopated positions. D: Re-syncopated pattern. The event on pulse 2 is now shifted to pulse 1 (dashed arrow). The events on pulses 3 and 4 cannot be shifted. Instead, the accents on pulse 3 and 4 are shifted (solid arrow).


CMMR2013 - 464


meter at 100bpm. First, they were de-syncopated according to the process described in 3.2. Second, the resulted de-syncopated patterns were re-syncopated twice (with 30% and 70% of the transformations applied) according the process described in 3.3. The “stylistic” rule was set to allow syncopation shifts only to two metrical subdivisions faster. We used the LHL metric to measure the syncopation in all versions of each of the 480 loops. An overview of the results is shown in Fig. 7.

The de-syncopation and re-syncopation algorithms performed as expected. The de-syncopation algorithm removed completely the syncopation in all 480 binary patterns (syncopation score = 0). This was expected as the de-syncopation process directly corresponds to the way onsets are matched to following silent pulses in the definition of syncopation in the LHL algorithm.

The re-syncopated process increased gradually the syncopation. The 30% transformation increased the syncopation in the majority of the patterns (in 469 out of 480 patterns). The 70% transformation increased further the syncopation for the majority of the patterns (in 445 out of 480 patterns). No pattern had less syncopation in the 70% than in the 30% transformations. In a few exceptions, the patterns had the same syncopation score for the de-syncopated and 30% transformation (11 patterns), or for the 30% and 70% transformations (35 patterns) (horizontal lines in Fig. 7). In 6 patterns out of the total 480 the re-syncopation transformations did not create any syncopation.

The exact relation between the syncopation scores and the percentage of the applied transformations depends on two factors: 1) the number of onsets per bar in the pattern that can actually be shifted and are not blocked by other onsets and 2) their metrical positions. When only a small number of events can be shifted there are accordingly few steps between the de-syncopated version and the fully re-syncopated version (100% of the transformations applied). The number of events that can be shifted in the pattern strongly depends on the density of events per bar. For very low density (e.g. only a couple of events per bar) or very high density (when almost all metrical positions are occupied) the onsets available for syncopation are very few. The evaluation shows that the re-syncopation transforms increases the syncopation with each step, even when the available steps are very few. In those cases, the dynamic accents are employed to increase further the syncopation.

Fig. 7: Syncopation measurements using the LHL algorithm on 4 different versions of the patterns found in the 480 binary patterns. Each circle represents a single measurement. The grey lines connect the measurements for the 4 different versions of each pattern. As the number of measurements is large, several lines naturally overlap. The darker the line is, the more the overlaps.


CMMR2013 - 465


5 The Syncopalooza Max4Live Devices

Our system comprises two Max4Live devices: 1) an audio device, loaded in an audio track and 2) a MIDI device loaded in a MIDI track. The audio device transcribes a real time audio stream of a drum performance into three separate streams of onsets, corresponding to the kick drum, snare drums and hi-hats. The MIDI device can receive these streams of onsets and stores one bar of one of the streams. It then transforms it according to the syncopation transformations of section 3. After applying the transformations it plays back the resulted pattern as a sequence of MIDI note events that can be fed to any standard virtual or external synthesizer. Alternatively, the MIDI device can be used independently of the audio device. In that case, the user can directly input a rhythmic pattern into the device, using the user interface.

In section 5.1, we provide a description of the audio device together with the drum transcription algorithm. In section 5.2, we describe the MIDI device.

5.1 Audio Device: Drum Transcription

Our audio device is driven by live audio comprising drum sounds. It assumes analyzing the input signal and labeling each detected event as either bass drum (BD), snare drum (SD) or hi-hat cymbal (HH), from a mixture of bass drums, snare drums, hi-hat cymbals and toms. Each time an event is detected, it is “broadcasted” to any other Max4Live device loaded in the same Ableton Live Set.

We are using the drum transcription algorithm described in [14]. Along the event detection and k-nearest neighbor classification, the method proposes an additional instance filtering stage (IF), which filters events in the corresponding bands for BD, SD and HH, before feeding them to the classifier.

The transcription assumes four steps. First, onset candidates are detected with a high frequency content onset detector (HFC). The window size is 11.6 ms and the hop size is 2.9 ms. Thus, this initial onset detector has the advantage that it can detect onsets quickly, with a delay of approximately 11 ms from the original onset.

Second, for every event detected by the HFC onset detector, a set of spectral features are extracted for BD, SD and HH.

In the third step, three additional sub-band onset detectors with higher time resolution are used to filter onsets for each of the BD, SD or HH classes. The audio is filtered with a low-pass filter with a cut-off frequency of 90 Hz for BD, a band-pass filter with a central frequency of 280Hz and a bandwidth of 20 Hz for SD, and a high-pass filter with a cut-off frequency of 9000 Hz for HH. Three complex onset detection functions are detecting onsets in these frequency bands. Because the window size is 23.2 ms and the hop size is 5.8 ms, these additional onsets are always coming after the initial HFC onset. An HFC onset must be matched with one of the sub-band onsets, in order to be fed to the classifier.

In the fourth step, the filtered events and the corresponding feature vectors are fed to three K-nearest neighbor classifiers, which label each event as a member or a non-member of the three classes, BD, SD and HH.


CMMR2013 - 466

14 George Sioros et al. Several factors make the drum transcription task challenging. First, the system

must transcribe events as fast as possible. The IF stage optimizes the performance with respect to real time-constraints by combining a fast onset detector with a sub-band onset filtering. Furthermore, we are dealing with a mixture of sounds which might overlap and not occur exactly at the same time, causing the system to over-detect events. The IF stage described in [14] is useful in eliminating instances which can be classified incorrectly. The evaluation showed that using the IF stage decreased the number of incorrectly labeled events and increased the performance.

5.2 MIDI Device: Syncopation Transformations

Our MIDI device manipulates effectively and in real time the syncopation in a binary pattern. It systematically removes the syncopation and creates new syncopations in the pattern according to the transformations described in section 3.

The user interface of the device is shown in Fig. 8. There are two main control areas on the device: 1) the step sequencer interface where the user inputs a pattern and, and 2) an area for the manipulation of the syncopation consisting mainly of a slider controlling the syncopation and a dial controlling the dynamic accents. In the following paragraphs we describe in more detail each of these controls.

Step Sequencer Interface. The user can input a pattern directly on the user interface or “record” the last pattern received through the MIDI input or from the drum transcription device. The length of the pattern is always a single bar. The device constructs automatically a metrical template that corresponds to the time signature and tempo settings of the Ableton Live Set. The template defines the quantization grid, thus the number of pulses or steps in the pattern. After a pattern is recorded from the received onset stream, it is treated in exactly the same way as if it was input manually by the user. The user can edit the input patter at any time and in real time. The transformations are applied automatically and playback continues without interruption.

The transformed pattern is shown directly below the input pattern. The transformed pattern is the one that is been played back and output as MIDI notes. The user cannot directly interact with this pattern since it is controlled by the syncopation controls described in the following paragraph.

Fig. 8: The MIDI Max4Live device that manipulates the syncopation of a binary rhythmic pattern. Dashed rectangles and arrows indicate the main controls


CMMR2013 - 467


Syncopation Controls. The user can control the syncopation transformations through a stepped slider and a dial found on the user interface (Fig. 8). The slider controls the syncopation transformations and the dial controls the amplitude of the dynamic accents.

The syncopation slider ranges between the de-syncopated version (top position), and the fully re-syncopated version (bottom position). In between those positions, there exist versions of the rhythmic pattern in which only a certain number of the available shifts is applied. Each step of the slider corresponds to a single shift of an onset (section 3.2 and 3.3). In-between the top (de-syncopated) and bottom (fully re-syncopated) positions, the original input pattern is found. Its position varies according to how many steps are required to de-syncopate the input pattern (moving towards the top) or to fully re-syncopate it (moving towards the bottom). For the onsets that cannot be shifted, their dynamic accents are shifted as described in section 3.4. When one reaches the bottom position, all onsets and accents have been shifted to their syncopating positions. The transformations are performed based on an automatically constructed metrical template.

The order in which the onsets or accents are shifted is random. However, the order is forced so that the transformations are passing through the original input pattern. The user can totally randomize the order of the transformations through the button above the syncopation slider. In that case, the input pattern will not appear on the slider; the available versions of the rhythmic pattern will be generated by gradually applying the re-syncopation transformations on the de-syncopated version in a totally random order, until a fully re-syncopated version is reached.

A secondary control, a number box found on the top left corner of the device, controls the syncopation style. It controls the stylistic rule described in section 3.3 and corresponds directly to the difference of metrical levels that the onsets are allowed to be shifted to.

The dial found on the left bottom corner of the device controls the dynamic accents applied. When the dial is turned to the left, no accents are applied and all onsets have maximum amplitude. When the dial is turned fully right, the amplitude of the onsets is relative to the metrical strength of their positions.

6 Conclusions / Future Work

In this paper we presented a set of formalized transformations that can manipulate the syncopation in binary patterns in an automatic way. We also presented an algorithm for applying dynamic accents in order to enhance the metrical and syncopation feels. The transformations can effectively remove and create syncopations by shifting the onsets according to a metrical template constructed for each time signature and tempo.

The transformations were evaluated using the Longuet-Higgins and Lee syncopation metric [4, 5]. We showed that the transformations can be used to gradually remove and create new syncopations.

We used the above transformations in a Max4Live MIDI device that manipulates in real time the syncopation of rhythmic performances, through a simple user interface. The device is accompanied by a Max4Live audio device that can transcribe in real time a drum performance into streams of drum onsets. The two devices can be


CMMR2013 - 468


used by musicians together or independently, in order to explore the uses of syncopations during music performances or in offline compositions.

We believe that syncopation styles can be modeled. To that end, we propose to use the syncopation transformations to “learn” syncopation styles and reapply them to a rhythmic pattern. The de-syncopation process can associate particular syncopation shifts to specific metrical positions. For example, the de-syncopation of a collection of patterns in a certain music style can build probabilistic rules about how onsets are shifted in that style. Applying the “learned” syncopation style on a pattern would then consist in shifting onsets according to the rules. The applications of the syncopation transformations to music and syncopation styles are left to be explored in the future work of our group.

The Max4Live devices and externals are available for download at our website: http://smc.inescporto.pt/shakeit/

Acknowledgments. This work is financed by the ERDF – European Regional Development Fund through the COMPETE Program (operational program for competitiveness) and by National Funds through the FCT – Fundação para a Ciência e a Tecnologia within project «PTDC/EAT-MMU/112255/2009-(FCOM-01-0124-FEDER-014732)».

7 References

1. Gómez, F., Thul, E., Toussaint, G.T.: An experimental comparison of formal measures of rhythmic syncopation. Proceedings of the International Computer Music Conference. pp. 101–104 (2007).

2. Huron, D.: Sweet anticipation: music and the psychology of expectation. The MIT Press (2006).

3. Randel, D.M.: The Harvard Dictionary of Music. Belknap Press of Harvard University Press, Cambridge, MA (1986).

4. Longuet-Higgins, H.C., Lee, C.S.: The rhythmic interpretation of monophonic music. Music Perception. 1, 424–441 (1984).

5. Fitch, W.T., Rosenfeld, A.J.: Perception and Production of Syncopated Rhythms. Music Perception. 25, 43–58 (2007).

6. Huron, D., Ommen, A.: An Empirical Study of Syncopation in American Popular Music, 1890-1939. Music Theory Spectrum. 28, 211–231 (2006).

7. Sioros, G., Guedes, C.: A formal approach for high-level automatic rhythm generation. Proceedings of the BRIDGES 2011 – Mathematics, Music, Art, Architecture, Culture Conference. , Coimbra, Portugal (2011).

8. Temperley, D.: Syncopation in rock: a perceptual perspective. Popular Music. 18, 19–40 (1999).

9. Lerdahl, F., Jackendoff, R.: A Generative Theory of Tonal Music. The MIT Press, Cambridge (1983).

10. Repp, B.H.: Rate Limits of Sensorimotor Synchronization. Advances in Cognitive Psychology. 2, 163–181 (2006).

11. Parncutt, R.: A perceptual model of pulse salience and metrical accent in musical rhythms. Music Perception. 11, 409–464 (1994).

12. London, J.: Hearing in Time. Oxford University Press (2012). 13. Gómez, F., Melvin, A., Rappaport, D., Toussaint, G.T.: Mathematical measures of syncopation.

Proc. BRIDGES: Mathematical Connections in Art, Music and Science. pp. 73–84. Citeseer (2005). 14. Miron, M., Davies, M.E.P., Gouyon, F.: An open-source drum transcription system for pure

data and max msp. The 38th International Conference on Acoustics, Speech, and Signal Processing. , Vancouver, Canada (2013).


CMMR2013 - 469

Manipulating the Syncopation in Rhythmic Performances

Documents

Transcript of Manipulating the Syncopation in Rhythmic Performances