Music Technology and Alternate Controllers for Clients with ...

151

doi:10.1093/mtp/miz006Advance Access publication July 18, 2019Music Therapy Perspectives, 37(2), 2019, 151–168

© The Author(s) 2019. Published by Oxford University Press on behalf of American Music Therapy Association. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Asha Ward is a research engineer currently undertaking an engineering doctorate (EngD). Her work focuses on making accessible musical instruments and interfaces for those with complex needs.

Dr. Tom Davis is a lecturer in Creative Technology with an interest in instrument building and improvisation.

Dr. Ann Bevan has been a registered nurse for over 40 years and now teaches children’s nursing, she is an action researcher.

Address correspondence concerning this article to Asha Ward, Centre for Digital Entertainment, Media and Communication, Bournemouth University, Fern Barrow, Poole BH12 5BB, UK. E-mail: [email protected].

Music Technology and Alternate Controllers for Clients with Complex Needs

ASHA WARD, EngD Centre for Digital Entertainment, Bournemouth University, Poole, UK

TOM DAVIS, PhD Creative Technology Department, Bournemouth University, Poole, UK

ANN BEVAN, PhD Child Health Nursing, Bournemouth University, Bournemouth University, Poole, UK

ABSTRACT: Music technology can provide unique opportunities to allow access to music-making for clients with complex needs. While there is a growing trend of research in this area, technology has been shown to face a variety of issues leading to underuse in this context. This literature review is a collation of information from peer-reviewed publications, gray literature, and practice. Focusing on active music-making using new types of alternate controllers, this re-view aims to bring together information regarding the types of tech-nology available, categorizes music technology and its use within the music therapy setting for clients with complex needs, catalogues work occurring within the field, and explores the issues and poten-tials surrounding music technology and its use in practice.

Keywords: music, music therapy, review literature, sound, technology

Introduction

This article provides a review of music technology used in music therapy, examining the types of technology currently used for sound exploration and music-making by both music therapy practitioners and researchers. The review highlights key developments within the field of music technology, with a focus on applications in music therapy for those with complex needs. The data gathered are a collation of peer reviewed and grey literature (institutional reports), alongside firsthand re-search carried out by the first author as part of an engineering doctorate.

The primary focus of this review is technology for active music-making, with a focus on alternate controllers that pro-vide control and potential for expression through sound and

music. For this review, active music-making is defined as playing instruments or actively exploring sound through inter-action with technology. The term complex needs refers to a spectrum of cognitive, physical, and/or sensory impairments or disabilities that can lead to individuals experiencing min-imal movement, disordered movement, altered states on con-sciousness, and/or no verbal communication (Magee, 2012).

Literature Review Strategy

Keyword searches of Google Scholar, Google, and the Bournemouth University Library Catalogue were used for article se-lection. The following keywords were used: music technology for music therapy, new interfaces for musical expression, music tech-nology and special education needs, music technology SEN, and music technology complex needs. The Nordoff Robbins Evidence Bank (2014) (specifically account no. 16) was also consulted as well as Research and Resources for Music Therapy 2016 (Cripps, Tsiris, & Spiro, 2016). This selection of papers expanded as litera-ture was reviewed. Papers were scanned for their significance as they pertained to the use of technology, both novel or off-the-shelf, with users with complex needs for active music-making or sonic exploration, or that they featured details of such technologies in use, or that they explored issues around and/or reviewed usage of such technology in use. Some grey literature was also consulted (Department for Education, 2011; Farrimond, Gillard, Bott, & Lonie, 2011; Ofsted, 2012; O’Malley & Stanton Fraser, 2004) as this provided a different perspective on technology usage in practice.

Context

Music technology reviews have been undertaken to address the use of music technology by music therapists (Cevasco & Hong, 2011; Clements-Cortes, 2013 Crowe & Rio, 2004; Hahna, Hadley, Miller, & Bonaventura, 2012; Knight & Krout, 2017; Knight & Lagasse, 2012; Magee, 2006; Magee & Burland, 2008; Streeter, 2007; Whitehead-Pleaux, Clark, & Spall, 2011), and to outline the aims of national music education plans within government policy (Department for Education, 2011; Ofsted, 2012), and to guide government policy (Farrimond et al., 2011). Magee (2014) edited a volume of articles drawing together uses of music technology in thera-peutic and health settings. These authors highlighted the im-portance of music technology, the types of music technology used, where technology is useful, and how technology could be improved to break down barriers and allow access to

Dow

nloaded from https://academ

ic.oup.com/m

tp/article/37/2/151/5533486 by guest on 06 July 2022

mailto:[email protected]?subject=

Music Therapy Perspectives (2019), Vol. 37152

music-making for those with complex needs. This literature review aims to take another step in this discussion, by further organizing this information and providing a timeline of devel-opment to the current state of the art.

The use of music technology for clients with complex needs in music therapy settings is broad, drawing from a variety of fields. Technology usage combines elements of human com-puter interaction (HCI), music therapy, music psychology, music education, and music technology. The scope of litera-ture featured in this review reflects this, with a focus on the ways technology can be used to increase access to active music-making opportunities for clients who are unable to use traditional musical instruments.

The review begins with an overview of the evolution of tech-nology, electronic music technology (EMT), and EMT for those with multifaceted needs. Following this are sections on the computer as the bridge, new developments, available tech-nology, and music technology used in music therapy practice, including issues related to these practices.

Evolution of Technology

Within the last two decades, technology usage in general has become increasingly accessible (Cevasco & Hong, 2011). This has proliferated into all areas of everyday life as devices such as smartphones, smart watches, tablets, and portable computers become more ubiquitous both inside and outside of the home (Nagler, 2011). In the broader view, this explo-sion of technology has led to changes in how we interact with music (Misje, 2013). Software has become more accessible, with programs for music-making such as Garageband coming preinstalled on every Apple Macintosh computer (Cevasco & Hong, 2011). Handheld devices like the iPad offer free apps for portable music access and making, both increasing the opportunity for the everyday user to create and share con-tent, and the opportunity to have a “band-in-a-box” with a variety of functions being achievable with one device. The uses and accessibility of technology has expanded the pos-sibilities for users with complex needs to participate in active music-making.

Electronic Music Technology

Electronic music technology is a wide-reaching branch of technology that has progressed over the last 30 years. Developments in hardware and software, and creation of new instruments that utilize technology, have pushed boundaries forward both in terms of the creation and production of music. While the history of the development of EMT, specifically elec-tronic instruments, is beyond the scope of this review, over-views can be found in literature (Bongers, 2000; Challis, 2009; Paradiso, 1997) along with proceedings from dedicated con-ferences like new interfaces for musical expression (NIME). Comprehensive introductions to the world of NIMEs can be found (Lyons & Fels, 2015) and books such as those by Miranda and Wanderley (2006), offering a reference point for the con-trol of sound using technology and issues surrounding the cre-ation of new instruments (Ward, Woodbury, & Davis, 2017).

EMT for Clients With Complex Needs

EMT that increases accessibility for clients with complex needs has been defined as a range of tools and devices which

are able to generate musical sounds through electronic, digital, or mechanical means (Magee, 2012). Definitions in-clude: “any equipment, device, or method that systematically fosters independent functioning, including the production of or response to music” (Crowe & Rio, 2004, p. 283); “the ac-tivation, playing, creation, amplification, and/or transcription of music through electronic and/or digital means” (Hahna et al., 2012, p. 456); and “a wide range of devices, equipment and software, spanning amplification devices, MIDI (musical instrument digital interface) devices and instruments, com-puter software, assistive devices, brain computer interfaces, as well as electronic musical instruments and specialist inter-faces such as switches and sensors” (Burland & Magee, 2013, p. 179). These types of technology, and their relationship to music therapy clinical practice, began being discussed in the late 1980s (Krout, 1987) and early 1990s (Krout, 1992), with the use of music technology for those with complex needs also being covered in popular music magazines (Thomas, 2012).

While the term EMT covers a wide range of technology to facilitate musical interaction within the field of music therapy (Magee & Burland, 2008), instruments created with tech-nology are often called digital musical instruments (DMIs) in the field of new interfaces for musical expression (NIME) (Poupyrev, Lyons, Fels, & Blain, 2001). Since the 1980s there has been a rapid expansion of EMT use with the field of music therapy (Whitehead-Pleaux et al., 2011) and many DMIs have been developed both commercially and for re-search purposes. DMIs can be aimed at a typical population or can be bespoke. Bespoke instruments use technology or combinations of technology to allow an individual access to active music-making. These technologies can include hard-ware and/or software.

DMI Development and Musical Genres

The expansion of DMIs can be in part attributed to the strong links between the type of music that has historically been created using music technology—that of hip-hop/rap and later, electronic dance music (EDM) (Crooke, 2018)—and client preference—particularly of children and adolescents—of those receiving music therapy (Viega, 2015). Music ther-apists striving to explore genres that heavily resonate with their clients have used hip-hop and rap and its strong links to the transformational power of music to “facilitate group experience and catalyse personal and social transformation” (Lightstone, 2012, p. 41). Themes explored within hip-hop culture and EDM, which took root from the oppressed ori-gins that forged the development of genres, can be likened to a “universal language” (Lightstone, 2012, p. 40), particu-larly when working with children and adolescents (Hadley & Yancy, 2012).

Types of Technology

Categories of Technology

Crowe and Rio (2004) completed a comprehensive his-torical literature review of technology and its implication in music therapy practice and research for music therapy edu-cation. From this, they organized the types of technology into taxonomical structures. They concluded that there are seven types of technologies: “(a) adapted musical instruments, (b)

Dow


ic.oup.com/m


Music Technology and Alternate Controllers for Clients with Complex Needs 153

recording technology (c) electric/electronic musical instru-ments, (d) computer applications, (e) medical technology, (f) assistive technology for the disabled and (g) technology-based music/sound healing practices” (Crowe & Rio, 2004, p. 291). These categories are exhaustive in terms of covering all types of technology used in the music therapy environment but do not focus on those used primarily for active music-making. The categories also include technology that is used for 1) ana-lysis and logging of data about client progress, 2) creating and hearing listening material, and 3) medical technology that in-volves sound waves.

The rate of change within the technological environment of electronic music has meant that there have been several de-velopments since the creation of these categories that are diffi-cult to place within them, and there is technology that crosses between them. Magee’s classifications (2006, 2012) reflect more up-to-date inclusions of self-contained music creating devices (such as synthesizers), music listening devices (such as mp3 players like the iPod), digital handheld music devices or DHHMDs (Nagler, 2011) (such as the iTouch app and iPads), and music games (such as Guitar Hero).

Krout (2015) subsequently provided four categories of electronic music resources based upon those that have been reported as being useful in music therapy clinical practice, and are also affordable and available. These included general or stand-alone products, computer software, electronic key-boards, and tablet computers (e.g., iPads).

Digital Handheld Music Devices

The category of DHHMDs have become ubiquitous aids for aiding in music-making. DHHMDs have become part of everyday life in an unanticipated convergence of technologies that has altered the practice of music therapy in a profound manner (Nagler, 2011). DHHMDs offer a new class of music listening experiences, predictive selections, and active music-making without need for therapeutic interventions. These de-vices have become multitasking musical companions allowing complex musical ideas to be created and shared without tech-nical training (Nagler, 2011). New technologies such as tab-lets featuring touch screens, particularly the iPad, have created a shift toward screen-based mobile music-making. The touch screen allows direct interaction to music apps using intuitive motion (Krout, 2015). Comprehensive reviews of iPad re-sources are available to help clinical practice (Knight, 2013). With each of the four methods of music therapy (recreating, improvising, listening, and composing) being able to be ac-centuated by apps (Knight, 2013).

iPads and Apps

iPads have become prolific in school settings, offering multifunctionality, the ability to tailor to individual styles of use, ease of use, portability, and high quality of graphics and sound (Krout, 2015). iPads have been used to create powerful and expressive controllers for digital music (Favilla & Pedell, 2014) with many music-based applications developed to meet different needs. Some apps tie into existing software to provide a new facet of access while others offer experiences unique to the device. Krout (2014b) provides an exploration of a number of apps for engaging young people with Autism Spectrum Disorders, the needs they address, and their efficacy

in music therapy. He suggests that the therapist must balance the advantages and disadvantages of using such technology against each client’s needs, abilities, and goals. Apps such as Beatsurfing (Lobby & De Ridder, 2018) allow the creation of custom graphical user interfaces (GUIs). These can be de-signed by the user through building with lines, polygons, cir-cles, and faders. Parameters such as size, color, orientation, 3D position, and value that can also be customized (Lobby & De Ridder, 2018). These GUIs can then be connected to MIDI-compatible software, hardware, or other MIDI-enabled apps to provide bespoke interaction and allow configurable sonic output. One such app is ThumbJam (Sonosaurus, 2018) which provides a vast array of features. Included in the app are over 40 sampled instruments, hundreds of scales, and an array of customization of how it can be played, and what is displayed on screen (including user uploaded backgrounds) (Matthews, 2018). ThumbJam also offers arpeggiating, looping, recording, effects selection and manipulation, instrument creation, and the ability to import and export data. The ever-growing app market means it is easier than ever to find screen-based appli-cations that fit the needs of the user and also offers access to the developers in terms of suggesting updates and tailoring for specific needs.

The Computer as the Bridge

Traditional acoustic instruments are “stand-alone” in the fact that they are composed of an excitation mechanism (string, reed, skin, etc.), a resonant capacity (the body of the instrument), and the specific timbre they produce. If, however, we add a computer as a bridge in this system, we arrive at DMIs. A DMI “implies a musical instrument with a sound gen-erator that is separable (but not necessarily separate) from its control interface” (Malloch, Birnbaum, Sinyor, & Wanderley, 2006, p. 49). DMIs break the coupling between the action used and the sound produced. This can be thought of as a three-layer system (Figure 1) consisting of the control inter-face, the processing (which can be achieved via a separate computer or an on-board system), and the effort mech-anism or output (audio/visual/haptic feedback) (Hunt, Kirk, & Neighbour, 2004).

Useful methods of classification can be adopted from the fields of HCI, music technology, and new interfaces for mu-sical expression (NIME), when categorizing new technology which uses the computer as the bridge. Wanderley (2001) suggested the term gestural controller to describe interfaces that consist of two elements. The first element is an interface that features one or more sensors to detect the physical inter-action of the performer (these can be in the form of body movement, empty-handed gestures, or object manipulation). The second element is the auditory, tactile-kinesthetic, and/

Figure 1. Three layers of digital musical instrument (DMI).

Dow


ic.oup.com/m



or visual feedback given to indicate the instrument’s status the performer.

Wanderley (2001) proposed a three-tier classification of such controllers as:

• Instrument-like controllers—where the input device design tends to reproduce each feature of an existing (acoustic) in-strument in detail (e.g., an electric keyboard)

• Augmented Instruments (also called Hybrid Controllers)—instruments augmented by addition of sensors (e.g., the Yamaha Disklavier)

• Alternate controllers—whose design does not follow one of an established instrument—an example would be the Hands (Waiswisz, 1985; Wanderley, 2001, p. 6)

Alternate controllers offer unique opportunities to create interactive musical systems from the ground up to specifically suit client need. Using new or bespoke modes of interaction and processing these interactions into meaningful content pro-vide unique potential to increase accessibility to active music-making. Alternate controllers can be designed with client cap-abilities at the center of the design process, can be built to assist both in terms of physical access and learning needs, and can be tailored to provide feedback to suit the client or con-text they are being used in.

Alternate Controllers

Alternative controllers take two forms: 1) those that require physical touch to control, which are referred to as touch-based, and 2) those that do not, which are referred to as empty-handed.

Touch-Based Alternate Controllers

Touch-based controllers use direct physical interaction with a control interface to acquire control data for musical systems. Notable developments in this area are discussed below.

MidiGrid. One of the first examples of using a touch-based alternate controller was explored by Hunt and Kirk (2003). In their long-term project (beginning in 1987) titled MidiGrid, they utilized the mouse and keyboard to control sound in software used by children and young people in a music therapy setting. Hunt and Kirk (2003) used the advent of MIDI (musical instru-mental digital interface) within their project. MIDI is a commu-nication protocol that was developed in 1983 to allow various pieces of technology to use a connective language. The MidiGrid project was furthered by the development of MidiCreator (Kirk, Abbotson, Abbotson, Hunt, & Cleaton, 1994), which con-verted signals from electronic sensors into MIDI. MidiCreator could then be connected to the MidiGrid software. A computer could be equipped with MidiGrid allowing users to explore the creation and composition of musical work without the need to learn a traditional instrument. MidiGrid has been used by a wide range of people, such as composers, schoolchildren, special needs teachers, and their clients (Hunt & Kirk, 2003).

Skoog 2. A more recent development is the Skoog 2 (Skoogmusic, 2018), a wireless Bluetooth-enabled tactile foam cube with companion app and software. Manipulation of the Skoog surface can be mapped to proprietary sounds within the software or can connect to external MIDI-compatible soft-ware. The system provides a wealth of resources “out-of-the-box,” allowing for user customizable sounds and notes as well

as controllable sensitivity settings for note triggering. This pro-vides a hands-on musical experience for those with no pre-vious musical knowledge affording individual exploration in a solo setting (Nath & Young, 2015).

Music production controllers. Music production con-trollers (MPCs) are generic devices developed for electronic music-makers that feature triggering pads often used with MIDI-compatible software. They provide another modality of interaction that can be used as a tool to increase accessibility; however, these devices require a person familiar with music technology to set them up. The configurability of these devices allows adjustment to fit specific client requirements; addition-ally, functionality allows user profiles to be stored and recalled as needed. In a setting where resources have to be shared, this is an important feature as it provides the flexibility to allow users with different abilities to dictate the media content being triggered by the pads. This also allows for different levels of support (from simple note triggering to timing support) de-pending once again on client needs and preferences. This type of music technology is often very attractive to children and young people providing a motivator for engagement.

Switches. Another touch-based alternate controller used extensively, particularly for clients with severe disabilities, is the switch (Bache, Derwent, & Magee, 2014; Crowe & Rio, 2004). Switches are electronic or mechanical devices which, via a control unit or cordless receiver, provide a simple mech-anism for choosing and communicating (Magee, 2012). Switches use physical action or gesture to give direct access to a variety of electronic music devices. There are a large range of switches that offer many forms of control. Bache and col-leagues (2014) provide a comprehensive overview of switches and their use with those with complex needs. Switches are a commonplace assistive technology that can be used in com-bination with specialist or commercial software. Custom built switches based on motor, cognitive, or sensory needs facili-tate interaction based on clinical need. Sounds triggered by a switch can give a sense of control to clients, reinforcing a sense of self and allowing for expression (Swingler, 1998). Communication by using switches is often a starting point for nonspeaking clients (Hunt et al., 2004; Magee et al., 2011)

Mogees. An alternate controller providing an out-of-the-box package is the Mogees (Mogees, 2018). Mogees is a contact microphone which when placed on any surface detects when the surface is “played.” Mogees has the potential to be used in a variety of settings and with objects that users are familiar with or motivated to interact with. It is highly portable and affordable.

Musii. Finally, another interesting alternate controller is the multisensory interactive inflatable Musii (Musii, 2017). Musii is a soft inflatable object that emits sound and illumin-ates with color when touched. It enables any nonmusician to experience the act of creating music by translating physical interaction with the device into stimulating audio, visual, and tactile sensation (Musii, 2017).

Empty-Handed Controllers

Empty-handed controllers do not require physical touch and use mechanisms such as infrared light, ultrasonic sensors, electromagnetic fields, radar, cameras, or microphones to de-tect sound or physical movement. Sonic parameters can be mapped and controlled from this information. This can be

Dow


ic.oup.com/m



particularly useful in facilitating clients with complex needs by providing high levels of control, especially for those with physical disabilities or impairments.

The earliest empty-handed controller is considered to be the Theremin, patented in 1928 by Leon Theremin, in which the player uses the proximity of their hands to two metal aerials to control frequency and amplitude of a sound. The earliest documented use of alternate controllers for music-making in music therapy can be traced back to 1987. Nagler and Lee (1987) used microcomputers in music therapy sessions to “investigate the possibility of enabling a se-verely physically handicapped person to create music with minimal assistance” (p. 72). Using an Apple II microcom-puter, Mountain Computer Music System, Express 3 infrared tracking device, and the Viewpoint optical indicator (an infrared light beam) clients could control the music based on their head movements, allowing them to achieve inde-pendent music-making.

Soundbeam. One of the first empty-handed commercially available alternate controllers for music therapy was the Soundbeam system (Soundbeam, 2018). Soundbeam is a tool that converts movement within an ultrasonic beam into MIDI information. Although it can be found in the equipment stores of many special educational needs schools in the United Kingdom, it has been described as poorly used (Magee, 2012). Factors that contribute to this may be that due to its complexity, specialist training is required to use the device and there is an inherent difficulty in placing the beams optimally to suit the movement of some users. The beams travel out linearly which can be unsuitable for users who cannot follow that axis of movement (Ellis & Van Leeuwen, 2000). The lack of tactile feedback can also mean a disconnect between cause and ef-fect for some users. Despite these drawbacks, Soundbeam has been extensively used in practice, possibly due to the unique mode of interaction it affords and the fact that there is a wealth of material and resources to enable people to use the system (Soundbeam, 2018).

Music Maker. Other motion capture systems use cameras to capture movement data. A notable development is Music Maker, which turns body movements into sound using a nonobtrusive camera. Music Maker uses displays of cartoon drawings or pictures of musical instruments to give an element of fun and can be adjusted according to patients’ levels of support needed, therapeutic goals, and type of equipment available in hospitals or patients’ homes (Gorman, Lahav, Saltzman, & Betke, 2007).

Eye gaze systems. Additionally, some control mechanisms include eye gaze systems. These detect the user’s direction of gaze as control information, often utilizing a “dwell” type eye event to elicit a mouse click. Eye gaze systems are often the only access method available to those with diagnosis of “locked-in syndrome” (Vamvakousis & Ramirez, 2016), they are used due to the efficient and less effortful way they can be used to provide access to the computer (Bache et al., 2014). Hardware and software developments by commercial com-panies such as Tobii, Sensory Guru, and Smartbox (Bache et al., 2014) have pushed forward the development of the mu-sical applications of eye gaze. One such example is EyeMusic, which provides a “system that transforms eye movement data into musical compositions and data sonifications” (Hornof &

Sato, 2004, p. 185). However, use of such systems do require skills developed over time by the client.

Clarion. A notable recent development in this area is the Clarion. The Clarion is a highly configurable software instru-ment developed as part of the Open Orchestras project (Open Orchestras, 2018). The Clarion allows the client to specify “the sound the instrument makes; the number of notes that are available to play; the shape, position and colour of the notes; and crucially the way in which [you] play them” (Farrimond, 2016). It integrates with eye gaze systems, SmartNav and the iPad, allowing use with existing hardware resources. Clarion comes as part of a package offered by Open Orchestras which includes the Clarion software, repertoire, training resources and support, and an evaluation framework.

New Developments

A number of recent related developments have impacted the world of DMIs. Microcontroller boards like Arduino (Arduino, 2015), affordable computers such as the Raspberry Pi (Raspberry Pi, 2015), and software such as Max/MSP (Cycling’74, 2018) allow for bespoke systems to be created. Sensors can directly capture a person’s input and then be in-tegrated as a control device for software, or stand-alone be-spoke devices can be created at a low cost. The development of the Internet of Things (Internet of Things Council, 2018), and Web portals and Webpages with tutorials such as Instructables (Autodesk, 2018) have provided a community of DIY develop-ments and assistance (in the form of forums) for those wishing to create bespoke instruments. Hacker communities are also providing space and tools, along with “hackathon” style com-petitions (often 24-hr themed competitions which are supplied and sponsored by companies), allowing for rapid prototyping of accessible instruments and new tools while also bringing together people with a range of skill sets to create and share information online. There are now also many intermediary ap-plications that allow for the quick creation of enticing inter-faces to trigger music and sound.

Makey Makey

Packages such as the Makey Makey (Makey Makey, 2018) allow conductive objects (e.g., fruit, putty, metal) to be con-nected to a microcomputer to emulate keyboard presses, which can then be used to trigger sound. For example, users could create a piano from bananas by using Makey Makey and connecting it to software such as Garageband or SoundPlant (Blum, 2018). Both these softwares allow sounds to be as-signed to keyboard presses.

Bare Conductive

Bare Conductive Touch Board (Bare Conductive, 2018) is another microcomputer featuring 12 touchpads that allow conductive materials to be connected via crocodile clips. The out-of-the-box setup allowed 12 sound samples to be triggered monophonically from a memory card placed in a slot embedded on the board which then play via an on-board headphone jack, or connected to a speaker. The board is well documented and designed to be used with minimal tech-nical knowledge. The board also offers expansion for those with more technical knowledge as it contains a built-in general MIDI chip for those wishing to reprogram the board

Dow


ic.oup.com/m



to allow polyphonic notes, or to allow creation of bespoke MIDI-enabled instruments. The Bare Conductive website (https://www.bareconductive.com/) features very comprehen-sive step-by-step guides for setting the board up and provides ideas for utilizing the board practically. These new develop-ments expand possible modes of interaction by providing off-the-shelf software and hardware that may be commonplace in music therapy settings, or simply using everyday items that the client may find enticing and motivating to engage with.

Leap Motion

Other new technologies such as the hand gesture tracker Leap Motion (Leap Motion, 2017) offer toolkits to build custom systems. The Leap Motion system converts hand movements to data, thus providing a flexible tool for mapping client-specific movements to sound (Uwyn.com, 2018).

Microsoft Kinect

The Kinect (KinectSEN, 2018) is a camera-based movement tracker made by Microsoft that allows body movement by skeletal tracking to be used to control data, thereby producing sound through movement.

GestureSEN

An excellent resource for gesture-based systems used in special education is the gestureSEN website (https://web.archive.org/web/20180723042755/https://kinectsen.wikispaces.com/home). The site, run by teachers in special schools, aims to explore how established and emerging gesture-based technology could help people with severe learning difficulties with their engagement, creativity, and in-dependence skills (Gesturesen.wikispaces.com, 2018). The site featured information on using eye gaze, Kinect, Leap Motion, iPad, and Virtual Reality in special education settings. Unfortunately, due to the closure of Wikispaces website, the content from the gestureSEN website is only viewable through internet archive websites such as https://web.archive.org/.

Games Controllers

Finally, game controllers such as the WiiMote and the Xbox controller alongside music-themed games can also provide unique mechanisms through which to access musical inter-action, with schools typically having these resources available for general use.

All of the above offer new methods of access to music-making with the computer that move away from the keyboard and mouse paradigm. These tools provide the flexibility to create systems that tailor to client capability, motivation, and curiosity.

Available Technology

As evidenced by the literature presented, there are many technologies available for aiding accessibility to music-making. As a growing field that crosses many disciplinary areas, challenges are created for music therapists. The pri-mary challenges are: knowing where to find this technology, examples of its use in similar contexts, and guidelines for integrating it into clinical practice.

Table 1 provides a summary of developments, including off-the-shelf DMIs, that have been used with clients who have complex needs. The DMIs included in the table were selected

because there is evidence that they been used with people with complex needs, through either peer-reviewed published literature, anecdotally, or observed firsthand by the first author. The DMIs reviewed are further organized into two categories: 1) commercially/freely available, and 2) research only. This decision was based on the fact that while some of the research and technology developed may show great promise for cli-ents with complex needs, they have not subsequently been made available for wider use. The two categories are then fur-ther divided into three subcategories: touch-based, software-based, and empty-handed. A final section provides some con-text to how the technology may be relevant to music-making in a music therapy setting.

Controlling Sound With Technology

Paine and Drummond (2009) suggest there are two distinct approaches to computer-assisted music: “control of predeter-mined sequences of sounds (such as the triggering of sound samples) or creation of sounds in real-time by the manipula-tion of software synthesis variables” (p. 2). Technology offers the ability to control and trigger sound in different ways that extend past that of acoustic instruments. Technology can also provide responses to interaction in ways that acoustic instru-ments cannot. It can offer physical and/or cognitive support, and scaffold capability to give users access in ways traditional instruments do not allow. Swingler (1998) suggests that few children have the physical coordination or control necessary for traditional performance. As such, technology can help to shift beyond traditional musical qualities toward a new and developing musical aesthetic, one enabled by the introduction of electricity to musical activity. He suggested that this allows the opening up of many musical doors so all can enjoy being expressive with sound: “Many techniques can be made easily available to virtually all kids through technology” (Swingler, 1998, p. 5). Through technology, small motions can lead to sound production and engagement. For example, with even something as simple as a microphone there are great oppor-tunities for utilizing feedback and amplification to allow the “tiniest voice and smallest nuances to be enhanced and ex-tended” (Ellis & Van Leeuwen, 2000, p. 8).

Music technology can therefore help to:

• Transduce movement and gestures into musical expression (Hunt et al., 2004)

• Make it possible for a client to realize a creative idea re-gardless of implementation or user and to give the oppor-tunity for an aesthetic experience (Misje, 2013)

• Allow people to lose themselves in artistic expression (with a quality of interaction so high that they are not aware they are using technology) (Hunt, Kirk, Abbotson, & Abbotson, 2000)

• Give initialization opportunities to usually passive users enabling the concept of selfhood, which can be inhibited for individuals with profound and multiple learning disabil-ities (PMLD)

• Provide, sometimes for the first time (Swingler, 1998), that “make something happen!” moment as described by Ellis (1997), which is a foundational experience of learning

These simple but crucial experiences may help users to en-counter and develop communication skills through sound. This

Dow


ic.oup.com/m


https://www.bareconductive.com/

https://web.archive.org/web/20180723042755

https://web.archive.org/web/20180723042755

https://kinectsen.wikispaces.com/home

https://kinectsen.wikispaces.com/home

https://web.archive.org/


Tabl

e 1

Alte

rnat

e C

ontr

olle

rs

Inte

ract

ion

mod

alit

yC

omm

erci

ally

/ fr

eely

ava

ilabl

eD

escr

ipti

onD

etai

ls

Touc

h

Gen

eral

tec

hnol

ogy

Switc

hes

Bro

ad r

ange

of

elec

tron

ic o

r m

echa

nica

l to

ols

to a

llow

on/

off

sele

ctio

ns

Tailo

rabl

e to

use

r, ea

se o

f int

egra

tion

to c

urre

nt r

esou

rces

, can

be

wir

ed o

r w

irel

ess,

tr

igge

r or

sta

rt/s

top

soun

d ef

fect

s, m

usic

al n

otes

/phr

ases

, rec

orde

d sa

mpl

es, o

r se

quen

ces

of s

ampl

es. “

For

exam

ple,

the

clie

nt m

ay a

ctiv

ate

a C

D p

laye

r or

pre

-re

cord

ed m

usic

trac

k” (K

nigh

t & L

agas

se, 2

012,

p. 1

92).

Bac

he e

t al.

(201

4) p

rovi

de

an in

-dep

th r

evie

w o

f pra

ctic

al s

witc

h us

e.

iP

adTo

uch

scre

en h

andh

eld

devi

ce s

peak

er,

mic

roph

one,

and

mot

ion

dete

ctio

n

Mul

tiuse

dev

ice,

tailo

rabl

e to

use

r, fa

mili

ar te

chno

logy

, ent

icin

g, d

irec

t int

erac

tion

with

app

s, s

elf-

cont

aine

d w

ith s

peak

er o

n-bo

ard,

com

mon

ly a

vaila

ble.

Use

ful f

or

quic

kly

reco

rdin

g m

ultit

rack

mus

ic a

rran

gem

ents

from

live

or

plug

ged-

in s

ourc

es.

Abi

lity

to c

aptu

re s

ound

usi

ng b

uilt-

in m

icro

phon

e. A

pps

can

emul

ate

inst

rum

ents

en

ablin

g th

e cl

ient

to p

lay

a st

ring

ed in

stru

men

t by

touc

hing

the

scre

en, o

r a

win

d in

stru

men

t by

blow

ing

into

the

mic

roph

one

(Kni

ght &

Lag

asse

, 201

2). “

App

s ca

n be

us

ed to

rec

ord,

syn

thes

ize,

man

ipul

ate,

or

prov

ide

feed

back

to c

lient

act

ions

and

so

unds

” (p

. 194

). K

nigh

t (20

13) p

rovi

des

an in

-dep

th r

evie

w o

f iPa

d ap

plic

atio

ns in

m

usic

ther

apy.

iPod

Tou

chA

s iP

adSi

mila

r to

iPad

but

sm

alle

r in

form

fact

or a

nd w

ith e

nhan

ced

secu

rity

(not

con

nect

ed

to a

net

wor

k)

A

pps

Piec

es o

f sof

twar

e fo

r us

e on

tabl

et o

r sm

artp

hone

dev

ices

Wid

e va

riet

y, s

ome

free

. App

s fo

r C

hild

ren

with

Spe

cial

Nee

ds (2

018)

is a

web

site

fo

r fin

ding

spe

cific

app

s ai

med

at c

hild

ren

with

spe

cial

nee

ds fe

atur

ing

mus

ic a

s w

ell a

s ot

her

apps

with

rev

iew

s an

d vi

deos

. Offe

ring

the

abili

ty to

tailo

r co

nten

t and

in

tera

ctio

n to

clie

nt r

equi

rem

ents

.

G

ener

ic m

usic

pr

oduc

tion

cont

rolle

rs (M

PCs)

Trig

ger

pads

with

ve

loci

ty s

ensi

ngM

IDI-

com

patib

le. S

ome

com

e w

ith o

wn

softw

are

inst

rum

ents

, req

uire

s te

chni

cal

know

ledg

e to

set

up.

Pro

vide

s ab

ility

to p

rese

lect

sou

nds

repr

esen

tativ

e of

a w

ide

vari

ety

of g

enre

s en

able

s cl

ient

s an

acc

essi

ble

way

to p

erfo

rm th

eir

cultu

ral o

r m

usic

al id

entit

ies.

Allo

ws

ther

apis

ts to

offe

r di

vers

e so

und

pala

tes

valu

able

for

play

ing

diffe

ring

feel

ings

or

emot

ions

dur

ing

impr

ovis

atio

n (C

rook

e &

McF

erra

n, 2

019)

. Can

al

so b

e us

ed in

com

posi

tion

to p

erfo

rm a

nd r

ecor

d dr

um b

eats

and

mel

odic

pat

tern

s (C

rook

e, 2

018)

, or

to tr

igge

r or

laun

ch a

ran

ge o

f loo

ps o

r sa

mpl

es, a

llow

ing

for

the

play

ing

of p

reco

mpo

sed

piec

es a

nd li

ve r

emix

ing.

Mus

ic R

adar

(201

8) p

rovi

de a

n ov

ervi

ew o

f ava

ilabl

e M

PCs.

M

usic

-bas

ed v

ideo

ga

mes

V

ideo

gam

e sy

stem

us

ing

gene

ric

or b

espo

ke

to th

e ga

me

cont

rolle

rs

Fam

iliar

to u

sers

, ent

icin

g te

chno

logy

, eas

y se

tup,

can

pot

entia

lly u

se e

xist

ing

reso

urce

s. B

lain

e (2

005)

pro

vide

d a

revi

ew o

f alte

rnat

e m

usic

-bas

ed v

ideo

gam

e co

ntro

llers

. Wik

iped

ia (2

017)

pro

vide

s a

list o

f mus

ic-b

ased

vid

eo g

ames

. Use

of

the

WiiM

ote

in m

usic

ther

apy

has

been

exp

lore

d (B

enve

nist

e, Jo

uvel

ot, &

Mic

hel,

2008

), an

d de

velo

pmen

ts s

uch

as th

e W

iinst

rum

ent c

an b

e do

wnl

oade

d to

util

ize

the

myr

iad

of d

ata

that

the

WiiM

ote

prod

uces

(The

Wiin

stru

men

t, 20

18).

Gam

es a

vaila

ble

incl

ude

Wii

Mus

ic b

y N

inte

ndo.

(Con

tinue

d)

Dow


ic.oup.com/m



Spec

ific

devi

ces

Des

crip

tion

Det

ails

Skoo

g 2

Wir

eles

s fo

am c

ube

with

fiv

e ta

ctile

pre

ssur

e-se

nsiti

ve p

ads.

Wir

eles

s, p

orta

ble,

eas

y to

set

up.

Sim

ple

and

intu

itive

. Ow

n ap

p an

d so

ftwar

e to

cu

stom

ize

sens

itivi

ty a

nd s

ound

cre

ated

, MID

I-co

mpa

tible

. Man

y re

sour

ces

for

use

in

spec

ial e

duca

tion

on w

ebsi

te. P

rovi

des

dire

ct c

orre

latio

n be

twee

n ph

ysic

al c

onta

ct

and

soun

d pr

oduc

ed, u

sing

vir

tual

mus

ical

inst

rum

ents

, sam

ples

, or

MID

I. O

fferi

ng

dyna

mic

con

trol

ove

r m

usic

al g

estu

res

(Sko

ogm

usic

, 201

8). “

The

inst

rum

ent d

oes

not

sim

ply

trig

ger

sam

ples

whe

n pr

esse

d bu

t use

s so

phis

ticat

ed s

ynth

esis

to d

ynam

ical

ly

man

ipul

ate

the

vari

ous

inst

rum

ent s

ound

s th

ough

pre

ssin

g, s

quee

zing

, rub

bing

, st

roki

ng, t

iltin

g or

man

ipul

atin

g th

e Sk

oog”

(Far

rim

ond

et a

l., 2

011,

p. 2

8).

Mak

ey M

akey

Mic

roco

ntro

ller

boar

d w

ith s

ix c

onne

ctor

s th

at e

mul

ate

QW

ERTY

ke

yboa

rd p

ress

es a

nd

mou

se c

ontr

ol

Turn

eve

ryda

y ob

ject

s in

to to

uchp

ads,

no

softw

are

to in

stal

l, fa

st s

etup

. Can

be

conn

ecte

d to

con

duct

ive

obje

cts

such

as

frui

t, co

nduc

tive

tape

, pen

cil g

raph

ite, a

nd

clie

nts

touc

hing

eac

h ot

her

in c

hain

s as

a m

eans

to tr

igge

r so

unds

(Mak

ey M

akey

, 20

18).

The

proc

ess

allo

ws

appr

opri

atio

n en

ablin

g cl

ient

s to

gai

n au

thor

ship

of t

heir

in

stru

men

ts, a

nd b

ecom

e an

intr

insi

c pa

rt o

f the

ir d

eplo

ymen

t (H

ayes

, 201

6).

Alp

haSp

here

G

lobe

-sha

ped

MID

I co

ntro

ller w

ith 4

8 pl

ayab

le v

eloc

ity-s

ensi

tive

pads

and

ow

n so

ftwar

e.

Alp

haSp

here

is a

tang

ible

con

trol

ler,

whe

n co

nnec

ted

to A

lpha

Live

sof

twar

e/us

ed w

ith o

ther

MID

I sof

twar

e ca

n be

set

up

to tr

igge

r an

d m

anip

ulat

e so

und

and

prov

ide

a un

ique

mod

ality

of a

cces

s th

at c

an b

e en

ticin

g to

clie

nts

(Pla

ce, L

acey

, &

Mitc

hell,

201

4).

MID

ICre

ator

Dev

ice

to c

onve

rt s

igna

ls

from

ele

ctro

nic

sens

ors

into

MID

I dat

a.

Clie

nts

can

cont

rol s

ound

s w

ith p

hysi

cal a

ctio

ns a

nd g

estu

res,

can

be

used

to d

etec

t si

mpl

e bo

dy m

ovem

ents

(Kro

ut, 2

014a

). C

an b

e co

nnec

ted

to M

IDI t

o be

use

d w

ith

othe

r sy

nthe

size

rs. S

enso

rs a

vaila

ble

incl

ude

pres

sure

, dis

tanc

e, p

roxi

mity

, dir

ectio

n,

etc.

(Mec

kin

& B

ryan

-Kin

ns, 2

013)

.

I-

Cub

eX

Softw

are

and

digi

tizer

fo

r cr

eatin

g sy

stem

s w

ith a

var

iety

of s

enso

rs

avai

labl

e

Tool

s to

cap

ture

the

hum

an a

ctio

ns a

nd/o

r env

ironm

enta

l var

iabl

es a

nd m

ake

thes

e si

gnal

s av

aila

ble

to o

ther

equ

ipm

ent s

uch

as a

com

pute

r or a

mus

ical

inst

rum

ent t

o tri

gger

sou

nd, m

usic

, vid

eo, g

raph

ics,

ani

mat

ion,

robo

tic m

ovem

ent,

etc.

MID

I dat

a ar

e us

ed, t

rans

mitt

ed v

ia M

IDI c

able

, USB

cab

le, o

r Blu

etoo

th w

irele

ss (I

-Cub

eX, 2

018)

.

Ky

ubEl

even

feat

her

touc

h ke

ypad

s on

five

sur

face

s of

a 3

-inc

h w

oode

n cu

be

with

acc

eler

omet

er

Mak

er fr

iend

ly, o

pen-

sour

ce D

IY M

IDI k

eybo

ard

feat

urin

g ca

paci

tive

sens

ing

and

acce

lero

met

er. F

ully

pro

gram

mab

le, s

et s

cale

, tw

eak

note

vel

ocity

cur

ves,

map

di

ffere

nt in

stru

men

ts to

diff

eren

t pad

s to

con

figur

e to

mus

ical

tast

e (K

yub

MID

I ke

yboa

rd, 2

018)

. Use

r ca

n al

so c

onfig

ure

the

way

the

inst

rum

ent l

ooks

and

sou

nds

by

desi

gnin

g th

eir

own

inte

rfac

e.

Su

zuki

QC

hord

El

ectr

onic

inst

rum

ent

with

on-

boar

d sp

eake

r an

d LC

D d

ispl

ay.

A d

evic

e fo

r co

mpo

sitio

n, te

achi

ng, a

nd th

erap

y. In

corp

orat

ing

tech

nolo

gy fr

om a

ba

sic

keyb

oard

and

ele

ctri

c gu

itar

and

com

bini

ng b

oth

in a

por

tabl

e, e

asy-

to-u

se

way

. Eig

hty-

four

diff

eren

t cho

rd c

ombi

natio

ns, 1

00 in

stru

men

t voi

ces,

orc

hest

rate

d rh

ythm

s. F

eatu

res

thre

e se

ctio

ns; a

touc

h-se

nsiti

ve 4

oct

ave

“str

um p

late

,” a

rhy

thm

se

ctio

n, a

nd c

hord

but

ton

sect

ion.

Eac

h of

thes

e ar

eas

can

be u

sed

inde

pend

ently

or

com

bine

d w

ith a

var

iety

of s

ound

s ob

tain

able

. Sou

nds

are

alw

ays

in tu

ne. C

an

be a

dapt

ed to

all

abili

ty le

vels

. Can

stim

ulat

e in

tera

ctio

n, in

crea

se c

oord

inat

ion,

st

imul

ate

gros

s an

d fin

e m

otor

ski

lls, a

nd in

crea

se s

elf-

expr

essi

on. P

itch

bend

whe

el

for

expr

essi

on. C

hang

eabl

e so

ng c

artr

idge

s. C

an c

onne

ct to

spe

aker

or

MID

I dev

ice

(Suz

uki Q

chor

d, 2

018)

.

Tabl

e 1.

Con

tinue

d

Inte

ract

ion

mod

alit

yC

omm

erci

ally

/ fr

eely

ava

ilabl

eD

escr

ipti

onD

etai

ls

(Con

tinue

d)

Dow


ic.oup.com/m



Yam

aha

DJX

-iiB

Gro

ove

mac

hine

in a

bo

x-sh

aped

des

ktop

m

odul

e fo

rm w

ith

scra

tch

pad

and

fade

r.

Can

sel

ect f

rom

70

patte

rns,

mut

e pa

rts,

add

pre

set h

its o

r fil

ls, a

nd s

hift

key

of th

e pa

ttern

s pl

ayin

g an

d us

e ef

fect

s. P

rovi

des

oppo

rtun

ities

for

the

clie

nts

to c

ompo

se

mus

ic b

y as

sem

blin

g lo

ops,

impo

rtin

g ot

her

song

s or

sec

tions

of s

ongs

, and

/or

reco

rdin

g he

r or

his

ow

n m

usic

, offe

ring

acc

essi

ble

and

dyna

mic

mea

ns o

f exp

ress

ing

emot

ions

(Whi

tehe

ad-P

leau

x et

al.,

201

1)

M

usii

Soft

infla

tabl

e se

lf-co

ntai

ned

port

able

un

it th

at e

mits

sou

nd

and

illum

inat

es w

hen

thre

e in

flate

d co

nes

are

touc

hed,

with

wir

eles

s co

ntro

l

Sim

ple

to u

se, n

eeds

no

extr

a eq

uipm

ent o

r tr

aini

ng to

ope

rate

. Abi

litie

s to

m

ake

mus

ic a

nd c

olou

r th

roug

h to

uch

and

mov

emen

t with

exp

ansi

ve li

brar

y of

so

unds

cape

s (o

ver

50) a

nd in

nova

tive

mus

ical

sys

tem

. Can

not p

lay

out o

f tim

e or

ou

t of t

une.

A n

umbe

r of

peo

ple

can

play

har

mon

ious

ly a

s a

grou

p or

an

indi

vidu

al

can

beco

me

an o

rche

stra

. Tac

tile

phys

ical

form

that

ena

bles

the

user

to s

ee a

nd fe

el

the

beam

that

they

are

inte

ract

ing

with

. “M

usii

has

been

des

igne

d fo

r de

velo

pmen

tal

play

in th

e SE

N s

ecto

r...T

he m

ulti-

sens

ory

expe

rien

ce o

f pla

ying

Mus

ii ha

s m

any

ther

apeu

tic p

ossi

bilit

ies

incl

udin

g en

cour

agin

g m

ovem

ent,

deve

lopm

ent a

nd

awar

enes

s of

pro

prio

cept

ion,

turn

taki

ng, c

ause

and

effe

ct, c

reat

ivity

, exp

ress

ion

and

com

mun

icat

ion.

It c

an b

e us

ed fo

r st

imul

atio

n or

for

calm

ing.

The

syn

chro

nise

d so

und

and

light

ing

as w

ell a

s th

e vi

sual

and

tact

ile fe

edba

ck o

f the

infla

tabl

e en

able

a

deep

er u

nder

stan

ding

of t

he m

usic

you

are

mak

ing”

(Mus

ii Lt

d, 2

017)

Rea

ctab

le

Tabl

e w

ith to

uch

scre

en

and

mov

eabl

e ob

ject

s (R

eact

able

Tec

hnol

ogy,

20

17)

Obj

ects

inte

ract

with

the

tabl

e su

rfac

e an

d ea

ch o

ther

to m

ake

mus

ic. A

llow

s in

tuiti

ve a

nd c

olle

ctiv

e cr

eatio

n of

com

plex

mus

ical

pie

ces,

col

labo

rativ

e sh

arin

g sp

ace

betw

een

user

s, p

rom

otes

imita

tion

gam

es, i

ncre

ases

vis

ibili

ty o

f act

ions

, en

able

s m

onito

ring

oth

er p

artic

ipan

t’s w

ork,

aid

s in

red

ucin

g so

litar

y pl

ay s

eque

nces

, fa

cilit

ates

ass

ocia

tive

play

(Vill

afue

rte,

Mar

kova

, & Jo

rda,

201

2)

Te

nori

-On

Han

dhel

d sc

reen

w

ith 1

6 ×

16

grid

of

LED

sw

itche

s. B

uilt-

in

spea

kers

, dia

l and

bu

ttons

con

trol

sou

nd

and

beat

s pe

r m

inut

e pr

oduc

ed

Cre

ate,

con

trol,

or p

erfo

rm m

usic

al m

ater

ial o

n vi

sual

ly ri

ch to

uch-

sens

itive

inte

rface

(F

arrim

ond

et a

l., 2

011)

. Sw

itche

s ac

tivat

ed in

diff

eren

t way

s cr

eate

mus

ic fr

om 2

56

soun

ds. E

ngag

ing,

mot

ivat

ing,

sen

sory

, wel

l sui

ted

for i

mpr

ovis

atio

n, e

asy

to u

se/h

ard

to m

aste

r. C

ombi

nes

visu

al a

nd m

elod

ic s

enso

ry in

form

atio

n, c

an s

timul

ate

cogn

ition

, m

emor

y, a

nd p

erce

ptio

n. C

an fu

nctio

n as

a rh

ythm

mac

hine

with

bas

ic o

r com

plex

rh

ythm

ic b

eats

that

can

be

loop

ed o

r cha

nged

. Not

es a

nd m

elod

ic p

hras

es c

an b

e ad

ded

as w

ell,

crea

ting

up to

16

laye

rs o

f sou

nd (C

lem

ents

-Cor

tes,

201

4). C

lem

ents

-Cor

tes

(201

4) p

rovi

ded

an in

-dep

th s

tudy

of t

he T

enor

i-On

in a

clin

ical

mus

ic th

erap

y se

tting

.

R

olan

d H

ands

onic

Dev

ice

with

13

ultr

a-se

nsiti

ve to

uch

pads

Ei

ght h

undr

ed a

nd fi

fty r

eady

-to-

play

sou

nds,

or

impo

rt c

usto

m s

ound

s. R

espo

nsiv

e an

d th

eref

ore

not o

verl

y de

man

ding

to p

lay,

with

eas

y ad

just

men

t of v

olum

e (C

halli

s &

Sm

ith, 2

012)

. Can

be

calib

rate

d to

clie

nt n

eed

in te

rms

of s

ensi

tivity

. Can

con

nect

to

oth

er M

IDI d

evic

es a

nd M

IDI c

an b

e re

cord

ed fr

om th

e de

vice

for

anal

ysis

.

Ko

rg K

aoss

ilato

r/K

aoss

ilato

r Pr

o/Ko

rg

Min

i Kao

ss P

ad

Aud

io e

ffect

s un

it w

ith

an X

/Y to

uch

scre

en.

Min

i Kao

ss p

ad fe

atur

es

on-b

oard

spe

aker

and

m

icro

phon

e

Posi

tioni

ng a

fing

er-t

ip o

n th

e to

uch

scre

en tr

igge

rs s

peci

fic s

ound

pro

gram

s. A

bilit

y to

trig

ger

indi

vidu

al n

otes

or

patte

rns

of n

otes

dep

endi

ng o

n th

e na

ture

of t

he c

hose

n so

und

and

setti

ngs

sele

cted

. Mov

ing

arou

nd th

e sc

reen

mov

es b

etw

een

note

s w

ithin

a

pred

efine

d sc

ale

or c

hang

es th

e na

ture

of t

he s

ound

. Allo

ws

the

resu

lts o

f act

ions

to

be

sam

pled

as

repe

atin

g lo

ops.

Par

ticul

arly

effe

ctiv

e fo

r th

ose

with

res

tric

ted

hand

an

d fin

ger

mov

emen

t, be

ing

easy

to in

tera

ct w

ith, t

o pr

oduc

e co

mpl

ex m

usic

al id

eas

and

patte

rns

(Cha

llis

& S

mith

, 201

2).

Tabl

e 1.

Con

tinue

d

Inte

ract

ion

mod

alit

yC

omm

erci

ally

/ fr

eely

ava

ilabl

eD

escr

ipti

onD

etai

ls

(Con

tinue

d)

Dow


ic.oup.com/m



Soun

dHou

se

Spec

ial A

cces

s K

it (B

anan

a Ke

yboa

rd)—

disc

ontin

ued

Sixt

een

keys

con

figur

ed

like

an o

vers

ized

pi

ano

with

sof

twar

e co

mpo

nent

Cur

ved

to s

uit t

he ra

dial

mov

emen

t of a

n ar

m. L

ight

touc

h ac

tivat

es m

usic

, sou

nds,

or

spe

ech

prog

ram

med

into

eac

h ke

y. A

llow

s co

nnec

tion

of u

p to

eig

ht s

witc

hes

for

activ

atio

n of

key

s on

the

keyb

oard

(Sou

nd H

ouse

, 201

7). I

nnov

ativ

e, a

dapt

able

con

trol

su

rfac

e th

at r

equi

re m

usic

ians

to p

ress

, squ

eeze

, or

stri

ke th

em to

cre

ate

and

cont

rol

mus

ic th

roug

h co

rres

pond

ing

mus

ical

sof

twar

e (F

arri

mon

d et

al.,

201

1). U

ser

frie

ndly

, ea

sy to

lear

n. A

rran

ge c

ombi

natio

ns o

f sou

nds

(MID

I sou

nds

or w

ave

files

). R

ecor

ding

fe

atur

e en

able

s re

al-l

ife p

erfo

rman

ces

to b

e sa

ved

and

voic

e an

d C

D s

egm

ents

to b

e re

cord

ed. A

rran

gem

ents

can

be

prin

ted

off a

nd fo

rmat

ted.

Var

iety

of g

loba

l set

tings

cu

tting

dow

n on

indi

vidu

ally

pro

gram

min

g ea

ch k

ey o

r sw

itch.

Aim

ed a

t fos

teri

ng

deve

lopm

ent o

f sw

itch

use,

cau

se-a

nd-e

ffect

, tim

ing,

cho

osin

g (S

ound

Hou

se, 2

017)

.

N

umar

k O

rbit

Wir

eles

s M

IDI c

ontr

olle

r Si

xtee

n ba

cklit

cus

tom

izab

le p

ads,

four

sel

ecta

ble

bank

s to

ass

ign

up to

64

cues

, lig

htin

g tr

ansi

tions

, vid

eo c

lips,

sam

ples

, etc

. Con

trol

whe

el a

nd o

n-bo

ard

acce

lero

met

er. C

an b

e co

nfigu

red

and

map

ped

to c

ontr

ol o

ther

MID

I sof

twar

e,

acce

lero

met

er fo

r m

otio

n co

ntro

l. C

omes

with

dem

o so

ftwar

e th

at s

how

how

to u

se

with

trac

ks a

nd e

ffect

s (N

umar

k.co

m, 2

018)

.

M

ogee

s R

eson

ance

con

tact

m

icro

phon

eEn

able

s in

stru

men

ts to

be

crea

ted

from

any

sur

face

/obj

ect a

long

side

con

figur

ing

of

the

soun

ds c

reat

ed w

ith d

edic

ated

iPho

ne/iP

ad a

pp. I

nteg

rate

s w

ith s

tand

ard

digi

tal

audi

o w

orks

tatio

ns v

ia A

udio

Uni

ts o

r VST

plu

g-in

. Can

be

used

to p

rovi

de e

xpre

ssiv

e in

stru

men

t by

usin

g di

ffere

nt a

reas

of s

urfa

ce tr

igge

r di

ffere

nt s

ound

(Mog

ees,

201

8),

allo

win

g fo

r ob

ject

s to

be

used

that

are

mot

ivat

ing,

fam

iliar

, or

enga

ging

to c

lient

s.So

ftw

are-

bas

ed

Gen

eral

tec

hnol

ogy

Des

crip

tion

Det

ails

Gen

eric

dig

ital

audi

o w

orks

tatio

ns

(Abl

eton

Liv

e/A

udac

ity/R

eape

r/C

ubas

e/Lo

gic/

Sona

re b

y C

akew

alk/

Gar

ageb

and)

Mus

ic r

ecor

ding

/co

mpo

sitio

n so

ftwar

e U

ses

pres

et o

r us

er d

eter

min

ed s

ettin

gs v

ia o

n-sc

reen

, or

pull-

dow

n m

enus

. Allo

w fo

r re

cord

ing,

com

posi

ng, p

layb

ack,

and

cre

atio

n of

mus

ic. S

ome

com

e w

ith c

onte

nt s

uch

as in

stru

men

ts a

nd s

ampl

es a

vaila

ble

for

inst

ant u

se. A

bilit

y to

use

sof

twar

e in

stru

men

ts

(VST

s), i

nput

mic

roph

ones

, or

elec

tron

ic in

stru

men

ts. G

arag

eban

d is

pre

inst

alle

d on

M

ac c

ompu

ters

with

out

-of-

the-

box

sam

ples

and

inst

rum

ents

ava

ilabl

e. R

eape

r is

af

ford

ably

pri

ced,

Abl

eton

is p

rize

d fo

r liv

e pe

rfor

man

ce. A

udac

ity is

free

war

e. C

an

expo

rt n

otat

ion

in s

ome

case

s. K

rout

(201

4a) p

rovi

des

a re

view

on

usin

g so

ftwar

e fo

r m

usic

com

posi

tion,

arr

angi

ng, n

otat

ing,

impr

ovis

ing,

and

seq

uenc

ing.

Sp

ecifi

c de

vice

sD

escr

ipti

onD

etai

ls

C

lari

on

Softw

are

inst

rum

ent

Allo

ws

user

to c

hang

e ev

ery

elem

ent i

nstr

umen

t inc

ludi

ng s

ound

, not

es, s

hape

, po

sitio

n an

d co

lour

of n

otes

, and

how

thos

e no

tes

are

play

ed. I

nteg

rate

s w

ith e

ye

gaze

sys

tem

s, S

mar

tNav

and

iPad

. Pac

kage

offe

red

by O

pen

Orc

hest

ras

incl

udin

g re

pert

oire

, tra

inin

g re

sour

ces,

sup

port

, and

an

eval

uatio

n fr

amew

ork

(Ope

n O

rche

stra

s, 2

018)

.

M

agix

Mus

ic M

aker

Dig

ital a

udio

w

orks

tatio

nPr

ovid

es 4

25 s

ound

s &

loop

s, s

even

free

Sou

ndpo

ols

(1,9

27 s

ound

s &

loop

s) th

ree

softw

are

inst

rum

ents

, eig

ht tr

acks

, and

eig

ht e

ffect

s. C

an b

e us

ed w

ith s

mar

t boa

rds.

Hyp

erSc

ore

A g

raph

ical

com

posi

tion

envi

ronm

ent

Use

rs d

raw

str

okes

and

line

s to

exp

lore

mus

ical

idea

s. G

raph

ical

ele

men

ts a

re

map

ped

to m

usic

al s

truc

ture

s, a

llow

ing

user

s to

sha

pe m

usic

al p

rogr

essi

ons

visu

ally

(G

rier

son

& K

iefe

r, 20

13; M

acho

ver,

2004

).

M

IDIG

rid

Mus

ic s

oftw

are

Use

s m

ouse

and

key

boar

d m

ovem

ents

with

in a

grid

to tr

igge

r not

es, c

hord

s, s

eque

nces

, or

pat

tern

s of

sou

nd th

at c

an b

e pl

ayed

bac

k an

d lo

oped

(Hun

t & K

irk, 2

003)

.

Tabl

e 1.

Con

tinue

d

Inte

ract

ion

mod

alit

yC

omm

erci

ally

/ fr

eely

ava

ilabl

eD

escr

ipti

onD

etai

ls

(Con

tinue

d)

Dow


ic.oup.com/m



Mic

roso

ft So

ngsm

ithM

usic

sof

twar

eG

ener

ates

mus

ical

acc

ompa

nim

ent t

o m

atch

a s

inge

r’s v

oice

usi

ng c

ompu

ter

mic

roph

one

inpu

t. M

usic

al s

tyle

and

feel

of s

ong

can

be s

elec

ted.

Son

gs c

an b

e po

sted

son

gs o

nlin

e, o

r us

ed to

cre

ate

mus

ic v

ideo

s

2S

impl

e m

usic

tool

kit

Softw

are

appl

icat

ions

Six

prog

ram

s th

at in

trodu

ce k

ey m

usic

al c

once

pts

in a

n in

tera

ctiv

e w

ay (2

Sim

ple,

201

7).

Empt

y-ha

nded

Cam

era-

base

dM

icro

soft

Kin

ect

RG

B c

amer

a, d

epth

se

nsor

, and

mul

ti-ar

ray

mic

roph

one

Prov

ides

full

body

3D

mot

ion

capt

ure,

voi

ce a

nd fa

ce r

ecog

nitio

n (K

inec

tSEN

, 201

8).

Eyes

Web

O

pen

deve

lopm

ent

softw

are

plat

form

R

eal-

time

mul

timod

al s

yste

m a

nd in

terf

ace

that

has

bee

n us

ed e

xten

sive

ly in

res

earc

h.

Syst

em s

uppo

rts

inpu

t dev

ices

incl

udin

g m

otio

n ca

ptur

e sy

stem

s, v

ideo

cam

eras

, ga

me

inte

rfac

es (e

.g.,

Kin

ect,

Wii)

, aud

io in

put,

anal

og in

puts

(e.g

., fo

r ph

ysio

logi

cal

sign

als)

. Out

puts

incl

ude

mul

ticha

nnel

aud

io, v

ideo

, ana

log

devi

ces,

rob

otic

pl

atfo

rms

(Cam

urri

et a

l., 2

000)

. Web

site

feat

ures

info

rmat

ion

on d

evel

opm

ent (

http

://w

ww

.info

mus

.org

/eye

sweb

_ita

.php

)

M

otio

nCom

pose

r (a

vaila

ble

2019

)Sy

stem

usi

ng tw

o ty

pes

of c

amer

a to

det

ect

mov

emen

t

Allo

ws

gest

ures

to b

e us

ed to

exp

lore

sou

nd e

nvir

onm

ents

(Ber

gsla

nd &

Wec

hsle

r, 20

16).

AU

MI

Free

sof

twar

e ap

plic

atio

n In

terf

ace

that

ena

bles

the

user

to p

lay

soun

ds a

nd m

usic

al p

hras

es th

roug

h m

ovem

ent

and

gest

ures

cap

ture

d vi

a w

ebca

m (L

arse

n, O

verh

olt,

& M

oesl

und,

201

6; O

liver

os,

Mill

er, H

eyen

, Sid

dall,

& H

azar

d, 2

011)

.

V

MI (

Vir

tual

Mus

ical

In

stru

men

t)Fr

ee s

oftw

are

Use

s w

eb c

amer

a to

det

ect m

otio

n. U

ser

virt

ually

“to

uch”

sha

pes

on s

cree

n to

trig

ger

soun

ds. R

equi

res

no s

peci

al e

quip

men

t, W

indo

ws

only

bas

ed c

ompu

ter.

Des

igne

d fo

r us

e by

ther

apis

ts a

nd e

duca

tors

, it i

s cu

stom

izab

le a

ccor

ding

to th

e pr

efer

ence

s an

d ne

eds

of th

e us

er, a

nd c

an b

e us

ed fo

r sp

ecifi

c th

erap

y or

edu

catio

nal g

oals

(Vir

tual

M

usic

al In

stru

men

t, 20

18).

Big

Eye—

disc

ontin

ued

still

ava

ilabl

e fo

r do

wnl

oad

Mac

into

sh o

nly

softw

are

prog

ram

U

ses

vide

o in

form

atio

n to

con

vert

into

MID

I mes

sage

s. A

llow

s tr

acki

ng o

f obj

ects

th

roug

h sp

ace

conv

ertin

g th

eir

para

met

ers

into

MID

I in

real

-tim

e (L

egac

y Pr

oduc

ts,

2018

).

C

amer

a Th

erem

in

Free

test

app

licat

ion

Cre

ate

soun

d fr

om m

ovem

ent u

sing

web

cam

(Cam

era

Ther

emin

, 201

8).

Mus

ical

Ges

ture

s To

olbo

xTo

olki

t for

exp

erie

nced

M

ax p

rogr

amm

ers.

C

olle

ctio

n of

mod

ules

and

abs

trac

tions

for

the

grap

hica

l pro

gram

min

g en

viro

nmen

t M

ax 5

to e

nabl

e ex

trac

tion

of m

ovem

ent d

ata

from

vid

eo (J

ense

nius

, God

oy, &

W

ande

rley

, 200

5).

Aer

odru

mPa

ckag

e fe

atur

ing

drum

stic

ks, s

oftw

are,

fe

et m

arke

rs, a

nd c

amer

a

An

air-

drum

min

g in

stru

men

t. R

uns

on c

ompu

ter

usin

g a

high

-spe

ed c

amer

a to

trac

k m

ovem

ents

to tr

igge

r dr

ums

(Kni

ght &

Kro

ut 2

017)

.

B

reak

-bea

mB

eam

zD

evic

e fe

atur

ing

four

br

eaka

ble

lase

r be

ams

Can

be

purc

hase

d as

a p

rofe

ssio

nal p

acka

ge fe

atur

ing

softw

are,

son

gs, s

truc

ture

d ac

tiviti

es, t

hera

py g

uide

s, a

nd le

sson

pla

ns

So

undb

eam

Dev

ice

feat

urin

g ul

tras

onic

bea

ms

and

switc

hes

(bot

h w

ired

or

wir

eles

s) a

nd a

sy

nthe

size

r un

it

Dev

ice

whi

ch u

ses

sens

or te

chno

logy

(up

to fo

ur u

ltras

onic

or

eigh

t sw

itche

s) to

tr

ansl

ate

body

mov

emen

t int

o m

usic

and

sou

nd u

sing

MID

I. N

ew u

nit f

eatu

res

touc

h sc

reen

inte

rfac

e, e

xten

sive

libr

ary

of s

ound

s, r

ecor

ding

and

com

posi

ng fu

nctio

ns,

trai

ning

pro

gram

s al

so a

vaila

ble.

Ext

ensi

ve s

uppo

rt fo

r us

e av

aila

ble

thro

ugh

onlin

e re

sour

ces

(http

s://w

ww

.sou

ndbe

am.c

o.uk

/).

Tabl

e 1.

Con

tinue

d

Inte

ract

ion

mod

alit

yC

omm

erci

ally

/ fr

eely

ava

ilabl

eD

escr

ipti

onD

etai

ls

(Con

tinue

d)

Dow


ic.oup.com/m


http://www.infomus.org/eyesweb_ita.php

http://www.infomus.org/eyesweb_ita.php

https://www.soundbeam.co.uk/


Ther

emin

(Mag

ee,

2006

)M

oog

Ther

emin

i—de

vice

feat

urin

g 32

w

avet

able

pre

set s

ound

s,

and

on-b

oard

spe

aker

an

d so

und

engi

ne

Can

be

used

at a

ny s

kill

leve

l, pr

ovid

ing

new

way

s to

exp

erim

ent w

ith m

usic

an

d ge

stur

al c

ontro

l. A

ssis

tive

pitc

h qu

antiz

atio

n al

low

s ea

ch p

laye

r to

adju

st th

e in

stru

men

t’s le

vel o

f pla

ying

diffi

culty

. “A

t the

max

imum

pos

ition

, the

The

rem

ini w

ill

play

eve

ry n

ote

in a

sel

ecte

d sc

ale

perfe

ctly

, mak

ing

it im

poss

ible

to p

lay

a w

rong

no

te. A

s co

ntro

l is

decr

ease

d, m

ore

expr

essi

ve c

ontro

l of p

itch

and

vibr

ato

beco

mes

po

ssib

le. W

hen

set t

o m

inim

um, t

he T

here

min

i will

per

form

as

a tra

ditio

nal T

here

min

” (T

here

min

i, 20

18).

Bui

lt-in

tune

r with

real

-tim

e vi

sual

feed

back

of p

laye

d no

tes

and

prox

imity

(use

ful f

or c

orre

ctin

g pl

ayin

g po

sitio

n). S

tore

sel

ecte

d sc

ale

& ro

ot n

ote,

se

t and

reca

ll a

spec

ified

pla

ying

rang

e, a

nd s

peci

fy p

atch

set

tings

. Sile

nt re

hear

sal

avai

labl

e vi

a he

adph

one

jack

. Tw

o lin

e le

vel a

udio

out

puts

, a p

itch

CV

out

put w

ith

sele

ctab

le ra

nge,

and

a m

ini U

SB ja

ck fo

r MID

I I/O

and

con

nect

ivity

(The

rem

ini,

2018

).

O

ptim

usic

/O

ptiB

eam

s (K

nigh

t an

d K

rout

201

7)

Inte

ract

ive

light

bea

ms

Pack

age

with

inte

ract

ive

light

uni

ts (t

he b

eam

s), l

apto

p w

ith O

ptiM

usic

sof

twar

e, U

SB

cont

rolle

r bo

x, u

ser

butto

n bo

x, r

eflec

tive

pads

/bat

s. In

tera

ct w

ith c

olor

ful b

eam

s of

lig

ht (2

, 4, 6

, 8, o

r 12

bea

ms)

, pas

s ha

nd o

r bo

dy th

roug

h th

e be

ams

or u

se r

eflec

tive

wan

d to

trig

ger

audi

ovis

ual e

vent

s in

rea

l-tim

e. C

omes

with

ove

r 80

inte

ract

ive

setti

ngs.

Pac

kage

als

o co

mes

with

trai

ning

(on-

site

or

e-tr

aini

ng) (

Opt

iBea

m, 2

018)

.

M

idiG

estu

reU

ltras

onic

bea

m s

enso

rSe

nsor

that

plu

gs in

to th

e M

IDIC

reat

or s

yste

m (s

ee M

IDIC

reat

or).

Leap

Mot

ion

(Lea

p M

otio

n, 2

017)

Smal

l dev

ice

to tr

ack

hand

mov

emen

t U

ses

two

mon

ochr

omat

ic IR

cam

eras

and

thre

e in

frar

ed L

EDs

to tr

ack

hand

and

fin

ger

mov

emen

t abo

ve d

evic

e. D

icke

ns, G

reen

halg

h, a

nd K

olev

a (2

017)

pro

vide

an

in-d

epth

des

crip

tion

of r

esea

rch

cond

ucte

d us

ing

the

Leap

Mot

ion

for

mus

ic

perf

orm

ance

with

use

rs w

ith c

ompl

ex d

isab

ilitie

s.

Bra

in c

ompu

ter

inte

rfac

e (B

CI)

Bra

infin

gers

(B

rain

finge

rs, 2

018)

Hea

dban

d fit

ted

with

se

nsor

s “D

etec

ts e

lect

rica

l sig

nals

from

faci

al m

uscl

es, e

ye m

ovem

ent a

nd b

rain

wav

es.

Bra

infin

gers

doe

s no

t dir

ectly

targ

et m

usic

cre

atio

n, a

s it

can

solv

e m

any

task

s su

ch

as s

impl

e cl

icki

ng, t

o co

mpl

ex c

ombi

natio

ns o

f con

trol

s. It

is s

oftw

are

that

con

vert

s al

l the

sen

sor

inpu

t dat

a in

to c

ontr

ols

term

ed B

rain

finge

rs. T

his

softw

are

is u

sefu

l for

a

broa

d ra

nge

of u

sers

, esp

ecia

lly p

eopl

e w

ith s

ever

e di

sabi

litie

s” (L

arse

n et

al.,

201

6,

p. 3

29).

Con

trol

s m

ost A

AC

sof

twar

e, e

duca

tiona

l sof

twar

e, a

nd v

ideo

gam

es.

Ey

e ga

zeEy

eMus

ic (L

arse

n et

al

., 20

16)—

lega

cy

files

ava

ilabl

e on

line,

re

quir

es te

chni

cal

skill

to in

stal

l (E

yeM

usic

, 201

8)

Softw

are

utili

zing

ge

neri

c ey

e ga

ze

equi

pmen

t

Syst

em th

at u

ses

eye

mov

emen

ts a

s in

put t

o el

ectr

onic

mus

ic c

ompo

sitio

ns. C

an b

e us

ed w

ith e

stab

lishe

d co

mpo

sitio

n so

ftwar

e al

low

ing

prer

ecor

ded

eye

mov

emen

t dat

a to

con

trol

mus

ical

com

posi

tions

(Hor

nof &

Sat

o, 2

004)

.

E-Sc

ape

(And

erso

n,

2018

)So

ftwar

e ut

ilizi

ng

gene

ric

eye

gaze

eq

uipm

ent (

or s

witc

hes)

Mus

ic s

oftw

are

spec

ifica

lly d

esig

ned

to b

e us

ed b

y pe

ople

with

dis

abili

ties

to c

reat

e or

per

form

mus

ic. S

yste

m o

pera

tes

via

larg

e gu

ided

pop

-up

men

us c

ontr

olla

ble

by

one

or m

ore

switc

hes,

mou

se, k

eybo

ard,

eye

gaz

e, o

r M

IDI c

ontr

olle

rs o

r se

nsor

s.

“At e

very

sta

ge, E

-Sca

pe a

sks

the

user

wha

t the

y w

ant t

o do

and

offe

rs a

ran

ge o

f op

tions

dep

endi

ng u

pon

whi

ch le

vel o

f com

plex

ity th

e us

er h

as c

hose

n to

wor

k at

” (F

arri

mon

d et

al.,

201

1, p

. 23)

. Tw

o m

odes

of o

pera

tion—

com

posi

tion

and

perf

orm

ance

. Can

out

put M

IDI d

ata

(Far

rim

ond

et a

l., 2

011)

.

Tabl

e 1.

Con

tinue

d

Inte

ract

ion

mod

alit

yC

omm

erci

ally

/ fr

eely

ava

ilabl

eD

escr

ipti

onD

etai

ls

(Con

tinue

d)

Dow


ic.oup.com/m



EyeH

arp

Free

sof

twar

e ut

ilizi

ng

gene

ric

eye

gaze

eq

uipm

ent

Gaz

e-co

ntro

lled

or h

ead-

cont

rolle

d m

usic

inte

rfac

e to

hel

p us

ers

lear

n an

d pl

ay

mus

ic. V

amva

kous

is a

nd R

amir

ez (2

016)

pro

vide

a c

ompr

ehen

sive

art

icle

on

the

deve

lopm

ent o

f the

Eye

Har

p. T

he w

ebsi

te th

eeye

harp

.org

als

o pr

ovid

es a

wea

lth o

f in

form

atio

n ab

out t

he p

roje

ct (T

he E

yeH

arp,

201

8).

Eye

Play

Mus

icFr

ee s

oftw

are

utili

zing

ge

neri

c ey

e ga

ze

equi

pmen

t

Trig

ger

note

s fr

om a

ran

ge o

f ins

trum

ents

ava

ilabl

e w

ith a

djus

tabl

e no

te le

ngth

and

tr

ansp

ositi

on. C

reat

e ow

n sc

ales

. Loa

d an

d sa

ve s

ettin

gs. W

ebsi

te fe

atur

es r

esou

rce

for

use

(MB

MM

, 201

7).

B

reat

hJa

mbo

xxM

IDI c

ontr

olle

r de

vice

Han

ds-f

ree

elec

tron

ic, b

reat

h-po

wer

ed in

stru

men

t. U

ses

sip

and

puff

to d

eter

min

e am

plitu

de o

f not

e. S

oftw

are

incl

uded

to c

onfig

ure

devi

ce a

nd o

n-bo

ard

mod

ulat

ion

whe

el, b

utto

n, a

nd ja

ck s

ocke

t to

allo

w s

witc

h co

nnec

tivity

(Jam

boxx

, 201

8).

Yam

aha

WX

5, W

X11

MID

I con

trol

ler

devi

ces

Bre

ath

pow

ered

MID

I con

trolle

r tha

t allo

ws

for o

ne-h

ande

d pl

ayin

g. W

X5

feat

ures

MID

I ou

tput

; how

ever

, WX

11 re

quire

s an

add

ition

al M

IDI c

onne

ctio

n bo

x (M

BM

M, 2

017)

.

M

agic

Flu

teSt

and-

alon

e in

stru

men

tSe

lf-co

ntai

ned

inst

rum

ent w

ith b

uilt-

in to

ne g

ener

atin

g ha

rdw

are.

Plu

gs in

to e

xter

nal

spea

kers

. Tw

o se

para

te p

arts

, the

flut

e an

d co

ntro

l mod

ule

with

dis

play

. The

flut

e be

ing

the

rem

ote

cont

rol f

or th

e co

ntro

l mod

ule.

Mus

icia

ns c

an s

elec

t diff

eren

t so

unds

or

acce

ss th

e us

er s

ettin

gs w

ithou

t the

hel

p of

ano

ther

per

son

(MB

MM

, 20

17).

The

volu

me

is c

ontr

olle

d by

blo

win

g in

a m

outh

piec

e an

d th

e pi

tch

by

mov

ing

the

mou

thpi

ece

up/d

own

with

the

mou

th (V

amva

kous

is &

Ram

irez

, 201

6).

“The

inst

rum

ent r

educ

es th

e ph

ysic

al a

nd c

ogni

tive

chal

leng

es in

here

nt w

ithin

co

nven

tiona

l win

d in

stru

men

ts. O

ne m

usic

ian,

with

ver

y lim

ited

lung

vol

ume,

is

none

thel

ess

able

to r

ealiz

e th

e fu

ll dy

nam

ic r

ange

of t

he in

stru

men

t” (F

arri

mon

d et

al

., 20

11, p

. 29)

.In

tera

ctio

n m

odal

ity

R

esea

rch

only

Des

crip

tion

Det

ails

Touc

h

Gen

eral

res

earc

h

C

olla

bora

tive

inte

rfac

es r

evie

w o

f lit

erat

ure

(Bla

ine

and

Fels

, 200

3)

Rev

iew

pap

er o

f in

terf

aces

use

d fo

r co

llabo

rativ

e m

usic

-mak

ing

Com

preh

ensi

ve r

evie

w o

f con

text

and

des

ign

of a

num

ber

mus

ical

exp

erie

nces

for

novi

ces

Spec

ific

rese

arch

Mus

icki

ng

Tang

ible

s—R

HY

ME

Proj

ect

Tang

ible

inte

rfac

es

cons

istin

g of

inte

ract

ive

digi

tal “

furn

iture

”

Exam

inin

g th

e de

velo

pmen

t and

ben

efits

of u

sing

inte

ract

ive

digi

tal m

usic

furn

iture

fo

r di

sabl

ed c

hild

ren

by u

sing

two

co-c

reat

ive

tang

ible

inst

rum

ents

. OR

FI-2

6 so

ft py

ram

id-s

hape

d, p

illow

-lik

e m

odul

es, i

n th

ree

diffe

rent

siz

es (3

0–90

cm

) fea

turi

ng

bend

sen

sors

and

ligh

ts, t

he u

nits

can

com

mun

icat

e w

irel

essl

y w

ith e

ach

othe

r. W

ave

Car

pet-7

-bra

nche

d, w

ired,

inte

ract

ive,

sof

t, da

rk c

arpe

t with

ora

nge

velv

et ti

ps th

at

glow

. Cen

tral a

rm c

onta

ins

mic

roph

one,

two

arm

s co

ntai

n ac

cele

rom

eter

s th

at c

hang

e th

e re

cord

ed s

ound

. Tw

o ar

ms

cont

ain

bend

sen

sors

that

cre

ate

rhyt

hmic

al b

ackg

roun

d m

usic

. One

arm

con

tain

s a

web

cam

era.

Con

tain

five

sof

twar

e pr

ogra

ms,

offe

ring

diffe

rent

mus

ic a

nd d

ynam

ic g

raph

ics

to s

how

via

pro

ject

or e

mbe

dded

in o

ne a

rm, o

r vi

a fu

ll w

all p

roje

ctio

n. C

ente

r con

tain

s tw

o sp

eake

rs a

nd s

trong

vib

rato

r in.

Con

tain

s IR

-sen

sors

allo

win

g in

tera

ctio

n w

ith R

GB

LED

ligh

ts (A

nder

sson

& C

appe

len,

201

3).

Tabl

e 1.

Con

tinue

d

Inte

ract

ion

mod

alit

yC

omm

erci

ally

/ fr

eely

ava

ilabl

eD

escr

ipti

onD

etai

ls

(Con

tinue

d)

Dow


ic.oup.com/m


http://theeyeharp.org


Noi

seB

ear

(Gri

erso

n an

d K

iefe

r, 20

13)

Mal

leab

le c

ontr

olle

rD

evel

opm

ent o

f rob

ust,

wir

eles

s, m

alle

able

con

trol

ler

for

child

ren

with

cog

nitiv

e or

ph

ysic

al d

isab

ilitie

s

B

ean

Ges

tura

lly c

ontr

olle

d di

gita

l ins

trum

ent

Dev

ice

desi

gned

aro

und

a W

ii nu

nchu

ck c

ontr

olle

r fo

r us

e in

a m

usic

ther

apy

setti

ng

(Kir

wan

, Ove

rhol

t, &

Erk

ut, 2

015)

MA

Wii

Dig

ital m

usic

al

inst

rum

ent u

sing

gen

eric

W

iiMot

e co

ntro

ller

Res

earc

h ex

plor

ing

WiiM

otes

as

virt

ual i

nstr

umen

ts fo

r ch

ildre

n w

ith b

ehav

iora

l di

sord

ers

(Ben

veni

ste

et a

l., 2

008)

.

Wam

Bam

Self-

cont

aine

d el

ectr

onic

ha

nd d

rum

Cre

ated

usi

ng p

iezo

sen

sors

. Pap

er d

escr

ibes

dev

elop

men

t and

test

ing

of d

evic

e us

ed

in fo

r m

usic

ther

apy

sess

ions

with

sev

erel

y in

telle

ctua

lly d

isab

led

clie

nts

(Jens

e &

Le

euw

, 201

5).

Touc

hTon

eD

igita

l mus

ical

in

stru

men

tD

evic

e fe

atur

ing

touc

h-se

nsiti

ve p

ads

desi

gned

to d

evel

op m

usic

al a

bilit

y, b

iman

ual

coor

dina

tion

and

incr

ease

soc

ial p

artic

ipat

ion

of c

hild

ren

with

hem

iple

gia

(Bha

t, 20

10).

Com

pute

r-A

ssis

ted

Mus

ic T

hera

pyA

ugm

ente

d re

ality

so

ftwar

eD

etai

ls s

yste

m d

evel

oped

with

Aug

men

ted

Rea

lity

tech

niqu

es a

llow

ing

mus

ic

com

posi

tion

and

crea

tion

activ

ities

usi

ng s

ound

and

col

our,

via

card

s (C

orre

a,

Fich

eman

, do

Nas

cim

ento

, & d

e D

eus

Lope

s, 2

009)

.

Se

nseE

ggW

irel

ess

cont

rolle

r de

vice

Dev

elop

men

t of a

han

dhel

d eg

g-sh

aped

dev

ice

feat

urin

g se

ven

on-b

oard

sen

sors

(b

utto

n, s

lider

, acc

eler

omet

er, w

ind

sens

or, u

ltras

onic

dis

tanc

e se

nsor

) and

a s

uite

of

sof

twar

e pa

tche

s ai

med

at f

or m

usic

al e

xplo

ratio

n an

d te

achi

ng. F

eatu

red

a co

mpo

nent

that

allo

wed

con

trol

of s

ettin

gs v

ia a

n iP

ad (B

lath

erw

ick

& C

obb,

201

5).

Soft

war

e-ba

sed

D

IYSE

sof

twar

eSo

ftwar

e th

at u

tiliz

es

Gui

tar

Her

o co

ntro

llers

Det

ails

dev

elop

men

t of s

oftw

are

allo

win

g co

nnec

tion

of e

xist

ing

cont

rolle

rs (G

uita

r H

ero

& W

iiMot

e, e

tc.)

to c

ompo

se a

nd r

esto

re m

usic

trac

ks, a

nd d

esig

n m

appi

ng

stra

tegi

es b

etw

een

inte

rfac

e an

d pl

ayed

sou

nds,

for

peop

le w

ho w

ith in

telle

ctua

l le

arni

ng d

isab

ilitie

s (L

uhta

la, K

ymäl

äine

n, &

Plo

mp,

201

1).

Empt

y-ha

nded

Cam

era-

base

dM

ovem

ent t

o M

usic

-MTM

Web

cam

era

and

softw

are

syst

emD

evel

oped

to a

ddre

ss th

e ne

ed fo

r af

ford

able

hom

e-ba

sed

mus

ical

pla

y sy

stem

, in

corp

orat

ing

auto

mat

ic m

ovem

ent r

ecog

nitio

n te

chno

logy

that

is n

onco

ntac

t and

no

ninv

asiv

e (T

am, S

chw

elln

us, C

eilid

h, H

amda

ni, L

amon

t, &

Cha

u, 2

007)

.

Bre

ak-b

eam

Ben

emic

/Oct

onic

Stan

d-al

one

inst

rum

ent

Dev

ice

with

arr

ay o

f eig

ht lo

w-c

ost i

nfra

red

dist

ance

sen

sors

. Ena

blin

g tr

igge

ring

and

m

anip

ulat

ion

of s

ound

s us

ing

MID

I mes

sage

s (C

halli

s, 2

011)

.

Eye

gaze

Eye

cond

ucto

rSo

ftwar

e us

es w

ebca

mSo

ftwar

e-ba

sed

mus

ical

inte

rfac

e to

pla

y m

usic

thro

ugh

eye

mov

emen

ts a

nd fa

cial

ge

stur

e us

ing

eye

trac

ker

equi

pmen

t and

web

cam

. Det

ects

gaz

e an

d se

lect

ed fa

cial

m

ovem

ents

ena

blin

g pl

ayin

g of

inst

rum

ents

, bea

t bui

ldin

g, s

eque

ncin

g m

elod

ies,

or

trig

geri

ng m

usic

al e

ffect

s (R

efsg

aard

, 201

8).

Not

e. M

IDI,

mus

ical

inst

rum

ent d

igita

l int

erfa

ce.

Tabl

e 1.

Con

tinue

d

Inte

ract

ion

mod

alit

yC

omm

erci

ally

/ fr

eely

ava

ilabl

eD

escr

ipti

onD

etai

ls

Dow


ic.oup.com/m



control can lead to changes in behavior patterns beyond the environment of a therapy session with individuals becoming more self-aware and interactive outside of the sessions, more tolerant, and with a growing awareness of others (Swingler, 1998). Hunt and colleagues (2004) suggested that technology offers access to real-time sound control to those with limited movement, along with new sound worlds and timbres (Ellis & Van Leeuwen, 2000; Hunt et al., 2000; Kirk, Hunt, Hildred, Neighbour, & North, 2002; Misje, 2013). Computer music can be intriguing, particularly to young people, who may find trad-itional instruments, which are often associated with strict dis-ciplined methods, “off-putting” (Hunt et al., 2004).

Technology can also offer the sense of control and au-tonomy (Crowe & Rio, 2004) removing the need for pre-requisite skills for learning to occur (Nagler, 2011). This can help clients reach peak experiences that would be difficult using traditional instruments (Misje, 2013). Technology can offer the ability to readily create music, learn to play an elec-tric instrument, use computer programs, and/or to write and record. These activities can be condensed into a small amount of equipment, by offering the potential for many instruments to be accessed from one set up. This provides a “blank sheet” (Kirk et al., 2002) onto which individual instruments can be built for different uses/users. “This aural richness and variety provides the internal motivation.…. In addition, the technology also provides physical access for [people with disabilities]” (Ellis, 1997, p. 176). In cases where affordability is an issue, technology could be beneficial, given how expensive acoustic instruments can be. “It is possible to create sounds with as much musical interest as familiar orchestra instruments but which could not be produced by a known instrument. A new dimension for interaction can then be opened up, offering rad-ical possibilities for performance” (Kirk et al., 2002, p. 1023) that allow for and support unconventional playing (Ellis & Van Leeuwen, 2000). DMIs do not need to sound or play like con-ventional instruments, and they can be created to be operated by any part of the anatomy with no right or wrong technique, only that which is appropriate to the individual (Ellis & Van Leeuwen, 2000).

Music Technology Used in Music Therapy Practice

Music technology offers up new possibilities for exploration within music as part of the larger framework of music therapy (Misje, 2013). It has been used for many music-making activ-ities both as an active music technique (singing, music com-position, instrument playing) and as receptive intervention such as listening. Technology has also enabled the exploration of activities such as songwriting, recording, improvisation, listening, recreative, and multimedia project development as well as studying, learning, and composing and serving the needs of individuals with disabilities both in medical prac-tice and research (Crowe & Rio, 2004; Viega, 2016). Music technology in music therapy has been used to address iden-tity development (Magee, 2006); express thoughts and feel-ings (Whitehead-Pleaux et al., 2011); promote empowerment (Burland & Magee, 2013; Cappelen & Andersson, 2013); construct meaning (McDowall, 2008); and develop agency (Kruger, 2007). The development of on-task behavior, con-centration, cooperation, communication, self-expression, problem solving, and decision-making have all been shown

to be supported through the use of technology (Crowe & Rio, 2004). Technology can be particularly useful for those with short attention spans as it can be set up to instantaneously provide a relevant and enticing response, leading to enhanced focus and the potential to transcend disability (Swingler, 1998). Technology can also be used to provide individual control by community participation (Misje, 2013). This can be seen in the work of Andersson and Cappelen (2013), and through the RHYME project, using tangible interfaces for musicking (Small, 2011).

Incorporating Music Technology Into Practice

Nagler (2011) suggests the next steps for the inclusion of technology (specifically digital handheld music-making de-vices) in music therapy clinical practice are:

1. The creation and development of applications that allow for music therapists to use musical methods analogous with practices achieved using traditional instruments, thus allowing for “demonstration of patient progress to-ward specific goal attainment” (p. 198)

2. The development of accepted, common guidelines from experts in the field with best practices needed to dictate methods. Nagler (2011) suggests that the development and sharing of technology could be spurred on by the use of Creative Commons licensing and open-source networks. This includes the need to create a taxonomy of understanding (to codify the pitfalls, methods, and potentials) incorporating the vocabulary, structure, and architecture of technology (specifically of handheld music devices) into clinical practice

Farrimond and colleagues (2011) suggest simplifying the complexity of available technology by distinguishing between access needs and learning needs to aid in finding technology that is suitable for providing musical possibilities for clients. This can then lead to an emphasis on the creative prefer-ences and needs of the individual. Magee and Burland (2008) echoed this by advising “recommendations from allied fields advise that access to music-making for an individual with disabilities needs to start with examining the variance of the individual’s abilities, the type of input required to achieve a task, and the possible mappings between the two” (p. 126).

Further, developments in music education such as the Sounds of Intent framework (Vogiatzoglou, Ockelford, Welch, & Himonides, 2011) seek to provide “evidence-based guidance on appropriate music pedagogy for all children in special edu-cation (thus informing policy and practice)” (Welch, Ockelford, Zimmermann, Himonides, & Wilde, 2015, p. 3). The resources they provide are aimed at mapping the musical development of children and young people in special education settings.

Finally, a key issue for designers of new technology to con-sider is the “musicality, usability, accessibility and afford-ability” of technology (Challis, 2011, p. 6). As such, there are several design considerations that should be reviewed when designing technology specifically for the special education set-ting (Ward et al., 2017). These considerations aim to maximize the potential for new developments to be incorporated into practice, make technological tools less daunting to everyday users, and foster creativity and communication among users.

Dow


ic.oup.com/m



Conclusion

While it is clear that incorporating music technology into clinical practice to enable active music-making has a myriad of potential benefits, it is also clear that the ever-changing landscape of technology can be overwhelming This can create gaps between the developer, clinician, and client. This ever-changing landscape may be particularly overwhelming for music therapists not already steeped in technology, as these systems often consist of several layers of technologies that re-quire technical skill to combine. Practitioners may find it dif-ficult to keep up with changes in technology, and figure out how to combine and integrate them into their practice. Still, despite these technical and financial challenges, the utiliza-tion of technology provides unique access to music-making for those that could not access traditional instruments or rep-ertoire. Alternate controllers, in particular, provide a means to explore new ways of utilizing an individual’s physical and learning abilities to provide meaningful and motivating mu-sical experiences. Using music technology in this manner, on its own or alongside traditional instruments, requires a dif-ferent approach to integration, repertoire, and skill set of the users. This approach must take into account the type of tech-nology, how it will be used, and also the intended outcome.

The potential in using technology is evident from the devel-opments presented in this article; this potential, however, must be discussed, shared, and best practices developed. There is still a growing need for a reexamination of the content of edu-cation and training that places technology at the forefront of music-making scenarios. The combination of this and new partnerships with those already steeped in technology would lead to a more established use of technology and would thereby build a new generation of clinicians.

Funding

Funding for the Centre for Digital Entertainment granted by the EPSRC (code: EP/L016540/1).

Acknowledgments

With thanks to the Three Ways School, Bath, UK, Luke Woodbury of DotLib. Dr. Tom Davis, and Dr. Ann Bevan of Bournemouth University for supervisory support.

References

2Simple. (2017, May 13). 2simple. Retrieved from https://www.2simple.com/product/music-toolkit

Anderson, T. (2018, June 25). Inclusive music – Music and computer access for people with disabilities. Retrieved from http://www.inclusivemusic.org.uk/

Andersson, A. P., & Cappelen, B. (2013, May). Designing empowering vocal and tangible interaction. Paper presented at the International Conference on New Interfaces for Musical Expression, Republic of Korea.

Apps for Children with Special Needs. (2018, May 12). Apps for children with special needs. Retrieved from http://a4cwsn.com/

Arduino. (2015, August 10). Arduino. Retrieved from https://www.arduino.cc/Autodesk. (2018, July 5). Let’s make. Retrieved from https://www.instructables.comBache, J., Derwent, G., & Magee, W. (2014). An introduction to using assistive de-

vices in music therapy. In W. Magee, (Ed.), Music technology in therapeutic and health settings. London, England: Jessica Kingsley Publishers, 63–82.

Bare Conductive. (2018, August 20). Bare conductive. Retrieved from https://www.bareconductive.com/

Benveniste, S., Jouvelot, P., & Michel, R. (2008). Wii game technology for music therapy: A first experiment with children suffering from behavioral disorders. Paper presented at the MCCSIS’08 - IADIS Multi Conference on Computer Science and Information Systems, Amsterdam.

Bergsland, A., & Wechsler, R. (2016). Turning movement into music – Issues and applications of the MotionComposer, a therapeutic device for persons with dif-ferent abilities. Sound Effects, 6(1), 23–47. doi:10.7146/se.v6i1.24911

Bhat, S. (2010, January). TouchTone: An electronic musical instrument for chil-dren with hemiplegic cerebral palsy. Paper presented at the fourth inter-national conference on Tangible, Embedded, and Embodied Interaction, Cambridge, MA.

Blaine, T. (2005, May). The convergence of alternate controllers and musical inter-faces in interactive entertainment. Paper presented at the 2005 conference on New Interfaces for Musical Expression, Singapore.

Blaine, T., & Fels, S. (2003). Collaborative musical experiences for novices. Journal of New Music Research, 32(4), 411–428. doi:10.1076/jnmr.32.4.411.18850

Blatherwick, A., & Cobb, J. (2015, June). SenseEgg: A wireless music controller for teaching children with special learning needs. Paper presented at ICSRIM: Music, Media, and Electronics, Leeds.

Blum, M. (2018, May 13). Soundplant: Computer keyboard sample triggering for Windows & Mac. Retrieved from http://soundplant.org/

Bongers, B. (2000). Physical interfaces in the electronic arts interaction theory and interfacing techniques for real-time performance. Trends in Gestural Control of Music, 41–70.

Brainfingers. (2018, February 21). Brainfingers.com. Retrieved from http://brainfingers.com/

Burland, K., & Magee, W. L. (2013). Music technology and identity in therapeutic contexts. In W. Magee (Ed.), Music technology in therapeutic and health set-tings. London, England: Jessica Kingsley Publishers, 327–348.

Camera Theremin. (2018, February 21). electroAT. Retrieved from https://code.google.com/archive/p/tukesprint/

Camurri, A., Hashimoto, S., Ricchetti, M., Ricci, A., Suzuki, K., Trocca, R., & Volpe, G. (2000). EyesWeb: Toward gesture and affect recognition in inter-active dance and music systems. Computer Music Journal, 24(1), 57–69. doi:10.1162/014892600559182

Cappelen, B., & Andersson, A. P. (2013, July). Towards an empowering tangible inter-action design for diversity. Paper presented at Include Asia 2013, Hong-Kong.

Cevasco, A. M., & Hong, A. (2011). Utilizing technology in clinical practice: A com-parison of board-certified music therapists and music therapy students. Music Therapy Perspectives, 29(1), 65–73. doi:10.1093/mtp/29.1.65

Challis, B. (2009). Technology, accessibility and creativity in popular music educa-tion. Popular Music, 28(3), 425–431.

Challis, B. (2011). Octonic: An accessible electronic musical instrument. Digital Creativity, 22(1), 1–12. doi:10.1017/S0261143009990158

Challis, B., & Smith, R. (2012). Assistive technology and performance behaviours in music improvisation. In A. L. Brooks (Ed.), Lecture notes of the institute for com-puter sciences, social informatics and telecommunications engineering (Vol. 101). Berlin, Germany: Springer, 63–70.

Clements-Cortes, A. (2013). High-Tech therapy: Music technology in music therapy. Canadian Music Educator, 54(4), 37–39.

Clements-Cortes, A. (2014). Getting your groove on with the tenori-on. Journal of Music Technology & Education, 7(1), 59–74. doi:10.1386/jmte.7.1.59_1

Correa, A. G. D., Ficheman, I. K., do Nascimento, M., & de Deus Lopes, R. (2009, July). Computer assisted music therapy: A case study of an augmented reality musical system for children with cerebral palsy rehabilitation. Presented at the conference on Advanced Learning Technologies, Riga.

Cripps, C., Tsiris, G., & Spiro, N. (Eds.). (2016). Research and resources for music therapy 2016: A compilation developed by the Nordoff Robbins research team. London: Nordoff Robbins. Retrieved from https://www.nordoff-robbins.org.uk/sites/default/files/research_and_resources_for_music_therapy_2016_final.pdf

Crooke, A. H. D. (2018). Music technology and the hip hop beat making tradition: A history and typology of equipment for music therapy. Voices: A World Forum for Music Therapy, 1–15. doi:10.15845/voices.v18i2.996

Crooke, A. H. D., & McFerran, K. S. (2019). Improvising using beat making tech-nologies in music therapy with young people: It’s all about the resources. Music Therapy Perspectives. Advance online publication. doi:10.1093/mtp/miy025

Dow


ic.oup.com/m


https://www.2simple.com/product/music-toolkit

https://www.2simple.com/product/music-toolkit

http://www.inclusivemusic.org.uk/

http://a4cwsn.com/

https://www.arduino.cc/

https://www.instructables.com



http://soundplant.org/

http://brainfingers.com/

http://brainfingers.com/

https://code.google.com/archive/p/tukesprint/

https://code.google.com/archive/p/tukesprint/

https://www.nordoff-robbins.org.uk/sites/default/files/research_and_resources_for_music_therapy_2016_final.pdf

https://www.nordoff-robbins.org.uk/sites/default/files/research_and_resources_for_music_therapy_2016_final.pdf


Crowe, B. J., & Rio, R. (2004). Implications of technology in music therapy prac-tice and research for music therapy education: A review of literature. Journal of Music Therapy, 41(4), 282–320. doi:10.1093/jmt/41.4.282

Cycling‘74. (2018, February 21). Cycling74.com. Retrieved from https://cycling74.com/Department for Education. (2011). The importance of music: A national plan for

music education. Retrieved from https://www.gov.uk/government/publications/the-importance-of-music-a-national-plan-for-music-education

Dickens, A., Greenhalgh, C., & Koleva, B. (2017, August). Creating space for fa-cilitated music performance gesture controlled sound for users with complex disabilities. Paper presented at Audio Mostly, London.

Ellis, P. (1997). The music of sound: A new approach with severe and profound and multiple learning difficulties. British Journal of Music Education, 14(2), 173–186. doi:10.1017/S0265051700003624

Ellis, P., & Van Leeuwen, L. (2000). Living Sound: Human interaction and children with autism. In ISME commission on Music in Special Education, Music Therapy and Music Medicine. Regina, Canada: ISME.

EyeMusic. (2018, February 21). Cs.uoregon.edu. Retrieved from https://www.cs.uoregon.edu/research/cm-hci/EyeMusic/

Farrimond, B. (2016, May 15). The Clarion - An accessible musical instrument 8 years in the making. Retrieved from https://network.youthmusic.org.uk/posts/clarion-accessible-musical-instrument-8-years-making

Farrimond, B., Gillard, D., Bott, D., & Lonie, D. (2011). Engagement with technology in special educational & disabled music settings. Retrieved from https://network.youthmusic.org.uk/file/5694/download?token=I-1K0qhQ

Favilla, S., & Pedell, S. (2014). Touch screen collaborative music: Designing nime for older people with dementia. Paper presented at the 2014 New Interfaces for Musical Expression Conference, London. doi:10.1145/2541016.2541088

Gesturesen.wikispaces.com. (2018, May 13). GestureSEN - Home. Retrieved from http://gesturesen.wikispaces.com/home

Gorman, M., Lahav, A., Saltzman, E., & Betke, M. (2007). A camera-based music-making tool for physical rehabilitation. Computer Music Journal, 31(2), 39–53. doi:10.1162/comj.2007.31.2.39

Grierson, M., & Kiefer, C. (2013). NoiseBear: A malleable wireless controller de-signed in participation with disabled children. Paper presented at the 2013 New Interfaces for Musical Expression Conference, Daejeon.

Hadley, S., & Yancy, G. (2012). Therapeutic uses of rap and hip-hop. New York, NY: Routledge.

Hahna, N. D., Hadley, S., Miller, V. H., & Bonaventura, M. (2012). Music tech-nology usage in music therapy: A survey of practice. Arts in Psychotherapy, 39(5), 456–464. doi:10.1016/j.aip.2012.08.001

Hayes, L. (2016). Sound, electronics and music: An evaluation of early em-bodied education. Unpublished manuscript, Department of Arts, Media and Engineering, Arizona State University, Tempe, AZ.

Hornof, A. J., & Sato, L. (2004). EyeMusic: Making music with the eyes. Paper pre-sented at the 2004 International Conference on New Interfaces for Musical Expression, Hamamatsu.

Hunt, A. D., & Kirk, R. (2003). MidiGrid: past, present and future. Paper presented at the New Interfaces for Musical Expression, Daejeon.

Hunt, A., Kirk, R., Abbotson, M., & Abbotson, R. (2000). Music therapy and elec-tronic technology. Paper presented at the 2000 EUROMICRO conference. doi:10.1109/EURMIC.2000.874506

Hunt, A., Kirk, R., & Neighbour, M. (2004). Multiple media interfaces for therapy. IEEE Multimedia, 11(3), 50–58. doi:10.1109/MMUL.2004.12

I-CubeX. (2018, June 16). I-CubeX. Retrieved from http://infusionsystems.com/catalog/

Internet of Things. (2018, February 21). Theinternetofthings.eu. Retrieved from http://www.theinternetofthings.eu/

Jamboxx. (2018, February 21). Jamboxx. Retrieved from http://www.jamboxx.com/Jense, A., & Leeuw, H. (2015). WamBam: A case study in design for an electronic mu-

sical instrument for severely intellectually disabled users. Paper presented at the International conference on New Interfaces for Musical Expression, Baton Rouge.

Jensenius, A. R., Godoy, R. I., & Wanderley, M. M. (2005). Developing tools for studying musical gestures within the max / msp / jitter environment. Paper pre-sented at the 2005 International Computer Music Conference, Barcelona.

KinectSEN. (2018, February 21). Kinectsen.wikispaces.com. Retrieved from https://kinectsen.wikispaces.com/

Kirk, R., Abbotson, M., Abbotson, R., Hunt, A., & Cleaton, A. (1994). Computer music in the service of music therapy: The MIDIGRID and

MIDICREATOR systems. Medical Engineering and Physics, 16(3), 253–258. doi:10.1016/1350-4533(94)90046-9

Kirk, R., Hunt, A., Hildred, M., Neighbour, M., & North, F. (2002). Electronic mu-sical instruments - A role in music therapy? Paper presented at the 10th World Congress of Music Therapy, Oxford.

Kirwan, N. J., Overholt, D., & Erkut, C. (2015). Bean: A digital musical instrument for use in music therapy. Paper presented at the 12th Sound and Music Computing Conference, Maynooth.

Knight, A. (2013). Uses of iPad® applications in music therapy. Music Therapy Perspectives, 31(2), 189–196. doi:10.1093/mtp/31.2.189

Knight, A., & Krout, R. (2017). Making sense of today’s electronic music technology resources for music therapy. Music Therapy Perspectives, 35(2), 219–225. doi:10.1093/mtp/miw025

Knight, A., & Lagasse, A. B. (2012). Re-Connecting to music technology: Looking back and looking forward. Music Therapy Perspectives, 30(2), 188–195. doi:10.1093/mtp/30.2.188

Krout, R. E. (1987). The microcomputing music therapist: A primer. Music Therapy Perspectives, 4(1), 64–67. doi:10.1093/mtp/4.1.64

Krout, R. E. (1992). Integrating technology. Music Therapy Perspectives, 10(1), 8–9. doi:10.1093/mtp/10.1.8

Krout, R. E. (2014a). Music technology used in therapeutic and health settings. In W. Magee (Ed.), Music technology in therapeutic and health settings. London, England: Jessica Kingsley Publishers, 45–62.

Krout, R. (2014b). Engaging iPad applications with young people with autistic spec-trum disorders. In W. Magee (Ed.), Music technology in therapeutic and health settings. London, England: Jessica Kingsley Publishers, 181–198.

Krout, R. E. (2015). Evaluating electronic music technology resources for music therapy. Dallas, TX: Barcelona Publishers.

Kruger, V. (2007, February 21). Music as narrative technology. Voices: A world forum for music therapy. Retrieved from https://voices.no/index.php/voices/article/view/492/399

Kyub MIDI keyboard. (2018, February 18). Kyub. Retrieved from http://kyubmusic.com/

Larsen, J. V., Overholt, D., & Moeslund, T. B. (2016). The prospects of musical in-struments for people with physical disabilities. Paper presented at the 2016 New Instruments for Musical Expression conference, Brisbane.

Leap Motion. (2017, February 21). Leap Motion. Retrieved from https://www.leapmotion.com/

Legacy Products. (2018, February 16). STEIM. Retrieved from http://steim.org/product/discontinued-products/

Lightstone, A. (2012). The importance of hip-hop for music therapists. In Hadley & G. Yancy (Eds.), Therapeutic uses of rap and hip-hop. New York, NY: Routledge.

Lobby, H., & De Ridder, Y. (2018, May 14). Beatsurfing, 81–98. Retrieved from http://beatsurfing.net/

Luhtala, M., Kymäläinen, T., & Plomp, J. (2011). Designing a music performance space for persons with intellectual learning disabilities. Paper presented at the New Interfaces for Musical Expression conference, Oslo.

Lyons, M., & Fels, S. (2015, November 2). How to design and build new musical interfaces [PowerPoint presentation]. Retrieved from https://dl.acm.org/citation.cfm?id=2818143.2818145

Machover, T. (2004). Shaping minds musically. BT Technology Journal, 22(4), 171–179. doi:10.1023/B:BTTJ.0000047596.75297.ee

Magee, W. L. (2006). Electronic technologies in clinical music therapy: A survey of practice and attitudes. Technology and Disability, 18(3), 139–146.

Magee, W. L. (2012, April 18) Music and medicine: Applications of electronic music technologies in addressing psychic pain [Video File]. Retrieved from https://www.youtube.com/watch?v=u9-OJs93naA

Magee, W. L. (2014). Music technology in therapeutic health settings. London, England: Jessica Kingsley.

Magee, W. L., Bertolami, M., Kubicek, L., LaJoie, M., Martino, L., Sankowski, A., … Zigo, J. B. (2011). Using music technology in music therapy with populations across the life span in medical and educational programs. Music and Medicine, 3(3), 146–153. doi:10.1177/1943862111403005

Magee, W. L., & Burland, K. (2008). An exploratory study of the use of electronic music technologies in clinical music therapy. Nordic Journal of Music Therapy, 17(2), 124–141. doi:10.1080/08098130809478204

Makey Makey. (2018, February 21). Makeymakey.com. Retrieved from http://makeymakey.com/

Dow


ic.oup.com/m


https://cycling74.com/

https://www.gov.uk/government/publications/the-importance-of-music-a-national-plan-for-music-education

https://www.gov.uk/government/publications/the-importance-of-music-a-national-plan-for-music-education

https://www.cs.uoregon.edu/research/cm-hci/EyeMusic/

https://www.cs.uoregon.edu/research/cm-hci/EyeMusic/

https://network.youthmusic.org.uk/posts/clarion-accessible-musical-instrument-8-years-making

https://network.youthmusic.org.uk/posts/clarion-accessible-musical-instrument-8-years-making

https://network.youthmusic.org.uk/file/5694/download?token=I-1K0qhQ

https://network.youthmusic.org.uk/file/5694/download?token=I-1K0qhQ

http://gesturesen.wikispaces.com/home

http://infusionsystems.com/catalog/

http://infusionsystems.com/catalog/

http://www.theinternetofthings.eu/

http://www.theinternetofthings.eu/

http://www.jamboxx.com/

https://kinectsen.wikispaces.com/

https://kinectsen.wikispaces.com/

https://voices.no/index.php/voices/article/view/492/399


http://kyubmusic.com/

http://kyubmusic.com/

https://www.leapmotion.com/

https://www.leapmotion.com/

http://steim.org/product/discontinued-products/

http://steim.org/product/discontinued-products/

http://beatsurfing.net/

http://beatsurfing.net/

https://dl.acm.org/citation.cfm?id=2818143.2818145

https://dl.acm.org/citation.cfm?id=2818143.2818145

https://www.youtube.com/watch?v=u9-OJs93naA

https://www.youtube.com/watch?v=u9-OJs93naA

http://makeymakey.com/

http://makeymakey.com/


Malloch, J., Birnbaum, D., Sinyor, E., & Wanderley, M. (2006). Towards a new con-ceptual framework for digital musical instruments. Paper presented at the Digital Audio Effects (DAFX) conference, Montreal.

Matthews, C. (2018, May 14). Customising ThumbJam for accessible music. Retrieved from http://www.drakemusic.org/blog/charles-matthews/customising-thumbjam/

MBMM. (2017, February 21). Mybreathmymusic.com. Retrieved from http://mybreathmymusic.com/en/

McDowall, J. (2008). Music technology: A vehicle for young children’s music learning. Australian Journal of Music Education, 2, 41–51. Retrieved from http://search.informit.com.au/documentSummary;dn=605192627489367;res=IELHSS

Meckin, D., & Bryan-Kinns, N. (2013). moosikMasheens: Music, motion and narrative with young people who have complex needs. Paper presented at the Interaction Design and Children Conference, New York. doi:10.1145/2485760.2485776

Miranda, E., & Wanderley, M. (2006). New digital musical instruments. Middleton,WI: A-R Editions.

Misje, R. (2013). Music technology in music therapy. Unpublished manuscript, University of Bergen, Bergen, Norway.

Mogees. (2018, May 15). Mogees. Retrieved from https://www.mogees.co.uk/Music Radar. (2018, May 15). The best MIDI pad controllers in the

world today. Retrieved from https://www.musicradar.com/news/tech/10-of-the-best-midi-pad-controllers-566293

Musii. (2017, February 21). Musii - Multisensory interactive inflatable. Retrieved from https://musii.co.uk

Nagler, J. C. (2011). Music therapy methods with hand-held music devices in con-temporary clinical practice: A commentary. Music and Medicine, 3(3), 196–199. doi:10.1177/1943862111407512

Nagler, J. C., & Lee, M. H. M. (1987). Use of microcomputers in the music therapy process of a postviral encephalitic musician. Medical Problems of Performing Artists, 2(2), 72. Retrieved from https://www.sciandmed.com/mppa/journalviewer.aspx?issue=1148&article=1483&action=1

Nath, A., & Young, S. (2015). VESBALL : A ball-shaped instrument for music therapy. Paper presented at the International Conference on New Interfaces for Musical Expression, Baton Rouge.

Nordoff Robbins. (2014). Music Therapy and Music & Health References and Resources. Retrieved from https://www.nordoff-robbins.org.uk/sites/default/files/the_nordoff_robbins_evidence_bank_3rd_edition_2014.pdf

Numark.com. (2018, May 15). Orbit wireless DJ controller with motion control. | Numark. Retrieved from https://www.numark.com/product/orbit

Ofsted. (2012). Music in school: Wider still, and wider. Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/413347/Music_in_schools_wider_still__and_wider.pdf

Oliveros, P., Miller, L., Heyen, J., Siddall, G., & Hazard, S. (2011). A musical im-provisation interface for people with severe physical disabilities. Music and Medicine, 3(3), 172–181. doi:10.1177/1943862111411924

O’Malley, C., & Stanton Fraser, D. (2004). Literature review in learning with tangible technologies. Retrieved from http://opus.bath.ac.uk/9506/

Open Orchestras. (2018, May 15). Open orchestras. Retrieved from https://www.openorchestras.org/instruments/

OptiBeam. (2018, May 15). Optibeam. Retrieved from http://optimusic.com/optibeam/Paine, G., & Drummond, J. (2009). Developing an ontology of new interfaces

for realtime electronic music performance. Paper presented at the 2009 Electroacoustic Music Studies conference, Buenos Aries.

Paradiso, J. A. (1997). Electronic music: New ways to play. IEEE Spectrum, 34(12), 18–30. doi:10.1109/6.642965

Place, A., Lacey, L., & Mitchell, T. (2014). AlphaSphere - From prototype to product. Paper presented at the New Interfaces for Musical Expression conference, London.

Poupyrev, I., Lyons, M., Fels, S., & Blain, T. (2001). New interfaces for musical expres-sion. Paper presented at the Computer Human Interaction conference, Seattle.

Raspberry Pi. (2015, February 21). Raspberry Pi. Retrieved from https://www.raspberrypi.org/

Reactable Technology. (2017, February 5). Reactable - Music knowledge technology. Retrieved from http://reactable.com/

Refsgaard, A. (2018, February 21). Eye conductor. Retrieved from https://andreasrefsgaard.dk/project/eye-conductor/

Skoogmusic. (2018, May 15). Skoogmusic. Retrieved from http://skoogmusic.com/Small, C. (2011). Musicking. Middletown, CT: Wesleyan University Press.Soundbeam. (2018, May 25). Home. Retrieved from http://www.soundbeam.co.uk/Sound House. (2017, February 4). Spectronics - Inclusive learning technologies. Retrieved

from http://www.spectronics.com.au/product/soundhouse-special-access-kitsSonosaurus. (2018, May 14). ThumbJam. Retrieved from https://thumbjam.com/Streeter, E. (2007). Reactions and responses from the music therapy community to

the growth of computers and technology - Some preliminary thoughts. Retrieved from https://voices.no/index.php/voices/article/view/1761/1521

Suzuki Qchord. (2018, May 17). QChord. Retrieved from http://www.suzukimusic.co.uk/qchord/index.html

Swingler, T. (1998). “That was me!” Applications of the Soundbeam MIDI con-troller as a key to creative communication, learning, independence and joy. Paper presented at the Technology for Persons with Disabilities conference, California.

Tam, C., Schwellnus, H., Ceilidh, E., Hamdani, Y., Lamont, A., & Chau. (2007). Movement-to-music computer technology: A developmental play experi-ence for children with severe physical disabilities. Occupational Therapy International, 14(2), 99–112. doi:10.1002/oti

The EyeHarp. (2018, May 1). EyeHarp. Retrieved from http://theeyeharp.org/Theremini. (2018, May 1). Theremini. Retrieved from https://www.moogmusic.com/

products/etherwave-theremins/thereminiThe Wiinstrument. (2018, May 17). Wiinstrument. Retrieved from https://the-

wiinstrument.soft112.com/Thomas, P. (2012). Music technology & special needs: Part 1. Retrieved from https://

www.soundonsound.com/techniques/music-technology-special-needs-part-1Uwyn.com. (2018, May 13). GECO - Music and sound through hand gestures.

Retrieved from http://uwyn.com/geco/Vamvakousis, Z., & Ramirez, R. (2016). The EyeHarp: A gaze-controlled

digital musical instrument. Frontiers in Psychology, 7, 1–14. doi:10.3389/fpsyg.2016.00906

Viega, M. (2015). Exploring the discourse in hip hop and implications for music therapy practice. Music Therapy Perspectives, 34(2), 138–146. doi:10.1093/mtp/miv035

Viega, M. (2016). Science as art: Axiology as a central component in methodology and evaluation of arts-based research (ABR). Music Therapy Perspectives, 34(1), 4–13. doi:10.1093/mtp/miv043

Villafuerte, L., Markova, M., & Jorda, S. (2012). Acquisition of social abilities through musical tangible user interface. Paper presented at the 2012 ACM Annual Conference Extended Abstracts on Human Factors in Computing Systems, Austin.

Virtual Musical Instrument. (2018, February 21). Flintbox.com. Retrieved from https://flintbox.com/public/project/25722/

Vogiatzoglou, A., Ockelford, A., Welch, G., & Himonides, E. (2011). Sounds of in-tent: Interactive software to assess the musical development of children and young people with complex needs. Music and Medicine, 3(3), 189–195. doi:10.1177/1943862111403628

Waiswisz, M. (1985). The hands, a set of remote midi-controllers. Paper presented at the International Computer Music Conference, San Francisco.

Wanderley, M. M. (2001). Gestural control of music. Paper presented at the International Workshop Human Supervision and Control in Engineering and Music, Kassel. Retrieved from http://recherche.ircam.fr/anasyn/wanderle/Gestes/Externe/kassel.pdf

Ward, A., Woodbury, L., & Davis, T. (2017). Design considerations for instru-ments for users with complex needs in SEN settings. Paper presented at the International Conference on New Interfaces for Musical Expression, Copenhagen. Retrieved from http://www.nime.org/proceedings/2017/nime2017_paper0040.pdf

Welch, G. F., Ockelford, A., Zimmermann, S.-A., Himonides, E., & Wilde, E. (2015). The provision of music in special education (PROMISE). Retrieved from http://discovery.ucl.ac.uk/1505981/1/The%20Provision%20of%20Music%20in%20Special%20Education.pdf

Whitehead-Pleaux, A. M., Clark, S. L., & Spall, L. E. (2011). Indications and counterindications for electronic music technologies in a pediatric medical setting. Music and Medicine, 3(3), 154–162. doi:10.1177/1943862111409241

Wikipedia. (2017, October 23). En.wikipedia.org. Retrieved from https://en.wikipedia.org/wiki/List_of_music_video_games

Dow


ic.oup.com/m


http://www.drakemusic.org/blog/charles-matthews/customising-thumbjam/

http://www.drakemusic.org/blog/charles-matthews/customising-thumbjam/

http://mybreathmymusic.com/en/

http://mybreathmymusic.com/en/

http://search.informit.com.au/documentSummary;dn=605192627489367;res=IELHSS



https://www.mogees.co.uk/

https://www.musicradar.com/news/tech/10-of-the-best-midi-pad-controllers-566293

https://www.musicradar.com/news/tech/10-of-the-best-midi-pad-controllers-566293

https://musii.co.uk

https://www.sciandmed.com/mppa/journalviewer.aspx?issue=1148&article=1483&action=1

https://www.sciandmed.com/mppa/journalviewer.aspx?issue=1148&article=1483&action=1

https://www.nordoff-robbins.org.uk/sites/default/files/the_nordoff_robbins_evidence_bank_3rd_edition_2014.pdf

https://www.nordoff-robbins.org.uk/sites/default/files/the_nordoff_robbins_evidence_bank_3rd_edition_2014.pdf

https://www.numark.com/product/orbit

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/413347/Music_in_schools_wider_still__and_wider.pdf



http://opus.bath.ac.uk/9506/

https://www.openorchestras.org/instruments/

https://www.openorchestras.org/instruments/

http://optimusic.com/optibeam/

https://www.raspberrypi.org/

https://www.raspberrypi.org/

http://reactable.com/

https://andreasrefsgaard.dk/project/eye-conductor/

https://andreasrefsgaard.dk/project/eye-conductor/

http://skoogmusic.com/

http://www.soundbeam.co.uk/

http://www.spectronics.com.au/product/soundhouse-special-access-kits

https://thumbjam.com/


http://www.suzukimusic.co.uk/qchord/index.html

http://www.suzukimusic.co.uk/qchord/index.html

http://theeyeharp.org/

https://www.moogmusic.com/products/etherwave-theremins/theremini

https://www.moogmusic.com/products/etherwave-theremins/theremini

https://the-wiinstrument.soft112.com/

https://the-wiinstrument.soft112.com/

https://www.soundonsound.com/techniques/music-technology-special-needs-part-1

https://www.soundonsound.com/techniques/music-technology-special-needs-part-1

http://uwyn.com/geco/

https://flintbox.com/public/project/25722/

http://recherche.ircam.fr/anasyn/wanderle/Gestes/Externe/kassel.pdf

http://recherche.ircam.fr/anasyn/wanderle/Gestes/Externe/kassel.pdf

http://www.nime.org/proceedings/2017/nime2017_paper0040.pdf

http://www.nime.org/proceedings/2017/nime2017_paper0040.pdf

http://discovery.ucl.ac.uk/1505981/1/The%20Provision%20of%20Music%20in%20Special%20Education.pdf



https://en.wikipedia.org/wiki/List_of_music_video_games

https://en.wikipedia.org/wiki/List_of_music_video_games

Music Technology and Alternate Controllers for Clients with ...

Documents

Transcript of Music Technology and Alternate Controllers for Clients with ...