Sound Association for Product-Sound Design Using Semiotics

491

Sound Association for Product-Sound

Design Using Semiotics

Kumari Moothedath Chandran , Prajakta Prabhune

and Dibakar Sen

© Springer India 2015 A. Chakrabarti (ed.) , ICoRD’15 – Research into Design Across Boundaries Volume 1 , Smart Innovation, Systems and Technologies 34 , DOI 10.1007/978-81-322-2232-3_43

Abstract Sounds from products which carry information about its performance can be called as product sounds. Product sounds can aid visual display and con-trols if designed effectively. In this work, the authors collected 6,190 sound samples. Out of this, 670 sounds were coded, based on three categories of abstrac-tions, namely index (on causality), icon (on similarity) and symbol (arbitrary). The coded sounds were used to associate them to selected examples of two product interfaces. Design exercises and user surveys were done to confi rm the coding of individual sounds, as well as the method of association. Three different meth-ods were used for the surveys. One—to associate sounds based on a given theme; two—to match the sounds provided to the product interface given; three—the sub-jects were made to listen to selected sounds and asked to describe the meaning they perceive of it. The results from the surveys tallied well with the method of categorization of sounds and their associations, and it provides a structured way of product-sound design.

Keywords Product sound · Semiotics · Sound association

1 Product Sounds

Sounds emanated from products which carry information about its functioning can be called as product sounds. A suitable and supportive product sound contribute positively to a product’s perceived quality by the user [ 1 ]. Consumers perceive auditory cues to be as important as visual cues in many product categories such as vehicles, domestic durables, and high-tech products [ 2 , 3 ]. Hence, product sounds

K.M. Chandran ( * ) · P. Prabhune · D. Sen Centre for Product Design and Manufacturing , Indian Institute of Science , Bangalore , India e-mail: [email protected]

492 K.M. Chandran et al.

can support user interaction and communication, and aid visual display and con-trols, if designed effectively.

There are two different kinds of sounds emanating from products, based on its source of origin. The fi rst kind of sounds are the structure-borne sounds, which are the audible sounds produced from the mechanical, electrical or manual function-ing of the product or its parts. For example, sound from a vacuum cleaner when it is switched ‘on’ and performing the cleaning action, or the sound produced while turning a mechanical switch ‘on’ or ‘off. The second kind of sounds are those used in product interfaces and controls which are intentionally added to a product. Examples are, the ring sound produced by a telephone as an incoming call alert, and beep sounds produced by a washing machine to alert user when a washing cycle is over. The fi rst type of sounds are named as consequential sounds and the second as intentional sounds by van Egmond [ 4 – 6 ]. In this paper, the authors focus on intentional sounds in a product which are digitally synthesized or recorded and refer to them as product sounds. The objective of this study is to sup-port the use and design of intentional sounds in product-user interfaces by taking examples of user-interfaces from two domestic appliances. While consequential sound design has been primarily a domain of acoustic engineers, and not entirely that of product designers, it is often a matter of production quality of the physical parts, materials involved, and the physical phenomena involved in the sound pro-ducing function. However, intentional sound design has been a domain of product designers. In this work, the authors aim to support product designers to systemati-cally design sounds on a product’s user interface.

2 Product-User Interface

A product interface is a medium between the core of a product and the user, to establish communication between the user and the product, by providing and receiving appropriate inputs and outputs needed for its effi cient intended function-ing and use. The user-friendliness of a product relies on its interface design, as it is on the overall ergonomic design of the product.

Communication through a product’s user-interface can happen through fi ve media—visual, tactile, audio, olfactory, gestures and brainwaves . The fi rst three media are widely used in user-interfaces of consumer products and machines worldwide. While, olfactory signs are yet to be explored as a communication medium in products, scented ads are used in print-magazines of cosmetic and beauty industry. Gestural control is in early stages of commercialization, and it is becoming common in commercial computer games. Brainwaves are being explored in actuating machines in its nascent stage. The input-output controls on the user-interfaces (Fig. 1 ) attract user attention and action using these media either alone or together.

Considering the fi rst three media for its design parameters—in visual interac-tion, use of texts, graphical icons and symbols, colour, shape and composition

493Sound Association for Product-Sound Design Using Semiotics

are the parameters. In tactile interaction, shape and form of the controls, material, texture, effort and movement of operating controls are the design parameters. In audio interaction, the mechanical sounds associated with the controls, the digital sounds of the interface, sound volume and quality, operation by user voices or recording of it are the parameters.

For product designers, the scope and use of digital sounds (intentional sounds) in product user-interfaces for product innovation are to be explored. The area of visual design is well explored in interface design, but not sound design. It has been shown that dual-coding, i.e., adding semantic (i.e., verbal) information congruent with perceptual (i.e., visual) information, has an additive effect on memory [ 7 , 8 ]. The studies of Tindall -Ford, Chandler, and Sweller have shown that participants studying instructional materials in multiple modalities (i.e., audio-text and visual diagrams/tables) perform better than those studying in a single modality (visual-only format) [ 7 , 9 ]. Hence, interface design may benefi t from adding multiple modalities for better user-interaction. The work presented here, aims to support product designers in assigning sounds to user-interfaces which are meaningful associations to the user.

3 Semiotics and Semiosis in the Context of Product Sounds

Semiotics is a theory of signs [ 1 , 10 , 11 ]. In semiotics, communication is based on the processing of perceptual events as sign events [ 1 ]. Jekosch [ 1 ] explains sign as a mental phenomenon and it denotes the processing of a perceptual object. Constituents of signs are the sign carrier, the object of prior experience to which it refers and the meaning which evolves [ 1 ]. The process of selection, organisa-tion, coordination and structuring, items of perception and objects of experience is termed as semiosis [ 1 ].

Consider an example of cooking food in a microwave oven. A bowl of milk was kept for reheating for 10 s by pressing a button, and immediately a visual countdown starts in the LCD display. The user can take out the bowl by seeing the visual feedback, or by hearing the consequential sound produced by the micro-wave stopping. In case, the bowl is still not taken, the microwave oven gives beep sounds at regular intervals to catch user-attention. The user then understands that

Fig. 1 Role of product-user interface


the reheating process is over, and the bowl is ready to pick up, or in other words semiosis takes place.

In the above example, there is the product, the user, the sound, the user-interface involving controls and visual display involved in completing the communication process (or for achieving semiosis) while reheating a bowl of milk. While assign-ing sounds to a product-user interface, there are four elements involved in a prod-uct-user communication process to be kept in mind viz., function of the control for which the sound is assigned, the feedback to be communicated to the user through the user-interface, the sound itself, and perception of the user. To achieve commu-nication or semiosis, the perception of the user is important. For a successful prod-uct-sound design, Jekosch explains the requirement of ‘no incongruity’ between user expectation and the perceived auditory image of the product [ 1 ]. However, ‘no incongruity’ can be a necessary bare minimum condition, and once it is achieved, the need for increasing degree of congruity begins. Therefore, the authors propose that need to achieve ‘congruity’ is a more natural direction to go in, for a success-ful product-sound design. Hence, the meaning of the function of a control, the meaning of the visual/tactile or other aids in an interface, the meaning of the sound assigned, as perceived by the user, should be congruent with each other for the user to be able to associate it with the information expected to be conveyed (see Fig. 2 ).

According to Jekosch , semiotic theory can be applied to product-sound design, as it is a basic theory for sciences which deal with phenomena that are perceived as sign carriers [ 1 ]. Jekosch [ 1 ] explains, for the sign carriers, here the product sound, there can be three categories of abstractions to its content: (a) Index—the sound refers the listener directly to its physical source, e.g., sound of a chisel on stone; (b) Icon—the relation between sound and source is based on similarity,

Fig. 2 Congruency of meaning to achieve semiosis


e.g., an artifi cial (digital) clack sound which sounds like the clack sound of a door being locked; (c) Symbol—the relation between sound and what it signifi es is based on conventions, e.g., a sound of an applause, where a person should know what it signifi es as (see Fig. 4 ). The next section explains how these categories are used in our work to support product-sound design.

4 Collection and Coding of Sound Samples

For sound related communication in design teams, product sound designers need to reproduce sound information from memory [ 7 ]. Several methods have been pro-posed to reduce the load on cognitive activities, which involves, verbalizing, audi-olising , imitating and using pictograms , to reproduce the perceived aspects of the sound [ 7 , 12 – 14 ]. For product-sound designers, it is diffi cult to explain graphically or lexically the particular sound they are looking for exactly. A database of sounds will be of help here. The authors felt the need for having a database of sounds, for product designers to fi nd sounds suitable for the product they are designing. Three set of sound samples were used as base for this study: (a) 125 sound samples were recorded from the environment around (both natural and man-made), (b) 68 sound samples were downloaded from Soundsnap website [ 15 ], (c) a free library called 6000 Sound Effects [ 16 ] was downloaded from their internet site. The samples from this library were primarily used for our study along with the samples the authors recorded.

1,000 different sound samples from various categories like natural (environ-mental) sounds (birds, animals, wind etc.), household sounds, speech sounds (expressions, disappointment, cheering etc.), machine sounds and cartoon sounds (synthesized sounds) were selected almost equal in number from each category from the above three sets. Sounds are given onomatopoeic names like cuckoo clock, buzz, clack sound, etc., to give the reader an understanding of the sample sounds being referred to. Out of the 1,000 sounds, 670 sounds were shortlisted based on sound quality and duration (3–5 s) of the recordings, for coding using the principles of semiotics. The abstractions of index , icon and symbol discussed in the previous section were mapped to each of these sounds. Each of these sounds were coded based on the dominating semiotic relation it carries. Figure 3 shows an example of how the coding was followed by authors with 3 sound samples given. If more than one relation was found, those too were coded to the sounds. For example, for the sound named ‘windup alarm clock’, it referred to its physical source—an alarm clock, which is index , and also it referred to conventions of rais-ing an alarm that it is time to perform some action, which is symbol . Table 1 shows a sample list of coded sounds by authors. This categorization and coding helps in the subsequent phase of sound selection and association, i.e., if one needs icon type sounds, one can pick that and use it to associate to a product user-interface control.


5 Sound Association to Products

With the database of categorized and coded sounds, experiments were conducted to see whether assigning sounds to product user-interface controls based on the semiotic relations will help in making meaningful associations to the user. Three approaches were followed for associating sounds to products: (1) sound associa-tion based on theme by index, icon or symbol ; (2) sound association for individual function or control by index, icon or symbol ; (3) single sound checked for promi-nent component out of index, icon , or symbol . For the cases 1 and 2, user inter-faces of a washing machine and a microwave were chosen. Interfaces from these house-hold appliances were chosen because the subjects were familiar with these products, and the interface was complex enough in terms of number of operations, and variability in input controls. For example, interfaces of an electric kettle or a vacuum cleaner has only very few input-controls such as POWER ON and OFF, whereas the interfaces of washing machine and microwave oven have at least more than four different controls.

Fig. 3 Abstraction of product sounds based on its content

Table 1 Categorisation of sound samples

Sound samples Index Icon Symbol

Chipmunk chatter ✓ ✓

Cannon fi res ✓ ✓

Water gushing ✓

Winding down theme ✓

Cola burp ✓ ✓

Cash register rings 2 ✓

Child toy duck quacks ✓ ✓

Offi ce phone ring ✓ ✓

Chop vegetables ✓

Brushing teeth ✓


5.1 Case 1—Theme Based Sound Association

5.1.1 Method

Fifteen postgraduate students of design and research and of age group of 20–30 years participated in the experiments. All subjects reported normal hear-ing and 6/6 corrected vision. An interface of a washing machine was used for this experiment (Fig. 4 ). The authors chose two themes for assigning sounds—fi rst theme ‘water’ based on ‘icon’ or similarity (as washing involved water), and sec-ond theme ‘orchestra’ based on ‘symbol’ or conventions (as washing involved an ensemble of actions and sounds like that of an orchestra). Four sound samples each for the two themes were chosen from the database by the authors. The sub-jects were given a PowerPoint slide with sounds’ links and a digital image of the washing machine interface. Names of the sounds were not shown to the subject and were merely numbered as sound 1, 2, etc. with the volume set at a comfort-able listening level. Function of four controls, WASH, RINSE, SPIN and DOOR LOCK, were explained. The interface was shown for a minute and then sounds were played. Sounds were present throughout the experiment for the subjects to playback if needed. After the sounds were heard, the participants were asked to do a matching task. Subjects were asked to match the sound of their choice to the control in the interface by drawing a line in the PowerPoint slide. The results of the matching task were checked to see if the associations by subjects have a pat-tern. One such outcome of a subject matching sounds to the interface given, is shown in Fig. 4 .

Fig. 4 Sound association diagram of a washing machine interface


5.1.2 Results

The tabulation of the matching task is shown in Figs. 5 and 6 . From Fig. 5 , it can be seen that for SPIN and DOOR LOCK, subjects favored two of the provided sounds in particular. The controls, WASH and RINSE, were matched with two dif-ferent sounds equally by the subjects. This refl ects that the interpretation of the meanings of the controls by the subjects were not distinct enough for WASH and RINSE, and the words also have a closely related meaning. Also, the two sounds matched for WASH and RINSE, were different from those matched for the other two controls, showing clear distinction of the perception of meanings of these con-trols and the sounds. In the symbol based orchestra theme, there was ambiguity in matching sounds to the fi rst three controls. A clear choice was made for the con-trol DOOR LOCK to one particular sound. This demonstrates the need for users to have prior associations to these sounds to fi nd meanings in symbol or convention based relations.

Fig. 5 Case 1: Experiment 1—Washing Machine. Theme: Water (Icon based)

Fig. 6 Case 1: Experiment 2—Washing Machine. Theme: Orchestra (Symbol based)


5.2 Case 2—Function Based Sound Association

5.2.1 Method

The same subjects as in Case 1 were involved in this experiment. An interface of a microwave oven was chosen for this experiment. Depending on the function of the individual controls, sounds with either ‘index’,’ icon’ or ‘symbol’ relations were chosen from the database by the authors. The subjects were given a digital image of the interface and functions of eleven controls were explained. The interface was shown for some time and then eleven sounds were given. After the sounds were heard, the participants were asked to do a matching task in a PowerPoint slide, similar to Case 1. Figure 7 shows the outcome of one such matching task by a subject. The results from this experiment were checked to see whether the sound association to the controls have a pattern.

5.2.2 Results

The tabulation of the outcomes of the matching task is shown in Fig. 8 . From Fig. 8 , one could see that out of the 11 sounds given—3 controls (CLOCK, BABYFOOD , KIDS MEAL) were associated with only 2 sounds; 5 controls (POWER LEVEL, AUTO REHEAT, AUTO DEFROST, MORE, KITCHEN TIMER) were associated with only 3 sounds; and the remaining 3 controls were associated with 4 or 5 sounds. In 10 out of the 11 controls, the subjects were predominantly matching a particular sound to a particular control. One control, KITCHEN TIMER, had 2 sounds preferred equally, which was a symbol based

Fig. 7 Case 2—Microwave Oven


association. The results from Experiment 2 show that if the sounds chosen are based on the semiotic relation the functions of individual controls are demanding, the associations are more meaningful to the listeners, and are easily understood and interpreted by them, to detect and discriminate the controls.

Fig. 8 Case 2—Microwave Oven


5.3 Case 3

5.3.1 Method

Ten sound samples, fi ve each from natural sounds and product sounds categories, were taken from the database by the authors. A survey was conducted to fi nd out whether a single sound has a dominating semiotic association (index, icon or sym-bol) as perceived by the subjects. Same subjects from the previous 2 studies par-ticipated in this study. The subjects were made to listen to the selected sounds and were asked to speak about what they felt, infer or identify, upon hearing the sound. These psychoacoustic descriptions were then categorized based on the semiotic relations of index, icon, or symbol.

5.3.2 Results

A sample of descriptions of sounds by subjects and categorisations are given below. Thunder cracks : crackers (icon); bomb explosion and drum (icon); lightening

(index); thunderstorm lightening (index); land mines blasting sound (icon); something

falling down (icon); lightening (index); crackers (icon); explosion (icon); bomb explo-

sion while a racial riot (symbol); computer game sounds (symbol); bullet fi ring (icon); bomb explosion destroyed the house (icon), thunder light rain (index); drum (icon).

Egyptian goose : horse driven carriage (icon); laughing man or turning old

wheel (icon); ghost or cruel laugh (icon); laugh with coughing of older people (icon);

Fig. 9 Case 3—Sounds checked for prominent relation


bird crying (index); a jungle bird (icon); mechanic testing horn sound (icon); pipani

fair noise (symbol); a man mocking or a bird or animal (icon); a sound human being

can’t make (icon); some bird sound in background or animal sound in focus (index); thought of squirrel but too loud for it, a big mammal’s sound (icon); dog barking,

cycle horns, blow horns or duck like bird (icon); ducks (index). The results of Case 3 is as shown in Fig. 9 . The results show that 8 out of the

10 sound samples have all the three semiotic relations, viz., index, icon and sym-bol. Nine out of the 10 sounds have one relation dominating substantially when compared to the remaining two. This shows that sounds have a prominent semi-otic relation, and subjects can distinguish, and perceive, the dominating relation of a sound. This categorization can hence be applied to associate sounds to product interfaces in an effective manner.

6 Conclusions

Understanding of the semiotic relations of sounds and product interface controls, can support product designers in assigning sounds to products in a systematic and innovative manner. However, there is a dichotomy that a product sound be ‘novel’ and yet ‘known’ to the user at the same time [ 1 ]. By use of themes for products, user surveys, and a database of sounds with semiotic categories, it is possible to design novel sound associations to product interfaces which the users can relate to. The need for a semiotic relations based database of sounds for product design-ers, is emphasized in this study. As seen from the experiments, user surveys can help in identifying ambiguous sounds and hence in refi ning the sound design of the interface. User surveys can also help in choosing sounds to assign, or in modi-fying and synthesizing new sounds by giving more prominence to the identifi ed semiotic relation by the user. It has been shown that synthesized sounds can also elicit semantic associations [ 5 , 6 , 17 – 21 ]. The approach presented in this work can bring in more structure to product-sound design process which has been done in an intuitive manner by product designers at present.

References

1. Jekosch , U.: Assigning meaning to sounds—semiotics in the context of product-sound design. Communication Acoustics, pp 193–221, 2005

2. Knöferle , K.: Using customer insights to improve product sound design. Mark. Rev. St. Gallen . 29 (2), 47–53 (2012)

3. Schifferstein , H.N.J.: The perceived importance of sensory modalities in product usage: a study of self-reports. Acta Psychol . 121 (1), 41–64 (2006)

4. van Egmond , R.: The experience of product sounds. In: Schifferstein , H.N.J., Hekkert , P. ( eds .) Product Experience, pp. 69–89. Elsevier, Amsterdam (2008)

5. Özcan , E., van Egmond , R.: Basic semantics of product sounds. Int. J. Des. 6 (2), 41–54 (2012)


6. Langeveld , L., van Egmond , R., Jansen, R., Özcan E.: Product sound design: intentional and consequential sounds. In: Coelho D. (Ed.) Advances in Industrial Design Engineering. ISBN : 978-953-51-1016-3, InTech , doi: 10.5772/55274 . Available from: http :// www . intechopen . com /books/advances-in-industrial-design-engineering/product-sound-design-intentional-and-consequential-sounds . (2013)

7. Özcan , E., van Egmond , R.: Memory for product sounds: the effect of sound and label type. Acta Psychol . 126 (3), 196–215 (2007)

8. Paivio , A.: Dual coding theory—retrospect and current status. Can. J. Psychol . 45 (3), 255–287 (1991)

9. Tindall -Ford, S., Chandler, P., Sweller , J.: When two sensory modes are better than one. J. Exp. Psychol . Appl . 3 (4), 257–287 (1997)

10. Eschbach , A., Trabant , J. ( eds .): History of Semiotics. Benjamins, Amsterdam (1983) 11. Eco , U.: Einfu ̈ hrung in die Semiotik . 38. Authorized German ed by J Trabant . Fink.

München (1994) 12. Özcan , E., van Egmond , R.: Pictograms for sound design: a language for the communication

of product sounds. In: Kurtgözü , A. (ed.) Proceedings of the 4 th International Conference on Design and Emotion. Middle East Technical University, Ankara (2004)

13. Özcan , E., van Egmond , R.: Product sound design and application: an overview. In: Desmet , P.M.A., Karlsson , M.A., van Erp , J. ( eds .) Proceedings of the 5 th International Conference on Design and Emotion. Chalmers University of Technology, Gothenburg (2006)

14. van Egmond , R.: Designing an emotional experience for product sounds. In: Desmet , P.M.A., Karlsson , M.A., van Erp , J. ( eds .) Proceedings of the 5 th International Conference on Design and Emotion. Chalmers University of Technology, Gothenburg (2006)

15. Soundsnap website. http :// www . soundsnap . com / 16. 6000 sound effects. http :// www .sound-ideas. com /sound-effects/series-6000-sound-effects-

library. html 17. von Bismarck, G.: Timbre of steady sounds: a factorial investigation of its verbal attributes.

Acta Acustica United Acustica 30 , 146–159 (1974) 18. Kendall, R.A., Carterette , E.C.: Verbal attributes of simultaneous wind instrument timbres:

I. von Bismarck’s adjectives. Music Percept. 10 (4), 445–468 (1993) 19. Björk , E.A.: The perceived quality of natural sounds. Acustica 57 (3), 185–188 (1985) 20. Solomon, L.N.: Semantic approach to the perception of complex sounds. J. Acoust . Soc. Am.

30 (5), 421–425 (1958) 21. Edworthy , J., Hellier , E., Hards , R.: The semantic associations of acoustic parameter com-

monly used in the design of auditory information and warning signals. Ergonomics 38 (11), 2341–2361 (1995)

Sound Association for Product-Sound Design Using Semiotics

Documents

Transcript of Sound Association for Product-Sound Design Using Semiotics