A Study on Vietnamese Prosody

Tang-Ho Le, Anh-Viet Nguyen, Hao Vinh Truong, Hien Van Bui, and Dung Le

Abstract. In the current paper, the Vietnamese prosody was analyzedin order to produce a natural Vietnamese synthesizer. Word duration intexts recorded by various human voices was measured to test several hy-potheses for logical segments of the utterances in a sentence. The soundfiles were analyzed and interpreted by several statistical calculations.

Keywords: Vietnamese, Text to Speech, TTS, Prosody, Ngữ điệu, Ngônđiệu.

1 Introduction

1.1 Naturalness of Voice Synthesizer

Two kinds of speech were distinguished: the natural speech produced by humanand the synthesized speech produced by computer. Evidently, an ideally syn-thesized speech should be natural. This becomes a criterion for the evaluationof a voice synthesizer in the actual literal. In order to reach the naturalness fora synthesizer, the prosody was studied. The basic functions of prosody are tosegment and to highlight. The domain variation of prosody extends beyond thephoneme into units of syllables, words, phrases, and sentences [3].

1.2 Prosody Definition and Particular Features of VietnameseProsody

In linguistics, prosody is the rhythm, stress, and intonation of connected speech.Prosody may reflect various features of the speaker or the emotional state of aspeaker that may not be encoded by grammar or choice of vocabulary [2].

There were two separate steps in this study of Vietnamese prosody. In the firststep (presented in this paper), we focused on words, phrases and words groups(usually regarded as “ngữ điệu” in Vietnamese). In essence, this study focusedon the local effect of speech. In the later work, the prosody will be examined inthe bigger units: the sentences, which involve the emotional expression (“ngônđiệu” in Vietnamese). This will focus on the global effect of speech.

There is a very important difference between the prosody of Vietnamese (asa monosyllabic and tone language) and the one of European languages (poly-syllabic languages). The Vietnamese prosody is concerned with rhythm (word’sduration) between words in a group of words or in a compound words, whileraising the tone (intonation) by augmenting the amplitude and/or the frequencyof all words has the global effect on the whole sentence (for example, in a ques-tion). The reason is that the Vietnamese speakers cannot change the intonation

N.T. Nguyen et al. (Eds.): New Challenges for Intelligent Information, SCI 351, pp. 63–73.springerlink.com c© Springer-Verlag Berlin Heidelberg 2011

64 T.-H. Lê et al.

(tone) of a word in order to highlight it, because each word has its own meaningthanks to one of the six accents. Moreover, some compound words (or phrases)must be pronounced without emission break between them (a very small silencelesser than 20 milliseconds is considered as no silence). The two phrases “sự mất-mát đáng tiếc” and “chúng ta đã mất đi một số thói quen” are as examples. Ifan utterance is correctly segmented, it is more natural and comprehensive. Forexample, the sentence "Học sinh học sinh học" (The student studies biology)cannot be spoken regularly because of the meaning variation. "Học sinh" is "thestudent" (a noun); "học" is "to study" (a verb); and "sinh học" is "the biology"(a noun). Therefore, a Vietnamese speaks this sentence with two breaks betweenthose words: "Học sinh—học—sinh học".

Another prosody’s feature of the Vietnamese is that there is no question to“make accent” on a syllable of a word (as in polysyllabic languages). Becauseeach word has only one syllable, it will be of no sense if one pronounces a wordstronger than other, except when one would express a special emotion; e.g. onecan speak some words slower and stronger than other ones in a sentence to makethem more important.

In order to segment an utterance naturally, a voice synthesizer must takeinto account the syntactic feature: whether the actual word is an article, apreposition or a conjunction; whether it is a compound word, or a word group.Section 3 below has different subtopics for each of these categories. In eachsubtopic, a hypothesis about the segmentation was presented, and tested throughexperimentation with sample sentences recorded by different speakers, most ofwhom were the Vietnamese television broadcasters.

1.3 Some Studies on Vietnamese Prosody

In the actual literal, some recent works and published papers studied the word’sduration in Vietnamese. According to Trần [6], the duration of a Vietnamesephonetic unit depends on its position, its pitch, its emphasis and its structure.Other authors [5] stated that the duration of word depends on the structureof the phonetic unit, the silence duration between words, and the structure ofthe previous word’s phonetic unit. Our study presented here examined also thismatter with more aspects and quantified measurement.

2 Experimental Description

2.1 Selection of Speakers

For this study of Vietnamese prosody, the audio files were collected via theInternet from the Voice of Vietnamese Broadcasters in several provinces. Thesedata samples include the voices of the north (Ha Noi city), the south (HoChiMinhcity) and the central land (Hue city). The original video files were extracted into36 wave audio files (.wav). There are 20,815 words and 20,779 silences withinthese audio files. Then, these words and silences were analyzed and measured.

A Study on Vietnamese Prosody 65

We would like to insist that our test data is not these 36 files (they are not theobjects for the test); instead, we aim at the content of these files which are morethan 20,000 words’ duration and silences between words. This number is veryenough for the objectivity of the conclusion or the formulated rules.

2.2 Pre-processing

After collecting the data, the typical audio segments were selected for processing.Only those speak by broadcasters without interviews and without backgroundnoise were used, which helped improve the validity of the result.

Fig. 1. A noisy audio segment

Fig. 2. A non noisy audio segment

2.3 Experiment

The beginning point and the ending point of each word on the spectrum (ex-plained in the section 4 below) were defined. The calculation of the silence be-tween words was automatically conducted with Microsoft Excel work sheets.Some macros in Excel were created to automatically analyze and total up thedata in order to prove the hypotheses.

3 Prosodic Hypotheses

Like other languages, Vietnamese has parts of speech such as conjunctions,prepositions, articles, numerals, etc. Also, there are some kinds of phrase (orgroup of words) called compound words. The distinctive feature about intonationis that each word can be pronounced with two or six tones (none, high-pitched,low-pitched, long-high-pitched, very-long-high-pitched, and short-high-pitched).The five following hypotheses about Vietnamese prosody were established.

66 T.-H. Lê et al.

3.1 Conjunction and Preposition

- Conjunction: using to join the same functional words in a sentence, or to jointhe main cause with the subordinate cause. There are two types of conjunction:associate conjunction (và, với, hoặc, rồi, lẫn, cùng...) and dependent conjunction(vì, thì, mà, nếu, tuy, mặc dù, vả lại...) [4]

- Preposition: using to join two words or two phrases having amain-subordinaterelation; for example, "của" in "cuốn sách của tôi" (the book of me). Some Viet-namese prepositions are: bằng, bởi, cho, chung quanh, do, dưới, giữa, gần, khỏi,không, lên, qua, ra, thành, trên, tại, tận, tới, từ, vào, về, với, xuống, đến, ở [2].

Hypothesis no.1: In Vietnamese, a conjunction/preposition and theword followed it must be pronounced without an emission break be-tween them, excepting the preposition at the beginning of a sentence.For example, “Tuy nhiên” (However) normally followed by a comma signifies anemission break.

- Let’s examine the sentence “ làm đẹp cho Huế, đồng thời tạo ra sân chơimới cho giới mỹ thuật ”. In this sentence, “cho” is a preposition. Based on themeasurement by Sound Forge 9.0c software, the followed silences are 1ms and2ms, two relatively short breaks.

Fig. 3. The spectrum of the sentence with two prepositions (”cho”)

3.2 Words Ending with Occlusive Syllables

Occlusive syllables are the phonetic units which terminate with one of the con-sonants p, t, c, ch (called stop consonants). This means the voice emission isstopped by a vocal physical element such as the lips, the tongue, and the glottis.

These phonetic units’ duration is naturally very short, and correspond onlywith the accents high-pitched (’) and short-high-pitched (.). The latter hasshorter duration than the former, e.g. “táp, tạp”. (See more explanation in [1]).

Hypothesis no. 2a: Consequently, all words ending with occlusive haveshorter duration than the ones with other accents. The following spec-trum of a sentence is an example: “Ngày mốt là một tháng hai.”

In this example, the word “một” has the shortest duration in spite of differentspeakers.

Hypothesis no. 2b: When pronouncing these words, the speakers au-tomatically add a silence to regularize the speaking rhythm (to be inharmony with other words) because the length of those words is veryshort.

A Study on Vietnamese Prosody 67

Fig. 4. The spectrum of the sentence

"Ngày mốt là một tháng hai"

3.3 Compound Words and Group of Words

Each Vietnamese word has only one phonetic unit (monosyllabic) and can be asimple word (meaningful) or an element of two or more combined words with adistinctive meaning. Compound words (từ kép) formed by two single words, andgroup of words (cụm từ) formed by more than two element words. For example,“tôi, bác, nhà, ngựa, cười, đẹp” are single words. “Chợ búa, bếp núc, xe cộ, sânbay, lấp ló, lóng lánh, xanh lè, đỏ rực” are compound words. Finally, “khít khìnkhịt, sạch sành sanh, vội vội vàng vàng” are group of words.

Hypothesis no. 3: To correctly pronounce a compound word or a groupof words, speakers do not break the vocal emission between the el-ement words because this will influence on the meaning, i.e. one can haveconfusion or an unclear meaning. For example, the sentence “Trời thu lạnh lẽonhưng trong lòng tôi không cảm thấy lạnh chút nào” has the following spectrum:

Fig. 5. The spectrum of

“Trời thu lạnh lẽo nhưng trong lòng tôi không cảm thấy lạnh chút nào”

The compound word “lạnh lẽo” is pronounced without silence between thesetwo words, while the next single word “lạnh” has the followed silence before theword “chút”.

3.4 Article and Numeral

An article precedes a noun; for example, “cái, con, cục, quyển, sợi, tấm, tờ, hạt,etc.” A numeral also precedes a noun to quantify the number of units (e.g. “bacon chó – three dogs”).

68 T.-H. Lê et al.

Hypothesis no. 4: Numeral words, as well as articles followed by anoun, were pronounced without vocal emission break between them.

3.5 Temporal Words

The duration of silences in sentences, which contain words such as day, month,year, hours, minutes, seconds, etc., was also measured. Some examples wereillustrated:

“Ngày hai mươi lăm tháng mười hai, ngày hôm qua, ngày mai” (the twentyfifth of December, yesterday, tomorrow); “năm nay, năm ngoái” (this year, lastyear); “sáu giờ mười phút” (six o’clock and ten).

Hypothesis no. 5: When pronouncing the temporal words, speakersdo not break the vocal emission; the silence between them is usuallyvery short.

4 Measurement

The SoundForge 9.0c was used to define the beginning point (bi) and the endingpoint (ei) of the ith word in a sentence which had n words. By default, thenumber of silence in this sentence was n-1. The length (l) of ith word was li = ei

- bi and the silence (s) following ith word was si = bi+1 - ei, where bi+1 was thebeginning of the next word (i+1th word). For the words with si>200 miliseconds(ms), their silence was adjusted to equal to 200 ms to standardize the speech.

Fig. 6. Spectrum with beginning and ending points of words

After that, the data was transfered to Excel, which was able to automaticallycalculate the duration, based on the operators and functions established.

The total length of n words in each data file was L =n∑

i=1li

The total llength of n-1 silence in each data file was S =n−1∑

i=1

si.

From that, the mean was calculated (L), the standard deviation (StDevL) ofthe word’s length as well as the mean (S) and the standard deviation (StDevS)of the silence length:

A Study on Vietnamese Prosody 69

L = Ln

and StDevL =

√

( 1n−1

)n∑

i=1

(li − L)2

S = Sn−1

and StDevS =

√

( 1n−2

)n−1∑

i=1(si − S)

2

By measuring the silence length which following a conjunction, a preposition,an article, a numeral and a temporal phrase, and then compare to S as well asthe word’s length of words ending with occlusives and short-high-pitched, thencompare to L, we can confirm the established hypotheses above.

5 Self-evaluation of the Current Experiment

The selected and recorded data file contained the voice of the well-trained broad-casters reading news and reports on the local and national broadcasting pro-grams. Therefore, these voices represented the popular voices of Vietnamese,which ensured the correctness and objectivity for the research. The voices of ma-jor regions in VietNam were carefully selected including Ha-Noi capital, HCMcity, Hue city, Da-Nang city, and Ca-Mau province.

In general, the calculated numbers from the experiment could sufficientlyprove the prosody hypotheses. However, there were some exceptions where thebroadcasters intentionally highlighted the speech’s content, and caused the ex-tension of some silence after the words with stop consonants. More time is nec-essary to analyze the data files. Further research will surely discover other rulesof Vietnamese prosody.

6 Experimental Results

The numbers in table 1, the overall result below, were automatically calculatedwith Excel. The total number of words in the 36 prosody data files was 20,815(the total number of silence was 20,815 - 36 = 20,779).

Table 2 indicates that there is not a vocal break (silence) after the conjunctionsand prepositions, excepting after the compound conjunction at the beginning ofsentence where the silence is equivalent to a comma punctuation mark (178 ms).

Table 3 indicated that the length of words with short-high-pitched & occlusivewas generally short (143ms) compared with the mean of all words (184ms).Hence, when a sentence was synthesized by concatenating a word of this categorywith a followed word, if a short silence is not inserted after them, then the readingwas a little odd, and unnatural. Indeed, the words of this category recorded inthe sample files had very short length about 100ms to 120ms, because the vocalemission was completely closed. However, when speakers talk, the rhythm wasregularized by naturally appending a short silence after them, which was verydifficult to be recognized

70 T.-H. Lê et al.

Table 1. Analysis of silence after conjunctions, prepositions, words with occlusive andcompound words

CONTENTSum/Mean

CONTENTSum/Mean

Words Compound & Normal wordsNumber of words 20,815 Compound wordsTotal length of words 3,847.507 Number of silence 774Mean of a word’s length (LMEAN ) 0.185 Total length of silence 17.893Standard deviation 0.064 Mean of a silence length 0.023Silence Percentage compared to CMEAN 47%Number of silence 20,779 Normal wordsTotal length of silence 1,011.617 Number of silence 118Mean of a silence length (SMEAN) 0.049 Total length of silence 5.519Standard deviation 0.062 Mean of a silence length (CMEAN ) 0.047Conjunctions & prepositions Other articlesNumber of silence 2,119 Number of silence 275Total length of silence 43.082 Total length of silence 5.747Mean of a silence length 0.020 Mean of a silence length 0.021Percentage compared to SMEAN 42% Standard deviation 0.019Words with Occlusive Percentage compared to SMEAN 43%Number of words 1,429Total length of words 204.696Mean of a word’s length 0.143Percentage compared to LMEAN 77%

Table 2. Analysis of silence after conjunctions & prepositions

Measureof

Total number andmean of length

Conjunctions andPrepositions

Conjunctions at thebeginning of sentence(e.g. tuy nhiên, tuyvậy, do đó, etc.)

Silence 20,779 2,119 9Words 20,815 2,119 9Mean of length 49 ms 20 ms 178 ms

Table 3. Analysis of words with occlusive and stop consonants

Measureof

Totalnumber andothermeasures

Words withocclusiveand stopconsonants

Words with short-high-pitched and occlusive

”p” ”t” ”c” ”ch”

Words 20,815 1,429 184 517 650 78Mean of length 184.8 143.2 150.9 151.1 136.5 128.9Max. duration 565 329 324 329 304 305Min duration 34 55 66 59 55 74

A Study on Vietnamese Prosody 71

Table 4. Analysis of silence between compound and simple words

Measure of Total/ Compound The same words usedMean words as simple words

Silence 20,779 774 118Mean of silence length 49 ms 23 ms 47 ms

Table 5. The mean of the silence length of numerals and temporal words

Type of word Notation Mean(ms)

The mean of silence length of article words Article 21The mean of silence length of numerals Num 23The mean of silence length of temporal words Temp 18The mean of silence length in 36 data files SMEAN 49

Table 4 indicated that the silence between the elements of compound wordsvaried around 23ms, 50% shorter comparing with the mean of all silence (49ms),which meant there were no vocal breaks between the element words. In the lastcolumn of this table, the same words used as simple words have the same followedsilence as other words in the sentence.

Table 5 compared the mean of the silence length between article words, tem-poral words or numerals to the mean of the silence length between all words inthe 36 data files. The former silence was shorter than the latter (>50%).

Fig. 7. The spectrum of “Statistical result chart”

Basing on the statistical results (6th column) in figure 7 above, the broadcast-ers were revealed to automatically adjust the silence length to regularize theirspeech.

72 T.-H. Lê et al.

Table 6. The percentage of the silence length between the numerals and the followednouns of these words compared to the mean of the general silence length (SMEAN)

Measure of không một hai ba bốn năm sáu bảy tám chín mười

Silence length (ms) 26 51 18 10 14 12 18 7 17 14 9Compared to SMEAN% 54 105 38 20 29 26 37 15 34 30 18

Table 7. The percentage of the silence length of these words within temporal phrasescompared to the mean of the general silence length (SMEAN)

Measure of không một hai ba bốn năm sáu bảy tám chín mười

Silence length (ms) 19 22 4 13 0 8 14 9 15 26 24Compared to SMEAN (%) 39 45 8 28 0 18 31 18 32 53 51

– The mean of the silence length following a conjunction or a preposition was20 ms and accounting of 42% was compared to the mean of the silence lengthin their sentences (SMEAN=49 ms)

– The mean of the word length having occlusive and short-high-pitched was143 ms and accounting of 77% was compared to the mean of all word length(LMEAN=185 ms).

– The mean of silence length between a compound words or noun phrase was23 ms and accounting of 49% was compared to the mean of silence length ofnormal words (CMEAN=47 ms).

– The mean of the silence length between articles words and a followed nounwas 19 ms and accounting of 39% was compared to the mean of silence lengthof all words (SMEAN=49 ms).

– The mean of the silence length between a numeral and a noun was 27 msand accounting of 56% was compared to the mean of silence length of allwords (SMEAN=49 ms).

– The mean of the silence length between temporal words was 19 ms andaccounting of 39% was compared to the mean of silence length of all words(SMEAN=49 ms).

Regarding to the article words, numerals and temporal words, the speakers au-tomatically adjust their speech by adding the silence after the words which havethe short-high-pitched and stop consonant" in the following statistical tables:

7 Conclusion

Basing on the calculated experimental results on 36 sound files of 20.815 wordsread by the broadcasters from several distinctive regions in Vietnam, five estab-lished hypotheses for Vietnamese prosody were tested and proved as illustratedin table 8 and the comparative chart

By applying these five rules, a speech synthesizer can improve the rhythmof speech to be more natural. To do that, we must integrate a dictionary of

A Study on Vietnamese Prosody 73

Table 8. Comparison of experimental results

Type of word Notation Mean (ms)

The mean of the silence length (SMEAN) SMEAN 49Mean of the silence length of conjunction/ preposition Conj-Prep 20Mean of the silence length of compound words (CMEAN) Comp 23Mean of the silence length of article words Article 21Mean of the silence length of numerals Num 23Mean of the silence length of temporal words Temp 18

articles, conjunctions, prepositions, compound words and group of words intothe system. When the program scans the text, it can easily recognize themthank to this dictionary. Then, we have elaborated a sophistical code to reduce,in runtime, the duration of the first element word without influencing its quality.And finally, we added a small silence (21ms) after the ending element word, sothat the reader clearly recognized that is a compound word or a group of words.For the temporal word phrase, the program can recognize them by a specialalgorithm, then add a silence (18ms) between these words.

8 Future Research

Further studies are necessary to understand and ascertain other rules relatedto the Vietnamese prosody (e.g. rules for rhythm of phrases, idioms, proverbs,etc.). The prosody feature (ngôn điệu) can be considered in the bigger units ofthe speech (sentences). For instance, the whole sentences involve the emotionalexpression introduced in the section 1.2 above. The results of this study may beefficiently applied to the second branch of the voice technology: the automatedspeech recognition, which is just at the initial phase in Vietnam.

References

1. Tang-Hồ, L.: Thử tìm một phương pháp hữu hiệu cho việc dạy đọc và viết tiếngViệt. Huong Viet Review, No. 11, Houston, USA (1999), Khảo cứu ngữ âm ViệtNam on http://vietsciences1.free.fr/vietscience/vietnam/

tiengviet/chuongtrinhtiengviet.htm

2. Bách khoa từ điển. Wikipedia, www.wikipedia.com

3. O’Shaughnessey, D.: Speech Communications Human and Machine, 2nd edn. IEEEPress, Los Alamitos (2000)

4. Đức, H.N.: Free Vietnamese Dictionary Project,http://www.informatik.uni-leipzig.de/~duc/Dict/

5. Minh, L.H., et al.: Phân tích và tổng hợp đặc tính trường độ của tiếng Việt. Mộtsố vấn đề chọn lọc của CNTT. Thái Nguyên, 29-31 tháng 8 năm (2003)

6. Đạt, T.D.: Analysis and Modeling of Syllable Duration for Vietnamese Speech Syn-thesis. In: O-COCOSDA 2007 (2007)