Proceedings of the 14th International Conference on Finite-State Methods and Natural Language Processing, pages 108–117Dresden, Germany, September 23-25, 2019. c©2019 Association for Computational Linguistics

108

Latin script keyboards for South Asian languageswith finite-state normalization

Lawrence Wolf-Sonkin, Vlad Schogol, Brian Roark and Michael Riley{wolfsonkin,vlads,roark,riley}@google.com

Google Research

AbstractThe use of the Latin script for text en-try of South Asian languages is common,even though there is no standard orthogra-phy for these languages in the script. Weexplore several compact finite-state archi-tectures that permit variable spellings ofwords during mobile text entry. We findthat approaches making use of transliter-ation transducers provide large accuracyimprovements over baselines, but that sim-pler approaches involving a compact rep-resentation of many attested alternativesyields much of the accuracy gain. This isparticularly important when operating un-der constraints on model size (e.g., on inex-pensive mobile devices with limited storageand memory for keyboard models), and onspeed of inference, since people typing onmobile keyboards expect no perceptual de-lay in keyboard responsiveness.

1 IntroductionMany of the world’s writing systems presentchallenges for machine readable text entrycompared with alphabetic writing systems(such as the Latin script used for the Englishin this paper). For example, a very large char-acter set, such as that used for Chinese, can beimpractical to represent on a keyboard requir-ing direct selection of characters; hence spe-cialized encoding methods are generally usedbased on smaller symbol sets. For example,the well-known pinyin system for text entryof Chinese relies on Latin alphabetic codes toinput Chinese characters. South Asian lan-guages, such as Tamil and Hindi, also usewriting systems that, while lacking the thou-sands of characters as in Chinese, are nonethe-less challenging for direct typing (particularlyon mobile devices), and hence are frequentlyentered using the Latin alphabet. In thoselanguages, however, unlike Chinese, there is

no single system that is used for romaniza-tion, rather individuals typically provide arough phonetic transcription of the words inthe Latin script.

The use of pinyin for Chinese is gener-ally part of a system for converting the textinto the native script, and this can also beachieved for keyboards in South Asian lan-guages (Hellsten et al., 2017). However, forthese languages, many individuals prefer tosimply leave the text in the Latin script ratherthan converting to the native script. To pro-vide full mobile keyboard functionality in sucha scenario – including, e.g., word predictionand completion, and automatic correction ofso-called fat finger errors in typing – languagemodel support must be provided. Yet in theabsence of a standard orthography, encod-ing word-to-word dependencies becomes morecomplicated, since there may be many possibleversions of any given word.

In this paper, we examine a few practicalalternatives to address the lack of a conven-tionalized Latin script orthography for use in afinite-state keyboard decoder. We use severaldifferent transducers that normalize input ro-manizations to either a native script word formor a “canonical” Latin script form1 in order tocombine with a word-based language model.To produce Latin script after this normaliza-tion, we must produce text from the inputtape of these transducers. We also present analternative method involving a compact rep-resentation of a large supplementary lexiconthat covers highly likely romanizations of in-vocabulary words. All of these methods pro-vide accuracy improvements over the baseline

1We use canonical in quotes here and elsewhere be-cause there is no standard orthography hence no truecanonical form; rather, for each native script word inour lexicon, we choose one romanization as “canonical”.

109

(fixed vocabulary) method.In the next section, we give some back-

ground on the problem before outlining ournew methods. We then present experimentalresults of keyboard entry simulation for Hindi,in which we demonstrate over 50% relative re-duction in error rate2 over existing baselines.

2 Background and preliminaries

2.1 South Asian romanizationRomanized text entry is widely used in lan-guages, such as Hindi and Arabic, for whichthere is no agreed-upon adherence to anyparticular conventionalized representation inthe Latin script such as is found in Chi-nese. South Asian languages written na-tively in a Brahmic script, including Hindi,Bengali, Tamil and many others, are heav-ily romanized mainly due to the complex-ity of typing the scripts. These scripts areabugida (or alphasyllabary) writing systemsthat are based on consonant/vowel “syllables”(akṣara) that pair consonants with a defaultvowel. Alternative vowels (as well as the lackof a vowel) are designated through the useof various diacritic marks that can appearabove, below, or on either side of the conso-nant (or consonant cluster). This, along withcomplex multi-consonant ligatures (known asconjuncts), makes direct use of native scriptkeyboards relatively uncommon. An exem-plar word in the Devanagari script contain-ing such complex graphemes is given in Fig-ure 1. Romanization is also used for Perso-Arabic scripts in South Asia, such as that usedfor Urdu, but not presumbably due to com-plexities in representing such scripts on nativekeyboards, but rather due to historical rea-sons3 and perhaps the influence of other re-gional languages.

As a result of having no conventionalizedromanization system, text in, say, romanizedHindi has no standardized orthography, butrather words are usually represented via rough

2Word-error rate in this setting means recovery ofthe intended form typed by the user. We take theromanized strings in the validation set as the intendedforms, despite spelling variation throughout.

3Languages using the Cyrillic script are also fre-quently romanized. There, and perhaps also for Perso-Arabic scripts, the issue is with historical lack of fontand encoding support in certain scenarios.

ब ⇒ ब /ba/ब+◌् ⇒ ब् /b/ब्+ र ⇒ /bra/+◌ा ⇒ ा /brā/

ह+◌्+ म+◌ी ⇒ ी /hmī/ा+ ी ⇒ ा ी /brāhmī/

Figure 1: Demonstration of how the Hindi word ा ी/brāhmī/, meaning Brahmic, is decomposed into itsunicode codepoints as written in Devanagari. Pronun-ciations are shown between slashes.

phonetic transcriptions in Latin script. Forexample, the Hindi words सं कृत and संपूण arecommonly romanized as sanskrit and sampu-ran, respectively. Both words begin in De-vanagari with the grapheme सं which is /sa/with a diacritic indicating a nasal consonant(such as /n/) in the coda. Note that thenasal becomes either /n/ or /m/ depending onthe following consonant,4 demonstrating howthese romanizations are driven by pronuncia-tion rather than from the native orthography.Urdu has the same words, written and

respectively in the Perso-Arabic script,and they are romanized similarly to the Hindiwords, also demonstrating the role of pronun-ciation rather than writing system in roman-ization for these languages.5 In general, dueto this lack of a standardized spelling in theLatin script, romanizations may vary due todialectal variation, regional accent, or simplyindividual idiosyncrasies.

As a concrete example, a blog entry on thegeneral topic of political corruption on a siterun by the India Today Group from 2011 hascomments in (1) English; (2) Hindi written inDevanagari (its native script and that is usedin the blog post itself); and also extensively in(3) romanized Hindi.6 One comment begins:“Bhrashtachar aam aadmi se chalu hota hai…”,which presumably corresponds to the De-vanagari: ाचार आम आदमी से चालू होता है androughly translates to: “Corruption starts withthe common man…” Given that corruption isthe overall topic of the blog post, it is unsur-

4This is a process known as assimilation.5Note that unlike in Devanagari, Perso-Arabic

script Urdu does in fact graphically differentiate thesephonetically differentiated onsets, spelling them re-spectively as ⟨sn⟩ and ⟨sm⟩.

6http://blogs.intoday.in/index.php?option=com_myblog&contentid=62323&show=Removal-of-corruption-from-the-beginning-itself&blogs=2

110

prising that the Hindi word for this shows upin many comments. It is, however, variouslyromanized. By our count: 16 times it is ro-manized as “bhrastachar”; 7 times as “bhrash-tachar”; and once each as “barashtachaar”,“bharastachar”, “bharstachar”, “bhastachar”and “bhrstachar”. Google Translate providesboth a translation and a romanization of theword (“bhrashtaachaar”7), a form which inter-estingly is not found in our (admittedly small)example blog comment sample.

2.2 Transliteration and romanized textThe need to transliterate between writing sys-tems comes up in many application scenar-ios, but early work on the topic was largelyfocused on the needs of machine translationand information retrieval due to loanwordsand proper names (Knight and Graehl, 1998;Chen et al., 1998; Virga and Khudanpur, 2003;Li et al., 2004). These approaches eitherexplicitly modeled pronunciation in the lan-guages (Knight and Graehl, 1998) or more di-rectly modeled correspondences in the writ-ing systems (Li et al., 2004). Models for ma-chine transliteration have continued to im-prove, through the use of improved mod-eling methods including many-to-many sub-string alignment-based modeling, discrimi-native decoding, and multilingual multitasklearning (Sherif and Kondrak, 2007; Cherryand Suzuki, 2009; Kunchukuttan et al., 2018),or by mining likely transliterations in largecorpora (Sajjad et al., 2017). Transliterationmodels are also being deployed in increasinglychallenging use scenarios, such as mixed-scriptinformation retrieval (Gupta et al., 2014) orfor mobile text entry (Hellsten et al., 2017).

The volume of romanized text in languagesthat use other writing systems is an acknowl-edged issue, one which has grown in impor-tance with the advent of SMS messaging andsocial media, due to the prevalence of roman-ized input method editors (IMEs) for theselanguages (Ahmed et al., 2011). The lack ofstandard orthography and resulting spellingvariation found in romanization is also foundin other natural language scenarios, such asOCR of historical documents (Garrette andAlpert-Abrams, 2016) and writing of dialectal

7translate.google.com/#en/hi/Corruption

Arabic (Habash et al., 2012).For this study, we make use of Wikipedia

data originally written in the native scriptthat has been romanized, and our task is topermit accurate text entry on mobile key-boards, rather than transliteration to the na-tive script or normalization for use in otherdownstream tasks. In this case “accuratetext entry” means fidelity to the intendedtext, even if that intended text is writ-ten without consistent spelling. If the usernoisily types “bgrashtachsr” while intending“bhrashtachar”, the keyboard should produce“bhrashtachar” not another romanization suchas “bhrastachar”. Given annotator-romanizedWikipedia text, we evaluate our ability to cor-rectly recognize the actual romanizations used.

2.3 Mobile keyboard decodingVirtual keyboards of the sort typically usedon mobile devices convert a temporal sequenceof interactions with the touchscreen (taps orcontinuous gestures) into text. Like speechrecognition or optical character recognition,the mapping of noisy, continuous input sig-nals to discrete text strings involves stochasticinference; further, given the low required la-tency during typing, models must be compactenough to run on the local device and inferencewith them must be fast. For this reason, thekinds of finite-state methods that have beenused for speech recognition and OCR have alsobeen used for this task (Ouyang et al., 2017).The work we present here will be in the con-text of such an FST-based keyboard decoder.

For touch typing, where the input consists ofa sequence of taps, we designate with the termliteral the string corresponding to the actualkeys touched. The intended string may dif-fer, due to such phenomena as so-called “fatfinger” errors, i.e., hitting a neighboring key,omitting a key or including an extra tap.

Analogous to the acoustic model in speechrecognition, which assigns probabilities tothe continuous waveform given a sequence ofphones, such a decoder makes use of a spatialmodel, assigning probabilities to the sequenceof taps (or gestures) given a sequence of letters.Taps, for example, are modeled in Ouyanget al. (2017) with Gaussians centered on themiddle of each key. Costs are thus assigned toalternative possible intended character strings

111

which may have substitutions, deletions andinsertions relative to the literal string.

In speech recognition, phones are typicallysplit in the acoustic model based on the sur-rounding context, in order to capture co-articulation effects and other influences onthe acoustics associated with a particular in-tended phone. Similarly, in the spatial model,keys are typically split based on the previoustouched key, which we will term bikey repre-sentation. So, at the start of a word, the letter‘b’ will be represented as ‘_b’, whereas afterthe letter ‘a’, it would be represented as ‘ab’.Later we will have methods that must be awareof the input representation.

The spatial model cost and the languagemodel cost are combined by the decoder toscore competing output strings. Typicallythese scored string alternatives will be com-pared to the literal string and only selectedif the difference in score is above some thresh-old, to avoid spurious changes to what the usertyped (Ouyang et al., 2017). To accept anystring (including any possible literal string), aloop transition for every character with somefixed cost can be included at the unigramstate (the base of the smoothing recursion, seeRoark et al., 2012), so that every string in Σ∗

has non-zero probability.In addition to decoding for auto-correction,

the language model may also be used for wordprediction and completion, i.e., showing sug-gestions in a small dynamic portion of the key-board. In this paper, we do not have much tosay about this part of the process, other thanto point out when its demands make certainapproaches more complicated than others.

Such an architecture has also been usedfor transliteration from Latin script input tonative script output (Hellsten et al., 2017),by interposing a finite-state transducer (FST)between the spatial model (defined over theLatin script) and the language model (definedover native script words). Some of our meth-ods are related to these, although the outputof the keyboard does not change script.

3 Methods

3.1 Word transliteration modelsFor both off-line model training and on-linetransliteration-based decoding methods, we

make use of pair n-gram (also known as“joint multi-gram”) modeling methods (Bisaniand Ney, 2008), which Hellsten et al. (2017)also use to train their transliteration mod-els. Given a lexicon with words in the na-tive script and possible romanizations of thosewords (see §4.1 for specifics on our data), ex-pectation maximization is used to derive pair-wise symbol alignments. For example, ाचारand “bhrashtachar” may yield a pairwise sym-bol alignment of:

भ:b ◌्:h र:r ϵ:a ष:s ◌्:h ट:t ◌ा:a च:c ϵ:h ◌ा:a र:rwhere each symbol is composed of an input(native script) unicode codepoint (or ϵ, denot-ing the empty string) and an output (Latinscript) unicode codepoint (or ϵ). These sym-bol pairs then become tokens in an n-gramlanguage model encoded as an automaton. Fi-nally, the automaton is converted to a trans-ducer with native script on one side and Latinscript on the other.

This model provides a joint probability dis-tribution over input:output sequence pairs,e.g., for a word ाचार and a romanization“bhrashtachar”, i.e., P ( ाचार,bhrashtachar).As Hellsten et al. (2017) note, within most de-coding settings that combine with a languagemodel on the native script side, a conditionalprobability is actually what is needed:P (bhrashtachar | ाचार). We refer readers tothat paper for details on how to incorporatethe appropriate normalization into an FST-based decoder. We use similar methods, per-mitting the model to be used in both on-lineand off-line scenerios.

Note that it is trivial to swap the input andoutput symbols for such a model, either bychanging the ordering of the pair symbols inthe training data or simply inverting the re-sulting WFST. The same model can thus beused for transliteration from input Latin scriptto native script forms; or from input nativescript to romanizations.

For example, suppose T is a transliterationtransducer (Latin script on the input side andnative script on the output side) and S is anautomaton that accepts a single native scriptword w for which we wish to find likely roman-izations. If we compose T ◦ S, this yields atransducer encoding alternative Latin/nativescript string relations with w as the output

112

string. We can convert this transducer intoan automaton accepting alternative romaniza-tions of w by projecting all transitions ontotheir input labels, i.e., preserving only the in-put label on every transition. Some transi-tions in this acceptor of alternative romaniza-tions might be epsilons, so we can remove ep-silon transitions (Mohri, 2002), then select then most likely paths (Mohri and Riley, 2002).All of these operations are general operationssupported by the OpenFst library (Allauzenet al., 2007). The n unique shortest paths inRmEpsilon(ProjectInput(T ◦S)) are the nmost likely romanizations of w.

3.2 Baseline fixed-vocabulary systemOur baseline system relies on automatic ro-manization of native script language modeltraining data. Using a transliteration trans-ducer, trained as described in §3.1, each wordin our fixed vocabulary is assigned a “canon-ical” (i.e., best scoring) romanization as itsLatin script representation. We then replacethe native script words in our language modeltraining corpus with their canonical romaniza-tions and retrain the model, yielding a lan-guage model over strings in the Latin script.

If the distribution over romanized alterna-tives for ाचार in the blog comments thatwere mentioned in §2.1 represented the dis-tribution provided by our model, then “bhras-tachar” would become its canonical romaniza-tion. Alternative spellings (e.g., bhrashtachar)would only match that word via the characterloop method (mentioned in §2.3) permittingthe omitted letter, generally with a cost.

3.3 Compact supplemental unigramNot every substituted, inserted or omitted tapis created equal when it comes to likely ro-manization variants. For example, as we haveseen in our running example, the use of ‘h’to indicate aspiration for consonants such asभ may or may not be used in romanizations.Similarly long vowels and geminates are some-times represented by doubling of Latin sym-bols, but often not. These variants are notrandom in the way that a character loop modelwould score them. One method for addinglikely alternative romanizations is to simplyadd them as alternative word forms to thelanguage model. These romanizations can be

computed using the method outlined in §3.1.However, adding many alternative roman-

izations of the same word can become spaceprohibitive, particulary for on-device methods,where storage and active memory usage areboth at a premium. It is possible, however, toprovide a very compact encoding specificallyof the words stored exclusively in the unigram,i.e., words that are neither prefixes nor suffixesof any higher order n-grams in the languagemodel. We achieve this in two steps. First, webuild two automata that accept all and onlythis set of words: a weighted automaton W ,which weights the path for each word with theappropriate cost for that word within the lan-guage model; and an unweighted automaton Awhich encodes the same set of words as W andhas been determinized and minimized. Nextwe create a weighted automaton Wm that hasthe same topology as A, but which is weightedto minimize the KL-divergence (Kullback andLeibler, 1951) between W and Wm, usingmethods from Suresh et al. (2019). This isan approximation of the distribution repre-sented in W over a much more compact topol-ogy. The methods to perform this approxi-mation are part of the open-source OpenGrmstochastic automata (SFst) library (availableat http://www.opengrm.org). We then inte-grate Wm into the larger language model au-tomaton, with the unigram state of the lan-guage model serving as both the start and fi-nal state for the paths corresponding to thosein Wm. This can be straightforwardly accom-plished by using the Replace operation in theOpenFst library (http://www.openfst.org).

3.4 Transducer to canonical formAs we noted in §3.1, we build pair n-gramtransliteration models between native scriptand romanized forms. In a similar way, wecan build a transducer between canonical ro-manized forms and alternative romanizations.To re-use our example, if “bhrastachar” is thecanonical romanization, and “bhrashtachar” isanother attested form, we can use expectationmaximization to derive an alignment:

b:b h:h r:r a:a s:s ϵ:h t:t a:a c:c h:h a:a r:rA pair n-gram model built from this would al-low weighted transduction from input roman-izations to the canonical form, which corre-sponds to tokens in the language model. We

113

…_b:b bh:h hr:r ra:a as:s st:t

bϵ:h br:r rϵ:a rs:s sh:ϵ ht:t

(a) bikey input to canonical romanization lattice

…_b:भ bh:◌् hr:र ra:ϵ as:ष

rs:ष

sϵ:◌् st:ट

bϵ:◌् br:र sh:◌् ht:ट

(b) bikey input to native script lattice

Figure 2: Lattices illustrating how reading from the input tape rather than the output tape can provide supportfor varied romanized input, for either native script or “canonical” romanized output.

can thus use a transducer much in the waydescribed in Hellsten et al. (2017) to translit-erate8 between variant romanizations and thechosen canonical romanizations.

There is, however, a complication with us-ing this method within the keyboard, in con-trast to the earlier described methods. Ifthe keyboard actually performed the transduc-tion from input romanization to the canonicalLatin script form, then it would enforce a nor-malization on the user’s spelling of the word.If the user types “bhrashtachar”, this system,as it has been described, will output “bhras-tachar” (without the ‘h’), since that is our cho-sen canonical form. However, there is no stan-dard orthography in the Latin script for Hindi,i.e., there is no correct spelling. Our canoni-cal form is chosen for convenience to be thehighest scoring romanization from the model.Even if we were to choose some kind of gen-erally common version as our canonical form,however, for any given individual we may endup coercing the output of a form that they dis-prefer. Instead, we would like to allow them tomaintain their preferred form, i.e., they shouldbe able to type their intended string.

Because the decoder is based on WFSTs, wehave a particularly straightforward solution tothis: output the string from the input taperather than the output tape. That is, we usethe transducer within the decoder just as isdone in Hellsten et al. (2017), however we out-put the string on the input side correspondingto the best scoring solution. In this way, wederive the modeling benefit from the language

8Note this isn’t quite transliteration in the usualsense since the strings stay in the same writing system,but we co-opt the term since Hellsten et al. (2017) usedan identical architecture for transliteration.

model without imposing a canonical roman-ization on the user. Note that it would bepossible to perform this large composition andproject onto input labels off-line rather thanon-the-fly, but the size of the off-line composi-tion is prohibitively large for on-device opera-tion, for reasons similar to those discussed inHellsten et al. (2017). The output labels arethus preserved in one of the transducers usedduring on-the-fly composition.

Figure 2a shows a WFST lattice represent-ing alternative paths through the decoder,with bikey inputs and canonical romanizationoutputs.9 For convenience, bikey representa-tions of outputs with no corresponding inputdisplay the previous key followed by an ep-silon, e.g., ‘bϵ’ signifies an omitted key follow-ing a ‘b’. Note that every path through thislattice has an output string corresponding to“bhrast”, the prefix of the canonical romaniza-tion of our running example. Different pathsrepresent different input string variations cor-responding to this word.

To read a string off of the input side of such alattice, we take the last symbol of the bikey ateach transition, with ϵ representing the emptystring. In such a way, the presented latticeencodes the alternatives bhrast, brast, bhrsht,bhrasht, etc. Whichever path has the lowestcost during decoding would be the version thatis produced by the keyboard.

3.5 Transducer to native scriptIf, as discussed in the previous section, we out-put from the input tape of our WFSTs, then

9Note that, during decoding, partial results maybe displayed to the user to improve responsiveness, sothese figures should be taken as an illustration not adepiction of the decoding process.

114

there is no strong reason10 to have Latin scripton the output tape. Instead, we use a nativescript language model and the same sort ofpair n-gram transducer as used in a translit-erating keyboard, then simply read the resultfrom the input tape.

Figure 2b has a set of paths, all of whichproduce the native script word prefix ( ) onthe output side. As with the other lattice, thepaths represent different input romanizationscorresponding to this string, which can be re-covered from the transition labels.

3.6 Native script OOV modelingOne question we have only briefly touchedupon is how to deal with out-of-vocabulary(OOV) items when using a transliterationtransducer, i.e., words not in the languagemodel being used for decoding. The simpledefault method is to have a character loop atthe unigram state of the language model thataccepts each character. That loop then getsan approximated language model cost fromthe transliteration transducer, to the extentthat that model provides a joint probabilityof input and output strings. Alternatively, wecan build a character language model, or evenmore complicated data structures, to assignprobabilities to OOV words.

We opt to follow an approach that providesflexibility to move between two extremes, themost permissive but least accurate being thecharacter loop, and the most restrictive butmost accurate being a weighted character trie.The trade-off is controlled by a single param-eter N , which is the number of states we wishto use to represent the OOV model. We startwith a large collection of native script wordsand their unigram probabilities. We first builda weighted character trie representing thesewords. The trie is weight-pushed so that theprobability mass of a state is seen as early aspossible by the decoder. Next we rank eachstate of the trie by the total probability massof all words reachable from that state. Fi-nally we remove any state beyond the first N

10This is not strictly speaking true, since, as ispointed out in §2.3, the models may also be used forword prediction and completion. In order to seamlesslyintegrate with such processes, predicted and completedwords would have to be presented in the Latin script,hence some additional information would need to beprovided for each word in the vocabulary.

states in the ranking. All transitions from re-tained states to removed states are redirectedto a state with the original character loop. Inthis way, we provide a mechanism to smoothlyscale between a full trie representation (nostates removed) down to a single-state charac-ter loop (all states removed), and everythingin between. This approach, like the characterloop baseline, permits arbitrary word-forms tobe typed, but it does so in a way that bet-ter captures the distribution of word forms inthe language. The pruned trie provides an ap-proximation to the distribution in the full trie,which permits a graceful tradeoff between thesize of the encoding and the quality of the ap-proximation.

4 Experiments

4.1 DataTransliteration models were trained from aproprietary lexicon of Hindi words and at-tested romanizations, consisting of approxi-mately 110,000 native script words and onaverage 3.1 romanizations per word. Thesealigned Devanagari–Latin word pairs wereused to build a WFST transliteration modelusing methods detailed in §3.1. We built pair3-gram models, pruned to contain just 110,000n-grams prior to conversion to a transducer.

Language models, both in Devanagari andcanonical Latin forms, were trained on a largeand diverse set of Devanagari Hindi text col-lected from the web, and were not trained forany specific domain. The 150,000 most fre-quent words in the training set were retainedin the language model, and trigram word-based models were trained and then pruned toretain just 750,000 n-grams, so as to fit withinon-device space limits.11 For a single exper-iment, we additionally considered a languagemodel containing 1,500,000 n-grams, doublethe n-gram count of the others.

For methods using a transducer to nativescript within decoding, the language modelis in the native script; whereas in other con-ditions, the language model is in the Latinscript. To train the Latin script language

11Our work as targeted South Asian languages,where inexpensive smartphones are the norm, hence,as mentioned in Hellsten et al. (2017) we have gener-ally targeted total model sizes around 10MB.

115

Word error Tap Avg active Model sizeMethod rate (%) avg ms states arcs (MB)Literal 45.0 - - - -Fixed vocabulary (canonical only) 22.6 0.95 164.8 417.9 4.9Fixed vocab + supplemental unigram 12.4 1.00 122.3 404.4 10.6Fixed vocab + transducer to canonical 12.9 1.01 75.6 225.6 9.3Transducer to native script 13.1 0.95 66.7 184.7 11.0Transducer to native + OOV model 10.5 1.08 67.5 181.8 11.2

(a) An operating point of approximately 1.0ms per tap

Literal 45.0 - - - -Fixed vocabulary (canonical only) 22.5 0.59 71.8 202.5 4.9Fixed vocab + supplemental unigram 12.1 0.53 38.1 132.0 10.6Fixed vocab + transducer to canonical 14.1 0.54 24.4 80.9 9.3Transducer to native script 14.0 0.60 31.8 91.6 11.0Transducer to native + OOV model 12.3 0.54 19.7 54.9 11.2

(b) An operating point of approximately 0.55ms per tap

Table 1: The word error rate for the decoding of noisy touchpoints into Latin script strings at two operatingpoints along the speed–accuracy tradeoff. The average number of milliseconds required per character as well asthe average number of states and transitions active during decoding and the model size in megabytes are listed.The best performing (lowest) word error rate method for each operating point is bolded.

model for Hindi, each word in the vocabu-lary (each of which is in Devanagari), is re-placed with its “canonical” romanization, i.e.,the highest probability romanization accord-ing to the trained transliteration model.

Devanagari script sentences from HindiWikipedia were manually romanized by na-tive speakers, and 4,000 of these (for a totalof 36,027 word tokens) were used as our de-velopment set for validation of the methodspresented above.

4.2 EvaluationTo evaluate our methods, we simulate touchpoints of a tapping keyboard as follows. Foreach symbol in the (Latin script) input strings,we sample a touch point from Gaussian distri-butions in two dimensions, with mean value atthe center of the key. To establish how muchnoise is introduced by this method, we evalu-ate the error rate of simply emitting the literalsequence, i.e., the symbols associated with thekeys that our noisy touch points actually fallwithin. Improvements over the literal baselineare due to decoder auto-correction.

The resulting touchpoints are then fed intothe decoder under each of our conditions, andthe strings output from the decoder are thencompared with the original text strings, whichare taken to be the intended strings. As men-tioned elsewhere in the paper, the goal is to al-low users to type their intended strings, with-out normalizing away their versions of the ro-manized words. Thus we measure word-error

rate versus the reference version in the roman-ized string. Note that the keyboard decoderhas various meta-parameters that can impact,e.g., the speed–accuracy trade-off. In additionto sweeping over such parameters for a givenmethod, as shown in Figure 3, we compare per-formance across the methods at comparableoperating points (in terms of average millisec-onds per character) in Tables 1a and 1b.

Note that the absolute numerical values ofthe latencies are not meaningful, just the com-parisons between the latencies. As discussedin Hellsten et al. (2017) and mentioned ear-lier, latencies must be low enough that no lagin keyboard responsiveness is perceived, andtarget values on device are often around 20msper tap. However this must be the case also forinexpensive devices with low processing power,and the decision to deploy a model would de-pend on device trials. For the purposes of thispaper, however, we just report values on a sin-gle device that can be used for comparison pur-poses. The operating points chosen for the Ta-bles are two that are plausible candidates foruse on such inexpensive devices.

4.3 ResultsWhile analyzing the entire operating curve asshown in Figure 3 gives us an idea of thefull potential of any particular model, in aresource-constrained scenario such as a mo-bile keyboard, we are ultimately restricted toworking at a particular operating point on thespeed–accuracy tradeoff. At a higher operat-

116

Average Latency per Character (ms)

Wor

d E

rror

Rat

e

0%

10%

20%

30%

40%

50%

0.25 0.5 0.75 1 1.25 1.5 1.75 2

Literal Fixed vocabulary (canonical only) Fixed vocab + supplemental unigram Fixed vocab + transducer to canonical

Transducer to native script Transducer to native + OOV model

Figure 3: Word error rate versus latency for the models presented in §3 as evaluated on simulated touchpointsfor Latin Hindi text. Various decoding-time parametrizations were explored to demonstrate the word error rate–latency tradeoff. The literal baseline shows how accurate the decoding procedure would be without a model atall on this data set.

ing point of 1.0ms per tap (as in Table 1a),the native script transducer with OOV modelis the best option, bringing the WER downfrom 22.6% for the fixed vocabulary to 10.5%,a substantial 54% relative decrease. This doescome at the cost of model size, with the modeltaking up 2.3x as much space. For the casewhere one chooses a lower operating point suchas 0.55ms per tap (as in Table 1b), the supple-mental unigram wins out providing a 46% rel-ative decrease in word error rate compared tothe fixed vocabulary baseline’s WER of 22.5%;all in a relatively compact model taking uponly 2.2x more space than the baseline. Addi-tionally, we note that at all operating points,a doubly-sized fixed-vocabulary system (de-scribed in §4.1) in fact achieves a slightly worseWER compared with a commensurately sized,otherwise identical language model. We takethis as evidence that this model’s inability tocapture the variant orthographic forms foundin this domain is not corrected by simply in-creasing the model’s n-gram count.

Looking at Figure 3, in the limit as latencyincreases, we find that while the supplemen-tal unigram, fixed vocabulary with transducerto canonical, and transducer to native scriptconverge to similar word error rates of about12.4 ∼ 12.8%, the transducer to native withOOV model can reach even 10.3% WER.

5 Summary

We have presented results for various ap-proaches for handling romanized text entry forSouth Asian languages within an FST-basedmobile keyboard decoder. Compared to base-line methods that naïvely rely upon a singlecanonical romanization for each word in thevocabulary, we can achieve 54% relative errorrate reduction by making use of a translitera-tion transducer and reading the output fromthe input tape. Even at very constrained op-erating points, our best method cuts the errorrate nearly in half.

Further, we have demonstrated that an al-ternative of compactly encoding a large sup-plemental lexicon in the language model, con-sisting of alternative romanizations of words,is competitive to the transducer-based normal-ization, at some space savings. This methodhas the further virtue of relatively straightfor-ward support for other parts of the keyboardapplication – such as word prediction and com-pletion, as well as personalization mechanisms– since the decoder outputs from its outputtape as in typical operation. Deploying meth-ods presented here that read from the inputtape into a keyboard app requires additionalintegration with these other modules.

117

ReferencesUmair Z Ahmed, Kalika Bali, Monojit Choudhury,

and VB Sowmya. 2011. Challenges in design-ing input method editors for Indian languages:The role of word-origin and context. In Proceed-ings of the Workshop on Advances in Text InputMethods (WTIM 2011), pages 1–9.

Cyril Allauzen, Michael Riley, Johan Schalk-wyk, Wojciech Skut, and Mehryar Mohri. 2007.Openfst: A general and efficient weighted finite-state transducer library. In International Con-ference on Implementation and Application ofAutomata, pages 11–23. Springer.

Maximilian Bisani and Hermann Ney. 2008. Joint-sequence models for grapheme-to-phoneme con-version. Speech Communication, 50(5):434–451.

Hsin-Hsi Chen, Sheng-Jie Hueng, Yung-Wei Ding,and Shih-Chung Tsai. 1998. Proper name trans-lation in cross-language information retrieval.In Proceedings of the 17th international con-ference on Computational linguistics-Volume 1,pages 232–236. Association for ComputationalLinguistics.

Colin Cherry and Hisami Suzuki. 2009. Discrimi-native substring decoding for transliteration. InProceedings of the 2009 Conference on EmpiricalMethods in Natural Language Processing: Vol-ume 3-Volume 3, pages 1066–1075. Associationfor Computational Linguistics.

Dan Garrette and Hannah Alpert-Abrams. 2016.An unsupervised model of orthographic varia-tion for historical document transcription. InProceedings of the 2016 Conference of the NorthAmerican Chapter of the Association for Com-putational Linguistics: Human Language Tech-nologies, pages 467–472.

Parth Gupta, Kalika Bali, Rafael E Banchs, Mono-jit Choudhury, and Paolo Rosso. 2014. Queryexpansion for mixed-script information retrieval.In Proceedings of the 37th international ACMSIGIR conference on Research & developmentin information retrieval, pages 677–686. ACM.

Nizar Habash, Mona T Diab, and Owen Rambow.2012. Conventional orthography for dialectalArabic. In LREC, pages 711–718.

Lars Hellsten, Brian Roark, Prasoon Goyal,Cyril Allauzen, Françoise Beaufays, TomOuyang, Michael Riley, and David Rybach.2017. Transliterated mobile keyboard input viaweighted finite-state transducers. In Proceed-ings of the 13th International Conference on Fi-nite State Methods and Natural Language Pro-cessing (FSMNLP 2017), pages 10–19.

Kevin Knight and Jonathan Graehl. 1998. Ma-chine transliteration. Computational Linguis-tics, 24(4):599–612.

Solomon Kullback and Richard A Leibler. 1951.On information and sufficiency. The annals ofmathematical statistics, 22(1):79–86.

Anoop Kunchukuttan, Mitesh Khapra, GurneetSingh, and Pushpak Bhattacharyya. 2018.Leveraging orthographic similarity for multi-lingual neural transliteration. Transactions ofthe Association of Computational Linguistics,6:303–316.

Haizhou Li, Min Zhang, and Jian Su. 2004. A jointsource-channel model for machine translitera-tion. In Proceedings of the 42nd Annual Meetingof the Association for Computational Linguistics(ACL-04), pages 159–166.

Mehryar Mohri. 2002. Generic ϵ-removal and inputϵ-normalization algorithms for weighted trans-ducers. International Journal of Foundations ofComputer Science, 13(01):129–143.

Mehryar Mohri and Michael Riley. 2002. An effi-cient algorithm for the n-best-strings problem.In Seventh International Conference on SpokenLanguage Processing.

Tom Ouyang, David Rybach, Françoise Beaufays,and Michael Riley. 2017. Mobile keyboard inputdecoding with finite-state transducers. arXivpreprint arXiv:1704.03987.

Brian Roark, Richard Sproat, Cyril Allauzen,Michael Riley, Jeffrey Sorensen, and Terry Tai.2012. The OpenGrm open-source finite-stategrammar software libraries. In Proceedings ofthe ACL 2012 System Demonstrations, pages61–66.

Hassan Sajjad, Helmut Schmid, Alexander Fraser,and Hinrich Schütze. 2017. Statistical mod-els for unsupervised, semi-supervised, and su-pervised transliteration mining. ComputationalLinguistics, 43(2):349–375.

Tarek Sherif and Grzegorz Kondrak. 2007.Substring-based transliteration. In Proceedingsof the 45th Annual Meeting of the Associationof Computational Linguistics, pages 944–951.

Ananda Theertha Suresh, Brian Roark, MichaelRiley, and Vlad Schogol. 2019. Distillingweighted finite automata from arbitrary proba-bilistic models. In Proceedings of the 14th In-ternational Conference on Finite State Meth-ods and Natural Language Processing (FSMNLP2019).

Paola Virga and Sanjeev Khudanpur. 2003.Transliteration of proper names in cross-lingualinformation retrieval. In Proceedings of theACL 2003 workshop on Multilingual and mixed-language named entity recognition-Volume 15,pages 57–64. Association for Computational Lin-guistics.