Font Forum. TUGboat, Volume 21 (2000), No Thai fonts Werner Lemberg

Transcription

1 TUGboat, Volume 21 (2000), No Font Forum Thai fonts Werner Lemberg Abstract This article describes how the Thai script works and how to implement the necessary ligatures for TEX using afm2tfm. 1 Some Historic Information The Thai script has been derived, similar to almost all other southern asian scripts not directly influenced by China, from an ancient version of Indic Sanskrit, Brahmi. Over the years, the original letters have been adapted to the peculiarities of the Thai language, one example would be superscripted digits used for tone marks. Regarding computers, Thailand followed other principles than those of India in the standardization of the script despite many similarities; all base letters are consonants with an inherent vowel (usually an a), and for the special case of Thai, an inherent tone. Vowel and tone can be modified by attaching other letters to the base consonant before and after the base consonant, but also above and below. The Indic standard (ISCII) uses the logical order to store text, which means that the vowel always comes after the consonant even if it appears before the consonant graphically (such vowels are called independent vowels). Contrary to India s logical order, Thailand defines its industrial standard TIS- 620 in that independent vowels must be stored in visual order. Unicode [8] follows TIS-620 in the processing of the Thai script. In the following, linguistic aspects will be completely ignored, referring to graphical features only. 2 The Structure of Thai Letter Clusters Since letter clusters are stored in visual order the graphical display of Thai is simplified to base letters, possibly with diacritical signs above and below no need to reorder vowels. There are five possibilities how diacritical signs can be positioned in Thai. 1. base consonant + vowel above: + = 2. base consonant + tone mark: + = 3. base consonant + vowel above + tone mark:

2 114 TUGboat, Volume 21 (2000), No = 4. base consonant + vowel below: + = 5. base consonant + vowel below + tone mark: + + = TIS-620 mandates that tone marks come last, but users sometimes ignore this. It is the duty of input methods for Thai to normalize incorrect input. Below, the standardized form is always expected. It can already be seen in the above examples that diacritical signs change its positions horizontally and vertically dependent on the shape of the base glyph resp. whether another diacritical sign is used. There are more peculiarities of the Thai script. 1. The vowel sara am will be split into the characters nikhahit and sara aa if it is appended to a consonant. The character interacts with the preceding character. + = + + = + + = = If necessary, nikhahit and the tone mark must exchange its positions. 2. The two consonants yo ying and tho than drop its lower part if combined with a lower vowel. + = + = 3. If sara aa follows the independent vowel ru or lu (those two letters are used for Sanskrit), it will be replaced by the sign lakkhangyao. + = + = 3 Glyph Classes To describe the necessary ligatures it is convenient to categorize Thai letters into various graphical glyph classes, ignoring all linguistical aspects. In the author s opinion, even incorrect or unrealistic combinations should be displayed in an optically pleasing way if possible. base normal Normal base glyphs without special features. base desc Base glyphs with descender. base desclike As described above, glyphs of this class consist of two parts, omitting the lower one if combined with a lower vowel. base asc Base glyphs with an ascender on the right side. base indic The two independent vowels ru and lu. base sign The sign lakkhangyao. base sara am The vowel sara am. base sara aa The vowel sara aa. lower Diacritical vowels below. upper vowel Diacritical vowels above. upper sign The sign nikhahit. top Tone marks. Now the glyph variant forms. base descless The glyphs of class base desclike without the lower part. lower low The glyphs of class lower shifted downwards. upper vowelleft The glyphs of class upper vowel shifted to the left. upper signleft The glyphs of class upper sign shifted to the left. top left The glyphs of class top shifted to the left. top low The glyphs of class top shifted downwards. top low-left The glyphs of class top shifted to the left and downwards. 4 Context Patterns Using the glyph classes defined in the last section it is easy to describe the context patterns for base glyphs with diacritical signs. Surprisingly, these patterns are quite systematic. Patterns in table 1 which are marked with an asterisk do nothing and are listed for completeness only. As mentioned above, these patterns cover more combinations as existing in the Thai script. Table 2 covers the ligatures of the character sara am. Finally, table 3 describes the letters specific to Sanskrit. 5 Intermezzo 1 A small introduction into the exotic variants of TEX s ligature mechanism which probably many users haven t seen before. Additionally, the documentation in the METAFONTbook is very sparse. In the following examples METAFONT s notation is used.

3 TUGboat, Volume 21 (2000), No base lower base lower * base upper base upper * base top base top low base lower top base lower top low base upper top base upper top * base desc lower base desc lower low base desc upper base desc upper * base desc top base desc top low base desc lower top base desc lower low top low base desc upper top base desc upper top * base desclike lower base descless base desclike upper base desclike upper * base desclike top base desclike top low base desclike lower top base descless lower top low base desclike upper top base desclike upper top * base asc lower base asc lower * base asc upper base asc upper left base asc top base asc top low-left base asc lower top base asc lower top low-left base asc upper top base asc upper left top left Table 1: Context patterns for diacritical signs. Here, base refers to the union of the subclasses normal, indic, sign, sara am, and sara aa of base; upper is the union of the subclasses vowel and sign of upper. base base sara am base upper sign base sara aa base asc base sara am base asc upper signleft base sara aa base top base sara am base upper sign top base sara aa base asc top base sara am base asc upper signleft top left base sara aa Table 2: Context patterns for sara am. Here, base denotes the union of subclasses normal, desc, and desclike of base. base indic base sara aa base indic base sign Table 3: Context patterns for ru and lu.

4 116 TUGboat, Volume 21 (2000), No. 2 The usual ligature action of two glyphs a and b is the replacement of both glyphs with another glyph c. a b =: c Another possibility is to retain the left or the right original glyph (before resp. after the ligature) or both. a b =: c a b =: c a b =: c The first rule creates ac, the second cb, and the last acb. In all three cases, the current point after appying the ligature rule is still at the first glyph of the replaced glyphs, and TEX simply restarts there to check ligatures (and kernings). A classical example is f f i ff i ffi To advance the current point to the right, append either > or >> (the latter is only possible if you retain both input glyphs). Here are the remaining four ligature rules. a b =:> c a b =: > c a b =: > c a b =: >> c For Thai ligatures, the most often needed rule is =: (i.e., retain the left glyph and stay at the same position before applying the next ligature rule). Note that using =:> instead is not a good idea since this would prohibit kerning between the left glyph and the ligature. 6 Ligature Rules As just explained, TEX can only handle context patterns of length 2, whereas Thai needs patterns of length 3. It was an interesting challenge to find out whether the problem can be solved with TEX s somewhat restricted ligature rules the gentle reader is invited to find a solution by herself! There won t be any difficulties in understanding ligatures afterwards. The tables 4, 5, and 6 use the same conventions as tables 1, 2, and 3, respectively. The current point isn t increased in any of the rules. Most of the ligature rules can be derived easily by handling the patterns sequentially (quite similar to logic puzzles found in various magazines), but at the end there remain two patterns which apparently contradict. base lower top base lower top low base asc lower top base asc lower top low-left After applying ligature rules for the first two glyph classes it is necessary to handle the context lower top, but depending on the previous glyph class top must be replaced with top left and top low-left, respectively. With a context pattern length of 3 this would be easy to solve, but TEX doesn t have this feature. What to do? The context base lower top must be distinguished from base asc lower top, i.e., two different lower classes are needed depending on the previous character since TEX is not able to forward information from one ligature cycle to the next. The idea is now to create an alias class, a class which behaves identically to the original one. The glyphs in this alias class are the same, but different glyph indices resp. glyph names are assigned to it. A closer look to table 4 shows that lower left isn t a typo but the alias class of lower. 7 Intermezzo 2 afm2tfm [6] uses two encoding vectors to create metrics files for TEX. The first maps from the Type 1 font to the raw font (converting glyph names to glyph indices): /ps_to_raw [... /bar... /bar... ] def The second encoding vector used for the virtual font (which will contain the ligatures) maps from glyph indices back to glyph names, so we can finally assign different glyph names to identical glyphs: /raw_to_vf [... /bar1... /bar... ] def bar1 and bar now both access the same glyph. Unfortunately, afm2tfm can only use glyph names for the virtual font which are already in the base font, so some manual work is needed to overcome this restriction. It is planned to reimplement Thai ligatures using the fontinst package [2] which doesn t have this problem. 8 The Implementation After solving the problem theoretically now the practical implementation. Only some interesting details are shown. Since it is not possible inafm2tfm to collect glyphs in classes the number of all ligature rules is quite big (464 in total). The file thai.enc, part of the CJK package [3], contains the complete solution. It also contains detailed installation instructions how to use afm2tfm. All glyph names follow the Adobe Glyph List (AGL) [1]. There are no predefined Adobe glyph names for Thai, so the prefix uni with attached Unicode value will be used for all glyphs which are encoded in Unicode. Example: The letter ko kai gets the name uni0e01. Glyph variants are identified by a postfix. Example: The left-shifted glyph variant of the vowel sara uee is called uni0e37.left. The following listing describes some of the ligatures, explaining its function.

5 TUGboat, Volume 21 (2000), No base top base top low base desc lower base desc lower low base desc top base desc top low base desclike lower base descless lower base desclike top base desclike top low base asc lower base asc lower left base asc upper base asc upper left base asc top base asc top low-left lower top lower top low lower low top lower low top low upper left top upper left top left lower left top lower left top low-left Table 4: Ligature rules for diacritical marks. base base sara am base upper sign base sara am base asc base sara am base asc upper signleft base sara am upper sign base sara am upper sign base sara aa upper signleft base sara am upper signleft base sara aa top low base sara am top low top base sara am top low top upper sign top top base sara am top base sara aa top low-left base sara am top low-left top left base sara am top low-left top left upper signleft top left top left base sara am top left base sara aa Table 5: Ligature rules for sara am. base indic base sara aa base indic base sign Table 6: Ligature rule for ru and lu.

6 118 TUGboat, Volume 21 (2000), No. 2 Rule: base top base top low This rule needs 225 ligatures (45 base glyphs 5 tone marks); this is almost 50% of all rules. % LIGKERN uni0e01 uni0e48 =: uni0e48.low ; % LIGKERN uni0e02 uni0e48 =: uni0e48.low ;... % LIGKERN uni0e01 uni0e49 =: uni0e49.low ; % LIGKERN uni0e02 uni0e49 =: uni0e49.low ;... % LIGKERN uni0e41 uni0e4c =: uni0e4c.low ; % LIGKERN uni0e46 uni0e4c =: uni0e4c.low ; Rule: base desclike lower base descless lower Here the left glyph will be replaced. % LIGKERN uni0e0d uni0e38 =: uni0e0d.descless ; % LIGKERN uni0e10 uni0e38 =: uni0e10.descless ;... Rule: base base sara am base upper sign base sara am The context pattern a b a c d has to be transformed to the following for TEX (as shown in table 5): a b a c b c b c d The first ligature is of interest: % LIGKERN uni0e01 uni0e33 =: uni0e4d ; % LIGKERN uni0e02 uni0e33 =: uni0e4d ;... 9 The Font Encoding The real encoding of the virtual font is irrelevant for ligature rules because glyph names have been used exclusively. Nevertheless, it has practical advantages to use TIS-620 as a font encoding also, filling unused positions with glyph variants (this is similar to the Unicode++ font encoding of Ω [5]). Especially users of Plain TEX will benefit if input and output encoding are identical. Care must be taken in L A TEX documents to avoid the use of \uppercase and \lowercase commands so that Thai letters aren t modified due to incorrect \lccode and \uccode values. Table 7 shows the used encoding of the Thai glyphs. 10 Problems From a typographical point of view, all problems are solved. To believe that it is now possible to simply enter Thai for getting correct output is an error, though. Two serious obstacles must be mastered first: finding word breaks and insertion of intercharacter glue. Words in Thai are not separated with spaces, and they aren t hyphenated either. 1 A space, usually much bigger than a space for the Latin script, has a similar function to an em-dash or to a semicolon; its primary use is to structure a sentence. Correct recognition of words in Thai is a very complex problem which can be solved without errors by sentence analysis only. The CJK package uses a relatively simple algorithm developed by Vuthichai Ampornaramveth which basically searches for the longest words in a dictionary (this is implemented as a Lisp package for Emacs it is assumed that the next major version of Emacs will contain this module directly). Due to missing context analysis it can t guarantee error-free results in all cases. Another complication is that Thai tends to very long words, making the search for good break points in justified paragraphs difficult. A legitimate solution is to moderately apply some intercharacter glue (cf. figure 1). An even better solution would be the use of Multiple Master fonts or similar fonts to enable small variations of the typeface. Newer versions of pdftex have experimental support for stretching and squeezing of fonts [7]. 11 Thai Support In The CJK Package The results presented in this paper will be part of the next version of the CJK package (daily snapshots of the development archive are available from ftp://ftp.ffii.org/pub/cjk/devel/ cjk-current.tar.gz). Included are (in addition to thai.enc and other auxiliary files) encoding, metrics, and font definition files for the freely available Thai font families DBThai and Norasi [4]. The latter uses glyphs created by Yannis Haralamous and Tereza Tranaka and are still under development; all Thai examples in this article have been typeset with it. Intercharacter glue and word break points will be inserted automatically by the Emacs interface cjk-enc. This package could be considered as a generalized inputenc package which is able to handle multiple character sets in Emacs simultaneously, and which does correct translation to L A TEX transparently to the user. 12 Acknowledgements Most of the development work on Thai ligatures has been done during a two-week research visit in Tsukuba (Japan) on invitation of the Electrotechnical Laboratory (ETL). 1 For narrow-column printing, hyphenation is used, but it isn t considered as good typography.

7 TUGboat, Volume 21 (2000), No Table 7: The Thai encoding of the CJK package. Range (0xA1 0xFB) without (0xDB 0xDE) is TIS-620, the rest are glyph variants are consonants , , and 347 are vowels. 337 is the Thai currency symbol, Baht are tone marks are the digits 0 to 9. The alias class to lower (at position ), lower left, is at position

8 120 TUGboat, Volume 21 (2000), No. 2 FAQ. FAQ. FAQ. FAQ. FAQ. FAQ. Figure 1: The same text, with and without intercharacter glue. To suppress warnings and error messages for the above variant, \tolerance had to be set to 8000 and \badness to \baselinestretch has the value 1.2. References [1] The Adobe Glyph List. http: //partners.adobe.com/asn/developer/ typeforum/unicodegn.html. [2] Allan Jeffrey et al. The fontinst package. Available from CTAN and its mirrors, e.g. ftp://ftp.dante.de/pub/tex/fonts/ utilities/fontinst. [3] Werner Lemberg. The CJK package. [4] Surapant Meknavin and Theppitak Karoonboonyanan. The thailatex package. ftp: //opensource.thai.net/pub/linux-tle/ updates/sources/thailatex tar.gz. The implementation for Thai in this package is incompatible to the one described in this article. For this reason, the Babel module of the CJK package is called thaicjk and not thai. [5] John Plaice and Yannis Haralambous. The Ω system. omega. Almost all modern TEX distributions contain support for Ω. [6] Tomas Rokicki. The afm2tfm program. Part of the dvips package which is available from virtually all TEX distributions. [7] Han The Thanh. pdftex. ftp: //ftp.cstug.cz/pub/tex/local/cstug/ thanh/pdftex/latest. pdftex is, similar to Ω, already part of most modern TEX distributions. The given URL specifies the primary address of pdftex since it still in development, sometimes with incompatible changes. [8] The Unicode Standard. http: // Werner Lemberg Kl. Beurhausstr Dortmund wl@gnu.org