Automatic Collecting of Text Data for Cantonese Language Modeling

Transcription

1 S4-4 Automatic Collectig of Text Data for Catoese Laguage Modelig Jiag CAO, Xiaoju WU, Yu Tig Yeug 2, Ta LEE 2, Thomas Fag Zheg *. Ceter for Speech ad Laguage Techologies, Divisio of Techical Iovatio ad Developmet, Tsighua Natioal Laboratory for Iformatio Sciece ad Techology, Tsighua Uiversity 2. Departmet of Electroic Egieerig, The Chiese Uiversity of Hog Kog *. Cotact: Room 4-46 Iformatio Sciece ad Techology Buildig, Tsighua Uiversity, Beijig, 00084, Chia; , Abstract It is hard to collect corpora used to trai good laguage models for may miority laguages. Catoese, oe of the most popular Chiese dialects, is such a kid of laguage, lackig of laguage materials for laguage model traiig. This is a very big obstructio for the processig of Catoese laguage. Ulike may other laguages, there are great differeces betwee writte ad colloquial Catoese. What s more, people i Hog Kog are usig mixed Catoese ad Eglish while they talk, which is also a special characteristic of this laguage. Beyod these, the materials collected from differet sources have differet proportio of colloquial Catoese seteces, which meas that differet sources should ot be equally treated. We developed a filter model, which was built up at lexical ad grammar levels. We traied this model usig a developmet set ad achieved a precisio rate of 99.89% ad a recall rate of 88.2% i the test set. With this model, we foud a method to defie the credibility for the differet material sources. It was a iterative process ad the proportio of the seteces chose from differet sources for model traiig is decided by its result. Itroductio Laguage modelig has bee successfully used for speech recogitio, part-of-speech taggig, sytactic parsig, ad iformatio retrieval recetly ad so o (Sog 999). I recet years, Grammar-Based Laguage Models (GLMs) ad Statistical Laguage Models (SLMs) are most widely used types of laguage models (Hockey 2005). For large-vocabulary cotiuous speech recogitio (LVCSR), SLM, typically N-gram, plays a importat role (Duchatcau 2005). I order to estimate model parameters, a large database of text of this laguage is ecessary. Lackig of laguage materials is geerally a serious problem i statistical laguage model traiig ad may affect the performace of a laguage processig system. For popular laguages like Eglish Spaish 30 ad Chiese, there are may existig laguage databases available for laguage model traiig (e.g. ATIS (Ward 990), CallHome (Ries 2000) ad SogouT ). It is also relatively easy to collect more text data for these lagauages. For those spoke laguages or dialects, the data availability is ot trivial. I Catoese, for example, there is very little speech data ad eve less text data available, simply because it is ot a official writte laguage (Fug 999). The research progress of Catoese laguage processig has bee limited by the lack of text databases. The task of collectig more laguage materials is a straightforward but feasible breakthrough poit i improvig the performace of existig systems. I this study, we first developed a filter model, which was built at lexical ad grammar levels, to decide whether a retrieved setece is i Catoese or ot. This model was traied usig a developmet set, which was made up of 500 colloquial Catoese seteces ad 500 stadard Chiese seteces. These seteces were all collected from web pages. The test set cotaied 2,25 colloquial Catoese seteces ad 7,72 writte seteces. The proposed filter model could achieve a precisio rate of 99.89% ad a recall rate of 88.2%, which was very ecouragig. It must be poited out that the precisio rate is more importat tha the recall rate i our iteded applicatio of data collectio because udesirable seteces would deteriorate the performace of laguage models. We desiged a method to assess the credibility of a source of text materials, e.g., a website, which was a iterative process. The credibility level of each source was updated i every iteratio accordig to the proportio of the colloquial Catoese seteces cotaied i the documets from this source. With differet iteratios, documets from differet sources were treated differetly. This method helped idetify more useful sources which attracted more attetio, ad thus improved the efficiecy of data collectio.

2 2 Catoese 2. Itroductio to Catoese Catoese or Yue ( ) is oe of the most popular Chiese dialects, ad is a member of the Sio-Tibeta laguage families (Li 2006). It is also kow as / (Madari: Gu gd g hu, Catoese: gwog dug waa). The populatio of Catoese speakers is over 55 millio. The areas with the highest cocetratio of speakers are i Guagdog Provice ad some parts of Guagxi Provice, Hog Kog ad Macau (See The Itroductio i Lau 999). It is the defacto official spoke laguage i Hog Kog. Most Chiese people i Southeast Asia ad North America also speak Catoese. The ame of this laguage, Catoese, has bee stated as beig derived from Cato, which is a Eglish ame for Guagdog Provice ad also a Eglish ame for Guagzhou. (Graham 2006). Catoese speech is geerally uitelligible to people who live i other provices. Catoese is a moosyllabic ad toal laguage. Each Chiese character is proouced as a syllable soud i Catoese (Cha 2005). There are about 5,500 commoly used characters i Catoese, ad this is a little less tha stadard Chiese spoke i Mailad Chia ad Taiwa. Some of the characters are uique to Catoese ad are ever beig used i stadard Chiese. But at the same time, there are also some characters used i stadard Chiese but ot i Catoese. I this paper, we maily focus o Catoese beig spoke i Hog Kog. 2.2 Writte Catoese ad Writte Chiese Stadard writte Chiese is, i essece, i agreemet with Madari i Taiwa. Whe stadard writte Chiese is read out with Catoese prouciatios, the speech would soud strage ad uatural. It is very differet from spoke Catoese. If we trascribe spoke Catoese ito Chiese characters, the resulted text is referred to as writte Catoese. There are may differeces betwee Catoese ad stadard Chiese. A stadard Chiese speaker will have difficulties to read writte Catoese. Catoese has some uique characters like,, ad uique words,,,,. There also exist some special grammatical forms ad phrase expressios i Catoese. Catoese is a laguage where code-switchig is quite commo (Cha 2005). I Hog Kog, people are used to embed Eglish words i a setece. Although this is becomig popular i Mailad Chia i recet years, code-switchig i Hog Kog Catoese is cosidered as a itegrated part ad a special feature of cotemporary Catoese. For laguage model traiig i Catoese speech recogitio, we eed text materials that match the cotet of real-life spoke Catoese. But i most Catoese-speakig regios, icludig Hog Kog SAR, Catoese is ot a official writte laguage (Zhag 999). I Hog Kog, stadard Chiese i traditioal characters ( ) is used i official commuicatios. Most ewspapers ad formal publicatios are prited i stadard Chiese. Over the iteret, which hosts a large amout of text materials, writte Catoese remais a miority. 3 The Catoese Filter Model As discussed earlier, there are quite a few differeces betwee writte Catoese ad stadard Chiese. Our goal is to develop a effective method to distiguish writte Catoese from stadard Chiese, so as to facilitate massive collectio of Catoese text materials from the iteret. 3. Costructio of the Filter Model The basic fuctio of the filter model is to determie whether a give setece is i writte Catoese or ot. This fuctio ca be implemeted i two steps. The first step is to idicate whether the setece is Chiese i a broad sese, i.e., either writte Catoese, stadard Chiese or other kids of Chiese text. This is doe by checkig the ecodig method of the text. The secod step is to idicate whether the sequece of Chiese characters is i Catoese or ot. Catoese text cotais a set of Catoese-specific characters. It may also cotai special grammar phrases ad code-switchig cotets. For Catoese laguage processig, methods of word segmetatio ad setece parsig are ot as well developed as for stadard Chiese. Relevat research is rare. I our filter model, a obvious feature is used to detect Catoese-specific cotet: Seteces that cotai at least oe of these characters is cosidered to be i Catoese. This simple method was foud to perform very well i our study. I (Cha 2005), some represetative Catoese seteces are listed. We chose 500 seteces from the list ad used them as the traiig data i our experimets. For stadard Chiese, there is a Commo Chiese Character 3

3 List published by the govermet i The we defied that the uique characters i Catoese from Chiese are the characters which appear i the traiig set but ot i the Commo Chiese Character List, ad of course, i the traditioal Chiese form. These characters are called positive characters. We also tried to defied egative characters as those which oly appear i Chiese but ot i Catoese. However, a simple experimet showed that almost all Chiese characters ca be foud i Catoese. So istead we chaged to fid out some words or phrases (i Chiese ad Catoese, a word or a phrase is built by oe or more characters). Without a big eough Catoese laguage database, this work was doe artificially. Ad the words ad phrases oly appearig i Chiese are called egative keywords. We fially had 32 egative keywords, such as ad. The whole filter model works like this way. If a setece does ot cotai ay positive characters, the filter model will judge it as a o-catoese setece. If it cotais ay oe of the egative keywords, it will ot be judged as Catoese either. Oly those cotaiig positive characters ad o egative keywords will be judged as Catoese seteces. 3.2 Experimets of the Filter Model We built a test set with the seteces collected from the web sites i Hog Kog. After a artificial cofirmatio, this test set cotaied 2,25 writte Catoese seteces. As a cotrast, we added 7,72 writte Chiese seteces (i the traditioal form) ito the test set. These writte Chiese seteces were chose from the SogouT database, which was built by the materials collected from the web sites i Mailad Chia, ad some ews web sites ad official web sites i Hog Kog. The test set was built up by D =2,25 Catoese seteces ad D 2 =7,72 Chiese seteces. After beig filtered, it was divided ito two subsets, with oe cotaiig X Catoese seteces ad the other X 2 Chiese seteces. Results are calculated this way. If there are C seteces i both D ad X, which meas these C seteces are correctly judged by the filter model, C the precisio rate should be P = ad the recall X C rate should be R =. D I our experimets, we foud that with differet traiig sets differet umbers of positive characters were foud ad thus the precisio ad recall rate were differet. All these experimets were doe i the same test set as we described above ad without the help of the egative keywords, as show i Table. Traiig Set Seteces No. of Positive Characters Precisio Recall % 49.7% % 6.79% % 78.45% % 82.50% Table : Performace Compariso for Differet Traiig Sets These experimets showed that whe the size of the traiig set icreased, the umber of positive characters icreased too. But the relatio was ot strictly liear sice some positive characters might occur i differet seteces. With the icrease of positive characters, the recall rate icreased too, because more positive characters covered more seteces. The precisio rate remaied i high level, as was also reasoable. Theoretically, the precisio rate should be.0 because of the way how the positive characters were foud out. Whe we checked the seteces which were actually Chiese but regarded as Catoese, we foud some differet error cases. Some of them cotaied a Catoese setece as a direct quotatio. Others cotaied positive characters i a wrog way. With the help of egative keywords, we removed some error cases. To verify this, we did a experimet with all the 64 positive characters, as show i Table 2. With Negative Keywords Precisio Recall No 98.20% 82.50% Yes 99.89% 82.50% Table 2: Performace Compariso: with or without Negative Keywords The results showed that after these two steps of filterig, our filter model achieved a great precisio rate with a high recall rate. I our model, the precisio rate is much more importat tha the recall rate. Wrog seteces may cause big problems i the traiig process of the laguage model ad the umber of the seteces collected from differet ways ca be quite large. 4 Automatic Collectig Method Oe of the best ways to collect laguage materials

4 for traiig a laguage model is to collect the materials from the iteret. But i Catoese areas, most web sites, the ews sites, the govermet sites ad eve the e-commerce websites, are i writte Chiese. Oly some iformal etertaimet sites or local forums are posted i Catoese. It ca be see that there will be a greater possibility to fid Catoese i those websites which have more iteractio with the users. It is impossible to artificially fid out all the websites which have higher possibility to cotai Catoese. So it will be useful to use a collectig method which ca automatically fid out those websites ad extract Catoese seteces. 4. Descriptio of the Collectig Method For a collectig method, there should be three steps to extract Catoese from the iteret. The first step is to choose the websites where it ca dowload web pages. After that it will decode the html documets to get the text cotets. At last it will filter out Catoese seteces from the whole text ad the filter model proposed ca be used. As metioed above, differet websites have differet laguage characteristics. Some are mostly i Chiese while others are mostly i Catoese. Assumig that web pages from the same site usually have the same laguage style, we developed a method to decide which sites are suitable for collectig. This method is a iterative process. Assume that there are some differet websites S, S 2, S 3,, S, ad each of them has a differet laguage style. Some of them may have more Catoese seteces while others less. I the begiig we treated them all the same ad dowloaded web pages from each site. For the website S i, we worked out the umber of Catoese seteces c i ad the umber of all seteces w i with the Catoese laguage filter ci model to calculated the Catoese rate as ri =. wi The higher the rate is, the better the website is suitable for collectig. So i the ext iteratio, we should get more web pages from this site. Let t i,j be the proportio of the web pages we have collected from the website S i i the j th iteratio, the proportio for S i i (j+) th iteratio will be: ri t' i, j+ = ti, j( + a( ti, j)), i=,2,3..., a> 0 ravg t ' i, j+ ti, j+ = t ' k = k, j+ where, r i meas the Catoese rate of the website S i i j th iteratio; r avg meas the average of these 33 differet websites, so it ca be computed as r avg = r i i=. The parameter a decides how fast the proportio chages. If the iteratio goes o ad o, the t i,j will coverge to the value which reflects the rate of the Catoese part i the whole website S i. 4.2 Experimet of the Collectig Method I our experimet, we chose 6 websites as sources. The websites are show i Table 3. Website Website ID appledaily.atext.com hk.myblog.yahoo.com 6 Table 3: Websites Used i the Experimet Websites ad 2 are both portal sites of Mailad Chia. Website 3 is a typical ews website i Hog Kog ad Website 4 is oe for Hog Kog s weather service, a typical official website. Website 5 is a very popular blog site i Hog Kog ad Website 6 is oe of the most popular forums i Hog Kog. It is reasoable to assume that Websites ad 2 cotai very few Catoese seteces ad Websites 3 ad 4 cotai oly a few Catoese seteces. After all, Websites 5 ad 6 are the sites where we wat to collect the laguage materials. The iterative process ra 00 iteratios. The results are listed i Table 4. s to 6 refer to the rates of the web pages which we should collect from these differet 6 websites i ext iteratio. From this experimet, we ca poit out that the rate will chage to reflect the actual importace of these 6 websites. The bigger the parameter a is, the more quickly these rates will chage. 5 Coclusio With the Catoese laguage model ad the automatic collectig method, we have developed a system to perform automatic collectig of Catoese laguage materials. I our experimets, this system performed well. But actually, there are still may difficulties to overcome. A efficiet spider used to dowload the web pages from the iteret, ad a good eough parser used to chage the html documets ito text, are both challeges before us. At the same time, our filter model still has rooms for improvemet. More features i the grammar level ca be used ad the recall rate ca be much higher. We

5 thik there is also room for improvemet of the iteractive process of our collectig method. 6 Ackowledgemet This work was supported by a project grated from the Shu Hig Istitute of Advaced Egieerig. Iteratio Times a= a= a= Refereces Y. C. J. Cha. (Cha 2005). Automatic speech recogitio of Catoese-Eglish code-mixig utteraces (PhD Thesis). Hog Kog, EE CUHK, J. Duchatcau, D. H. V. Uytsel, H. V. Hamme, ad P. Wambacq. (Duchatcau 2005). Statistical laguage models for large vocabulary spotaeous speech recogitio i Dutch. I Proceedigs of Iterspeech, 2005, pp P. Fug ad L. Y. Yee. (Fug 999). Uderstadig Chiese spotaeous speech: are Madari ad Catoese very differet. I Proceedigs of ISSPIS 999, Guagzhou, Chia. T. Graham. (Graham 2006). Socioliguistics ad cotact-iduced laguage chage: Haia Cham, Aog, ad Pha Rag Cham. Teth Iteratioal Coferece o Austroesia Liguistics, pp B. A. Hockey ad M. Rayer. (Hockey 2005). Compariso of grammar-based ad statistical laguage models traied o the same data. I Proceedigs of the AAAI Workshop o Spoke Laguage Uderstadig. K Y. Lau. (Lau 999). Catoese phrase book. Loely Plaet Publishig. (ISBN ). Itroductio. Table 4 Results of the Iterative Process 34 P Li. (Li 2006). The hadbook of East Asia psycholiguistics. Cambridge Uiversity Press (ISBN ), pp.3-4. K Ries. (Ries 2000). Shallow discourse gere aotatio i CallHome Spaish. I Proceedigs of the Iteratioal Coferece o Laguage Resources ad Evaluatio (LREC-2000), Athes, Greece. F. Sog ad W. B. Croft. (Sog 999). A geeral laguage model for iformatio retrieval. Research ad Developmet i Iformatio Retrieval, pp W Ward. (Ward 990). The CMU air travel iformatio service: uderstadig spotaeous speech. I Proceedigs of the DARPA Speech ad Natural Laguage Workshop, pp (Zhag 999)....