The challenge of collecting and evaluating LRs for commercial use

Transcription

1 Language Technologies Observatory The challenge of collecting and evaluating LRs for commercial use Bente Maegaard, CLARIN ERIC (and University of Copenhagen)

2 Overview of the challenges 2 Which LRs are needed? Where to find them Quality measures Improving LRs Awareness

3 LRs needed 3 (Many more) Language Resources for commercial applications are needed! Large scale LRs for e.g. SMT For many languages and domains Which languages? All European, many important international languages Which domains? Difficult to point to particular domains that are suffering as so little is available LRs are scarce and information about them is scattered LTO has created the LR Catalogue

4 Related work this is not a challenge! 4 Related work META-SHARE, ELRA, CLARIN VLO, OPUS, TAUS, PANACEA, etc The LTO focus in on LRs for commercial use

5 Evaluating LRs, usability 5 No shared method for evaluating the quality of an LR ELRA has validation procedure (Operational) Usability easier to define

6 Elements of quality/usability 6 Ease of access Ease of download and implementation Domain relevance Languages available Source and target language? is it really a translation? Cf. Multilingual websites Is it recent and/or updated? Availability/cost Sometimes size can be a quality

7 Selection criteria 7 Selection criteria for the LTO catalogue Availability Languages covered Validation Longevity (is it updated) Ease of download

8 Metadata 8 Metadata are indispensable Nothing can be found without metadata Time-consuming, find good balance

9 LTO metadata 9 Title Creator Contact person Description Language (mono-, bi- ) Resource type Availability Tags Modality - text URL Domain Format Format descr. Size Production date Comment

10 Challenge: insufficient metadata - Valorisation process 10 Many LRs detected could not go directly into the LTO Catalogue Insufficient metadata Metadata have been added/adapted/improved

11 Challenge: creation of new resources. Tools for new resources 11 If LRs don t exist, create them! Tools for corpus creation Tools for terminology extraction Have been investigated, guides written IPR a problem

12 Number of parallel corpora per language 12

13 Results of data collection investigations - challenges 13 Many LRs exist for academic purposes Not available for commercial use Companies don t share Many bilingual corpora are also multilingual Tools exist which help the creation of new LRs BUT: IPR rules make it very time-consuming to obtain the rights to use what you find on the web. Fair use would really help! Only small developments compared to META- White Papers

14 Results of implementation 14 The LTO catalogue exists The LTO catalogue only contains references to useful LRs We are happy to learn about more LRs that could be part of the LTO catalogue

15 Gap description summary of challenges 15 Coverage gap Languages, domains Awareness gap Demand side often unaware of what actually exists, which makes the LTO Catalogue particularly useful Supply side does not know which are the most needed LRs Quality gap Valorisation is needed Quantity gap Large quantities are needed for modern LT

16 Thank you! LT-Observatory Observatory for LR and MT in Europe Funded by the