CEN WORKSHOP AGREEMENT

Size: px

Start display at page:

Download "CEN WORKSHOP AGREEMENT"

Myra Watts
5 years ago
Views:

1 CEN WORKSHOP AGREEMENT Fourth draft for CWA XXXX English version Information Technology - Multilingual European Subsets in ISO/IEC Technologies de l information - Informationstechnologie - Jeux partiels européens multilingues Mehrsprachige europäische Untermengen dans l ISO/CEI in ISO/IEC This CEN Workshop Agreement has been drafted and approved by a Workshop of representatives of interested parties, the constitution of which is indicated in the foreword of this Workshop Agreement. The formal process followed by the Workshop in the development of this Workshop Agreement has been endorsed by the National Members of CEN but neither the National Members of CEN nor the CEN Central Secretariat can be held accountable for the technical content of this CEN Workshop Agreement or possible conflicts with standards or legislation. This CEN Workshop Agreement can in no way be held as being an official standard developed by CEN and its Members. This CEN Workshop Agreement is publicly available as a reference document from the CEN Members National Standard Bodies. CEN Members are the National Standards Bodies of Austria, Belgium, Czech Republic, Denmark, Finland, France, Germany, Greece, Iceland, Ireland, Italy, Luxembourg, Netherlands, Norway, Portugal, Spain, Sweden, Switzerland and United Kingdom. CEN EUROPEAN COMMITTEE FOR STANDARDIZATION COMITÉ EUROPÉEN DE NORMALISATION EUROPÄISCHES KOMITEE FÜR NORMUNG Central Secretariat: rue de Stassart 36, B-1050 Brussels CEN 1998 All rights of exploitation in any form and by any means reserved worldwide for CEN national Members Ref.No. CWA XXXX:1998 E

2 Page 2 Fourth Draft for CWA XXXX:1998 Multilingual European Subsets of ISO/IEC This page has been intentionally left blank.

3 Multilingual European Subsets of ISO/IEC Contents Page 3 Fourth Draft for CWA XXXX:1998 Foreword 4 0. Introduction 5 1. Scope 6 2. Normative references 8 3. Definitions and abbreviations 8 4. The Repertoire of EN 1923 LL The Multilingual European Subset No. 1 (MES-1) The Multilingual European Subset No. 2A (MES-2A) The Multilingual European Subset No. 2B (MES-2B) 11 Annex A (Informative) Bibliography 14

4 Page 4 Fourth Draft for CWA XXXX:1998 Multilingual European Subsets of ISO/IEC Foreword This CEN Workshop Agreement has been prepared by the CEN/ISSS WS/MES, of which the secretariat was held by STRÍ. It was approved in the month of November The members of the CWA were: (Wolf Arfvidson) (Statskontoret, wolf.arfvidson@statskontoret.se) Daniel Brickley (ILRT, daniel.brickley@bristol.ac.uk) Elżbieta Broma-Wrzesień (Telekomunikacja Polska S.A., ebroma@cbr.tpsa.pl) (Bernard Chauvois) (French Ministry of Education, chauvois_bernard@wanadoo.fr) John Clews (SESAME Computer Projects, european@sesame.demon.co.uk) Jean-Louis Cortot (AFNOR, jeanlouis.cortot@ .afnor.fr) (Viktoras Dagys) (Centre of Informatics and Prognosys, viktoras.dagys@ipc.lt) Paul Dettmer (DIN, paul.dettmer@tonline.de) Michael Everson (Project editor) (Everson Gunn Teoranta, everson@indigo.ie) Gintautas Grigas (Institute of Mathematics and Informatics, grigas@ktl.mii.lt) Marion Gunn (Everson Gunn Teoranta, mgunn@egt.ie) John Hudson (Tiro Typeworks, tiro@tiro.com) Indrek Hein (Institute of the Estonian Language, kiisu@eki.ee) Erkki Kolehmainen (TIEKE, erkki.kolehmainen@tieke.fi) (Mike Ksar) (Hewlett-Packard, mike_ksar@hp.com) Markus Kuhn (Computer Lab, Cambridge University, markus.kuhn@cl.cam.ac.uk) Marc Wilhelm Küster (Zentrum für Datenverarbeithung,Universität Tübingen, marc.kuester@zdv.unituebingen.de) Karl Ivar Larsson (LWP Consulting, kil@lwpc.a.se) Knut Lindelien (NTS/NSF, knut.lindelien@nts.no) Chris Makemson (Project leader) (The Peer Association, cmakemson@compuserve.com) Ciarán Ó Duibhín, (Queen s University Belfast, csg0070@queensbelfast.ac.uk) William Ross Mills (Tiro Typeworks, tirotech@tiro.com) Klaas Ruppel (Research Institute for the Languages of Finland, ruppel@domlang.fi) Keld Jørn Simonsen (Danish Unix User Group, keld@dkuug.dk) Baldev Soor (IBM, soor@ca.ibm.com) Monica Ståhl (IBM Svenska AB, monica.stahl@its.se) Alexandrina Stătescu (IRS, Romanian Institute for Standardization, ndstat@pcnet.pcnet.ro) Trond Trosterud (Universitetet i Tromsø, trond.trosterud@hum.uit.no) V. S. Umamaheswaran (IBM Toronto Laboratories, umavs@ca.ibm.com) Johan van Wingen (Van Wingen Consultancy, precal@rulmvs.leidenuniv.nl) orgeir Sigur³sson (Project secretary) (STRÍ, Icelandic Council for Standardization, thorgeir@stri.is) orvar³ur Kári Ólafsson (Project reviewer) (STRÍ, Icelandic Council for Standardization, thorvardur@stri.is) ¹Àîîé»áýóôòï (National Technical University of Athens, maistros@cs.ntua.gr) ùõàççåìï»åìáçòàëè (ELOT, eem@elot.gr) ºÞîóôáîôéîï ªôáõÞðïùìï (Iris Media Internet Solutions, cstath@irismedia.gr) Names (in parentheses) here are people who participated at the meetings in Reykjavík and/or in London but did not make comments on the second draft. Names struck out here are people who have signed up for the CWA but who have not yet participated in it. Before final publication of this CWA, the names of individuals who have not actually participated will be removed from this list.

5 Multilingual European Subsets of ISO/IEC Introduction Page 5 Fourth Draft for CWA XXXX:1998 There is a growing need for IT-communication across the national boundaries of Europe, with public administrations cooperating in large data systems, and commerce and trade between countries increasingly using IT-techniques. In addition, legal requirements regarding the spelling of personal names for individuals moving around in Europe must be considered. This leads to extensive requirements on the character set repertoires used in European IT-equipment. In Europe there is a large number of official languages and officially-accepted indigenous minority languages. These employ a large number of letters. In addition, a large number of other graphic characters are required for day-to-day computer use in Europe. Prior to ISO/IEC Universal Character Set (the UCS), there was no character set standard that included all the characters and scripts needed by Europe. The possibilities to combine several standardized character sets coded in 7-bits or 8-bits in the same IT-systems with existing code extension techniques have proven to be impractical and insufficient. Part 1 of ISO/IEC , the BMP, provides a good base for European character coding, since it defines fixed code positions for almost all presently known letters and a very large number of symbols and other characters for non-specialized use. It is likely that various compression schemes and transformation formats will be used for data storage and transmission of UCS encoded data. Required storage space and communication time will thus not double compared with single-octet codes. Implementation of ISO/IEC characters requires resources for supporting, for example: font definitions tables for code table translation, sorting, matching, and upper/lower casing functions The resources needed are material or human, for example: system memory space for definitions, tables, and functions licence costs for fonts documentation working time for program development learning time convenient and standardized keyboard input methods Some of the implementations problems discussed above could be solved by subsets of the UCS. This work on European subsets is particularly aimed at solving the problems of outputting the full character set of the UCS. It is estimated that implementing the full character set of the UCS will be costly in the first stages of UCS use, and that many manufacturers will implement in subset-stages. To ensure that a common subset usable to the vast majority of European users be available for a reasonable price, and as a guide to manufacturers, it will be helpful to specify, to users and procurers of systems, European subsets of the UCS encompassing all characters for use in European languages plus other frequently used and specialist characters. Such subsets may also be useful with regard to further standardization work (for example, on sorting specifications), so that the work is reasonably limited and still useful in a European environment.

6 Page 6 Fourth Draft for CWA XXXX:1998 Multilingual European Subsets of ISO/IEC Scope 1.1 This CEN Workshop Agreement specifies European Subsets of ISO/IEC : EN 1923 LL8 A Latin repertoire based on ISO/IEC 6937:1992 (a limited subset, fixed collection) MES-1 A Latin, Greek, Cyrillic repertoire based on ENV 1973:1996 (a limited subset, fixed collection) MES-2 A repertoire needed to write the languages of Europe and transliterate between them (a selected subset (a non-fixed collection) and a limited subset (a fixed collection). These repertoires are intended for, for example, governmental administrations in transborder transactions, the European Union and EFTA in use with the Schengen area and within the European Economic Area, CEN (the European Committee for Standardization) to define sets to work with in European standardization, and the Council of Europe in support of its treaty on the rights of minority and regional languages in Europe. Procurement guidance and conformance requirements are outside the scope of this CWA. 1.2 EN 1923 LL Coverage EN 1923 LL8 is a Latin repertoire of 333 characters, based on ISO/IEC 6937, but identical to the LL8 repertoire of EN 1923:1998. It covers at least the languages listed in Annex D of ISO/IEC 6937:1992. It is a superset of the Latin 1 through Latin 5 character sets and similar vendor coded character sets (not listed here) Field of application This repertoire is mainly intended for use in information systems that have 8-bit coding restrictions and/or are restricted by users to Latin characters. Example: name registries in countries using the Latin script. 1.3 MES Coverage MES-1 is a Latin, Greek, and Cyrillic repertoire of 9xx characters, similar to ENV 1973:1996. It covers the languages listed in Annex B of ENV 1973:1996 (plus Azerbaijani and Romany) and is a superset of EN 1923 LL8, all repertoires of EN 1923, and a large number of commonly used coded character sets (not listed here, but see examples in Annex C of ENV 1973) Field of application This repertoire is intended for: general purpose data and text processing applications in typical office environments in Europe governmental administrations in transborder transaction the European Union and EFTA in use within the Schengen area and within the European Economic Area CEN the European Committee for Standardization to define processing rules.

7 Multilingual European Subsets of ISO/IEC MES-2 Page 7 Fourth Draft for CWA XXXX: Coverage MES-2 is a repertoire covering all the characters belonging to European scripts. This coverage is guaranteed by defining MES-2A as a script-based and non-fixed collection. Thus it is a superset of any Latin, Greek, Cyrillic, Armenian, and Georgian repertoire, including EN 1923 LL8 and MES Field of application This repertoire is needed by the Council of Europe in support of its treaty on the rights of minority and regional languages in Europe. It is intended to be used in information systems that allow writing in the languages of Europe and transliterating between them. It also aims to provide for specialist administrative, scientific, technical, educational, and publishing requirements in Europe. NOTE 1: This repertoire is based on selected scripts of European origin with an inherent left-to-right directionality. Although the repertoire thus includes some characters that are not necessarily used in European languages, it does not include the characters needed for all the indigenous languages of Europe: the Karaim, Ladino, Tat, and Yiddish languages, for instance, are indigenous European languages using the Hebrew script. As this script is right-to-left, complete multilingual support for all the indigenous languages (either official or those with a recognized status as a minority language) of Europe would require bi-directional support, which requirement would probably be seen excessive for the majority of multilingual European IT systems. The requirement to support Yiddish, however, is a real one, not just in principle, but also in a multilingual environment. Collections 12, 13, and 63 in ISO/IEC should be used to provide full support for these languages, though they are not a part of the MES-2 specification itself. NOTE 2: Other ancient scripts of importance to Europe, but not part of this CWA, have been identified. Ogham (Collection 82) and Runic (Collection 83) have already been encoded in the UCS. Gothic and Etruscan have been provisionally accepted for encoding in part 2 (plane 1) of ISO/IEC 10646, and Cypriot, Glagolitic, Iberian, Linear B, Old Hungarian, and Old Permic are under study by ISO/IEC JTC1/SC2/WG2 and the Unicode Technical Committee for eventual inclusion in the UCS. The MES-2 does not broadly correspond to any single existing character set: it does correspond to the sum of many existing character sets. It is a collection-based subset of the UCS. The MES-2 will help developers worldwide to implement all characters belonging to European scripts. The requirement for support of European scripts is not intended to imply that European users have no need for non-european scripts; but it is logical to specify a subset defining the collections in ISO/IEC which contain European scripts. Non-European scripts are being provided for by non-europeans. The MES-2 is mainly intended to provide guidance to product developers to facilitate cost-effective provision of fonts for rendering devices, for instance. 2. Normative references This CWA incorporates by dated or undated reference, provisions from other publications. These normative references are cited at the appropriate places in the text and the publications are listed hereafter. For dated references, subsequent amendments to, or revisions of, any of these publications apply to this CWA only when incorporated in it by amendment or revision. For undated references the latest edition of the publication referred to applies. EN 1923:1998 European character repertoires and their coding - 8 bit single byte coding.

8 Page 8 Fourth Draft for CWA XXXX:1998 Multilingual European Subsets of ISO/IEC ENV 1973:1996 Information technology - European subsets of ISO/IEC ISO/IEC 6937:1992 Information technology - Coded graphic character set for text communication - Latin alphabet. ISO/IEC Information technology Universal Multiple-Octet Coded Character Set (UCS) Part 1: Architecture and Basic Multilingual Plane. 3. Definitions and abbreviations 3.1 Definitions For the purposes of this CEN Workshop Agreement the following basic definitions apply: coded character: A character together with its coded representation (ISO/IEC ). coded character set: A set of unambiguous rules that establishes a character set and the relationship between the characters of the set and their coded representation (ISO/IEC ). combining character: A member of an identified subset of the coded character set of ISO/IEC intended for combination with the preceding non-combining graphic character, or with a sequence of combining characters preceded by a non-combining character (ISO/IEC ). limited subset: A limited subset consists of a list of graphic characters in the specified subset. This specification allows applications and devices that were developed using other codes to interwork with this coded character set. A claim of conformance referring to a limited subset shall list the graphic characters in the subset by the names of graphic characters or code positions as defined in ISO/IEC (ISO/IEC clause 13.1). script: A set of graphic characters used for the written form of one or more languages (ISO/IEC clause 4.34). selected subset: A selected subset consists of a list of collections of graphic characters as defined in ISO/IEC The collections from which the selection may be made are listed in Annex A of each part of ISO/IEC A selected subset shall always automatically include the Cells 20 to 7E of Row 00 of Plane 00 of Group 00. A claim of conformance referring to a selected subset shall list the collections chosen as defined in ISO/IEC (ISO/IEC clause 13.2). 3.2 Abbreviations The following abbreviations apply: BMP: Basic Multilingual Plane LL8: Large Latin 8-bit character repertoire MES-1: Multilingual European Subset No MES-2: Multilingual European Subset No UCS: Universal Character Set

9 Multilingual European Subsets of ISO/IEC Page 9 Fourth Draft for CWA XXXX:1998 NOTE: Clauses 6 and 7 specify two repertoires based on the MES-2. The first, MES-2A, is a non-fixed collection and will allow a user to implement the MES-2 dynamically with reference to the version of The second, MES-2B, is a fixed collection, and is a snapshot which can be implemented without reference. 4. The Repertoire of EN 1923 LL8 4.1 Specification for the Repertoire of EN 1923 LL8 Collection Name: EN 1923 LL8 Type of Collection (Fixed or Non-fixed): FIXED Number of characters 333 Plane 00 Rows Positions (Cells) E, A0-FF B 2E-4D 50-7E 02 C7 D8-DB DD C 1D B-5E A 4.2 Collections containing the repertoire of EN 1923 LL8 The following UCS collections from Annex A of ISO/IEC contain the characters of the MES-1: 1 BASIC LATIN E 2 LATIN-1 SUPPLEMENT 00A0-00FF 3 LATIN EXTENDED-A F 6 SPACING MODIFIER LETTERS 02B0-02FF 32 GENERAL PUNCTUATION F 36 LETTERLIKE SYMBOLS F 37 NUMBER FORMS F 38 ARROWS FF 47 MISCELLANEOUS SYMBOLS FF

10 Page 10 Fourth Draft for CWA XXXX:1998 Multilingual European Subsets of ISO/IEC The Multilingual European Subset No. 1 (MES-1) 5.1 Specification for the Multilingual European Subset No. 1 Collection Name: Type of Collection (Fixed or Non-fixed): MES-1 FIXED Plane 00 Rows Positions (Cells) E A0-FF F 8F 92 B7 DE-EF FA-FF B 1E-1F 59 7C 92 BB-BD C6-C7 C9 D6 D8-DD DF A 7E 84-8A 8C 8E-A1 A3-CE D F E A-0B 1E-1F A-6B B F2-F3 1F D D B 5D 5F-7D 80-B4 B6-C4 C6-D3 D6-DB DD-EF F2-F4 F6-FE E A 3C 3E 44 4A 7F 82 A3-A4 A7 AC (AF) E 5B-5E A F A 1E-1F 27-2B A C C 24 2C 34 3C 50-6C C A0-A1 AA-AC B2 BA BC C4 CA-CB CF D8-D9 E6 26 3A-3C A-6B FB FF FD 5.2 Collections containing the MES-1 The following UCS collections from Annex A of ISO/IEC contain the characters of the MES-1: 1 BASIC LATIN E 2 LATIN-1 SUPPLEMENT 00A0-00FF 3 LATIN EXTENDED-A F 4 LATIN EXTENDED-B F 5 IPA EXTENSIONS AF 6 SPACING MODIFIER LETTERS 02B0-02FF 8 BASIC GREEK CF 9 GREEK SYMBOLS AND COPTIC 03D0-03FF 10 CYRILLIC FF 30 LATIN EXTENDED ADDITIONAL 1E00-1EFF 31 GREEK EXTENDED 1F00-1FFF 32 GENERAL PUNCTUATION F 33 SUPERSCRIPTS AND SUBSCRIPTS F 34 CURRENCY SYMBOLS 20A0-20CF 36 LETTERLIKE SYMBOLS F 37 NUMBER FORMS F 38 ARROWS FF 39 MATHEMATICAL OPERATORS FF 40 MISCELLANEOUS TECHNICAL FF 44 BOX DRAWING F 45 BLOCK ELEMENTS F 46 GEOMETRIC SHAPES 25A0-25FF 47 MISCELLANEOUS SYMBOLS FF LTR PRESENTATION FORMS FB00-FB17 70 SPECIALS FFF0-FFFD NOTE: A collection identifier for LTR Presentation Forms has not yet been defined by ISO/IEC JTC1/SC2/WG2. It will have to be requested.

11 Page 11 Multilingual European Subsets of ISO/IEC Fourth Draft for CWA XXXX: The Multilingual European Subset No. 2A (MES-2A) 6.1 Specification for the Multilingual European Subset No. 2A Collection Name: Type of Collection (Fixed or Non-fixed): MES-2A NON-FIXED The following UCS collections from Annex A of ISO/IEC comprise the MES-2A: No. Collection name hex range rationale 1 BASIC LATIN E Latin script 2 LATIN-1 SUPPLEMENT 00A0-00FF Latin script 3 LATIN EXTENDED-A F Latin script 4 LATIN EXTENDED-B F Latin script 5 IPA EXTENSIONS AF Latin script 6 SPACING MODIFIER LETTERS 02B0-02FF used with European scripts and IPA 7 COMBINING DIACRITICAL MARKS F used with European scripts and IPA 8 BASIC GREEK CF Greek script 9 GREEK SYMBOLS AND COPTIC 03D0-03FF Greek script 10 CYRILLIC FF Cyrillic script 11 ARMENIAN F Armenian script 27 BASIC GEORGIAN 10D0-10FF Georgian script 30 LATIN EXTENDED ADDITIONAL 1E00-1EFF Latin script 31 GREEK EXTENDED 1F00-1FFF Greek script 32 GENERAL PUNCTUATION F used with all European scripts 33 SUPERSCRIPTS AND SUBSCRIPTS F publishing and science 34 CURRENCY SYMBOLS 20A0-20CF banking and general use 35 COMBINING DIACRITICAL MARKS FOR SYMBOLS 20D0-20FF publishing and science 36 LETTERLIKE SYMBOLS F publishing and science 37 NUMBER FORMS F publishing and science 38 ARROWS FF publishing and science 39 MATHEMATICAL OPERATORS FF publishing and science 40 MISCELLANEOUS TECHNICAL FF publishing and science 42 OPTICAL CHARACTER RECOGNITION F banking 44 BOX DRAWING F legacy character sets 45 BLOCK ELEMENTS F publishing and science 46 GEOMETRIC SHAPES 25A0-25FF publishing and science 47 MISCELLANEOUS SYMBOLS FF publishing 48 DINGBATS BF publishing LTR PRESENTATION FORMS FB00-FB17 publishing 65 COMBINING HALF MARKS FE20-FE2F bibliographic use 70 SPECIALS FFF0-FFFD replacement characters NOTE: A collection identifier for LTR Presentation Forms has not yet been defined by ISO/IEC JTC1/SC2/WG2. It will have to be requested.

12 Page 12 Fourth Draft for CWA XXXX:1998 Multilingual European Subsets of ISO/IEC The Multilingual European Subset No. 2B (MES-2B) 7.1 Specification for the Multilingual European Subset No. 2B Collection Name: Type of Collection (Fixed or Non-fixed): MES-2B FIXED Plane 00 Rows Positions (Cells) E A0-FF FF F AD B0-EE E A 7E 84-8A 8C 8E-A1 A3-CE D0-D7 DA-F E-CE D0-F5 F8-F F A 10 D0-F6 FB 1E 00-9B A0-F9 1F D D B 5D 5F-7D 80-B4 B6-C4 C6-D3 D6-DB DD-EF F2-F4 F6-FE D 6A E A0-AE (AF) D0-E A F F B 7D-9A A A0-F A C B 4D 4F E AF B1-BE FB FE FF FC FD 7.2 Collections containing the MES-2B The following UCS collections from Annex A of ISO/IEC contain the characters of the MES-2B: No. Collection name hex range rationale 1 BASIC LATIN E Latin script 2 LATIN-1 SUPPLEMENT 00A0-00FF Latin script 3 LATIN EXTENDED-A F Latin script 4 LATIN EXTENDED-B F Latin script 5 IPA EXTENSIONS AF Latin script 6 SPACING MODIFIER LETTERS 02B0-02FF used with European scripts and IPA 7 COMBINING DIACRITICAL MARKS F used with European scripts and IPA 8 BASIC GREEK CF Greek script 9 GREEK SYMBOLS AND COPTIC 03D0-03FF Greek script 10 CYRILLIC FF Cyrillic script 11 ARMENIAN F Armenian script 27 BASIC GEORGIAN 10D0-10FF Georgian script 30 LATIN EXTENDED ADDITIONAL 1E00-1EFF Latin script 31 GREEK EXTENDED 1F00-1FFF Greek script 32 GENERAL PUNCTUATION F used with all European scripts 33 SUPERSCRIPTS AND SUBSCRIPTS F publishing and science 34 CURRENCY SYMBOLS 20A0-20CF banking and general use 35 COMBINING DIACRITICAL MARKS FOR SYMBOLS 20D0-20FF publishing and science 36 LETTERLIKE SYMBOLS F publishing and science 37 NUMBER FORMS F publishing and science 38 ARROWS FF publishing and science 39 MATHEMATICAL OPERATORS FF publishing and science 40 MISCELLANEOUS TECHNICAL FF publishing and science 42 OPTICAL CHARACTER RECOGNITION F banking 44 BOX DRAWING F legacy character sets 45 BLOCK ELEMENTS F publishing and science 46 GEOMETRIC SHAPES 25A0-25FF publishing and science

13 Multilingual European Subsets of ISO/IEC Page 13 Fourth Draft for CWA XXXX: MISCELLANEOUS SYMBOLS FF publishing 48 DINGBATS BF publishing LTR PRESENTATION FORMS FB00-FB17 publishing 65 COMBINING HALF MARKS FE20-FE2F bibliographic use 70 SPECIALS FFF0-FFFD replacement characters NOTE: A collection identifier for LTR Presentation Forms has not yet been defined by ISO/IEC JTC1/SC2/WG2. It will have to be requested. Annex A. Bibliography (Informative) Unicode Consortium The Unicode Standard. Version 2.0. Reading: Addison-Wesley. ISBN

Flerspråkiga delmängder i ISO/IEC

CEN Workshop Agreement SIS- Utgåva 1 Juli 2004 Flerspråkiga delmängder i ISO/IEC 10646-1 Information Technology Multilingual European Subsets in ISO/IEC 10646-1 ICS: 35.040 Språk: engelska Copyright SIS.