UPENET (UPGRADE European NETwork) CEPIS NEWS. 2 Editorial: UPGRADE in Top Position of Google PageRank for ICT Journals Geoff McMullen

Transcription

1

2 UPGRADE is the European Journal for the Informatics Professional, published bimonthly at < Publisher UPGRADE is published on behalf of CEPIS (Council of European Professional Informatics Societies, < by Novática < journal of the Spanish CEPIS society ATI (Asociación de Técnicos de Informática, < UPGRADE monographs are also published in Spanish (full version printed; summary, abstracts and some articles online) by Novática UPGRADE was created in October 2000 by CEPIS and was first published by Novática and INFORMATIK/INFORMATIQUE, bimonthly journal of SVI/ FSI (Swiss Federation of Professional Informatics Societies, < UPGRADE is the anchor point for UPENET (UPGRADE European NETwork), the network of CEPIS member societies publications, that currently includes the following ones: Informatik-Spektrum, journal published by Springer Verlag on behalf of the CEPIS societies GI, Germany, and SI, Switzerland ITNOW, magazine published by Oxford University Press on behalf of the British CEPIS society BCS Mondo Digitale, digital journal from the Italian CEPIS society AICA Novática, journal from the Spanish CEPIS society ATI OCG Journal, journal from the Austrian CEPIS society OCG Pliroforiki, journal from the Cyprus CEPIS society CCS Pro Dialog, journal from the Polish CEPIS society PTI-PIPS Editorial Team Chief Editor: Llorenç Pagés-Casas, Spain, Associate Editors: François Louis Nicolet, Switzerland, Roberto Carniel, Italy, Zakaria Maamar, Arab Emirates, <Zakaria. zu.ac.ae> Soraya Kouadri Mostéfaoui, Switzerland, Rafael Fernández Calvo, Spain, Editorial Board Prof. Wolffried Stucky, CEPIS Former President Prof. Nello Scarabottolo, CEPIS Vice President Fernando Piera Gómez and Llorenç Pagés-Casas, ATI (Spain) François Louis Nicolet, SI (Switzerland) Roberto Carniel, ALSI Tecnoteca (Italy) UPENET Advisory Board Hermann Engesser (Informatik-Spektrum, Germany and Switzerland) Brian Runciman (ITNOW, United Kingdom) Franco Filippazzi (Mondo Digitale, Italy) Llorenç Pagés-Casas (Novática, Spain) Veith Risak (OCG Journal, Austria) Panicos Masouras (Pliroforiki, Cyprus) Andrzej Marciniak (Pro Dialog, Poland) Rafael Fernández Calvo (Coordination) English Language Editors: Mike Andersson, Richard Butchart, David Cash, Arthur Cook, Tracey Darch, Laura Davies, Nick Dunn, Rodney Fennemore, Hilary Green, Roger Harris, Michael Hird, Jim Holder, Alasdair MacLeod, Pat Moody, Adam David Moss, Phil Parkin, Brian Robson Cover page designed by RFCalvo, Rafael Fernández Calvo 2006 Layout Design: François Louis Nicolet Composition: Jorge Llácer-Gil de Ramales Editorial correspondence: Llorenç Pagés-Casas Advertising correspondence: UPGRADE Newslist available at < Copyright Novática 2006 (for the monograph) CEPIS 2006 (for the sections UPENET and CEPIS News) All rights reserved under otherwise stated. Abstracting is permitted with credit to the source. For copying, reprint, or republication permission, contact the Editorial Team The opinions expressed by the authors are their exclusive responsibility ISSN Monograph of next issue (February 2007) "Next Generation Web Search" (The full schedule of UPGRADE is available at our website) 2 Editorial: UPGRADE in Top Position of Google PageRank for ICT Journals Geoff McMullen 3 Presentation: OpenDocument Standard for Digital Documents Jesús Tramullas-Saz and Piedad Garrido-Picazo 5 Open by Design: The OpenDocument Format Standard for Office Applications Erwin Tenhumberg, Donald Harbison, and Rob Weir 14 Is OpenDocument an Open Standard? Yes! David A. Wheeler 25 OpenDocument Hidden Traps and their Side Effects on Free/Open Source Software Marco Fioretti 29 ISO (OpenDocument) vs. MS-Office Open XML Alberto Barrionuevo-García 38 Interoperability: Will the Real Universal File Format please Stand Up? Sam Hiser and Gary Edwards 47 ODF: The Emerging Document Format of Choice for Governments Marino Marcich 50 Promotion of the Use of Open Document Formats by the IDA and IDABC Programmes Miguel A. Amutio-Gómez 53 A Brief History of Open Standards in Denmark John Gøtze 57 Standard Open Formats and Libre Software in the Extremadura Public Administration Luis Millán-Vázquez de Miguel UPENET (UPGRADE European NETwork) 60 From Mondo Digitale (From AICA, Italy) Programming Languages Programming Languages: An Introduction Carlo Ghezzi 64 From Pro Dialog (PTI-PIPS, Poland) E-Commerce Organization and Economics of Entertainment Services Networks Exchanging Virtual Goods Andrzej P. Urbañski CEPIS NEWS The European Journal for the Informatics Professional Vol. VII, issue No. 6, December 2006 Monograph: Open Document Format (ODF) (published jointly with Novática*) Guest Editors: Jesús Tramullas-Saz, Piedad Garrido-Picazo, Marco Fioretti 70 Harmonise Project: Assigning Responsibilities among the Partners and Planning Next Steps François-Philippe Draguet 71 News & Events: European Funded Projects and News Updates * This monograph will be also published in Spanish (full version printed; summary, abstracts, and some articles online) by Novática, journal of the Spanish CEPIS society ATI (Asociación de Técnicos de Informática) at <

3 Editorial UPGRADE in Top Position of Google PageRank for ICT Journals UPGRADE has just completed six years of existence. It is not a long time in terms of informatics history. In the Computer Science field, we have indeed a lot of competitors, publications with a quite long tradition and big budgets to spend in order to attract readers through appealing marketing campaigns. In these circumstances, it appears to be quite extraordinary that UPGRADE is currently at the seventh place by "PageRank" in the category "Computer Science/Publications/Journals" of Google (out of more than 500 computer journals that are estimated to be published all around the world). PageRank is the core component of the Google search engine in order to denote "the most relevant and highlyregarded sites", in their own words. It is considered one of the most relevant "thermometers" to measure the interest and success of the Internet websites. Therefore, at this point, we must be very glad and think that very are walking in the right way, as the number of accesses to our website shows too. These are, without any doubt, good indicators that UPGRADE contents are relevant and useful for the ICT community worldwide. Now, our commitment is to follow up and, where possible, improve the scope and quality of our materials and their relevance in our field. We remain extremely grateful to the found editors of UPGRADE, François Louis Nicolet and Rafael Fernandez Calvo, for putting in so much effort to get UPGRADE started and develop it into a success, and to their member societies for supporting them so strongly. In particular, we acknowledge with thanks the role of ATI, our Spanish member, in championing this important initiative. Geoff McMullen President of CEPIS <president AT cepis DOT org> UPENET UPGRADE European NETwork The network of CEPIS member societies publications Current partners Informatik Spektrum (GI, Germany, and SI, Switzerland), ITNOW (BCS, United Kingdom), Mondo Digitale (AICA, Italy), Novática (ATI, Spain), OCG Journal (OCG, Austria), Pliroforiki (CCS, Cyprus), Pro Dialog (PTI-PIPS, Poland) 2 UPGRADE Vol. VII, No. 6, December 2006 Novática

4 Presentation OpenDocument Standard for Digital Documents Jesús Tramullas-Saz and Piedad Garrido-Picazo The boom experienced by the libre software movement and its unlimited possibilities, together with the growing success enjoyed by software libre tools in the commercial and business world, has overshadowed a number of key issues that are inherent to the movement. We are of course referring to open standards, which are core elements to achieve the interoperability of software applications. Standards are what set the rules of the game for many aspects and functionalities of software tools. Compliance with standards, whether de facto or de iure, is what can make or break an application. Much of the software development going on today is guided by standards, as is the much vaunted issue of "software quality". For this reason the scant interest aroused by standards in recent years is even more surprising. Obviously, an application is more exciting (and useful) than a technical standard, especially if it is an open standard. However, the former are impossible without the latter. Most of the protocols on which the communication and transfer of information over the Internet is based are open (or nearly open) standards. The World Wide Web Consortium directs its efforts towards formulating and accepting standards in the knowledge that without standards it would be impossible to carry on developing the Web and its advanced information services. But this seems to be contradicted by the lack of standards for what is the most widely created and disseminated information worldwide: the information stored in office automation documents. Whoever controls users desktop controls their applications. And 80% of all computer users work with classic desktop applications, with a word processor or a spreadsheet. Office work in companies, the work done in public administrations or in educational environment, are examples of the intensive use of office automation applications. The information they use, generate, and transform is stored in office automation documents. The formats of these documents have become de facto standards which have cunningly imposed their rules on all users. These rules, which are often abusive, limit freedom of choice, compatibility, and interoperability, and oblige users to consider as normal a series of costs that, in any other context, would immediately be denounced as monopolistic abuse and contrary to the principles of fair competition. To have a standard is, by definition, recommendable. A standard establishes requirements and the rules of the game. The Guest editors Jesús Tramullas-Saz is an associate professor in the Dept. of Documentary Sciences, Universidad de Zaragoza. He is a member of the GRIO (Gestión de Recursos de Información en las Organizaciones, Information Resource Management in Organizations) research group and a coordinator of the Thematic Network on Digital Documentation (National R&D&I Plan, and ). He is the principal researcher on the project "Semantic Web and digital libraries: the development information services based on RDF and Topic Maps" ( ). His main research interests are digital libraries and digital information services, content management, and libre software tools for information management. He is responsible for maintaining the Spanish translation of the libre software Greenstone Digital Library Software. <jesus@tramullas.com>. Piedad Garrido-Picazo is an assistant professor in the Dept. of Computing and Systems Engineering, Universidad. de Zaragoza. She is a member of the GRIO (Gestión de Recursos de Información en las Organizaciones, Information Resource Management in Organizations) research group. She belongs to the Thematic Networks on Information Retrieval from Digital Texts and Libraries, Digital Documentation, and Soft-Computing-based Web Information Access Systems. Her main research interests are xml databases, libre software for information management, digital libraries, RDF and Topic Maps (XTM) in the context of the semantic Web. <piedad@unizar.es>. Jesús and Piedad have coordinated together the book Software libre para servicios de información digital, Prentice Hall, Madrid Marco Fioretti is a hardware systems engineer and a part time freelance writer for several Linux and other IT magazines. He is the co-founder (in 2002) and current coordinator of the Run Up to date Linux Everywhere Project < which makes modern Free Software easily installable on older computers. Marco Fioretti has always been interested in truly Open file formats and protocols, from e-books to portable databases and the Universal Business Language, and in their impacts on economy, education and civil rights. In 2004, for example, he investigated the philosophical links between Free Software and Scouting. In the same spirit, in 2006 he co-founded Elèutheros < which aims to promote a wider, official adoption of Free Software and non proprietary formats within the Catholic Church. More recently, Fioretti has also started to follow the attempts of the Free Software and disabled users communities to communicate more effectively. His more recent initiative to promote the importance of Free Standards and Software among all citizens is the Family Guide to Digital Freedom < digifreedom.net>. Marco Fioretti is also member and contact for Italy of the OpenDocument Fellowship < fellowship.org>, a volunteer organization that promotes the use and development of the OpenDocument format, and the author of Everybody s Guide to OpenDocument < article/8616>. <marco.fioretti@open documentfellowship.org>. Novática UPGRADE Vol. VII, No. 6, December

5 However, a standard may be used in such a way as to create undesirable results, especially when patents and legal constraints that favour one party over another are involved. For this reason it is essential for standards to be open, developed in a spirit of collaboration among equals, and for their specification not to contain hidden constraints that make it hard to use. If these standards also ensure all citizens the right to access, store, and transform digital information, regardless of the platform used, they immediately acquire an incalculable economic, social, and political value. OpenDocument is a standard (the only standard) for office automation documents which sets out all the desirable characteristics that should be found in such documents. Its version 1.0 has been endorsed by the International Organization for Standardization (ISO), as ISO/IEC standard 26300:2006. It is the result of the open, collaborative work of all the main players; software developers, solution implementers, and end-users. It is public and free of charge, and the legal requirements included in the standard prevent any partial or abusive use. It also makes use of other open standards within its own specification, such as XML, SVG, and Dublin Core. OpenDocument had already been developed and approved as a standard by the Organization for the Advancement of Structured Information Standards, OASIS, in 2005, which ensures its continuing support and development by the main industry players. However, OpenDocument is not just a format standard for office automation documents. The underlying philosophy is different. Some may argue that a standard is nothing more than a technical issue, and in purely formal terms they may be right. But the technical excellence of OpenDocument, which is undoubtedly superior to that of any existing office document format, is a product of the underlying philosophical approach and the way in which this philosophy has been applied to working methods and techniques, and to the resultant end-product. The adoption of an open and democratic collaborative working method has made it possible to involve all interested parties, thereby ensuring that all their many and varied demands are addressed. The involvement of industry interests ensures the presence of standard-compliant products on the market, which in turn ensures inter-platform portability and interoperability. Any citizen can access the document containing the technical specification, which means that the information contained in OpenDocument format documents are not subject to arbitrary decisions by third parties. This open approach also encourages competition between products and, given a range of options, users are free to choose the one they think is best suited to their needs, or the one that is most technologically advance. This independence is not only desirable; it is essential and inevitable. The OpenDocument format is prepared for the Semantic Web. All tagging complies with the XML standard and documents may include the Dublin Core metadata standard (ISO 15:836:2003). Various output can be obtained using XSLT. As they are XML-tagged text documents, the tools and libraries for search engines such as Lucene or Xapian are able to process them with a minimum load. For example, by combining with other standards it is possible to generate Topic Maps (ISO/IEC 13250:2003) from document content, with all that can mean for the development of information extraction and representation systems. One issue that is of paramount importance, which is often completely neglected, is the preservation of digital information. Although many organizations seem still to be unaware of it, the medium- and long term retention of digital assets, most of which are stored in office automation documents, is a growing concern, both due to knowledge management issues affecting the organization itself, and due to legal issues related to their activity. While there is a standard for the long-term preservation of electronic documents (ISO :2005) using PDF, the fact is that it is only valid for final versions of a document. OpenDocument is able to keep an activity log of the content of a document, and can "remember", say, the formulas used to reach a mathematical result. Add to this the growing demand for records and document management in the field of public administrations, companies, etc., which also have their own set of standards (ISO /2:2001 Records Management; UNE/ISO /2:2006 Documents Management), and it would seem clear that OpenDocument has a long future ahead of it and plenty of ground to cover. Translation by Steve Turpin Useful References on ODF OpenDocument version 1.0 specification: < OpenDocument-v1.0-os.pdf>. OASIS OpenDocument Essentials: < Opportunities for Innovation with OpenDocument Format XML: < USEN-00.pdf>. OpenDocument - Formula TC: < committees/tc_home.php?wg_abbrev=office-formula>. OASIS OpenDocument Format for Office Applications: < home.php?wg_abbrev= office>. The OpenDocument Foundation: < foundation.us/>. OpenDocument Format Alliance: < OpenDocument Fellowship: < fellowship.org/>. Technorati: OpenDocument: < tag/opendocument>. OpenDocument xml.com: < Open Interoperative Document Initiative: < OpenDocument Format Viewer: < fellowship.org/odfviewer>. OpenOffice: < Koffice: < O Reilly ONLamp: What Is OpenDocument: < onlamp.com/pub/a/onlamp/2006/07/27/what-is-open document.html>. Sam Hiser s blog: < Andy Updegrove s blog: < org/ standardsblog/>. Bob Sutor s blog: < Charles H. Schulz s blog: < 5/charles>. David A. Wheeler s blog: < Erwin Tenhumberg s blog: < Ron Weir s blog: < index.html>. 4 UPGRADE Vol. VII, No. 6, December 2006 Novática

6 Open by Design: The OpenDocument Format Standard for Office Applications Erwin Tenhumberg, Donald Harbison, and Rob Weir This article describes the history of the Open Document Format (ODF) which is now published as the ISO standard ISO/ IEC It covers subjects like the value of openness in file formats, its short and long-term benefits, interoperability and innovation. It is a collaborative essay written by some outstanding members of the OASIS OpenDocument Format Adoption Committee whose purpose is "to create awareness and demand for a new class of applications and solutions designed specifically to support and promote the OpenDocument Format". Keywords: Document Format, ECMA, OASIS, ODF, Office Applications, Office Open XML, OpenDocument, Open Standard, XML File Format. ments, which stay valid and relevant over decades, or even centuries. But it is no less the case for personal documents. Just as paper and pens have been available from multiple vendors and not just one single source, document file formats and the applications creating these file formats need to be supported by and available from multiple vendors. This guarantees long-term access to data even if companies cease to operate, change their strategies, or dramatically raise their prices. In effect, with choice, the user retains control Authors Erwin Tenhumberg is a member of the Open Source Group at Sun Microsystems which is led by Sun s Chief Open Source Officer Simon Phipps. Among his roles and responsibilities are community development and marketing for OpenOffice.org. In addition, Erwin focuses on open source business models and open source in the public sector. Besides his role in the Open Source Group, he also co-chairs the OASIS OpenDocument Format (ODF) Adoption TC (Technical Committee). As part of his work, Erwin is involved in various ODF efforts such as at the Commonwealth of Massachusetts in the US or within the European Union. In addition, he collaborates with the OASIS OpenDocument Format TC, the ODF Alliance and the OpenOffice.org open source community. <erwin.tenhumberg@ sun.com>. Donald Harbison co-chairs the OASIS ODF Adoption Technical Committee with Erwin Tenhumberg. Donald has been with IBM for 10 years and has worked in the software industry for over 25 years. He has extensive industry experience in software business development in technical publishing, document and content management, collaboration services including enterprise messaging, and object oriented application communications services. In addition to these roles, Donald was a member of the core strategy team that developed and led the development and execution of IBM s Linux middleware strategy beginning in 1998 through He is currently Program Director, Emerging Standards, in the IBM Software Group. <donald_harbison@us.ibm.com>. Rob Weir is a 16-year veteran of IBM and Lotus Development Corporation. He has extensive experience working with office 1 Why an Open File Format Matters In a world where paper documents are increasingly being replaced by electronic records, ensuring that long term access and usability of these records is critical. This is especially the case for legal contracts and government docufile formats, from the old binary formats in Lotus SmartSuite and Microsoft Office, to the new generation of XML formats undergoing standardization. He is a member of the OASIS ODF TC, Metadata and Formula Subcommittees, and ODF Adoption TC. He is also a U.S. delegate to ISO/IEC JTC1 SC34. <robert_weir@us.ibm.com>. OASIS (Organization for the Advancement of Structured Information Standards) is a not-for-profit, international consortium that drives the development, convergence, and adoption of e-business standards. Members themselves set the OASIS technical agenda, using a lightweight, open process expressly designed to promote industry consensus and unite disparate efforts. The consortium produces open standards for Web services, security, e-business, and standardization efforts in the public sector and for application-specific markets. OASIS was founded in More information can be found on the OASIS website at < The purpose of the OASIS OpenDocument Format Adoption Committee is to create awareness and demand for a new class of applications and solutions designed specifically to support and promote OpenDocument XML (commonly referred to as the OpenDocument Format or ODF). The Adoption Committee dedicates its energy and resources to creating wide-scale understanding of the benefits of OpenDocument Format support within organizations and governmental bodies through education and promotion. The Adoption Committee aligns and supports the activities of the OASIS OpenDocument Technical Committee by providing market-based requirements. These requirements help guide future development of the OpenDocument specification by the OASIS OpenDocument Technical Committee. Novática UPGRADE Vol. VII, No. 6, December

7 over and ownership of the documents she authors; she is no longer dependent on a single vendor to read and edit her work. Open standards that are equally accessible and do not favor one particular vendor can help maintain a diverse ecosystem of vendors. This also fosters competitive pricing, thus creating the conditions for the best use of money from investors to tax payers. In the case of public documents that governments provide to their nation s citizens, it is also important that no resident be excluded from data access. Public data should be accessible to citizens independent of their income and their physical abilities. Accessibility in this case, has an entirely different meaning for Persons with Disabilities (PwD). An open standard dealing with document data must also be designed to enable the addition of a range of assistive technologies such that a person with no vision, or low vision, paralysis and even severe motor skill limitations have sufficient access to the software and document data to be able to author and read effectively. The recent ODF v1.1 Committee Specification 1 addresses these requirements. In the tradition of open standards development, the OASIS ODF Technical Committee established a Sub-Committee of technical experts skilled in the field of accessible technology. The Accessibility SC set an ambitious goal to meet and exceed the accessibility support currently available in the industry s dominant file format, as well as what is specified in the W3C Web Content Accessibility Guidelines 1.0. The Accessibility SC also recognized the need to provide application developers with implementation guidelines to ensure that their ODF supporting solutions fully meet the needs of people with disabilities by including the full range of requirements. The result is the "Accessibility Guidelines for Implementations of OpenDocument Format v1.1." [1]. Open standards lower the barrier to entry, allowing new companies to join the ecosystem. For example, the SQL standard for relational databases allowed the emergence of various implementations, including free and open source and very specialized high-end database management systems. As long as only standard SQL features are used, data stored in database management systems can be exchanged without much effort. A user may choose a SQL implementation that includes unique, vendor-specific elements in addition to the basic, but that is her choice. Thus, vendor lockin becomes a choice, not an unfortunate necessity. 2 Approved by OASIS and ISO: An Overview of ODF The OpenDocument Format (ODF) is an open, XMLbased document file format for office applications that create and edit documents containing text, spreadsheets, charts, and graphical elements. The file format makes transformations to other formats simple by using and reusing existing standards wherever possible. ODF is defined via an open and transparent process at OASIS (Organization for the Advancement of Structured Information Standards) and has been approved unanimously by the Joint Technical Committee 1 (JTC1) of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) as an International Standard (IS) in May In November 2006 ISO/ IEC announced the publication and availability of ISO/IEC 26300:2006. It is available for implementation and use free of any licensing, royalty payments or other restrictions. By providing an alternative to proprietary technologies, OpenDocument allows end users to embrace an open-standards approach to managing vital documents. It helps assure that end users, such as governments and their citizens, are able to access and share information now and for generations to come without having to continue to pay unnecessary licensing fees to view or edit information stored in proprietary formats. Organizations or individuals can deploy any word processing application, thereby giving them greater control of their documents by decoupling file formats from the applications used to create them, especially proprietary formats with accompanying limitations and restrictions. OpenDocument promotes long-term information retrieval by entrusting the format to an independent standards body that operates as a community. This is in contrast to a history of single vendor control, wherein backward file format compatibility has not been guaranteed. Adoption of OpenDocument avoids reliance on the life span of a piece of software to maintain access to vital information. Unfortunately, experience has shown the life span of a software application to be only a small fraction of the life span of critical documents, such as birth or financial records. In technical terms, the OpenDocument Format specification defines an XML schema for office applications and their semantics. The schema is designed to be suitable for a range of office documents including text documents, presentation documents including charts, drawings, animations, spreadsheet documents for financial calculations and access to external datasets. The schema provides for high level information enabling interactive editing of the document data. It defines supporting XML structures for the full range of office documents and is easily transformed using XSLT or similar XMLbased tools. The ODF specification itself describes the structure of documents, the meta data information possible to store in such documents, text and paragraph content, text fields, text indices, table content, graphical content, chart content, form content, content common to all document types, integration of SMIL (Synchronized Multimedia Integration Language) animation markup content, style information, formatting properties used within styles, and data types used by the OpenDocument schema. It is complete, mature, simple and elegant, and designed to be implemented and supported by multiple vendors serving a range of customer requirements. From a packaging perspective, ODF is a ZIP archive 6 UPGRADE Vol. VII, No. 6, December 2006 Novática

8 Date / Time Frame Event / Milestone 1999 The Development of an XML default file format begins at StarDivision. Limitations of the old binary format and a need for Unicode support trigger the change. The goal is to create an open, interoperable file format that can be used and implemented by other vendors as well. August 1999 Sun Microsystems, Inc. acquires StarDivision. 13 October 2000 Sun Microsystems, Inc., releases the source code to StarOffice under open licenses to the recently founded (July 2000) OpenOffice.org project. 13 October 2000 The XML community project is set up in OpenOffice.org with the goal of defining the specification of the OpenOffice.org XML file format as an open community effort Definitions for CJK (Chinese, Japanese, Korean) and complex text layout languages are added to the OpenOffice.org XML file format specification Collaboration with the KOffice project begins. 16 December 2002 The OASIS Open Office Technical Committee (TC) has its first conference call. May 2002 August 2003 OpenOffice.org 1.0 and StarOffice 6 are released, both using the OpenOffice.org XML file format as the default file format. KOffice decides to use ODF as its default file format / 2004 The original OpenOffice.org XML file format specification is modified to reflect recent developments in the XML and office application area, e.g.: * Introduction of XML namespaces that conform to the OASIS naming rules * Switching from XML DTDs to Relax-NG as the schema language * Improvements of the schema to better support the validation of documents * Adaptation of the schema to new versions of standards * Adaptations for additional office applications (KOffice) * Adaptations for new office application versions (OpenOffice.org 2.0) * Removal of inconsistencies in the specification * Error corrections December 2004 January 2005 A second committee draft is approved, and the title of this draft is changed from OASIS Open Office Specification to OASIS Open Document Format for Office Applications (OpenDocument) The TC is renamed to OASIS Open Document Format for Office Applications (OpenDocument) TC. Novática UPGRADE Vol. VII, No. 6, December

9 March 2006 March 2006 April 2006 The ODF Alliance is founded with 35 initial members in order to promote ODF in the public sector. The OASIS ODF Adoption TC is founded with the aim of educating the marketplace of the value of ODF. KOffice 1.5, which uses ODF as the default file format, is released. May 2006 ISO approves ODF as ISO/IEC June 2006 September 2006 The ODF Alliance already has more than 200 members including companies, organizations, and municipalities such as BBC, Corel, EDS, EMC, IBM, Novell, Red Hat, Oracle, Software AG, Sun Microsystems, and the City of Vienna. ODF 1.0 Second Edition completed bringing in editorial changes identified in the ISO review process. October 2006 ODF 1.1 approved as Committee Specification; to be submitted for an OASIS Standard vote in January 2007 Continuing development of formula, accessibility and metadata deliverables planned for publication in 2007 as ODF 1.2. ODF Alliance membership surges past 300 members from over 40 countries. Table 1: The History of ODF that contains a collection of XML files that describe the document s content and presentation. Binary files are only used for such things as embedded images. The use of XML makes accessing the document content simple, because content can be opened and changed with simple text editors, if necessary. In contrast, the previously used proprietary binary-only file formats were cryptic and difficult to process. The ZIP compression guarantees relatively small file sizes, which reduce file storage and transmission bandwidth requirements; this makes it easier to exchange files, regardless of bandwidth. (ODF was the first broadly used document file format that used a ZIP package containing different XML files.) ODF uses the same set of XML files for different application types. In addition, definitions for elements like tables are consistent across application types. 3 A Long Tradition of Openness: The History of ODF The OpenDocument Format has a long tradition of openness. The first work on the file format started as early as Right from the beginning, ODF was designed as an open and implementation neutral file format. The open specification process started in 2000 with the foundation of the OpenOffice.org open-source project and the community efforts within its XML development project. An even higher level of openness was established in 2002 with the creation of the OASIS Open Office Technical Committee (TC). During the last seven years, an increasing number of organizations and companies have joined the ODF specification process. In addition, a growing number of applications implement the OpenDocument file format. Table 1 provides an overview of the history of the OpenDocument Format. 4 Open by Design: The Benefits of ODF The OpenDocument Format was designed to be vendor neutral and implementation agnostic. It was designed to be used by as many applications as possible. In order to simplify transformations and to maximize interoperability, the format reuses established standards such as XHTML, SVG, XSL, SMIL, XLink, XForms, MathML, and Dublin Core. ODF files of different application types (e.g., the word processor, spreadsheet) include the same set of XML files within the ZIP packages. Figure 1 shows a simple ODF text document and the contents of the corresponding ZIP package. Figure 2 shows a simple ODF spreadsheet document and its ZIP archive contents. Both the text document and the spreadsheet document have the same structure, e.g., both contain a content.xml, a styles.xml, and a meta.xml file. 8 UPGRADE Vol. VII, No. 6, December 2006 Novática

10 Figure 1: An ODF Text Document unzipped. Figures 3 and 4 illustrate that tables within text documents are defined by the same XML elements as tables within spreadsheet documents. Using the same set of XML files within ODF documents as well as defining similar document elements across application types with the same XML elements makes transforming and processing ODF documents simple. Figure 3 shows the content.xml file with a table definition of a text document. Figure 4 shows the table definition of a spreadsheet document. The same XML elements are used to define tables in spreadsheet documents as in text documents. Table 2 highlights the key features and benefits of the OpenDocument Format. 5 Innovation Opportunities with OpenDocument 5.1 Integration through Programming Today, programming with document data is too complex and platform dependent. Microsoft Office software requires developers to use Microsoft Visual Basic for Applications, or Visual Studio Tools for Microsoft Office (now in it s 2nd edition), both of which only support proprietary Microsoft Windows and Microsoft Office software. Alternatively, developers working with StarOffice or OpenOffice must rely on the Universal Network Objects (UNO) application programming interface, that is only available within those office application suite applications; e.g. Writer, Calc, Impress, but supported on multiple operating systems supporting multiple programming languages such as C++, Python, etc. However, none of these technologies interoperates well with independently developed third-party technologies, in the open standards sense of the Internet; e.g. HTML, CSS, DOM and JavaScript. The Document Object Model (DOM) used by all modern Web browser applications is a powerful way to functionally (not just visually) integrate various kinds of documents. It is also widely used in the context of Web applications on the server side in languages such as Java. Thus, it is one of the few interfaces known and understood by browser script-based programmers as well as by traditional programmers who use procedural languages such as Java. A new, simplified DOM-based programming model for ODF is emerging. It uses the ODF XML format but, more importantly, it uses a DOM as the document s run-time model. This means that it s now possible to dynamically control an ODF document using a variety of scripting and other languages. It is also possible to programmatically integrate run-time behavior of an ODF document with other DOM-based open-standard documents such as XForms and Scalable Vector Graphics (SVG). And all browser-based technologies such as Cascading Style Sheets (CSS) can be used for personalization or accessibility. Moreover, with a truly open format that has open access to document elements at all levels, accessibility itself becomes open and Figure 2: An ODF Spreadsheet Document unzipped. Novática UPGRADE Vol. VII, No. 6, December

11 Figure 3: Content.xml File of a Text Document viewed in the Mozilla Firefox Browser. programmable no longer constrained by a static realization of predetermined policies. This will truly enable ODF documents to participate in and contribute to a wide ecosystem of documents and deliver an enriched user experience obtained via easy, open composition of standard elements. 5.2 Document-centric Collaboration There is an emerging trend toward online rich documentbased collaboration. Google Docs and Spreadsheets, Zoho Writer, ajaxwrite and social networking and content management startups Zimbra, Socialtext, and Alfresco are moving in this direction. In the past, commercial document processing systems (e.g., Microsoft Office, IBM Lotus SmartSuite software) supported some forms of collaboration. Today, wikis and blogs are beginning to represent new approaches to collaboration on the so-called "Web 2.0" platform. However, wikis and blogs do not have a structured information model below them. Without this foundation, it is difficult to support content-based access control, history, versions, views and live collaboration. Coupled with the integration technology above, ODF s XML-based document model can unlock new paradigms in document-based collaboration on the Web. This easily allows multiple authors to interact in real time with the document and its information, allowing role-based access control, views, versions and history. Combine this concept with specific business templates for documents, spreadsheets and presentations, and the document lifecycle model evolves to one where interaction and collaboration over content or information (data) in the context of a business document are radically different. For example, authoring teams can easily come together and edit their documents in real time over the network using their preferred ODF editor(s) in any combination. Or teams can simply edit within the Web browser. To facilitate this editing, the ODF document is treated as a shared data model and is "rendered" into different forms: one used by the native ODF editor and the other into HTML for the rich text editor. Modifications to the content flow both ways and users can productively collaborate on the content freed from the Figure 4: Content.xml File of a Spreadsheet Document viewed in the Mozilla Firefox Browser. 10 UPGRADE Vol. VII, No. 6, December 2006 Novática

12 Feature OASIS standard Approved by ISO as ISO/IEC ISO standard Relax-NG schema types (ISO/IEC :2003) Benefit Open, transparent specification process with multi-vendor participation Well known and broadly accepted standard Well known and broadly accepted standard Supported by multiple applications Broad industry support Shipping products since September 2005 Free open source reference implementations ODF implementations available for all major desktop platforms Open standard W3C XForms technology is used for forms Reuse of existing standards where possible Choice between free, open-source and commercial implementations including OpenOffice.org, StarOffice, KOffice, IBM Workplace, Textmaker, Abiword/Gnumeric, Google Docs & Spreadsheet, and AjaxWrite. ODF guarantees long-term viability. The OASIS ODF TC, the OASIS ODF Adoption TC, and the ODF Alliance include members from Adobe, BBC, Bristol City Council, Bull, Corel, EDS, EMC, GNOME, IBM, Intel, KDE, MySQL AB, Novell, Oracle, Red Hat, Software AG, Sun Microsystems, and the City of Vienna. As of December 2006, the ODF Alliance already has more than 350 members. ODF files can already be created and used today. The first products with ODF support started shipping in September ODF is supported by multiple free, open-source office applications including OpenOffice.org, KOffice and Abiword/Gnumeric. OpenOffice.org, for example, is developed by a large community including vendors like Sun Microsystems, Novell, Intel, and Red Hat. Because the source code is available, anyone can add support for additional platforms. Applications with ODF support are available for Microsoft Windows, Linux, the Solaris OS, Apple Mac OS X, and FreeBSD. The forms concept integrated into ODF is based on the W3C XForms standard which is supported by multiple applications and vendors. In order to make interoperability as simple as possible, ODF reuses established standards such as XHTML, SVG, XSL, SMIL, XLink, XForms, MathML, and Dublin Core. Well established The first work for the ODF file format started as early as 1999 (see the ODF history in Table 1). Table 2: Benefits of ODF. Novática UPGRADE Vol. VII, No. 6, December

13 document format. This is possible because open standards are developed and specified with the help and contributions of multiple stakeholders in an open community. The openstandards process plays an important role. As standards are defined and evolve, developers increasingly recognize the opportunity for new marketplaces for these tools and run times. With this revolution in the open-document standards domain, industry leaders will pave the way for content-based collaboration across different types of users, editors and devices. This is the same phenomenon that accelerated the development of the Internet and its subsequent adoption in commerce and daily life. 5.3 Implications for Enterprise Document and Content Management Systems Today s enterprise document and content management solutions manage large repositories of all types of information documents, images and multimedia. Banking and insurance companies depend on these systems for missioncritical business processes such as claims processing and credit approval. Business information does not always appear in machine-readable structured forms; often, it exists as semistructured templates, such as claim documents and loan documents. Document information must be indexed to be efficiently searched. Indexing technology typically depends on the degree of metadata associated with the document, since search engines are challenged to crawl and retrieve meaningful information when the internals of a document or image are stored in opaque, binary data form. With the advent and anticipated wide-scale adoption of ODF, and the view of the future where document data will be stored in XML format, these systems will be much more effective in terms of their ability to programmatically index, query, search, retrieve and assemble through transformation operations into new compound documents. These new techniques and methods open up new horizons for developing business solutions distinctly set apart from yesterday s office application suite model. More significantly, they create the opportunity for the development of new software that will programmatically converge many different data sources into a new document, further automating business processes and creating new efficiencies. An open-standards format is critical because it enables the creation of relational or XQuery-type operators on a document; it also guarantees the document s semantics. For example, in an insurance scenario you could select all claim documents where the claim was about US$20,000 or, join a set of auto claims and home claims documents to create a document with claim amount, claim type and customer name; then assemble the new compound document on the fly. In fact, document management systems can provide mini-search/relationship mining engines and can suggest new links between projects or assets within the organization, and contribute to the overall efficiency of the enterprise. 6 The Future of the OpenDocument Standard It s important to note that ODF is currently in its firstversion form. As a de jure open standard, ongoing stewardship and development of the ODF specification continues at OASIS with many vendors and individuals from diverse organizations participating and providing leadership. Significant new work in three subcommittees concluded before the end of 2006 and moved through the open standards ballot and submission processes resulting in an update to the ISO/IEC 26300:2006 in the second half of Accessibility, formula and metadata extensions will update the ODF specification and continue to support ongoing creative innovation. Therefore, we are just seeing the first entry of the specification with much more to look forward to in the near future. An open standard, with multivendor stewardship in a bona fide open standards consortia ensures the technology will evolve and continue to provide value for years to come. 6.1 Twenty Things you Can Do with the OpenDocument Format Robert Weir, a member of the OASIS ODF Technical Committee enumerated a variety of usage patterns for ODF, demonstrating that it has wide applicability beyond the traditional heavy-weight office-like editors. We include it here to stimulate your imagination and continued curiosity. 1. Interactive creation in a heavy-weight client application. This is the traditional mode of operation in KOffice, OpenOffice.org, etc. 2. Interactive creation in a light-weight web-based application. We are starting to see this in Google Docs Spreadsheets. 3. Collaborative (multi-author) editing. This includes the traditional "comment and merge" style of collaboration as well as real-time, multi-user editing where multiple authors edit the same document at the same time. 4. Automatic creation of document in response to a database query. This is the report generation model of use. Data source could be a web service rather than a database. 5. Indexing/scanning of document for search engine. 6. Scanning by anti-virus software. 7. Other types of scanning, perhaps for regulatory compliance, legal or forensic purposes. 8. Validation of document, to specifications, house style guidelines, accessibility best practices, etc. This goes beyond RELAX NG validation, beyond Schematron, into content validation that is beyond XML structure. 9. Read-only display of document on machine without the full editor, for example a lightweight viewer as a browser plugin or extension. 10. Conversion of document from one editable format to another, i.e., convert ODF to OOXML. 11. Conversion of document into a presentation format, such as PDF, PS, print or fax. 12. Rendering of a document via other modes such as sound or video (speech synthesis). 13. Reduction/simplification of document to render on a sub-desktop device such as cell phone or PDA. 12 UPGRADE Vol. VII, No. 6, December 2006 Novática

14 14. Import of ODF into a non-office application, i.e., import of spreadsheet data into statistical analysis software. 15. Export from a non-office application into ODF, such as an export of a spreadsheet from a personal finance application. 16. An application which takes an existing document and outputs a modified version of that presentation, e.g., fills out a template, translates the language, etc. This has some nice benefits since it allows separation of concerns, where a business user can control the look of the document, but leave place holders that can be filled in by automation, perhaps based on a web service query. 17. Adds or verify digital signatures on a document in order to control access (DRM). 18. Software which uses documents as part of a workflow, but treats the document as a black box, or perhaps is aware of only basic metadata. This is the way most current systems work. 19. Software which treats documents as part of a workflow, but is able to inspect the document and make decisions based on the content. This relies on the transparency of the ODF format, and the ability of software to see what is inside. 20. Software which packs/unpacks a document into relational database form, i.e. XML-relational mapping. 7 Summary History has demonstrated that by adopting common standards, society achieves uncommon results. Standardization in electricity, train switches, emergency firefighting equipment and maritime applications have transformed our world. The Internet, based on broad participation and availability of the standard specifications, has opened peoples lives and created boundless opportunities for growth, exploration and innovation, creating new value far beyond what any single vendor is capable of. As reflected by this experience, open standards provide vital benefits in the areas of: collaborative innovation - where communities of organizations, governments, and individuals come together to address serious problems such as providing relief after natural disasters; flexibility - standards provide more technology options for citizens, users, and implementers to easily configure information systems, procure technology from a competitive marketplace, and more easily adapt to ever-changing requirments and procedures; interoperability - eliminating the barriers that inhibit communications and information sharing, within and across governments, especially in healthcare, public safety and education; cost effectiveness - where the adoption of policies in support of open standards avoids single vendor lock-in, increases competitive choice while lowering prices; and freedom of action - where users are empowered to benefit from a level playing field, lessening the risk that a single vendor can pace, control, or block technology. References [1] OASIS Accessibility Guidelines for Implementations of OpenDocument Format v1.1, < org/committees/download.php/20977/odf_ Accessibility_Guidelines_14_2Nov06.odt>. OASIS Open Document Format TC Homepage, < php?wg_abbrev=office>. OASIS OpenDocument Format Accessibility Sub Committee Homepage, < OASIS OpenDocument Format Meta Data Sub Committee Homepage, < OASIS OpenDocument Format Formula Sub Committee Homepage, < tc_home.php?wg_abbrev=office-formula>. OASIS ODF Adoption TC Homepage, < oasis-open.org/committees/tc_home.php? wg_abbrev=odf-adoption>. ODF Information Web Site, < xml.org/>. ODF Alliance Homepage, < about.php>. ODF Wikipedia Page, < OpenDocument>. OASIS OpenDocument Essentials (Online book), < /books.evc-cit.info/>. ODF Perl Module, < OpenOffice-OODoc/>. Abbreviations CJK = Chinese, Japanese, Korean CSS = Cascading Style Sheets DOM = Document Object Model DRM = Digital Rights Management DTD = Document Type Definition HTML = HyperText Markup Language IEC = International Electrotechnical Commission INdT = a research group belonging to Nokia ISO = International Organisation for Standardisation OASIS = Organization for the Advancement of Structured Information Standards ODF = OpenDocument Format OOXML = Object-Oriented Extensible Markup Language PDA = Personal Digital Assistant (an electronic device) PDF = Portable Document Format PS = PostScript PwD = Persons with Disabilities SC = Sub-Committee SMIL = Synchronised Multimedia Integration Language SQL = Structured Query Language SVG = Scalable Vector Graphics TC = Technical Committee UNO = Universal Network Objects XHTML = Extensible Hypertext Markup Language XML = Extensible Markup Language XSL = Extensible Stylesheet Language ZIP = a format for compressed files Novática UPGRADE Vol. VII, No. 6, December

15 Is OpenDocument an Open Standard? Yes! David A. Wheeler This paper shows that OpenDocument truly is an open standard. It identifies the most important definitions of the term "open standard", and merges their requirements to create a more complete definition of the term "open standard". It then shows that OpenDocument strongly meets all of their requirements. The paper particularly focuses on the issues of "No Discrimination" and "No Domination", requirements that some other specifications fail to meet but OpenDocument succeeds. David A. Wheeler, This article is released under the Creative Commons "Attribution-NonCommercial 2.0" license < It was previously published on Groklaw < and on the author s web site < Keywords: Discrimination, Domination, European Union, Interoperability, Krechmer, Open Document, Open Format, OpenDocument, Open Standard, Perens. 1 Introduction Ever since the OpenDocument standard was ratified by OASIS in May 2005, it s been gathering steam. In May 2006, ISO formally approved a draft of OpenDocument as standard ISO/IEC [1]. KOffice has completely switched to OpenDocument as their native format (OpenOffice.org did this long before), and more implementations are being announced each month. Massachusetts continues to plan to switch to it in spite of some nasty politics, and all evidence suggests increasing use worldwide. Wikipedia s article on OpenDocument [2] gives more information about it, including OpenDocument adoption [3]. But is OpenDocument really an open standard, or not? For example, can anyone implement it? Was its development process completely controlled by a single party (which would not be open), or is there evidence that it s a consensus result by many? It s generally accepted that OpenDocument is an open standard, but recently I ve been told that some people are claiming otherwise. So let s figure out what the criteria are for an open standard, and then see if OpenDocument meets those criteria. 2 What s an Open Standard? There s no single definition of the term "open standard". That s true for most words and phrases, actually. But lots of documents hint at what it means, for example: Europe s "Valoris report" [4] defines an open standard this way: "The minimum requirements for an open standard are that the document format is completely described in publicly accessible documents, that this description may be distributed freely and that the document format may be implemented in programs without restrictions, royalty-free, and with no legal bindings." In an earlier paper [5] I noted that, "When you think open, think open competition or open market. An open Author David A. Wheeler has long worked to improve software and is an expert in computer security, open source software / Free Software, and open standards. His publications include the book "Secure Programming for Linux and Unix HOWTO", the developerworks series "Secure Programmer", "Countering Trusting Trust through Diverse Double-Compiling (DDC)" (ACSAC), the book "Software Inspection: An Industry Best Practice" (IEEE Computer Society Press), and "Why OSS/FS? Look at the Numbers!". Mr. Wheeler was so impressed with OpenDocument that after OASIS completed OpenDocument 1.0, he joined OASIS and now leads the OpenDocument formula subcommittee. His personal website is < dwheeler.com>. <dwheeler@dwheeler.com>. standard should allow competition based on merit, instead of limiting customers suppliers to one particular supplier or a subset of suppliers based on their business model, development model, licensing model, and so on. You should be able to replace one product that does the same function for another, as long as they meet the same open standard, and achieve at least the same basic function provided by the standard (though some may perform better or have additional features think of plug-replaceable components).". Let s first look at the two definitions of "open standard" that seem to be the most widely used. The first is by Bruce Perens; the second is by Ken Krechmer (Fellow of the International Center for Standards Research). These two are so widely used that when I did a Google search on "open standards" these were the second and fourth results respectively (the first and third were OASIS and the W3C, two standards bodies that create open standards). We ll then look at the European Commission s definition of open standards, which is a formally approved definition of the term (and one that European governments use). Then, after we ve looked at these three definitions, we ll create a merged definition that includes all of their requirements (from all three sources). That way, if the specifica- 14 UPGRADE Vol. VII, No. 6, December 2006 Novática

16 tion meets this merged set of requirements, we can be very confident that we have an open standard; such a specification would meet all three definitions. 2.1 Perens A very popular definition of the term "open standards" (according to Google the most popular ) is Bruce Perens "Open Standards: Principles and Practice" [6]. You re best off reading the actual paper for its full content, of course. Let me summarize it by quoting its list of principles that it states a specification must meet to be an open standard: 1. Availability: Open Standards are available for all to read and implement. 2. Maximize End-User Choice: Open Standards create a fair, competitive market for implementations of the standard. They do not lock the customer in to a particular vendor or group. 3. No Royalty: Open Standards are free for all to implement, with no royalty or fee. Certification of compliance by the standards organization may involve a fee. 4. No Discrimination: Open Standards and the organizations that administer them do not favor one implementor over another for any reason other than the technical standards compliance of a vendor s implementation. Certification organizations must provide a path for low and zerocost implementations to be validated, but may also provide enhanced certification services. 5. Extension or Subset: Implementations of Open Standards may be extended, or offered in subset form. However, certification organizations may decline to certify subset implementations, and may place requirements upon extensions (see "Predatory Practices"). 6. Predatory Practices: Open Standards may employ license terms that protect against subversion of the standard by embrace-and-extend tactics. The licenses attached to the standard may require the publication of reference information for extensions, and a license for all others to create, distribute, and sell software that is compatible with the extensions. An Open Standard may not otherwise prohibit extensions. 2.2 Krechmer Another popular definition is the set of requirements for open standards created by Ken Krechmer, Fellow of the International Center for Standards Research (University of Colorado) [7]. He s published several versions; here I ll summarize the February 7, 2005 version of "Open Standards Requirements". He looked at standards from the viewpoint of recognized standards-setting organizations, implementors, and users, and tried to find some middle ground merging their desires. He claims that an open standard must meet the following requirements: 1. Open Meeting - all may participate in the standards development process. 2. Consensus - all interests are discussed and agreement found, no domination. 3. Due Process - balloting and an appeals process may be used to find resolution. 4. Open Intellectual Property Rights (IPR) - how holders of IPR related to the standard make available their IPR. 5. One World - same standard for the same capability, world-wide. 6. Open Change - all changes are presented and agreed in a forum supporting the five requirements above. 7. Open Documents - committee drafts and completed standards documents are easily available for implementation and use. 8. Open Interface - supports proprietary advantage (implementation); each interface is not hidden or controlled (implementation); each interface of the implementation supports migration (use). 9. Open Access - objective conformance mechanisms for implementation testing and user evaluation. 10. On-going Support - standards are supported until user interest ceases rather than when implementer interest declines Clearly, these definitions have a lot in common. Ken Krechmer wrote his paper after Perens, and compares his list to Perens. Krechmer maps each of Perens 6 points to his own list of ten as shown in Table 1. Krechmer in particular notes that Perens doesn t include any requirement about wanting "One World" or ongoing support. Rick Jelliffe claims that Krechmer s "open interface" doesn t map to Perens "ability to create extension or subset" [8], but I think it does... and so does Krechmer! However, Krechmer s list has very serious failure. Perens Krechmer Availability Open Documents Maximum end-user choice Open Access No royalty Open IPR No discrimination Open Meeting, Consensus and Due Process Ability to create extension or subset Open Interface Ability to prevent predatory practices Open Change Table 1: Comparison between Krechmers and Perens Requirements according to Krechmer. Novática UPGRADE Vol. VII, No. 6, December

17 Krechmer s list omits one of the most important factors of all: the ability of anyone to implement the standard. The whole point of open standards is to allow anyone to implement the standard, and to allow any user to have unfettered selection and switching between many implementations. Krechmer s list notes the importance of patents and copyright ("IPR"), but his definition allows "open standards" to forbid competitors from implementing the standard. This is a fundamental flaw in his definition; defining "open standard" as "a standard that some competitors are forbidden from implementing" is nonsense, and conflicts with most other sources. Perens definition explicitly forbids this, as does the Valoris report [4], which required as a minimum that the format "may be implemented in programs without restrictions, royalty-free, and with no legal bindings." It also conflicts with the European Commission s definition of open standards, which also required royalty-free use (we ll get to that definition in a moment). The most economically obvious example of this conflict is open source software (OSS). Today, in a vast number of software markets, the dominant or #2 program is OSS, including web servers, web browsers, mail servers, and DNS servers. Yet OSS are legally forbidden from using royaltybearing patented works, so obviously specifications requiring royalty-bearing patents or other legal restrictions preventing OSS or proprietary implementations are obviously not open standards. A standard cannot be "open" if it is illegal for the dominant or #2 supplier to implement it. This conflict between patents and open standards only makes sense, when you think about it. The purpose of patents are to prevent competition, while the purpose of open standards is to enable open competition. The purposes of patents and open standards are fundamentally in conflict. Open standards are not the same as open source software; you can choose open standards and use only proprietary software. But selecting open standards lets you choose between implementations, including OSS, and lets you switch to another implementation later. 2.3 European Commission The European Commission (EC) has defined the term "open standards" as part of the final version 1.0 of the European Interoperability Framework [9]. Newsforge ran a short article about it [10]. The EC declared that "to attain interoperability in the context of pan-european egovernment services, guidance needs to focus on open standards", - in other words, the EC views the use of open standards as a significant policy issue. They define the following as "the minimal characteristics that a specification and its attendant documents must have in order to be considered an open standard", and I quote them here: The standard is adopted and will be maintained by a not-for-profit organisation, and its ongoing development occurs on the basis of an open decision-making procedure available to all interested parties (consensus or majority decision etc.). The standard has been published and the standard specification document is available either freely or at a nominal charge. It must be permissible to all to copy, distribute and use it for no fee or at a nominal fee. The intellectual property i.e. patents possibly present of (parts of) the standard is made irrevocably available on a royalty-free basis. There are no constraints on the re-use of the standard. The document also suggests that open source software (OSS) compliments open standards. It says: "Open Source Software (OSS) tends to use and help define open standards and publicly available specifications. OSS products are, by their nature, publicly available specifications, and the availability of their source code promotes open, democratic debate around the specifications, making them both more robust and interoperable. As such, OSS corresponds to the objectives of this Framework and should be assessed and considered favourably alongside proprietary alternatives." Both the definition and explanatory text make it clear that the intent was that any open standard must be implementable by both OSS and proprietary programs, especially given the requirements for royalty-free use and lack of constraints on re-use. 2.4 Merging the Definitions So let s use a definition of "open standard" merging the best of each, and so that a specification would meet all of these definitions to qualify. Comparing Krechmer s list to Perens, Perens list is shorter, clearer, and doesn t have the serious defect of forbidding open competition. We ll then add the two points of "One World" and "Ongoing support" stated by Krechmer as important issues. Most of the EC s requirements also map well to Perens, except that the requirement for a free or nominal-cost specification isn t explicit (see Table 2). So again, let s add "No or nominal cost for specification". Perens doesn t explicitly say that a standards maintainer has to be not-for-profit, but he does require no discrimination, with essentially the same notion; we ll add that as an implicit requirement to no discrimination. The result is a slightly stricter definition of "open standard" than any of the three definitions by themselves. Thus, so if any specification meets this merged definition, then it s clearly an open standard. 3. Definition of "Open Standard" Here s a definition that merges the points of these various widely-cited documents: An open standard is a specification that enables users to freely choose and switch between suppliers, creating a free and open competition between suppliers. To accomplish this, an open standard must have the following properties: 1. Availability: Open Standards are available for all to read and implement. 2. Maximize End-User Choice: Open Standards create a fair, competitive market for implementations of the standard. They do not lock the customer in to a particular vendor or group. 16 UPGRADE Vol. VII, No. 6, December 2006 Novática

18 Perens No discrimination (though doesn t explicitly require not-for-profit) (No match) No Royalty Availability Table 2: Mapping of Perens and EC s Requirements. European Commission The standard is adopted and will be maintained by a not-forprofit organisation, and its ongoing development occurs on the basis of an open decision-making procedure available to all interested parties (consensus or majority decision etc.). The standard has been published and the standard specification document is available either freely or at a nominal charge. It must be permissible to all to copy, distribute and use it for no fee or at a nominal fee. The intellectual property -- i.e. patents possibly present -- of (parts of) the standard is made irrevocably available on a royalty-free basis. There are no constraints on the re-use of the standard. 3. No Royalty: Open Standards are irrevocably free for all to implement, with no royalty or fee {or other legal tricks to inhibit implementation}. Certification of compliance by the standards organization may involve a fee (as long as it s not required for implementation). 4. No Discrimination: Open Standards and the organizations that administer them do not favor one implementor over another for any reason other than the technical standards compliance of a vendor s implementation. Certification organizations must provide a path for low and zerocost implementations to be validated, but may also provide enhanced certification services. The standard is adopted and maintained by a not-for-profit organisation, with open meetings (all interested parties may participate in the standards development and maintenance process), consensus (all interests are discussed and agreement found, with no domination by any party), and an open decision-making process with due process (balloting and an appeals process may be used to find resolution). 5. Extension or Subset: Implementations of Open Standards may be extended, or offered in subset form. However, certification organizations may decline to certify subset implementations, and may place requirements upon extensions (see Predatory Practices). I would add that users definitely should be told when an implementation only implements a subset, and what the nonstandard extensions of an implementation are, since these can cause interoperability problems. But it is critical that implementations be allowed to implement subsets or supersets of standards, because without this permission, standards cannot respond to changing conditions, and many open source software licenses are incompatible with software that cannot be modified. In practice, useful standards are updated, based on experience based on real implementations that create subsets and supersets (XML is a subset of SGML, for example, and all useful updated standards are based on extensions created by implementors). There are lots of ways to address the needs of both users (who need to know about the subsets and supersets) and implementors (who need to create them). For example, a standards body might say that an implementor of only a subset would have to say "implements a subset of standard X" instead of "implements standard X", and they can require implementors to document their extensions. The biggest danger here are intellectual rights (patent) agreements that grant patent rights to implementors "only to implement this standard, exactly". These agreements are often a trap; the organization granting the intellectual rights then becomes the only organization that can make improvements on the standard, and can entrap everyone else who uses the standard. 6. Protection from Predatory Practices: Open Standards may employ license terms that protect against subversion of the standard by embrace-and-extend tactics. The licenses attached to the standard may require the publication of reference information for extensions, and a license for all others to create, distribute, and sell software that is compatible with the extensions. An Open Standard may not otherwise prohibit extensions. 7. One World: The same standard should be applicable for the same capability, world-wide. It must not be devised as a "barrier to entry" by those from other regions. 8. On-going Support: The standard is supportable until user interest ceases rather than when implementer interest declines. 9. No or nominal cost for specification. (This may soon become a requirement for no-cost specifications that can be copied further. Fees innately discriminate against many users and implementors, particularly in the third world; with the rise of the Internet fees have become a completely unnecessary discrimination.) 4 Is OpenDocument an Open Standard? 4.1 Walking through the Requirements Novática UPGRADE Vol. VII, No. 6, December

19 So, is OpenDocument an open standard? Let s walk through the list of requirements. 1. Availability: Open Standards are available for all to read and implement. This one is easy. You can download the OpenDocument specification for free from OASIS [11]. Absolutely anyone can implement OpenDocument; if you look at the IPR statements, they clearly state that absolutely anyone can implement it, and there are no limits at all. 2. Maximize End-User Choice: Open Standards create a fair, competitive market for implementations of the standard. They do not lock the customer in to a particular vendor or group. The fundamental question here is, is this designed to permit multiple interoperable implementations? The most obvious way to determine if this is true is to determine if there multiple implementations available on multiple different platforms. The answer to the latter question is emphatically yes; there are already multiple implementations of OpenDocument, with many more in progress. OpenOffice.org/StarOffice and KOffice are office suite applications that support reading and writing OpenDocument, and are completely independent implementations. There are also many specialized products that implement their relevant portions, for example, AbiWord and Writely are word processors that can read and write the word processing portion of OpenDocument; Gnumeric is a spreadsheet application whose implementation of the spreadsheet portion of OpenDocument is in development. (Wikipedia has a list of applications supporting OpenDocument [12]). You can also look at other evidence to help gain more confidence in this: a) Were multiple implementors involved in its specification to help prevent lock-in? Yes; Sun, KDE, Corel (vendor of Word Perfect), IBM (vendor of IBM Workplace and Lotus SmartSuite), and others. I ll discuss more about the participants later, but that s enough to prove the point. b) Does the specification maximally reuse other open standards (otherwise it may end up creating unnecessary dependencies on nonstandard components)? Yes; other standards OpenDocument reuses include SVG, SMIL, XSL, XForms, MathML, XLink and the Dublin Core Meta Initiative. 3. No Royalty: Open Standards are free for all to implement, with no royalty or fee. Certification of compliance by the standards organization may involve a fee. There s no royalty or fee to implement OpenDocument, so this is met 100%. 4. No Discrimination: Open Standards and the organizations that administer them do not favor one implementor over another for any reason other than the technical standards compliance of a vendor s implementation. Certification organizations must provide a path for low and zerocost implementations to be validated, but may also provide enhanced certification services. I believe this one is also a clear "yes", however, it s more difficult to measure this than most other points. We ll explore this one further in a moment, to see why I believe this is "yes" as well. 5. Extension or Subset: Implementations of Open Standards may be extended, or offered in subset form. However, certification organizations may decline to certify subset implementations, and may place requirements upon extensions (see Predatory Practices). Implementors may implement subsets or supersets of OpenDocument, without any special requirements, so this is clearly met too. 6. Predatory Practices: Open Standards may employ license terms that protect against subversion of the standard by embrace-and-extend tactics. The licenses attached to the standard may require the publication of reference information for extensions, and a license for all others to create, distribute, and sell software that is compatible with the extensions. An Open Standard may not otherwise prohibit extensions. The OpenDocument developers chose to not embed any protective measures against subversion by external parties. However, all the members of the technical committee that created the standard were required to grant royaltyfree licenses to implement it. This prevented anyone from quietly inserting a requirement in the standard and after ratification announce that they would extort payments from implementors. Thus, this requirement is easily met. 7. One World: Same standard for the same capability, world-wide. The OpenDocument specification was specifically designed to be used worldwide, and not limited to any region. It includes capabilities to support arbitrary locales and arbitrary languages, and in fact was developed by representatives from many different countries. For example, OpenDocument supports Unicode/ISO characters (whose purpose is to support the characters of all languages), Ruby text (important in supporting some Asian languages), and text written from right to left (important in supporting Arabic and Hebrew). Reusing many open standards helps this, too. Also, by working to avoid patented techniques, the developers of OpenDocument made it more likely that everyone in the world could use the standard (otherwise the standard might only be usable in countries that don t tolerate software patents). We ll discuss more about patents in a moment, because patents fundamentally conflict with the previously-noted requirement for no discrimination. 8. On-going Support: Standards are supported until user interest ceases rather than when implementer interest declines. The OpenDocument standard is supported by OA- SIS, a standards body, rather than any one particular vendor. Thus, as long as there are users who wish to support the standard, they can work with OASIS to continue its support. There s absolutely no indication that this is a problem anyway; there is a massive amount of interest in OpenDocument. <LINo or nominal cost for specification. OASIS posts the specification at no charge on their website, clearly meeting this requirement. In short, we get "yes" answers for all of these points. All but one of them are trivially answered as "yes". One point, however, is harder to measure - the "No discrimination" point. This isn t because there s a problem with OpenDocument on this point; the challenge is that "no discrimination" is harder to measure for any standard. So let s drill into the "no discrimination" point to see if OpenDocument meets this requirement if it does, then it is clearly and unambiguously an open standard. 18 UPGRADE Vol. VII, No. 6, December 2006 Novática

20 4.2 No Discrimination Perens requires that open standards have "no discrimination", that is, that "Open Standards and the organizations that administer them do not favor one implementor over another for any reason other than the technical standards compliance of a vendor s implementation. Certification organizations must provide a path for low and zero-cost implementations to be validated, but may also provide enhanced certification services.". There s no certification issue, so we don t have to deal with that. The European Commission had an explicit requirement that the administering organization must be a not-for-profit, which is easy to show in this case: OASIS is a non-profit! But there s more to preventing discrimination than simply creating a specification via a not-for-profit organization. What about the whole first part of Perens requirements? Krechmer maps this single requirement to three requirements: Open Meeting - all may participate in the standards development process. Consensus - all interests are discussed and agreement found, no domination. Due Process - balloting and an appeals process may be used to find resolution. We can deal with "due process" easily enough. OpenDocument was developed in OASIS, which has clear balloting and appeals process, so that s clearly met. The "open meeting" requirement that "all may participate in the standards development" is a little more interesting, but it seems to be met, too: Note that this does not mean that everyone has to participate sometimes organizations who should participate don t do so. In 2004, Europe formally asked in very strong language that Microsoft participate in OpenDocument development; while it was an observer on the technical committee for a long while, and is part of OASIS, Microsoft never chose to become actively involved with this standard. The definition doesn t require that all parties get involved, merely that they are given the opportunity. Like many old standards bodies, OASIS typically requires payments from organizations interested in developing the standard. The problem is that some people who have a legitimate interest in a standard may not be part of an organization willing to make these payments. If there were no recourse, this would cause the OASIS group to fail to meet this requirement. However, OASIS has worked hard to make sure that there are mechanisms to allow such organizations to be represented through non-profit organizations; as a result, this requirement is met too. But that leaves us a mixture of requirements: "do not favor one implementor over another," and "Consensus - all interests are discussed and agreement found, no domination." If there s a single vendor who controls all real decisions, then clearly we have a problem. Thankfully, I think we have good evidence that there wasn t domination in the case of OpenDocument. If there was no domination, then OpenDocument is an open standard without question. So let s look at the evidence, shall we? 4.3 No Domination How can you tell if there s domination by a vendor in a standards body? There are several signs that if present give strong evidence of vendor domination. For example, vendor domination is very likely if the rules or processes controlling the standards development strongly limit the range of technical changes, forbid changes that would affect only one particular vendor, or give one particular vendor a sole veto power. The OpenDocument development process had no such problem; the rules in placed allowed anyone to propose changes, even if they forced any or all vendors to change their implementations. But there can be unstated rules that effectively limit others participation, even if there s no obvious written rule enforcing it. You could also check to see if other implementors involved in the process are complaining about the process locking them out, though that isn t always a valid indicator. In this case, there seems to be no such problem. IBM s Bob Sutor reported that, "IBM and Sun are working together happily and effectively on the OpenDocument Format. I think we ve made a terrific amount of progress in the last year and that s because of the broad cooperation by the community." In fact, Andy Updegrove [13] reported that it s central to Sun and IBM s strategy "to have many applications that support ODF. Remember - it s a good thing, and not a bad thing, for there to be many different implementations - both proprietary and open source - so long as they all support ODF. That s one of the big reasons why ODF matters - to have multiple choices (and not just one - Office), each with its own independently valuable features." What we need is direct evidence that there was no domination in the development of OpenDocument. One easy way is to see if there s only one party making essentially all the changes, or if in fact others are making proposals that cause technical changes to the specification that affect implementors. Particularly telling are changes that cause all implementors to make significant changes to their products. If all implementors are changing their product to meet the specification, then clearly no one implementor is calling all the shots. So let s start by looking at who proposed the original base document, and who proposed changes that were accepted. The original base document was contributed by Sun and the OpenOffice.org group, so clearly they were involved. Their base document was based on actual experience of using the format as their primary format, which is absolutely perfect... here we have a base document based on actual experience. But did Sun and OpenOffice.org control everything, or were changes made to the specification by others? Sun and OpenOffice.org did contribute the original document, as well as some later changes, but I asked a number of TC members and they easily showed that many other organizations made substantive changes to the specification. In some cases Novática UPGRADE Vol. VII, No. 6, December

21 it s hard to tell who was the proposer or proponent, but there s more than enough evidence to show that many others were involved. Even in the cases were I have not identified the proposer or proponent, it s obvious that changes were made that caused implementors to change their implementations. A long list of examples of the many changes made to the original base document contributed by Sun/ OpenOffice.org on the way to its becoming OpenDocument is shown in Table 3. You don t need to read this information in detail; the very point is that it s a long set. Anyway, here s a table showing some of the changes: I am sure that there are many other changes not listed above, but I think I have enough listed here to make the point: this specification was a work of many, not a specification controlled by any one vendor. Various other changes were made to the document to improve it as a document, too. A big change was that the schema definition language changed to RELAX-NG, a standard specification language which is more powerful than the older DTD language yet is very easy to understand. One advantage of this additional power is that automated validation tests of OpenDocument files using the specification can be much more rigorous. It s also more likely to result in interoperability; RELAX-NG can be much more specific about the permitted values in constructs, eliminating many potential misunderstandings by implementors. Clearly, there were many changes made after the initial submission of the base document, due to interaction of all the members. The first draft of the original specification was about 108 pages and its schema was 69K. The final version is about 723 pages and its schema is 529K. This growth to a full standard happened through careful review by a large number of experts from different backgrounds. A risk of making many changes to a specification is that the result may be too hard to implement. The OpenDocument process avoided this problem by having implementors actually implement the changes as they went, and in particular used multiple open source reference implementations as proving grounds. One participant commented, "it was clear that the proprietary [vendors] had gone into the source code or at the least downloaded [OpenOffice.org], and were studying the implementation and methods.... Explanations, rationale, and techniques were exchanged in ways that would be impossible if not for a common reference application and code base everyone had access to." 4.4 Another Tack: Look at the Participants I stated above that the most important measure was to see if there were changes made from many different sources, and I think I ve proven that very conclusively. But for some additional evidence, let s look at the participants themselves do we see the multiple implementors, multiple users, and diversity of views that we d expect to see from a standard not dominated by any one organization? Once again, I think the answer is "yes". The OASIS website gives lots of details; Wikipedia summarizes the people involved in creating the OpenDocument standard [14]. First, a quick background. Version 1.0 of the OpenDocument specification was developed after lengthy development and discussion by multiple organizations. The first official OASIS meeting to discuss the standard was ; OASIS approved OpenDocument as an OA- SIS standard on This is about two and a half years of hard work; it s not easy to create a good specification. The standardization process included the developers of many office suites or related document systems, including (in alphabetical order): Adobe (Framemaker, Distiller). Arbortext (Arbortext Enterprise Publishing System). Corel (WordPerfect). IBM (Lotus 1-2-3, Workplace). KDE (KOffice). SpeedLegal (SmartPrecedent enterprise document assembly system); both product and company later changed names to Exari. Sun Microsystems / OpenOffice.org (StarOffice/ OpenOffice.org). That s a lot of implementors, some of whom are direct competitors, and they represent a very broad range of concerns. That s a very good sign, when competing implementors work together on a standard. Another good sign is that there were users with specific issues who were involved: Boeing (complex large documents). CSW Informatics. Drake Certivo. Intel (complex large documents; they are developing sample documents as a test suite). National Archives of Australia (retrieve documents long after development). New York State Office of the Attorney General (complex large documents retrieved long after development). Novell. Society of Biblical Literature (large multilingual documents, long-term retrieval). Sony. Stellent. My personal favorite is the "Society of Biblical Literature", because they re so unexpected I would never have thought to invite them! Yet this group worries about dealing with large multilingual documents in any living language as well as many long-dead ones, and they worry about long-term retrieval issues in terms of millenia. The National Archives of Australia was represented, too. If their needs are met for internationalization and long-term retrieval, then my needs will be met too. Another way to show that they really had diversity is to see the different goals of different groups all getting met. Michael Brauer, chair, insists that the only objective of the group was to support "Desktop Office Suite work". Others, such as Gary Edwards, believe OpenDocument really cov- 20 UPGRADE Vol. VII, No. 6, December 2006 Novática

22 Change Modified to allow multiple metadata fields. The original specification did not allow list, e.g., there could only be one author for a document. At the first face to face meeting, strongly urged that this be changed, and the first OASIS draft included this change. Added SVG to support vector graphics using an already-existing standard, and worked to resolve issues involving SVG use. Set of requirements for business charts. Embedded support for a subset of the XForms standard was added. XForms add the capabilities typically desired for custom schemas, which were requested by Europe, but without the horrific interoperability problems that uncontrolled custom schemas can cause. Gary Edwards reports, It turns out that XForms is an elegant solution to the custom defined schema problem, able to address both the collaborate with yourself... model, and, the infinitely more important shared business process schema model. The binding model in XForms is extraordinary... we were quite unaware of this potential [at first]... [but once we began to understand it] oh what a treasure we found. In particular, its approach manages to be portable among many different systems, and not tied to any one. XForms is also a standard in its own right, which is a good thing; it s generally a very good sign if a standard tries to reuse existing standards instead of recreating its own incompatible components. Attribute fo:margin was added; this improved margin handling. Numbered paragraphs/headings without number and the text:numbered-paragraph element (an alternative to <text:list>) added more flexibility, for example, numbered-paragraph is better suited for independently numbered paragraphs. Table templates were added; this makes it easier to control and modify the appearance of tables. A sequence of page styles and support for copyframes was added, a new wrapping mode was added for graphics, and a desktop publishing mode was added (using the draw:page element). All of this improved support for desktop-publishing-based layout. Hyphenation became a boolean character property, Proposer/Proponent Patrick Durusau (Society of Biblical Literature) Paul Langille (Corel) Paul Langille (Corel) European Commission requested support for custom schemas ; Daniel Vogelheim (Sun) recommended adding XForms. David Faure (KDE) David Faure (KDE) David Faure (KDE) David Faure (KDE) David Faure (KDE) Novática UPGRADE Vol. VII, No. 6, December

23 Diagonal lines in table cells were added, providing another helpful option. Added a line style for the footnote separator. Added a number:denominator for fractions with a fixed denominator. Added a new date formatting option (number: month number: possessive-form= true ), Added draw:regular-polygon, improving the shapedrawing capabilities. Added more document statistics (sentence-count and syllable-count). The original specification s ordered-list and unorderedlists were replaced by a single list element, with ordered/unordered information conveyed entirely through styles. This made the specification simpler, and also made it easier to switch a list from one form to another. A per-paragraph list wrapper element Support for the SMIL standard was added as well, even though Sun and OpenOffice.org s implementations didn t support it. These aren t the sort of things you add if you just wanted a rubber stamp. OpenDocument changed the older OpenOffice.org format in how style:properties were handled. Originally there was a single properties element, which contained a mix of all style properties. OpenDocument has properties per style family (paragraph-properties, etc.), which allows for a cleaner separation of properties and better handles dependencies within style families A style:display-name attribute was added and a rule that style names had to be XML tokens was added. The structure of documents was changed so that the type of document (word processing, spreadsheet, etc.) could be trivially determined without looking at the filename ending. This wasn t in the original base document, and required a structural change, but it was added. The original format had only footnotes and endnotes; this was generalized to notes with a general note-class value, which makes it much easier to support additional type of notes. The original format had several different frame-like elements (e.g. images or text-boxes) that shared certain attributes. This was generalized to support a single draw:frame element which would then contain the actual frame content (textbox, image, etc.) plus an optional replacement image. This eliminated unnecessary David Faure (KDE) David Faure (KDE) David Faure (KDE) David Faure (KDE) David Faure (KDE) David Faure (KDE) - Phil Boutros (Stellent) - Proposed by Phil Boutros (Stellent) Supporters included Phil Boutros (Stellent) and Paul Grosso (ArborText). Phil (Stellent) recommended this UPGRADE Vol. VII, No. 6, December 2006 Novática

24 The original format had several different frame-like elements (e.g. images or text-boxes) that shared certain attributes. This was generalized to support a single draw:frame element which would then contain the actual frame content (textbox, image, etc.) plus an optional replacement image. This eliminated unnecessary complexity (by eliminating duplication) and would make it much simpler to implement consistent user interfaces. Frames were changed so that they could list a sequence of different format options (e.g., a vector drawing and a bitmapped drawing) so the viewer/processor can choose the best format that it can support. OpenDocument formulas now use a namespace prefix to identify the formula language used. This makes it much less likely that the wrong meaning would be applied to a given formula. Lots of other small changes. The style:default-outlinelevel attribute for heading styles was added, making it easier for users to use templates. The original style:properties was refined into style:paragraphproperties, style:text-properties, and so on. Also, pagemaster was renamed to page-layout for clarity (and to avoid mixing with master-page) (David Faure thinks that he commented that the name page-master was confusing, and that Michael Brauer (Sun) then offered to rename it.) Table 3: A Long List of Changes Made to the Original Base Document Contributed by Sun/OpenOffice.org. ers three areas: Desktop Productivity Environment: desktop file format. Service-oriented architecture (SOA): universal transformation layer bridging desktops into SOA XML information flows. Open Internet: ODF as the successor to HTML/ XHTML language. Gary Edwards has claimed that Boeing was actually primarily interested in the second area (SOAs), for example. Interestingly enough, there s no need to declare which set of goals is "right". Different organizations joined the group because they had different goals, unsurprisingly, yet in the end all believed their (different) goals had been achieved. That is very good news! A standard that can meet the different goals of different organizations tends to be a good standard as long as it s implementable... and since it s been implemented, there s no question there. Also, the fact that Boeing wanted the specification to be so good that it could be a "universal transformation layer" is excellent evidence of OpenDocument s capabilities. To fulfill this role, OpenDocument has to be so expressive that it s able to capture the information from many different document formats, not just Microsoft Office or any other single format. The result is a more general and flexible standard. Although it s not part of the definition of "open standard" I can t help but be impressed by the expertise of many of the participants. Tom Magliery (Corel/XMetal) was on the original W3C XML 1.0 working group (as well as the original NCSA Mosaic browser team). Phil Boutros (Stellant) is an expert on Microsoft s file formats. Paul Grosso (founder, ArborText) was chairman of the XSL:FO effort. Doug Alberg (Boeing) well-understood the needs for enterprise publication and content management software systems. Patrick Durusau is a well-known SGML expert. And so on. This is why it s a good idea to create standards through an open process by gathering the experts together, in a way that lets them truly fix problems they find, a group of experts can develop a very good result. 5 Conclusions Without doubt, OpenDocument is an open standard. It meets every requirement of a rigorous (merged) definition of "open standard," with lots of evidence. In particular, there is significant evidence that it was not controlled by any single vendor. And that s a good thing. When I send a pure text file, nobody asks "did you create that using WordPad, vim, or emacs?" When I send a bitmapped PNG file, nobody asks "did you create that using the GIMP or Paint Shop Pro?" Why? Because it doesn t matter. Data formats that are fully open standards can be implemented and understood by all. By using them, I m free to use whatever tool I want. Novática UPGRADE Vol. VII, No. 6, December

25 Governments have started to realize how important open standards are: 1. EU Commissioner Erkki Liikanen said, "Open standards are important to help create interoperable and affordable solutions for everybody. They also promote competition by setting up a technical playing field that is level to all market players. This means lower costs for enterprises and, ultimately, the consumer." 2. Europe s Telematics between Administrations Committee (TAC) said: "Because of its specific role in society, the public sector must avoid that a specific product is forced on anyone interacting with it electronically. Conversely, any document format that does not discriminate against market actors and that can be implemented across platforms should be encouraged. Likewise, the public sector should avoid any format that does not safeguard equal opportunities to market actors to implement format-processing applications, especially where this might impose product selection on the side of citizens or businesses. In this respect standardization initiatives will ensure not only a fair and competitive market but will also help safeguard the interoperability of implementing solutions whilst preserving competition and innovation." [15]. In short, open standards create freedom from control by another. That s a freedom we should all experience. sid=04/11/19/148236>. [11] OASIS. < [12] Wikipedia. OpenDocument software. < wikipedia.org/wiki/list_of_applications_ supporting_ OpenDocument>. [13] Andy Updegrove s blog. < org/standardsblog/article.php? story= >. [14] Wikipedia. OpenDocument standardization. < wikipedia.org/wiki/standardization_of_ OpenDocument>. [15 IDABC Programme. TAC approval on conclusions and recommendations on open document formats. < europa.eu.int/idabc/en/document/2592/5588>. Acknowledgments My thanks to the many people who helped me find the information in this paper, including Daniel Carrera, Patrick Durusau, Gary Edwards, J. David Eisenberg, David Faure, and Daniel Vogelheim. References [1] Andy Updegrove. OpenDocument Approved by ISO/ IEC Members. < standardsblogarticle.php?story= >. [2] Wikipedia. OpenDocument < wiki/opendocument>. [3] Wikipedia. OpenDocument adoption < wikipedia.org/wiki/opendocument_adoption>. [4] Valoris. Comparative Assessment of Open Documents Formats. Market Overview. < servlets/doc?id=17982>. [5] David A. Wheeler. Answering Microsoft: Comments on Microsoft s Letter to MA. < article.php?story= >. [6] Bruce Perens. Open Standards: Principles and Practice. < html>. [7] Ken Krechmer. The Meaning of Open Standards. < [8] Rick Jelliffe. A little freaked out by ODF s definition of Open Standard. < blog/2006/09/freaked_out_by_odfs_definition.html>. [9] Comisión Europea. The European Interoperability Framework for pan-european egovernment Services. < [10] Rishab Aiyer Ghosh. EC announces Open Standards Definition. < 24 UPGRADE Vol. VII, No. 6, December 2006 Novática

26 OpenDocument Hidden Traps and their Side Effects on Free/Open Source Software Marco Fioretti OpenDocument is widely considered, especially within the Free/Open Source Software (FOSS) community, one of the most important tools for the promotion of FOSS itself and for a truly free Information Technology market. OpenDocument is also rightly considered essential, as far as IT is concerned, for the construction of a more free society and culture. The very nature of OpenDocument, however, is not enough to overcome some obstacles to those goals and, in any case, is very likely to have a deep and sometimes unexpected effect on the future of FOSS. This article introduces these obstacles and side effects and, when they create actual problems, briefly describes the best kind of solution to deal with them. Keywords: Certification, Data Ownership, Free Software, Long Term Archiving, OpenDocument, Open Source, Open Standards, Public Administrations. 1 The Advantages of OpenDocument A highly structured, metadata rich, application independent XML (extensible Markup Language) file format like OpenDocument can finally offer two huge advantages to all computer users and to Society as a whole. The first is complete interoperability among many software applications, regardless of their user interface, license or development model. It is worth noting that this capability is not limited to office productivity software, but will be usable and useful with a much wider range of products. Being XML, OpenDocument files can be generated on the fly, analyzed or indexed automatically by any number of independent server-side applications. The other really important thing that OpenDocument makes possible, on a large scale and for the first time, is reliable long term archiving, without any loss of information, of digital office documents. This is something which is not possible with other standards, including those specifically developed for archiving like PDF/A (Portable Document Format Archive). The latter "only addresses those files that might be described as a digital representation of a paper document" and "will ensure that a PDF document will be rendered as it was created 50 years from now" [1]. Consequently, PDF/A is enough only when the only available open version of an already existing document is a PDF file, or with documents created by scanning paper. PDF/A files are not meant, for example, to preserve the history or, above all, the internal algorithms of the original document. They do not preserve different versions, or of the formulas from which the printable numbers in a spreadsheet are generated. This would make the future historical or scientific analysis of, among others, law proposals, election reports or global warming studies much harder than it could be if the original had been stored in OpenDocument format. Of course, reliable long term archiving depends on many more variables than the file format alone, from the physical Author Marco Fioretti is a hardware systems engineer and a part time freelance writer for several Linux and other IT magazines. He is the co-founder (in 2002) and current coordinator of the Run Up to date Linux Everywhere Project < which makes modern Free Software easily installable on older computers. Marco Fioretti has always been interested in truly Open file formats and protocols, from e-books to portable databases and the Universal Business Language, and in their impacts on economy, education and civil rights. In 2004, for example, he investigated the philosophical links between Free Software and Scouting. In the same spirit, in 2006 he co-founded Elèutheros < which aims to promote a wider, official adoption of Free Software and non proprietary formats within the Catholic Church. More recently, Fioretti has also started to follow the attempts of the Free Software and disabled users communities to communicate more effectively. His more recent initiative to promote the importance of Free Standards and Software among all citizens is the Family Guide to Digital Freedom < Marco Fioretti is also member and contact for Italy of the OpenDocument Fellowship < a volunteer organization that promotes the use and development of the OpenDocument format, and the author of Everybody s Guide to OpenDocument < <marco.fioretti@opendocumentfellowship.org>. media to the file system specifications, but they are outside the scope of this paper. 2 Impact of OpenDocument on Free/Open Source Software Many FOSS supporters, including politicians who hope that FOSS will create more local IT jobs, seem to have not yet realized that a general adoption of OpenDocument would delay the adoption of FOSS in several scenarios or, in any case, change the way in which FOSS desktop applications are to be developed and supported. Almost any large public or private organization already has an equally large, installed and partially paid for IT infrastructure, which would take years to migrate. The only thing that really prevents interoperability with such organiza- Novática UPGRADE Vol. VII, No. 6, December

27 tions, or among them, is not which software they run. It is the fact that they (can) only produce or accept proprietary file formats, for which, by definition, FOSS filters can never be perfect. In addition to this, in many countries only interoperability and data ownership can be realistically included as mandatory requirements in public tenders. It is immensely easier and much more coherent, at least in a Free as in Freedom [2] society, to lobby for laws which only require free formats and protocols, rather than any specific software "product", whatever its license might be. Imposing the adoption of OpenDocument is the easier and most effective line of action in this context. To read or produce OpenDocument files, however, a Microsoft Office user has no need whatsoever to migrate to OpenOffice or Linux. All he or she needs is to install, or have installed, a plugin for Microsoft Office like the one being development by the OpenDocument Foundation [3]. In other words, by just adding OpenDocument support to their existing infrastructure, many private and public users may have much less reasons or external pressures to migrate to FOSS. Once data ownership and interoperability are guaranteed, migration does not just become much easier, it also becomes much less urgent and attractive. Therefore, FOSS will have to compete in other fields in order to be adopted by desktop users, regardless of license prices. Performances on limited hardware, usability, end user documentation, and free online support for non-programmers will have to be consistently better than those of proprietary software to convince those users to migrate to, and keep using, FOSS solutions. Some FOSS advocates will probably be upset by these new trends, but it would be wrong to fight them. First of all, mandatory adoption of OpenDocument would immediately make the 100% FOSS desktops of any student or small business owner completely compatible with the proprietary ones already existing in governments or corporations. Through OpenDocument, FOSS-only desktops would become much more usable for business, research and active politics than they are today and much earlier. Secondly OpenDocument will eventually improve the quality of FOSS, for the reasons just mentioned. 3 Hidden Traps It is very likely that, eventually, all major producers of both proprietary and Free software in the office files space will support OpenDocument. In and by itself, however, there are several ways in which the standard enables monopolies to continue, or its usefulness to be nullified for long term archiving. Technically speaking, OpenDocument is very powerful and useful because it can be extended. Robert Weir, a Software Architect of the IBM Software Group, recently discussed with the health care standards consortium MedBiquitous [4] an OpenDocument based format for physician certification which would have to add to each file digitally signed XML metadata. However, extensibility is neutral with respect to intentions. It is possible to have a perfectly Free as in Freedom XML container full of patentridden components. If anything, the fact that an office file standard is not owned and controlled by one vendor may make this even easier, not harder, as proprietary extensions appear to keep end users locked in, at least in some scenarios. 3.1 General Issues Where the Open Document Format for Office Applications (OpenDocument) v1.0) specification (OpenDocumentv1.0-os,swx) allows a new namespace to be introduced for externally-defined elements is an entry point for proprietary extensions, if the namespaces are not publicized and the schemas and semantics for the elements are not entirely published and legally usable. For example, arbitrary introduction of foreign tags below the paragraph level is permitted, as well as arbitrary document metadata (section 2.2), formatting properties (section 15) and their application-determined defaults. The standard also specifies how unknown tags should be treated and, above all, states that at least some of them should be preserved. As a practical example, OpenOffice already preserves OLE objects and alternative renderings for images, in order to maintain compatibility with Microsoft Office if the document is exported back to Office. All these "foreign" elements, however, are intrinsically not safe for use in interchange, because there is no predictable treatment of them beyond such an optional, passive preservation [5]. 3.2 Multimedia Formats The OASIS standard "Open Document Format for Office Applications (OpenDocument) v1.0" specification obviously allows the inclusion of images, audio or any other multimedia object in texts, spreadsheets and presentations. It says nothing, however, about the license or any IP restriction of the corresponding file formats, so it is perfectly legal (as far as the standard is concerned) to include multimedia content in proprietary or patent-restricted formats inside OpenDocument files. 3.3 Macros Macros are the only realistic way, for non-programmer users of office suites, to achieve two intrinsically different goals: one is to extend the functionality of one specific software program on all the files it operates: dictionary interfaces, spelling checkers, word counters and similar fall into this category. The other is to embed some kind of data processing capability or interactivity into one specific document: a real-world example may be an e-learning course, usable even without an Internet connection, made of interactive forms. This is similar to what happens with Web browsers. There are both Firefox specific extensions and web pages containing JavaScript applets that can run in any browser. Macros of the first kind have nothing to do with OpenDocument, but the interactive course scenario is quite different. If that course consists of forms inside 26 UPGRADE Vol. VII, No. 6, December 2006 Novática

28 OpenDocument texts or spreadsheets, then its users and distributors will expect all the buttons, form fields, checks on user input and so on, to work without problems inside any OpenDocument-compliant program. The standard, however, specifies how macros must be embedded in the document, but not what their language should be [6]. 3.4 In-file Databases Single-file, server-less SQL databases, which besides the data also contain schemas, indexes and forms structures are not as powerful as full-fledged RDBMS solutions, but they are extremely useful in a very common and important case. They make it possible for all computer users, including those without programming skills, root password or permission to install extra software on their computers, to create and above all exchange databases just as easily as text or images [7]. The popularity of MS Access in small offices is a proof of this fact. OpenDocument files may include such single-file databases. OpenOffice, for example, uses HSQLDB as its default format for this. In general, OpenDocument files may also be dynamically linked to external databases. In both cases, the same issues as with multimedia objects apply: nothing in the standards prevent linked or embedded databases from being proprietary or undocumented Digital Signatures and other Security Related Extensions Both governments and large corporations have interest in ways to give or deny access to different sections of the same document. An internal report, for example, may be published on the Internet with only some paragraphs encrypted. An OpenDocument file may also contain its own digital signature, to certify its authenticity. Even individual metadata may be signed, as in the MedBiquitous case. Again, any cryptographic algorithm, regardless of its license or the completeness of its public documentation, may be used for this purpose. 3.6 Formulas Version 1.0 of the OpenDocument standard did not mandate a common format for the formulas embedded in spreadsheets, and this created well known interoperability problems [8]. This issue, however, will be fixed in version 1.2 of the standard, so it only exists for OpenDocument files already archived today or in the next 12 months. 3.7 Fonts Fonts are referenced by name, e.g., "Times New Roman". Of course, they are only used for presentation, not encoding of information. They will not create substantial interoperability or archiving problems, but using fonts which are proprietary or available on only a single platform can create unnecessary trouble, for example making distribution of printed copies without modifications impossible. This is especially true for scientific or technical documents with many complex formulas. 4 Nature of the Solution As already described, "100% ISO26300 Compliance" is not enough to guarantee that an OpenDocument report or law proposal stored today will be completely readable and usable 20 or more years from now. In order for that to happen, all the components of such files should be at least as open and documented as OpenDocument itself, but it is possible and relatively easy do the contrary, thereby keeping the end users locked into a specific software product. Both benefits of OpenDocument, that is complete interoperability and long term archiving without information loss, would be lost in such cases. Generally speaking, only part of the solution is technical and can be placed in the applications. For example, a good option to have in OpenOffice (or any other word processor) would be to automatically convert all imported graphics into an open format like PNG (Portable Network Graphics) or SVG (Scalable Vector Graphics). The same could be done with fonts. When databases are concerned, archiving procedures should perform, when necessary, an automatic inclusion of the database linked from the OpenDocument file(s). Automatic XML checkers which scan the content of OpenDocument files and signal any closed component, comparing its format against an accepted list, would also be easy to implement and very useful. Public Administration and libraries could run them much like antivirus software, refusing to accept any file whose interoperability, now and in the future, cannot be guaranteed. At another, but still technical level, well defined interoperability testing is also essential to distinguish the implementations which are willing to cooperate. The University of Central Florida, for example, is already working on an OpenDocument test suite which would allow implementations to verify the conformance of a generic OpenDocument-enabled software program with the specification [9]. At the deepest level, however, this is not a format specification issue. The specification of OpenDocument or its license should not and could not contain, allow or forbid every conceivable extension, not to mention those which do not even exist yet. This is implicit in the very concept of open and extensible standard: it must facilitate data exchange between implementations which are willing to exchange them, but it can not force compatibility when this does not happen. The problem must be solved in another way, which is only partly technical. The first step is to raise the awareness, among system administrators and policy makers, that these issues do exist and must be faced before they start to become a serious problem. The second is to implement the automatic checkers mentioned above. The third step may be to officially define an "OpenFile" or similar trademark which is applicable only to files in which no component has restrictive licenses or incomplete documentation. Part of the problem here is of course to define who should take this action. In the European Community, a likely candidate may be the Novática UPGRADE Vol. VII, No. 6, December

29 Interoperable Delivery of European egovernment Services to public Administrations (IDABC) [10]. The last and most important step would then be to require by law that all the OpenDocument files stored or exchanged with public Administrations, libraries and so on be compliant with the "OpenFile" trademark, regardless of the software application which created them. There are obviously several key cases where it may be unavoidable to grant exceptions to such a rule, at least in the short and medium term. Making sure that public documents are and remain completely open, of course, remains a mandatory prerequisite for a truly open society, and OpenDocument alone is not enough to reach this goal. References [1] Martin Bailey. Why Would Anyone Write a PDF Standard Just for Archivists?, < July 29, [2] Sam Williams. Free as in Freedom: Richard Stallman s Crusade for Free Software. O Reilly Media, ISBN: [3] The OpenDocument Foundation, Inc. < documentfoundation.us>. [4] MedBiquitous. < [5] Dennis E. Hamilton Unofficial analysis of the OpenDocument specification < October 11, [6] Marco Fioretti. Macros an obstacle to office suite compatibility, < pl?sid=05/09/09/ >. September 17, [7] Marco Fioretti. The Lack of a Small Unified Database, < September 28, [8] Marco Fioretti. OpenDocument office suites lack formula compatibility. < software/05/09/09/ shtml?tid=93>. September 20, [9] Open Document Fellowship. Open Document Sample Documents. < fellowship.org/>. [10] European Comission. Interoperable Delivery of European egovernment Services to public Administrations (IDABC). < 28 UPGRADE Vol. VII, No. 6, December 2006 Novática

30 ISO (OpenDocument) vs. MS-Office Open XML Alberto Barrionuevo-García Until recently, Humankind has been using electronic document formats that were exclusive and which locked users into a specific IT provider and their software. These providers have been changing over the years, with the exception of the last ten years. These changes have resulted in millions of unusable and inaccessible documents due to formats that are obsolete or not properly readable by the new applications. These days, and for the last ten years or more, MS-Office has been the dominant operator, tying users to an operating system from the same manufacturer and at the same time shutting out most other operators. However, the arrival of open standard ISO 26300, OpenDocument, looks like changing this outlook and releasing application formats and their users from the stranglehold of a small number of applications and systems. In response to this radical sea change in the market, the company that up until now has dominated the office automation market is proposing an alternative, semi-open format. Much of the future of information technology and the path that the Information Age will take now depends on which format gets the upper hand in this fascinating battle, the like of which has never been seen in the history of computing. In this article the two formats are compared from a number of viewpoints and one basic conclusion is reached: only one should survive. Opentia, S.L., This article is distributed under the "Creative Commons Attribution-ShareAlike-NoDerivs- NoComercial 2.5 Spain License" Creative Commons licence, available at < by-nc-nd/2.5/deed.en>. Keywords: ECMA, ECMA-Office Open XML, Electronic Document, ISO/IEC, Libre Software, MS-Office Open XML, ODF, Office Automation Format, OpenDocument, Open Standards, Standard. 1. Historical Context 1.1. The Middle Ages of Electronic Documentation From the moment when we stopped using "vi", "emacs", and "edit" to draft documents in ASCII with no more layout than could be achieved in a rudimentary and artistic manner by inserting or removing spaces and by using some special characters as decoration, office automation has been dominated by document formats with four main characteristics: 1) The format was dependent on the application that generated it and, therefore, on the company that owned that application. 2) The format was secret and exclusive to each particular manufacturer and, at most, other competitors or developers managed to support it by using costly reverse engineering techniques or by handing over large sums of money and signing agreements not to reveal the secrets of the format. 3) In practice, "in order to be compatible", everyone was obliged to use the format (and the application) of whichever was the dominant operator in the market at that time. 4) Generally speaking, these applications were sold under non-libre licences, and their code was as much a secret as the format itself. Since the eighties, barring a few exceptional cases such as LaTeX, DocBook (for technical applications) and, much later, the OpenOffice formats, all users and buyers of office Author Alberto Barrionuevo-García is a graduate in Computer Science from the Universidad Politécnica de Madrid. He is a partner and CEO of OPENTIA, S.L. specialized in consultancy services related to computer standards, document management and archival, and open solutions in general, and also advanced migrations libre software through its subsidiary VIRTUA, S.L. He is Vice-President of the Munich-based European association, Federation for Free Information Infrastructures (FFII), and is the coordinator of the association s "Open Standards" working group. He is also working group and web coordinator of EstandaresAbiertos.org, member of the OpenDocument Fellowship, corporate member of the ODF Alliance, and member of Hispalinux. He is "Linux Registered User #130136". He has been director of operations and consulting at OpenText Corporation for Latin America; managing director, director of operations, and director of consulting at IXOS Software AG for Latin America; and technical consultant at SAP Spain and Portugal. <abarrio@ffii.org>. automation software have effectively been forced always to use whichever application was dominating the market at that time so as to be able to exchange documents with the rest of society. This happened with applications such as WordStar first of all, followed by WordPerfect, and then, to a lesser extent, AmiPro, and finally the software which ended up dominating the market thanks to the virtual monopoly of its manufacturer in PC operating systems: MS-Office. The feature common to all these formats is that if someone ever sent you a document in a format and/or version other than the application that you were using, you were forced to go out and get hold of the application behind that Novática UPGRADE Vol. VII, No. 6, December

31 format to be able to access the information it contained. In the home computer market the truth is that getting access to new software would not, in all probability, be much of a problem because you could always trot along to your local purveyor of illegal software who would supply you with whatever you needed very cheaply or even for free. However, in the institutional and business market, the acceptance of a new document format has always meant taking a fairly important step, involving a mass migration of applications at every workstation, not to mention the consequent disbursement of the by no means insignificant cost of licences, training, and even the adaptation of existing documents and applications. By way of a curious aside, at this point I always remember the notary I usually work with who "still" uses the nongraphical version of WordPerfect. They probably don t have to exchange much documentation with the outside world (I know for a fact that they don t use ). But these are customs from bygone days and should have no place in our post-industrial era. 1.2The Information Revolution: the Internet and New Opportunities The obscurantist model of binding the application to the format persisted, without any real alternative, until the general population started getting hooked up to the Internet en masse, and formats that were application-independent, and which in many cases were not subject to any royalties, began to become popular. The reason for this popularity was that their original creators knew that it would be impossible to compete with the dominant application (by now a virtual monopoly) unless they adopted the tactic of breaking the link between the format and the application, and released the format specification so that other applications could also use it, thereby benefiting from a greater economy of scale. The most paradigmatic case is perhaps the situation of Sun in 1999, when it employed 40,000 people each with their corresponding workstation. When acquiring an office automation application, Sun was faced with the choice of either paying Microsoft for 40,000 licences, while also having to duplicate hardware for its many employees who needed to continue using Unix, or doing what they actually did: they bought the code of an office automation application on the German market, StarDivision, which also provided them with a marketable alternative to Microsoft s dominant product. But a problem arose: an application like StarOffice was costly to maintain and keep up to date and would require a large department of developers devoted exclusively to it if they wanted to compete at the same level as other rival applications who did have such a department. So, as a solution to that problem came the second major milestone of the short history of electronic document formats. It happened in the late nineties, just as a new model of development and distribution of desktop software was reaching maturity: libre software. Until a short time previously, libre software had been focusing its efforts on very technical applications, mainly aimed at servers and developers. But the time had come for libre software to take the plunge into the world of the desktop. Thus, just one year after buying StarOffice, Sun took the eminently financial and commercial decision to release not only the specifications of the StarOffice format, but also the code used to implement those specifications in order to create a parallel libre software project that would maintain, use, and disseminate their formats. It was the launch of OpenOffice.org project, the same format as I am using to write this article. The fact of the matter is that it was the only economically viable way that a hardware company such as Sun could compete against the practically monopolistic software that had all users locked in tight: i.e. Microsoft s. At the same time, Sun decided to go on marketing their closed-source version of StarOffice based on OpenOffice.org s libre code with some extra functionalities. This dual licence and product model had already been used by another application that had been driven out of the market by Microsoft s stranglehold on operating systems: the forgotten Netscape, the seed from which, phoenix-like, the current successful Mozilla-Firefox web browser arose. All this finally led to the commercial availability of a number of open document formats covering all the functionalities of the dominant application which could also be used by many of the other alternative office automation applications that already existed on the market..., as indeed has happened. 2 Document Standards 2.1 And why not an Open Standard?: Open Document After a few years of putting up with some much vaunted computing standards that were too theoretical, complex, and flawed to work with, the official international standardization organizations decided to create some standards based on market-driven protocols or formats which had already proven their worth in the real world. This was probably helped by the fact that many companies were getting together to create their own alternative, non-governmental organizations for developing industrial standards. This was the case, for example, of the organization OASIS (Organization for the Advancement of Structured Information Standards), made up of almost all the major software developers around the world. And this has also been the case of the document format created and maintained by OASIS, one which is independent, open and intended for use by anyone, and is based on the old StarOffice and OpenOffice.org formats: the OpenDocument format. OASIS itself describes the purpose of the OpenDocument Committee 1 since its inception in late 2002 as "to create an open, XML-based file format specification for office applications". In other words, the format should be generic and aimed at all office automation applications 30 UPGRADE Vol. VII, No. 6, December 2006 Novática

32 which, of course, voluntarily decide to use it. Later in this article we will see that today this is practically all of them, with one notable exception. But in any event, what is really important and what makes a difference is that the standard was not created in the of the format and interests of any one application or manufacturer, but was corrected and reshaped where necessary in a manner that was totally open to all interested parties. And the result is that now, just over a year later, it is already being used natively 2 by at least 5 office suites covering the 5 most commonly used platforms (in descending order: Symbian, Windows, MacOS, Linux and FreeBSD), as well as several dozen applications more that use some of its parts (text document, presentation, spreadsheet, and vectorial drawing) or use it for some additional functionality (document management, indexing, forms, operating flows, etc.) One factor that has definitely contributed to the ease with which many existing office automation applications have been able to adopt OpenDocument is that, as far as possible, its specification used any existing standards that were of interest. Also, for many of those standards there was a great deal of free source code and libraries available on the market. This made it possible to make implementations of the specification cheaper and easier. Examples of reused standards in OpenDocument are XML, ZIP, Dublin Core, SVG, XSL, SMIL, XLink, XForms, MathML, HTML, etc. This clearly shows there was no unnecessary re-inventing of the wheel. And testimony to this is the fact that the specification document 3 is just a tad over 600 pages long But aside from its many technical virtues which we will not go into right now, much of the interest that the format has aroused is down to its ratification as a (de iure) international standard by the ISO/IEC (International Organization for Standardization / International Electrotechnical Commission) on May 1, Since then, despite not being officially published until December 2006, OpenDocument has been approved as official standard ISO After receiving such a unanimous "blessing", and in a period of a little over six months, a not inconsiderable number of governments and official bodies have decided to adopt OpenDocument as their sole document standard. This is only logical, as there is no other standard of similar characteristics to serve as an alternative. In fact, it could even be argued that society has no need for there to be an alternative, in the same way that, for example, society has no need whatsoever for there to be an alternative to the metric measurement system. Here, a single standard fulfils its purpose perfectly; measurements are standardized and can be exchanged between entities and people without creating any unnecessary confusion due to the existence of a duality of systems. And if you need any proof of that, we refer you to the huge bungle that caused NASA s Mars Climate to crash 4. Among the most noteworthy cases of OpenDocument adoption are: The Massachusetts state government, perhaps the most conflictive and publicized case due to the great political and media pressure brought to bear by Microsoft in an attempt to overturn the decision. The Belgians who adopted the OpenDocument format throughout their federal government. Denmark s open standards law, on the back of the Ramboll report 5 on the savings to be made by adopting OpenDocument as a standard format. The Motion of the Extremadura Regional Government in Spain mandating the exclusive use of OpenDocument and PDF/A in its government offices. The announcement by the French government that it is to migrate all its administration to OpenOffice.org software supporting OpenDocument, in the light of the conclusions of the Carayon Report 6. Similar announcements by Norway s Ministry of Science. Several mass migrations in Brazilian government agencies. The mass migration of the Government of Tamil Nadu in India. The adoption of OpenDocument by the Zaragoza City Council in Spain. The exclusive adoption by the Misiones Province of Argentina. The pioneering legislation of the Universidad de Cádiz (Spain) governing electronic document exchange. Etc. 1 OASIS s OpenDocument committee, created in November < (last accessed ). 2 Software applications supporting OpenDocument or use it natively, according to the OpenDocument Fellowship. < (last accessed ). 3 OpenDocument specification by OASIS. < (last accessed ). 4 CNN: "Metric mishap caused loss of NASA orbiter.". < OpenDocument-v1.0ed2-cs1.odt> (last accessed ). 5 Danish Government: "Ramboll Report" on the cost savings from implementing OpenDocument instead of MS-Office Open XML. < (last accessed ). 6 French Government: "Carayon Report" on the cost savings from implementing OpenDocument. < (last accessed ). Novática UPGRADE Vol. VII, No. 6, December

33 The OpenDocument Fellowship 7 and the OpenOffice.org project 8 are monitoring these and other large scale adoptions of the standard, especially in the public sector. It is in the public sector where it is most necessary to use open standards in order to avoid discriminating among citizens on the basis of whether or not they are customers of a particular private provider by using a format that is exclusive to their own commercial application. 2.2Do we Need a Second Standard? Microsoft s Strategy When taking any decision it is normally impossible to satisfy all interests. But it is especially hard, and not entirely fair, to satisfy economic interests that are not only in private hands but, what is worse, are also contrary to the interests of society at large. The company that holds a practical monopoly on desktop operating systems and office automation applications with its well-known duo Windows/Office, has decided not adopt the OpenDocument standard adopted by practically all the other players in the market, and is now engaged in a process of intense lobbying to standardize their alternative format: Microsoft-Office Open XML. As part of their strategy, Microsoft has used a European standardization agency with a reputation for being a soft touch: the ECMA (European Computer Manufacturers Association). Thus by means of a small technical committee, chaired by MS, in which most of the members were appointed expressly and solely for the purpose of this process 9, they have pushed through the approval of the format under the name of ECMA-Office Open XML (EOOX), though not without a resounding vote against from IBM, a historical member of the association. Now the next step is to push for its approval as an international standard by ISO, since ECMA is ultimately no more than an association of companies issuing their own private standards with no official status. However, many important voices have been raised against the idea of unnecessarily duplicating formats by having two standards 10 that are completely overlapping and meet an identical need. Neither should the predictions of leading analysts 11 in this regard be ignored. But the most interesting thing as far as this article is concerned is that the ECMA-Office Open XML standardization effort carried out by ECMA has revealed a number of details that we need to bear in mind and which require studying in some detail. Firstly, we should take a close look at the composition of ECMA s Technical Committee 45 convened to standardize Microsoft s format. As we have already mentioned, only four of its members are from ECMA (Microsoft, Intel, Toshiba and Apple) and the rest are external committee members expressly and solely appointed to the committee for the purpose of this standardization process. But what is even more significant and revealing is the officially declared goal of the committee convened to standardize ECMA-Office Open XML format. It especially contrasts with the fair and open purpose declared by the OA- SIS committee that created OpenDocument, quoted earlier. The declared goal of the ECMA committee is none other than "to produce a formal standard for office productivity applications within the Ecma International standards process which is fully compatible with the Office Open XML Formats." The first thing that springs to mind when reading this strange declaration of purpose is that ECMA should at least have left out the adverb "fully"; that way we might have entertained the slightest of doubts as to the impartiality and neutrality of the committee s purpose. Thus, in the light of these declared intentions, the obvious question is: in that case, why were the other entities on the committee at all, if it was merely a question of ratifying, without discussion, everything that Microsoft put on the table arising out of that "full compatibility" with the application belonging to the monopolistic provider? For example, the case of Novell, one of the non-ecma members of the committee, is significant. After a multimillion dollar patent agreement with Microsoft, perfectly interpretable as a "financing agreement", Novell designated themselves as developers of the support to the EOOX format for OpenOffice.org. The idea is probably to avoid the impression that only one office automation application on the market will actually support it: the new, about to be released (and bold 12 ) version of MS- Office. However, Novell will have no easy task since, as opposed to the scarcely 700 pages of the OpenDocument specification, the final specification of the format proposed by 7 OpenDocument Fellowship: examples of the adoption of OpenDocument around the world. < (last accessed ). 8 OpenOffice.org: market share figures. < (last accessed ). 9 ECMA: Presentation "ECMA TC45 Office Open XML Formats" (p 16). < Office%20Open%20XML%20Formats/TC45_GA_Dez05.pdf> (last accessed ). 10 European Commission, "PEGSO Conclusions and Recommendations on Open Document Exchange Formats", < 11 Report in which the Gartner Group predicts that the Microsoft format will not be standardized by ISO. < (last accessed ). 12 Wall Street Journal, "Bold redesign improves Microsoft Office 2007", in which Microsoft is referred to as "bold" due to the high training costs that the major changes to the new MS-Office s user interface will involve. < shares/international_news/ htm> (last accessed ). 32 UPGRADE Vol. VII, No. 6, December 2006 Novática

34 Microsoft and released by ECMA runs to over 6,000 pages to which will have to be added a welter of support documentation, all to cover no more and no less than the same functionality as the ODF. Although the fact of the matter is that they do not work as well as each other, whatever well publicized claims may have originally been made. The fact is that one of them works significantly less well as a document format than the other. This may be one of the reasons why its specification is so inordinately hefty. 3. OpenDocument vs. ECMA-Office Open XML (EOOX) When comparing the two formats the first thing that stands out is the different purposes of the standardization committees that released them. As we have mentioned, while one format is aimed at the industry as a whole with no reference or preference to any particular party, the other is aimed exclusively at being compatible with a specific application which is the exclusive property of the company that created the format. And, what is more, the sole purpose of that format is, in all probability, to keep the present users of their applications locked in to their products. In fact it has gone to such lengths to ensure compatibility with MS-Office that EOOX s specification includes historical errors that are known by all, and makes no attempt to correct them 13. This is probably one of the few times in history that a standardization body has not removed known bugs from a format or protocol during the standardization process but has instead perpetuated them so that all other implementers of the specification will also be forced to artificially reproduce them in their respective applications. This leads us to the typical situation that arises when a dominant company has a problem. It has two ways of preventing negative repercussions, which is the real issue, not the problem in itself. The first option would be to simply solve the problem, while the other option is to export the problem to all the other competitors so that no one gains an advantage, even though this means that the users suffer in the process. Another thing is that, to a large extent, the EOOX specification reinvents the wheel by avoiding the reuse of existing standards to cover included functionalities. This is probably for reasons of self-interest, as it is a way to inflict an additional effort and investment on the competition, while Microsoft requires no extra effort whatsoever as they already have the application and the fact that it works in such an unorthodox manner does not affect them. A good example is the case of hyperlinks. Rather than use the very widely and commonly used XLink standard, it encodes them in this peculiar and cryptic manner: <w:hyperlink w:rel="rid1" w:history="1"> <w:r> <w:t>this is a hyperlink</w:t> </w:r> </w:hyperlink> It is necessary to fetch the hyperlink in question from an additional file. OpenDocument on the other hand is designed with human beings in mind and uses the XLink standard. Let us compare the previous code with the same hyperlink as encoded in OpenDocument: <text:a xlink:href=" This is a hyperlink </text:a> An example of the reuse of anything already existing and useful is that for any programmer familiar with XHTML, DocBook or even HTML, OpenDocument is quite similar and easy to implement. EOOX, on the other hand, encodes everything in a completely different and cryptic way, a way that any professional would find very difficult to understand and, what is worse, one that is practically impossible to translate into any other standard format such as XHTML or OpenDocument. Hence the relatively well-founded suspicion that EOOX is simply a translation to XML of the secret binary formats that Microsoft have been using up to now, which are ultimately no more than simple and extremely technified dumps of the representation of the document to the memory of the MS-Office application; i.e. machine readable, but not human friendly. Something akin to programming a large software application in Assembler. This view gains weight if we look at the different XML encoding methods used in the two formats. While OpenDocument uses a "mixed content" XML of the type used for non-structured content (as text documents are), EOOX organizes the inside of its files using an "unmixed content" method more typical of databases and structured data formats. To be more precise, in an XML document you can find two kinds of inputs, tags and text, and in between the opening tag and the closing tag, other tags and text can be included. Thus, in the mixed content method, in between the opening tag and the closing tag you can add either tags or text or a mixture of both. However, in the unmixed content method, you can only add more tags or more text, but not a mixture of both. Let s see how this affects the representation of the same short text in OpenDocument and in EOOX. First of all, here is the original text with the format we wish to represent: 13 MS-Excel contains an error in the sequential calculation of the days of the calendar because it is based on the false premise that the year 1900, the origin year, was a leap year. In ECMA-Office Open XML this error is perpetuated. More information in Rob Weir s "A Leap Back". < (last accessed ). Novática UPGRADE Vol. VII, No. 6, December

35 Esto es un documento muy simple con un poco de formato y un hiperenlace. 14 And the hyperlink, which we cannot see in the example, is pointing to a URL < Thus, OpenDocument would represent this text in this way: <text:p text:style-name="standard"> Esto es un <text:span text:style-name="t1">documento</ text:span> muy simple <text:span text:style-name="t2">con un poco de</text:span> formato y un <text:a xlink:href=" EstandaresAbiertos.org">hiperenlace</ text:a> </text:p> which, as we can see, could even be modified manually using a text editor on any text terminal, as it is reasonably intelligible for anyone remotely familiar with HTML, XHTML. or Docbook. Now let s see what EOOX encoding makes of the same text: <w:p> <w:r> <w:t>esto es un </w:t> </w:r> <w:r> <w:rpr> <w:b /> </w:rpr> <w:t>documento</w:t> </w:r> <w:r> <w:t> muy simple </w:t> </w:r> <w:r> <w:rpr> <w:i /> </w:rpr> <w:t>con un poco de</w:t> </w:r> <w:r> <w:t> formato y un </w:t> </w:r> <w:hyperlink w:rel="rid4" w:history="1"> <w:r> <w:rpr> <w:rstyle w:val="hyperlink" /> </w:rpr> <w:t>hiperenlace</w:t> </w:r> </w:hyperlink> </w:p> As we can see, it would be difficult for anyone to even understand the representation of our simple text, let alone change it manually. But the cryptic nature of EOOX is not only down to this method of encoding XML; it is also due to encodings that are specifically the result of how Microsoft s operating system, Windows, stores information in its memory. In the following example, taken from section (page 759) of Volume 4 of the final draft of EOOX, we can see the following half XML, half binary expression: <w:font w:name="times New Roman"> <w:sig w:usb0="2002a87" w:usb1=" " w:usb2=" " w:usb3=" " w:csb0="000001ff" w:csb1=" " />.. </w:font> As we can see, the hexadecimal numbers assigned to the variables "usb" and "csb" are mere bit-for-bit dumps of the Windows memory data structure with no intervening abstraction to make them suitable for other platforms or remotely intelligible at a higher level. Didn t someone say that EOOX was a pure XML format? Hadn t we already agreed that binary formats were a thing of the Middle Ages of software development? But the worst thing is that this kind of binary, platform dependent encoding (for example, only for big-endian systems or only for little-endian systems?) is repeated for various EOOX functionalities, such as paragraph conditional formatting, table-cell conditional formatting, conditional table-row formatting, table style conditional formatting settings exception, etc. Interested readers can find more on the matter of bitmasks in EOOX in an article written by IBM professionals 15. But EOOX s dependence on a specific system and application does not end here; it is patently obvious that EOOX is not a specification of a format, but rather a journey through the history of MS-Office, an accumulation of all the patches and fixes over the last ten years. Just take a look at some of 14 Note of the Editor: This Spanish sentence means "This is a very simple and a bit formatted document, and a hyperlink". 15 An Antic Disposition, Rob Weir, "A bit about the bit with the bits". < (last accessed ). 34 UPGRADE Vol. VII, No. 6, December 2006 Novática

36 the exceptions in the specification 16, such as: "interpret this bit as X when it comes from Mac, as A when it comes from Win32 and even different when it s written before date Z". And there is still more. The specification requires you to take into account the operational peculiarities and exceptions (not even described in the specification and known only to Microsoft) of all the different legacy versions of MS-Office there have been, combined with the successive versions of MS-Windows in which they have run 17. The chapter titles alone of some of these historical gems as they appear in EOOX s official specification should be testimony enough: autospacelikeword95 (Emulate Word 95 Full-Width Character Spacing) chapter footnotelayoutlikeww8 (Emulate Word 6.x/95/97 Footnote Placement) chapter mwsmallcaps (Emulate Word 5.x for the Macintosh Small Caps Formatting) chapter suppresstopspacingwp (Emulate WordPerfect 5.x Line Spacing) chapter linewraplikeword6 (Emulate Word 6.0 Line Wrapping for East Asian Text) mwsmallcaps (Emulate Word 5.x for Macintosh Small Caps Formatting) shapelayoutlikeww8 (Emulate Word 97 Text Wrapping Around Floating Objects) truncatefontheightslikewp6 (Emulate WordPerfect 6.x Font Height Calculation) useword2002tablestylerules (Emulate Word 2002 Table Style Rules) useword97linebreakrules (Emulate Word 97 East Asian Line Breaking) wpjustification (Emulate WordPerfect 6.x Paragraph Justification) shapelayoutlikeww8 (Emulate Word 97 Text Wrapping Around Floating Objects) These few examples alone should more than explain why EOOX s specification is over 6,000 pages long, not including appendixes. Furthermore, another design problem affecting EOOX encoding is that it mixes pure content with its presentation, while OpenDocument, although it is not entirely blameless in this respect, does so to a much lesser extent and mostly keeps data separate. This separation is comparable to the way XHTML encoding uses CSS files. Let s take another example of formatted text and the ways that each format represents it: texto en negritas (text in bold type) To represent it in OpenDocument, firstly the information relating to content is stored: <text:span text:style-name="enfasis_10_ Negrita"> texto en negritas </text:span> then the manner in which the content is to be presented is encoded. As we can see, the style to be used is defined by "Negrita_20_Enfasis": <style:style style:name="enfasis_10_ Negrita" style:display-name="enfasis con Negrita" style:family="text"> <style:text-properties fo:fontweight="bold" /> </style:style> Notice that OpenDocument is using "styles" to format the document. This is very useful, particularly when altering the presentation of an entire document, since it is only necessary to change the style, without having to navigate through the entire document changing the appearance of every chunk of text on a case-by-case basis. Meanwhile, EOOX encodes this same text and format in the following manner: <w:r> <w:rpr> <w:b /> </w:rpr> <w:t>texto en negritas</w:t> </w:r> We can see here the odd result of mixing content with format, since <w:b /> means "bold type (negritas)". To continue with our technical analysis of the two formats, if we compare the size of files and their elements 18 required to store the same text of some size, we see that OpenDocument generates files that are around 90% smaller than equivalent files in Microsoft s old.doc format, and around 50% smaller than those generated by the EOOX format. From an economic point of view, the cost of implement- 16 KDE Developer Journals, Zander, "office document formats". < (last accessed ). 17 Rob Weir, "How to hire Guillaume Portes". < (last accessed ). 18 O Reilly XML.com: "Comparing XML office document formats: using XML metrics". < 2006/08/comparing_xml_office_document_3.html> (last accessed ). Novática UPGRADE Vol. VII, No. 6, December

37 ing Microsoft s monstrous format specification is also far in excess of what it will cost to implement OpenDocument. An employee of the Microsoft s Mac division estimated 19 that it would take 5 developer-years to program just a quarter of what was needed for just the word processor. Meanwhile, and expert from Adobe estimated 20 the effort required to implement the entire specification at 150 developer-years. Finally, the VP of the OpenDocument Foundation estimated 21 that it would cost a thousand dollars just to print out the documentation required to implement it. Another serious problem that raises its head when implementing or adopting EOOX is that there is no guarantee that either Microsoft or any of the other members of the ECMA technical committee have declared, let alone waived, the software patents that the format infringes 22. This leaves the door open to the risk of lawsuits being filed later against all the other applications that adopt the format but, of course, only once the standard is sufficiently well established, as happened in the case of Unisys and GIF 23. In fact Microsoft s licence does not cover all the patents necessary to implement the format in its entirety 24. Fortunately for Europeans, software patents are not legal, but the same cannot be said for the citizens and companies of other countries, starting with the USA. And obviously we need to bear in mind that the scope of international standards must be global; it makes no sense calling them "international" if they only apply to a specific region. To conclude this comparison between the two formats, we need to take into account another, highly controversial, factor that has recently become very important. Let us suppose that the EOOX format is patent free and can be freely implemented by anyone regardless of the not insignificant difficulties that we have just described. However, up until now our analysis has ignored a key factor: DRM 25 (Digital Rights Management or, perhaps more appropriately, Digital Restrictions Management). There can be no problem for Microsoft and their monopoly to open up their formats if they can then close them up again because the documents generated in their operating systems are subject to DRM unlock codes that are impossible to read by other platforms and that are actually illegal to circumvent under US 26 and European 27 laws. And Windows Vista together with MS- Office 12 fully integrates DRM and the unlock codes 28 that lock in the content so it can be accessed only by certain hardware and software. Finally, we should mention the factor that most militates against EOOX, which is that after just over a year of development, compared with the five years of OpenDocument, there is currently not one application on the market that supports EOOX. Not even its sponsor and promoter, Microsoft, has a commercially available product that supports EOOX. Right at this moment it is nothing but a promise: pure vapourware Conclusions The facts outlined above answer a great many questions: should we really be returning to the times when standardization bodies came up with convoluted theoretical standards that were practically impossible to meet? Or, should we be doing so merely to satisfy the commercial interests of a company that has been declared a monopoly by the authorities of the world s two largest markets? Hadn t we put that mistake behind us? What is the point of having a specification of thousands upon thousands of pages that merely reflects the way that one particular application and one particular company does things, a specification for which there is no practical possibility of any other competitor applications supporting it? Is the idea to open up and ensure free markets in which com- 19 Effort estimation for implementing EOOX by Rick Schaut of Microsoft: "Open XML converters for Mac Office". < blogs.msdn.com/rick_schaut/archive/2006/12/07/open-xml-converters-for-mac-office.aspx> (last accessed ). 20 Effort estimation for implementing EOOX by Andrew Shebanow of Adobe: "Is Office Open XML a one-way standard? Ask Microsoft". < (last accessed ). 21 Information Week: "Microsoft s XML Standard Needs Fast Track Approval To Halt Defections", with commentary by Sam Hiser. < (last accessed ). 22 Groklaw s analysis of intellectual and industrial property rights in ECMA Open Office XML. < articlebasic.php?story= > (last accessed ). 23 Unisys demands payment from all users of the GIF image format. Explanation from the FSF. < (last accessed ). 24 Sam Hiser, "Analyzing the Microsoft Office Open XML license". < analyzing_the_m.html> (last accessed ). 25 Wikipedia, DRM. < (last accessed ). 26 US, Digital Millennium Copyright Act (DMCA) analysed by the EFF. < (last accessed ). 27 EU Directive 2001/29/EC "on the harmonisation of certain aspects of copyright and related rights in the information society" according to Wikipedia. < _copyright_and_related_rights_in_the_information_society> (last accessed ). 28 Jeff Farris, "Remote Attestation". < (last accessed ). 29 Daniel Eran, "Microsoft s Yellow Road to Cairo". < AD D7D.html> (last accessed ). 36 UPGRADE Vol. VII, No. 6, December 2006 Novática

38 petition lowers prices and drives innovation or, conversely, is it a matter of locking everyone into a single application driven by the commercial interests of a monopolistic manufacturer thanks to its patents, DRM, whopping specifications, etc.? As we have seen in the last point of this article, EOOX doesn t even have any technical advantages over ODF. In fact, as is mentioned on the OpenDocument Fellowship website, we have yet to find a single thing that EOOX can do that OpenDocument cannot. Not one. Finally, in this day and age it makes no sense for the public authorities to consider having two standards when practically nobody, with the exception of one company in particular, will benefit in the slightest from such a measure. It would be like reintroducing Turkish miles just so the metric system had some competition. Is the metric system in need of competition? I think not. Official open standards that do the job required of them, as OpenDocument does, should not have, nor do they need, unnecessary alternatives; on the contrary, they should be demanded and promoted by the public authorities. Where there should be competition and freedom of choice is between manufacturers and their applications, not between standards of more than sufficient quality. What is the point of causing errors, chaos, and confusion where there is a magnificently productive harmony? All these conclusions are drawn in the context of a situation in which OpenDocument has the backing of practically all the office automation market. This is the best reflection of what future electronic document formats for Humankind: open, universal, and free standards, like the alphabets, languages, and paper making techniques of the past which have allowed us to benefit from the wisdom of our forebears. Meanwhile, ECMA/MS-Open Office XML s only contribution is to prolong unnecessarily a history of users being locked into the formats and applications of a series of single companies, regardless of the greater or lesser market share they may have had or whether or not they were monopolies at the time. Thus the possible acceptance of EEOX as and international standard and its success on the market can only mean a return to the Middle Ages of electronic documentation. To continue living in an age in which Humankind suffered the hijacking and censorship of knowledge at the hands of a few feudal lords with special rights that are denied to the rest of us mortals. In short, possibly one of the worst obstacles to the development of the Information Society that we have ever known. In short, software should be designed to meet the standard, not the other way round. Humankind already has a document standard. Its name is ISO 26300, OpenDocument. and Tim Bray (Sun Microsystems), to Marino Marcich (ODF Alliance), Sam Hiser (OpenDocument Foundation), to Rafael Rodríguez (Universidad de Cádiz) and any others that I may have inadvertently left out, without whose tireless efforts in the study and promotion of open document formats this article would not have been possible. Translation by Steve Turpin Acknowledgements My thanks go to Alex Hudson, J. David Eisenberg, Bruce D Arcus, Daniel Carrera and Marco Fioretti (OpenDocument Fellowship), to Rob Weir and Bob Sutor (IBM), to Simon Phipps Novática UPGRADE Vol. VII, No. 6, December

39 Interoperability: Will the Real Universal File Format please Stand Up? Sam Hiser and Gary Edwards This article compares the two solutions that are currently competing to become the universal file format standard: ODF (OpenDocument Format), from the OpenDocument Foundation, and EOOXML (ECMA Office Open XML) from Microsoft. The article analyses the two XML-based formats both from a technical point of view and in terms of their future impact on enterprises, focusing on their true interoperability and their ability to achieve maximum fidelity in the conversion of legacy files. The large print giveth and the small print taketh away. Tom Waits, Small Change, 1976 Keywords: EOOXML, File Conversion, Interoperability, ODF, Standards, Universal Format, XML. 1 Introduction Here we are at a crossroads in standards, with the connectivity future of desktops, devices and enterprise systems up for grabs. There are two implementations of XML that can span the horizons of this emerging universe, and the world is waiting to see how these solutions shake out. It s decision time. Enter our two contenders: OpenDocument Format (ODF) and ECMA 1 Microsoft Office Open XML (EOOXML). Microsoft states they are interoperable. XML s promise of fluid transformation will be fulfilled. The ODF crowd says no way, ODF is the only universal file format worth considering. There is no need to make a decision, Microsoft insists. If you re using Microsoft applications, running the EOOXML stack, then EOOXML will be your format of choice. For the 485 million users of MS Office on Windows, locked into decades of legacy binaries, building out their daily business processes based on those binaries, then EOOXML is for you. And only EOOXML. If you re using anything else, ODF will be your choice. Microsoft is pleased to accommodate the choice between document standards [sic] because Microsoft is working as we speak on ensuring the two formats interoperate. After all, both ODF and EOOXML are XML. Right? Wrong! And therein lies the tale behind a decision that will rule the future. Interoperability and easy transformation is the hallmark of XML. But that s not what s happening here. Microsoft of course will do everything possible to make the interoperability dream seem real, including forge an 1 ECMA stands for the European Computer Manufacturers Association. Authors Sam Hiser is Vice President and Director of Business Affairs at the OpenDocument Foundation. He was OpenOffice.org marketing project leader and along with Tom Adelstein he is the author of "Exploring the JDS Linux Desktop". He contributes regularly to the Financial Times Digital Business < section. Sam holds an MBA is from Duke University s Fuqua School of Business. <sam.hiser@opendocument.us>. Gary Edwards is President of the OpenDocument Foundation, and founding member of the OASIS Open Office XML Technical Committee. He has 20 years of network technology experience. He has spent 12 of those years in the financial services and real estate industries, specializing in the design and implementation of new business models based on emerging Internet technologies. His technology consulting work is that of designing and deploying business process reengineering solutions. <gary.edwards@opendocument.us>. elaborate set of agreements with a weak Linux and OpenOffice vendor to demonstrate Microsoft s commitment to customer requirements. Look closely and you ll see that the only customer requirement Microsoft is focused on is extension of the old document lockins into the new world of the Internet, XML and SOA. Step back, take it all in, and you ll see that the ultimate objective is to extend the desktop monopoly into servers, devices and globally connected information systems. This time, it s for all the marbles. Are there important technical and legal issues with portions of the Microsoft-Novell technology-sharing deal? Unequivocally, yes. Different constituencies each have distinct reservations about different facets of the deal. Open source developers and the GPL license are adversely affected by the patent stand-still agreement; competing Linux distro and open source developers are understandably sore about the spurious patent protections as well as the virtualization and sales cross-collaboration arrangements; and users and governments are rightly worried that the Microsoft-Novell of- 38 UPGRADE Vol. VII, No. 6, December 2006 Novática

40 fice suite "interoperability" commitments will limit their options. Our concerns today focus in fine detail on the latter aspect of the Microsoft-Novell deal, and place things into the larger context of ISO, SOA, and the future of highly advanced and extremely productive information processing chains that span desktops, servers, and devices. Office Open XML ("The companies joint project to develop a facility for converting between OpenDocument, ODF, files and Microsoft/Ecma, EOOXML, files") is fraught with problems. It offers itself as an "interoperability" solution between Microsoft Office and OpenOffice.org. It is not 2. Even on the face of it, Microsoft s specification for EOOXML is gravely flawed and impossible for non- Microsoft developers to fully implement. So it will have serious problems being accepted by ISO as an international standard. But if you understand only a few of the severe problems with EOOXML, the anti-competitive underpinnings of this "interoperability" project, and the deal between Microsoft and Novell, show through. 2 Migrating Microsoft s Legacy Files to XML Everyone agrees that XML is certain to be the next-generation format for documents. XML is after all both the future of the Internet, and, the heartbeat of any SOA. It s not just "the future", it s also the bridge between where we are today (our legacy infrastructure) and what we need to do to get to that future. Microsoft wants to "pull up" customers into their own proprietary version of XML ("proprietary XML" is undoubtedly a non sequitur, but we ll let that slide for the timebeing); the ODF community wants to get everyone using its universal & portable implementation of XML; and customers are rightly confused because the Microsoft option looks acceptable, if expensive. Besides, the "EOOXML or ODF" non-decision decision (as in both are acceptable) might preserve a CIO s career. Assuming people trust Microsoft, and while people agree Microsoft is only looking after customers interests (as usual), this non-decision decision looks safe. Even if people are skeptical of Microsoft s motives, the well-entrenched belief exists that the Microsoft option is simply unavoidable due to the market deployment power of the Monopolist. And, on the other side, the ODF option (despite its two year lead in the ISO standardization process) still looks a little under-articulated and lacking integration across the business processing chain. Fair enough. But ODF is credible and offers early adopters today (governments in particular) some really appealing benefits of 2 Interoperability normally imports a sense of a complete absence of barriers. See e.g., ISO/IEC , as quoted in Wikipedia: ("[t]he capability to communicate, execute programs, or transfer data among various functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of those units"). perpetual data access, lower cost, and new flexibility in systems procurement (more on SOA below). The benefits really light up for an organization when it successfully gets through a migration and starts originating documents in an open implementation of XML. The case for ODF is well-understood by those who have announced commitments to it (they know who they are). Yet their first concerns include a) how do I get my legacy Microsoft documents into ODF; b) how do I cope with mixed formats and mixed systems during and even long after I transition my organization; c) how do I cope with irregularities introduced through document round-tripping in regular business processes across different systems in- and outside my organization; and d) how can I use open standard XML documents to shift my development efforts away from application-layer APIs particularly those owned by a sociopathic company known for cooking a second set of books on the APIs it owns? This is why the "interoperability" word is out and about. With XML on the horizon, customers are all over interoperability. They are hammering on Microsoft and the ODF community to solve the file format incompatibility problems that have plagued document behaviors inside the Microsoft family of formats and out. Accordingly, Microsoft has fallen in love with the word "interoperability" if not the actions to achieve it. The mandate driven by customers is that they want server and device side systems to integrate fully with the Windows MS Office desktop. At the exact point of integration, they want an open and application- and platform-independent XML file format connecting those MSOffice desktops to server and device side systems. In short, information technology customers want full and uncompromised ownership over both their information and their information processes, control over the day to day information and business processes on which their workflows and services are based. But Microsoft is reluctant to make it easy for customers to leave formats it controls. In truth, Microsoft knows its own secret formats perfectly. Perfectly. Any company that could sync perfect conversion fidelity between the billions of binaries and EOOXML, either natively or using a plugin, could easily provide a 100% fidelity with ODF within a few weeks of development. The comparative structural advantages of ODF over EOOXML are so compelling that, if not for the business case of advancing and extending the monopoly, there is technically every reason for Microsoft to have chosen ODF over EOOXML. For customers "interoperability" means 100% file fidelity from one application to another, either of the same or different kind, regardless of underlying platform. The requirement is not complicated and has been articulated clearly in Massachusetts in the ITD Request for Information about an ODF Plugin for MS Office. One of the trickiest jobs in Christendom is Microsoft s challenge to "pull up" (as they euphemistically call it) all Microsoft half-a-billion-odd customers into a modern, supportable version or two of their OS, application and file Novática UPGRADE Vol. VII, No. 6, December

41 format technologies. Due to lusty appetites, hard work, unscrupulous leadership and a sociopath ic competitive culture, the company has revved the customer base on so many versions that their principal competitor is themselves. (A sub-text of the main challenge, that of selling, is the need to master the intricacies of EOL-ing (end-of-life-ing) older versions without drawing attention from regulators or infuriating customers beyond a certain tipping point.) Maybe Redmond feels it s pushed this game once to often, but we re now witnessing a change in strategy. MSOffice 2007 is a transitional application, very ambidextrous in that it has one EOOXML for legacy Win32 API bound BoBs (billions of binary documents), and a different EOOXML for native, Vista -.NET 3.0 API-bound documents. In fact, it s possible to have the exact same document in MSOffice 2007, one version a native and the other a legacy, with different EOOXML internals and dependencies. What gives? While no one knows for sure, it does look like the point of lockin has moved from that of MS Officebound documents and business processes to an information processing chain comprised of MSOffice 2007 <=> EOOXML <=> IE 7.0 <=> and the Exchange/SharePoint Hub. It is the E/S Hub where the new lockin occurs because all EOOXML documents are converted here to the Vista -.NET 3.0 API-based dependencies. MS Office is just the head-point of this processing chain. From the E/S Hub, expect accelerated EOOXML connectivity to MSLive, MSN, MS-ERP, SAP, MSSQL Servers and backend transaction processing systems, Active Directory, collaboration Server and MSOffice Server systems (to name but a few). Expect also that the EOOXML ready VSTO 2005 will grease the skids for migrating existing MS Office-bound business processing systems to the E/ S Hub, as well as provisioning the rapid development of EOOXML-ready line of business applications able to fit nicely into the EOOXML-dominated information processing chains. Now that s integration! Or, as they say in Redmond, "Interoperability by Design". At this point we have to ask ourselves if competitive, non-microsoft directed, software systems and services can interoperate within this extraordinary processing chain design? I mean, imagine an OpenOffice desktop, or MAC Office desktop participating in this chain? How about Google, Yahoo, or Amazon Web 2.0 mashup services? Or Oracle backend ERP, relational databases, and transaction processing systems? Or IBM s Lotus Notes? Or maybe BEA s SOA model? Now, do you understand what s at stake? Oh yeah. Bye bye Adobe. So what s at work here anyway? Interoperability (real round-trip interoperability, such as that provided by a universal file format) is important to business processes of all kinds based on the way people move documents across a widening variety of systems. The business process of greatest relevant concern from the standpoint of antitrust law and the effects on competition is an organization s migration of its legacy documents stored in Microsoft Office binary formats to the OpenDocument Format ("ODF"). If such a migration can be accomplished then it can be said that competition with Microsoft s new business processes stack is feasible. But if such migrations are infeasible, Microsoft maintains its vendor lock-in on existing customers - not only in the office suite market but also in the business processes arena. Yes, the BoBs, again (those "Billions of Binaries"): those billions of legacy Microsoft Office documents stored in binary formats, the BoBS that Microsoft reminds us of so frequently when arguing for adoption of EOOXML as a standard. The Novell-Microsoft-CleverAge file converter is limited to EOOXML-to-ODF conversions and does not directly convert between ODF and the secret Microsoft legacy binary file formats, still used by Microsoft Office for internal processing. 3 The Novell-Microsoft-Clever Age ODF Translator is Intentionally Half-Baked For at least four reasons, the file converter being developed by Clever Age, paid for by Microsoft, and inserted into the Novell-only version of OpenOffice, will never be capable of full interoperability between Microsoft Office and applications supporting ODF like OpenOffice.org. The reasons have to do with fundamental characteristics of both the EOOXML format as well as the Clever Age-Microsoft technical approach, XSLT. Microsoft knows in advance that XSLT is inadequate. This comes as no surprise. Bill Gates never bets on anything unless he s holding all the cards. And as long as people are thinking EOOXML is just XML, and that any XML can be easily and universally transformed with XSLT, his three ace s in the hole are safe as can be, (95% desktop marketshare, perfect fidelity conversion BoBs, and a EOOXML bound to both the legacy Windows API and the Vista -.NET 3.0 API internals). 3.1 Other Developers Cannot Feasibly Implement all of EOOXML Alongside the problem of EOOXML s dependence on obscure RTF instructions, there are a host of tags in the EOOXML specification that can not feasibly be implemented by anyone but Microsoft. Here are only a few examples collected by Ben Langhinrichs < geniisoft.com/showcase.nsf/archive/ >: autospacelikeword95 (Emulate Word 95 Full Width Character Spacing) - pages footnotelayoutlikeww8 (Emulate Word 6.x/95/97 Footnote Placement) - pages linewraplikeword6 (Emulate Word 6.0 Line Wrapping for East Asian Text) - pages mwsmallcaps (Emulate Word 5.x for Macintosh Small Caps Formatting) - pages shapelayoutlikeww8 (Emulate Word 97 Text Wrapping Around Floating Objects) - pages suppresstopspacingwp (Emulate WordPerfect 5.x Line Spacing) - pages truncatefontheightslikewp6 (Emulate 40 UPGRADE Vol. VII, No. 6, December 2006 Novática

42 WordPerfect 6.x Font Height Calculation) - pages useword2002tablestylerules (Emulate Word 2002 Table Style Rules) - pages useword97linebreakrules (Emulate Word 97 East Asian Line Breaking) - pages wpjustification (Emulate WordPerfect 6.x Paragraph Justification) - pages These are deprecated features from old Microsoft Office versions that are being carried forward only for the purpose of backward compatibility. Nearly all such tags in the EOOXML specification are accompanied by a bit of repeated "guidance" that only a cynic might view with some sense of hilarity or which, in fact, prompts belly laughs from the file format cognoscenti: "To faithfully replicate this behavior, applications must imitate the behavior of that application, which involves many possible behaviors and cannot be faithfully placed into narrative for this Office Open XML Standard. If applications wish to match this behavior, they must utilize and duplicate the output of those applications.". The EOOXML specification and file format unquestionably contain hundreds of processing instructions that in the aggregate can only be processed feasibly by Microsoft Office. It is a single-vendor file format, a file format that treats software as an endpoint rather than as a router of information in any business process software stack other than Microsoft s own tightly-integrated stack. Those application-specific tags might as well be binary blobs as far as any non-microsoft application is concerned. The tags functionality is concealed in the unspecified Microsoft Office application layer rather than being specified in the EOOXML standard adopted by ECMA. The existence of such tags belies any Microsoft claim that EOOXML was designed for interoperability, a claim expressly made in the Ecma final draft specification (< TC45_current_work/TC _final_draft.htm>), Part 1: "The goal is to enable the implementation of the Office Open XML formats by the widest set of tools and platforms, fostering interoperability across office productivity applications and line-of-business systems, as well as to support and strengthen document archival and preservation, all in a way that is fully compatible with the large existing investments in Microsoft Office documents.". (Bold letters added by the authors). Right, just blink past the fact that neither the specifications for the legacy binary formats nor the APIs used for the conversions will be found in the EOOXML specification; they remain a closely-guarded Microsoft secret. All of 3 For a more detailed critique of EOOXML within the context of a Service-Oriented Architecture, see the article "IBM s potential MS-Office killer to roll out by year s end" < talkback.zdnet.com/ html?forumid= 1&threadID=13561&messageID=273162&start=-4> by Gary Edwards. 4 IBM s Rob Weir has since published a satirical counterpoint to Sutor s earlier article < 2006/12/how-to-write-standard-if-you-must.html>. that "compatibility" with the binary formats is reserved for the exclusive use by the products of a single vendor, Microsoft. You can begin to see the modus operandi resolving clearly. Through systematic obscurity and audacity s sleight of hand, Microsoft would have ISO grant it an exclusive monopoly on migrating legacy Microsoft Office files to XML. To what flavor of XML? Why, to Microsoft s own. Never mind that Microsoft refused to participate in developing the OpenDocument XML international standard that ISO has already adopted. Never mind that Microsoft refused to provide native support for OpenDocument in Microsoft Office. Never mind that since February of 2003, ODF has had the capability of rendering perfectly any and all BoBs, application specific processing instructions and binary blobs included. Blink and you ll end up back in 1995: software as an end-point. Obviously, if one follows the logic employed by Microsoft and Ecma, software consumers will need yet a third and maybe a fourth file format implementation of XML for office documents. One to translate WordPerfect Office files to XML, and yet another for IBM s Lotus Notes. How then is the new age of XML office formats any substantial improvement over the cacophony of binary formats that have stymied full interoperability these many decades? The inclusion of hundreds of vendor-specific tags is not EOOXML s only weakness, of course. As only three more examples, the EOOXML specification also suffers from inappropriate reliance on Windows operating system bitmasks, on Microsoft application-specific bug work-arounds < and a Microsoft proprietary custom XML parser < 3 There is scant wonder that Microsoft found it necessary to pay Novell a large sum of money for even a partial implementation of EOOXML in OpenOffice.org. 3.2 EOOXML is a Vendor Lock-in Format The sheer complexity of the EOOXML specification, some 6,000 pages, atop its reliance on application-specific tags, Windows bitmasks, and Office bug work-arounds translates to vendor lock-in. It means that no one but Microsoft will be able to support the specification anywhere near completely. That has obvious consequences, as explained by IBM s Bob Sutor 4 < blog-open/?p=1145>: "Fully and correctly implementing [EOOXML] will require the cloning of a large portion of Microsoft s product. Best of luck doing that, especially since they have over a decade head start. Also, since they have avoided using industry standards like SVG and MathML, you ll have to re-implement Microsoft s flavor of many things. You had better start now. So therefore I conclude that while Microsoft may end up supporting most of [EOOXML] (and we ll have to see the final products to see how much and how correctly), other products will likely only end up supporting a subset.". Novática UPGRADE Vol. VII, No. 6, December

43 That means that other products and software, in practice, will NOT be able to understand arbitrary [EOOXML] that might be thrown at them. There is just too much. Therefore they will only create a bit that they need and send that off. Send it off to whom? The only software that might understand it, namely Microsoft Office. So this is how I see this playing out: [EOOXML] will be nearly fully read and written by Microsoft products, but only written in subset form by other software. This means that data in [EOOXML] form will be largely sucked into the Microsoft ecosystem but very little will escape for full and practical use elsewhere. (See also Sutor s later article including graphics depicting the above concepts, discussed in more general terms and exploring the difference in meaning between, "interoperability" and "intraoperability". < newsite/blog-open/?p=1260>). It is accurate only in theory for Microsoft to claim that the EOOXML file format can be implemented by anyone. As a practical matter, full implementation by other developers will not occur. Unless perhaps, Novell will commit to reverse engineering all of those features of previous Office versions invoked by those hundreds of tags, Windows bitmasks, and Microsoft Office bug work-arounds. 3.3 Novell and Microsoft Do Not Intend to Achieve Interoperability In light of the gratuitous complexity of the EOOXML specification, it is especially significant that Microsoft s Steve Ballmer said of EOOXML-to-ODF file converter being jointly developed by Novell and Microsoft < WEWEMNL103006EP17A>: "Nor will the collaboration team attempt to build file converters that can make files 100 percent compatible between the two file formats", he said. "But it will achieve the level of interoperability that customers can work with", he said. Novell s OpenOffice.org development team leader Michael Meeks agrees: "what Ballmer says is true in a way; sure - it s likely that 100% interoperability between any two non-trivial apps is never going to be possible. That of course is misleading, as long as interop is -good-enough- people are unlikely to miss the last 2% (or whatever). At some people the return on investment of adding the next 1% becomes so low that people don t bother :-)". However, at least one developer who has actually grappled with converting Microsoft s binary file formats into OpenDocument disagrees. The OpenDocument Foundation s Gary Edwards, who also represents the OpenOffice.org Community in the OASIS OpenDocument Technical Committee, says that Mr. Meeks "or whatever" is substantially larger than 1 or 2 percent when migrating Microsoft binaries to OpenDocument using EOOXML as an intermediary format: "The ODF transformers Ballmer and Novell both refer to are actually the same CleverAge/Microsoft ODF Translator filters that are based on XSLT with supplemental C # routines - routines that prove beyond doubt that EOOXML is not XSLT-ready. The CleverAge converter will be the central part of the plug-in being developed for Novell OpenOffice. Ballmer is right in that no one will ever get anywhere near 100 percent fidelity that defines interoperability using XSLT methods.". The reason why XSLT will never work in this situation is that XSLT needs highly structured XPath-perfect XML to perfect a transformation. ODF was written to be this XPath-perfect XML structure that XSLT needs. However, EOOXML is anything but. "The structural deficiencies in EOOXML that render XSLT next to useless (say an optimal 60% fidelity as compared to the lossy 85% fidelity or so of the traditional reverse engineered filters in OpenOffice.org considered by all who have ever tried to be an extraordinary achievement), center on the loosely structured styles model (presentation) in EOOXML. And this loose half structure is in turn directly related to the simple layout engine of MSOffice. (Simple in comparison to the rich and very complex layout engine in OpenOffice).". "The Ballmer statement that neither Novell, Microsoft, nor CleverAge will attempt to hit 100% fidelity is in effect a statement that they intend on sticking to XSLT as the transformation method.". Moreover, when it comes to software as a router of information in a business process, where the same data must be read and written by a series of different applications, only full interoperability is workable. A business process that allows loss of data on every trip from one app to another is not "interoperability." Saying "the level of interoperability customers can work with" makes about as much sense in context as saying "partially pregnant." Indeed, the Ecma white paper on EOOXML < openxmldeveloper.org/archive/2006/10/24/764.aspx> tells us in section 4.6 that only "high fidelity" translations will work when migrating the legacy Microsoft binary formats to XML: "High Fidelity Migration OpenXML is designed to support all of the features in the Microsoft Office binary formats. It is difficult to overstate the difficulty of accomplishing this goal, and the consequent uniqueness of OpenXML in doing so.... OpenXML is intended to permit future editing or manipulation at the same level of abstraction available to the original creator[.]". But only if Microsoft Office is the editing tool. See also id., section 4.7 ("Integration with business data"). 3.4 Market Requirements Will Not Be Fulfilled by the Joint Development Project Despite Mr. Ballmer and Novell s claim that lossy "good enough" migration from Microsoft formats to ODF is sufficient, laws around the world effectively mandate 100 percent fidelity in migrating binary documents to XML. See e.g., E- SIGN Act, 15 U.S.C. 7001(d)(1)(B) (electronically preserved records must "accurately reflect[] the information set forth in 42 UPGRADE Vol. VII, No. 6, December 2006 Novática

44 the contract or other record" and be "in a form that is capable of being accurately reproduced for later reference, whether by transmission, printing, or otherwise"); Sarbanes-Oxley Act, 15 U.S.C. 7261(b) (financial information must "not contain an untrue statement of a material fact"). The "good enough" lossy migration of legacy records to ODF contemplated by the Novell-Microsoft deal is not nearly good enough in such a legal environment. A single user may be able to convert a single file and then manually check to see that no data was lost. It is an entirely different situation when documents must be successively roundtripped or in the case of mass- or wholly-automated conversions of files. 4 The Market Requirement for Interoperability Can Be Fulfilled Novell s Miguel de Icaza evaded rather than answered the following important question that Pamela Jones asked: "Also, please tell us all what is wrong, in your view, with the Sun or the ODF Foundation [sic] solutions and can you please explain to us the differences between what yours will do and what they do?". Icaza answered: "Michael [Meeks], the head of our OpenOffice team which happens to be visiting town says: the ODF Foundation Solution is not free software; The Sun one is unpublished ". Contrary to Mr. Meeks statement, the OpenDocument Foundation has not yet made a final decision on licensing of its file conversion products under development, but is decidedly leaning toward, and has entered into discussions with ODF community leaders, licensing under the free and open source software GPL license. As to the differences among the solutions, Sun s solution is a C# wrapper around OpenOffice.org that uses OOo s traditional file import-export facilities. According to Edwards, it is able to achieve only around 85 percent fidelity at best in translating between Microsoft binaries and OpenDocument. At the heart of this method is the same filter process now used in OpenOffice 2.0. There is no edge where "improvement" might occur. The Microsoft/Clever Age group is working on XSLT-C # filters that translate between EOOXML and OpenDocument. Given the structural challenges for any XSLT based method, CleverAge ought to get a technology of the decade prize if they can hit at best 60 percent fidelity. The Foundation tools however are designed to achieve high-fidelity by bypassing the EOOXML trap, working directly with the Microsoft Office modified RTF conversion formats. One Foundation tool is the ODF Plug-in for Microsoft Office, a plug-in dubbed "da Vinci" that uses the same Office internal APIs Microsoft uses for file formats it supports natively. The other is a standalone ODF XML Infoset API that can be used in applications (on the serverside, too) that support ODF to import and export Microsoft binary formats. A third is a da Vinci plugin for OpenOffice.org. And a fourth is a "interoperability wizard" designed to guarantee that OpenOffice workstations caught up in a MSOffice driven business process - workflow will produce perfectly interoperable documents. Edwards said that the da Vinci plug-in and OpenDocument Infoset API will be able to achieve 100 percent fidelity in translating between Microsoft binary formats and ODF for applications that correctly implement OASIS OpenDocument Format specification version 1.2. Version 1.1 of the ODF specification focuses on assistive technologies, and was recently accepted under a vote of the OASIS OpenDocument Technical Committee. ODF version 1.2 is the equivalent of interoperability on steroids, and will have the attention of the TC in January and February of Edwards advises that Microsoft Office 2007 users change the suite s default settings and save their work in the traditional binary formats rather than EOOXML in order to simplify migrations to ODF after the Foundation s tools are released. But he says there are even stronger reasons for avoiding EOOXML: "Once Microsoft Office-bound business processes are moved over to EOOXML, those processes are ready for migration to the Microsoft Exchange/ Sharepoint Hub centered information processing chain. Organizations who fall into this business process migration trap will not be coming off the Microsoft processing chain anytime soon. They might as well cut a long-term leasing deal with Microsoft good for at least the next fifteen years. They should also make certain the deal covers the expense of the eight different desktop productivity applications and nine different server-side systems needed to expand and enhance these highly productive, but EOOXML chain-bound, business processes.". The Foundation s tools are the only ones thus far brought to public attention that promise full interoperability between Microsoft Office (and its binary formats) and applications that support ODF. Foundation developers began with the last public documentation of the Microsoft Office file conversion APIs before Microsoft decided all future enhancements would be secret, then from there the Foundation deduced the nature of the secret tags. Our da Vinci plug-in adds native support for ODF to Microsoft Office, something that none of the other forthcoming tools will offer. Microsoft Office can then be used in batch mode as a "pump" to migrate documents from the legacy Microsoft binary file formats to OpenDocument format with unmatched fidelity. The salient point in the Foundation s strategy is this; boost the creation of alternative ODF information processing chains that can compete with the EOOXML dominated chains. MSOffice bound business processes are going to migrate to highly productive E/S style hubs. Nothing is going to stop or alter this migration. The only question is, will these hubs be EOOXML or ODF ready? So it doesn t make sense to totally disrupt current workflows and services to re write these business processes to ODF ready desktop alternatives. What does make sense is to first, get these MSOffice bound business processes into ODF. Get the documents and document/data flows into ODF. This is the job of da Vinci. Second, migrate those MSOffice bound business processes to ODF ready Hubs! This is the primary Novática UPGRADE Vol. VII, No. 6, December

45 purpose of our ODF InfoSet Engine - API; a lightweight server and device side engine that can ODF automate applications instantly. Full fidelity in such migrations is feasible only because ODF was intentionally designed for interoperability and includes features specifically designed for interoperability with Microsoft Office. ODF implements a system whereby unrecognized and foreign elements and attributes are preserved by a conforming application. See generally OpenDocument specification section 1.5 < develop.opendocumentfellowship.org/spec/>. (But note that occurrences of "should" and "may" in that section are expected to change to "must" in version 1.2 and further interoperability refinement of the specification is planned.). Those requirements (generally referred to by the ODF development community as foreign elements and unknown elements, or simply as the Microsoft tags) moot Microsoft s claim that a separate standard is needed to ensure compatibility with those billions of legacy Microsoft Office binary files. The da Vinci plug-in has been developed well beyond proof of concept stage and has been demonstrated for multiple government bodies around the world, including one demonstration in Europe that was attended by Microsoft officials; who publicly announced their lossy ODF XSL Transformer project the very next day, a somewhat inconsistent action for a company that claims to be a champion of software interoperability. 5 The Novell-Microsoft Deal Raises Antitrust Issues When the Novell-Microsoft deal was announced, and to this day, both companies proclaimed that customers were demanding interoperability between the Microsoft and ODF stacks of software. That, and the need to share virtualized operating system technology, are the major justifications for the deal, both companies said. However, it is also beyond question that both companies realized their deal will not deliver what the market requires as to interoperability. Instead, the deal cements non-interoperability into place for five more years, actually degrading the level of fidelity presently achievable by OpenOffice.org s import/export filters by diverting customers from conversions of the binary formats onto even more lossy XSL Transformation tools. Any candid discussion of the Novell-Microsoft deal must begin by asking both companies: [i] why they are refusing to supply what the market requires; and [ii] why they have cemented their joint refusal into a binding contract. Both companies are very apparently aware that full interoperability is feasible. But the Novell-Microsoft deal so far looks less like an "interoperability deal" and far more like a prohibited agreement not to compete and an unlawful allocation of the office productivity software market, a Sherman Act combination in restraint of trade. 5 Under the deal, Novell is allocated the market for OpenDocument-based office productivity software and Microsoft is allocated the market for corresponding EOOXML-based software. The apparent conspiracy has each company s agreed share of the market shielded from competitors by an undefined patent thicket publicly asserted by the other company in the form of a covenant not to sue the other s paying customers. The highly-publicized covenants not to sue implicitly threaten anyone who uses other products with expensive patent infringement litigation. Novell s knowledge that far better interoperability is feasible is amply demonstrated by the fact that two days before the Novell-Microsoft deal was signed and announced, Novell head-hunted the OpenDocument Foundation s chief technology officer, and world re known document conversion - document processing expert, Florian Reuter. Novell unquestionably knew what he was working on before it signed on the dotted line with Microsoft. Moreover, Microsoft made no secret of the fact that it could easily provide full native support for OpenDocument in Microsoft Office. Former Massachusetts Secretary of Administration & Finance Eric Kriss: "... said technical people at Microsoft told him it would be "trivial" to add support for ODF to the new Office The resistance to doing so came from the vendor s business side, according to Kriss". < viewarticlebasic&articleid=273815&pagenumber=2>. It thus appears that both Novell and Microsoft knew that full interoperability was achievable and knew it was a market requirement, but conspired to ensure that software consumers would not be provided with the full interoperability they required, knowingly using joint development of inadequate XSL transformation tools as antitrust "cover" for an agreement to maintain separate non-interoperable software 5 Palmer v. BRG of Georgia, Inc., 498 U.S. 46, 50 (1990) (per curiam) < getcase.pl?court=us&vol=498&invol=46> ("[s]uch agreements are anticompetitive regardless of whether the parties split a market within which both do business or whether they merely reserve one market for one and another for the other"); see also 15 U.S.C. 1: < "Every contract, combination in the form of trust or otherwise, or conspiracy, in restraint of trade or commerce among the several States, or with foreign nations, is declared to be illegal. Every person who shall make any contract or engage in any combination or conspiracy hereby declared to be illegal shall be deemed guilty of a felony, and, on conviction thereof, shall be punished by fine not exceeding $10,000,000 if a corporation, or, if any other person, $350,000, or by imprisonment not exceeding three years, or by both said punishments, in the discretion of the court.". The courts have imposed a requirement that restraints of trade thus prohibited must be "unreasonable." A finding that a Sherman Act restraint of trade is unreasonable may be "based either (1) on the nature or character of the contracts, or (2) on surrounding circumstances giving rise to the inference or presumption that they were intended to restrain trade and enhance prices." NCAA v. Board of Regents of Univ. of Okla., 468 U.S. 85, 103 (1984), quoting National Society of Professional Engineers v. United States, 435 U.S. 679, 692 (1978). 44 UPGRADE Vol. VII, No. 6, December 2006 Novática

46 stacks and not to compete in the divided market. The Microsoft-Clever Age-Novell EOOXML-to-ODF transformer admittedly will not achieve full interoperability and therefore is not what the market requires. An agreement not to supply what the market requires in order to divide a market between two companies is blatantly anti-competitive. Novell could still salvage some of its shredded goodwill among the free and open source software community by solidly and publicly backing the OpenDocument Foundation solutions. Novell might also thereby avoid an otherwise predictable wave of antitrust litigation both in the U.S.and in Europe that Novell is far less likely to survive than Microsoft. Novell does, after all, have a freshly acquired set of deep pockets that will attract the sharks. But regardless of its course of action, Novell should stop spinning its deal with Microsoft as an "interoperability deal." The real basis of the deal is embarrassingly transparent and the company can only shed further credibility by continuing the charade. It should be remembered that in a business process, a file format is every bit as much a communications protocol as a method of storing data. The European Commission s DG Competition previously found < comm/competition/antitrust/cases/decisions/37792/en.pdf> that Microsoft s refusal to disclose its communications protocols for Windows and Windows Server to competitors was an antitrust violation. But DG Competition did not limit its remedial order to just the Windows communications protocols; it ordered Microsoft to refrain "from any act or conduct having... equivalent object or effect," an issue raised by the European Committee for Interoperable Systems when it filed its complaint < story= > with DG Competition because of Microsoft s refusal to support ODF and its refusal to disclose the specifications for its Office binary file formats. As Novell was one of the companies that instigated and participated as a party in the European Windows antitrust litigation, it seems more than somewhat incongruous for Novell to subsequently become a co-conspirator in Microsoft s bid to use the secrecy of its office file formats and file conversion APIs to thwart full interoperability with software that supports ODF. That is especially so in a deal also intended to share technology for virtualization of each companies operating system. What good is virtualization of operating systems if the business applications running on those concurrent operating systems can not interoperate? And is that not an "equivalent object or effect" of Microsoft s refusal to disclose its Windows communications protocols? EOOXML is open in name only. Novell knows full well what the law has to say on this issue. Indeed, Microsoft paid Novell 536 million to withdraw from further participation in the European Commission s antitrust case. Having successfully advocated to establish the injunction, Novell is in a poor position to contest its implementation. An argument that "we couldn t afford to turn down a second pay check" is not a strong defense to an antitrust conspiracy case, where a defendant is responsible not only for its own but also for its co-conspirator s acts and omissions. 6 Conclusion - The XML Cold War in a Changing Market for Software To fully grasp the vendor lock-in and legal bind Microsoft and Novell are devising, it is important to understand how ISO/IEC ("ISO") adoption of EOOXML one or two years from now would result in a legally sanctioned extension of Microsoft s monopoly in office document formats. Through its technology-sharing arrangement with Novell and its elaborate messaging around "interoperability," Microsoft audaciously seeks to reassert the old lock-ins while draping its behavior in the "open" language that is today in vogue. The fundamental legal problem is that ISO adoption of EOOXML as an international standard would make its use mandatory in many situations, in both the private and government sectors. By the same token, its use would be prohibited in many of the same situations if not approved by ISO. (See generally, Article 2 of the Agreement on Technical Barriers to Trade < booksp_e/analytic_index_e/tbt_01_e.htm#rticle2> and Article VI of the Agreement on Government Procurement < analytic_index_e/gpa_02_e.htm#rticle6>). Those treaties are intended, inter alia, to stimulate competition by promoting the development of de jure open standards, eliminating multiple standards where a single standard will suffice, and requiring use of adopted standards. Therefore, the ability of non-microsoft developers to implement EOOXML in their software is a crucial issue in assessing the suitability of EOOXML as a candidate international standard. But to comprehend the war raging over OpenDocument and EOOXML, one must first understand that the office productivity software market s ground rules are in a state of flux. We live in the age of the Internet. The age of universal access and exchange. The age of universal connectivity and collaborative computing. Things are changing. In the era we are leaving behind, office software was designed as an end point. The way to achieve interoperability was for everyone in an office and those exchanging files with that office to use the same software. Different vendors programs used different and incompatible file formats. Exchanges and info streams were entirely API bound. It was a Tower of Babel situation that ultimately resulted in one vendor s software Microsoft Office attaining a monopoly largely through a combination of file format incompatibilities and bundling of the software with new computers. But even as Microsoft achieved dominance in the office suite market, forces were at work that would undermine its dominance. One such factor was the ascendancy of the open standards-based Internet and ubiquitous networking. Another major factor was a growing crisis of complexity < /www-128.ibm.com/developerworks/library/wsmigratesoa/>: "Over the last four decades, software architectures have attempted to deal with increasing levels of software complexity. But the level of complexity contin- Novática UPGRADE Vol. VII, No. 6, December

47 ues to increase, and traditional architectures seem to be reaching the limit of their ability to deal with the problem. At the same time, traditional needs of IT organizations persist; the need to respond quickly to new requirements of the business, the need to continually reduce the cost of IT to the business, and the ability to absorb and integrate new business partners and new customer sets, to name a few. As an industry, we have gone through multiple computing architectures designed to allow fully distributed processing, programming languages designed to run on any platform, greatly reducing implementation schedules, and a myriad of connectivity products designed to allow better and faster integration of applications. However, the complete solution continues to elude us...". Now you find more complex environments. Legacy systems must be reused rather than replaced, because with even more constrained budgets, replacement is cost-prohibitive. You find that cheap, ubiquitous access to the Internet has created the possibility of entirely new business models, which must at least be evaluated since the competition is already doing it. Growth by merger and acquisition has become standard fare, so entire IT organizations, applications, and infrastructures must be integrated and absorbed. In an environment of this complexity, point solutions merely exacerbate the problem, and will never lead us out of the woods. Systems must be developed where heterogeneity is fundamental to the environment, because they must accommodate an endless variety of hardware, operating systems, middleware, languages, and data stores. The cumulative effect of decades of growth and evolution has produced severe complexity. With all these business challenges for IT, it is no wonder that application integration tops the priority list of many CIOs. To meet the challenge posed by that crisis of complexity, a new Service Oriented Architecture (SOA) began to emerge. SOA is in large part built around XML, the open and humanreadable extensible Markup Language as the building block for future expansion. As to data stored in relevant legacy file formats, SOA requires endless repetitions of the following workflow: [i] identify the location and form of the data specified by the requesting application; [ii] convert the data from the legacy file formats into a validxml format; [iii] programmatically extract the portions of the data to be repurposed; [iv] input the data to an XML transformation process; [v] output the data in the requested XML format; and [vi] serialize that data to the specified application in the workflow for subsequent processing. Notice in that somewhat over-simplified workflow that applications are waypoints, i.e., routers of information, rather than end points. Applications architected in the days of the sneaker net such as Microsoft Office have distinct disadvantages. Flawless migration of data between formats is integral to a SOA process. Workflows incorporating such steps are commonly referred to as business processes. Microsoft is able to read the tea leaves of industry trends as well as anyone else. Not surprisingly, Microsoft is developing its own stack of proprietary business processes software. That stack, the infamous information processing chain, uses EOOXML not only as a file format, but also as a communications protocol among the various applications. So EOOXML is far more than just a file format for an office suite. Like OpenDocument, it is designed as an interoperability tool for business processes within solutions such as a SOA. As will be shown below, the difference is that EOOXML is a proprietary closed specification. On the other hand, ODF is completely open. It is also no accident that the war between proponents of OpenDocument XML and of Microsoft Office 2003 XML (an antecedent of EOOXML) first came to wide public attention < &query=donnybrook> during the design of a Service Oriented Architecture by the both the EU Valoris Report, and the Commonwealth of Massachusetts with its Information Technology Division s Enterprise Technical Reference Model. The process of designing that architecture created the need to choose which XML file format would be the destination format for Microsoft Office legacy binary file formats. For a variety of reasons, Massachusetts ITD chose OpenDocument as that file format. Of course, SOA is only part of the story. A growing host of Web and Web-distributed applications support OpenDocument, including Software as a Service and nextgeneration Web 2.0 < fellowship. org/applications>. So here we are. It s decision time. Microsoft is offering a very compelling, fully populated, EOOXML based information processing chain that greatly leverages the existing monopoly base of installed MSOffice desktops, bound business processes, and locked in legacy BoBs. It is no doubt a business objective designed exactly to extend the monopoly from the desktop to servers, devices, and beyond. Awesome to say the least. Incredibly audacious. ODF on the other hand is designed and destined to be a universal file format transcending applications, platforms, archival needs, and information technology advances unknown. It is a universal file format servicing the needs of information domains as diverse yet begging interoperable connectivity and exchange as: desktop productivity environments, enterprise publication, content and archive management systems, SaaS, SOA, the Web 2.0, and beyond. EOOXML decidedly breaks the promise of XML; the ease of universal transformation and cross-generation interoperability. But fear not, perfect interoperability between ODF and the billions of binary documents is available in ODF 1.2. And then there is the ODF information processing chain promise of the Foundation s da Vinci plugin (its ODF Plugin for MS Office) and its ODF InfoSet API. ODF is ready. It s about 2 years ahead of EOOXML in the ISO standardization process. Let the battle begin! Acknowledgment Marbux, a retired lawyer, who is a volunteer member of the OpenDocument Fellowship, contributed this article s legal analysis. 46 UPGRADE Vol. VII, No. 6, December 2006 Novática

48 ODF: The Emerging Document Format of Choice for Governments Marino Marcich A growing number of governments worldwide have expressed support for the OpenDocument Format (ODF) since its adoption by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) as an international standard in May Marino Marcich, Managing Director of the ODF Alliance, provides a comprehensive overview of government policy decisions to move to ODF, the reasons why governments are doing so, and the broader effect of having governments committed to deploying ODF. Keywords: ISO/IEC 26300, ODF, ODF Alliance, Open Document Format. 1 Government Policy Adoptions of ODF No one favored way to implement ODF has emerged from the myriad public authorities that have announced support for ODF. To date, these policy actions have taken the form of laws, executive decisions, interoperability frameworks, or policy statements. The early-adopter governments or government entities are at varying stages of implementation. Some governments have made policy announcements, while others are in the process of transitioning government agencies to ODF in their electronic interaction with each other and the general public. Gartner, a provider of IT analysis and consulting services, believes that there is a 70 percent probability that by 2010, ODF document exchange will be required by 50 percent of governments and 20 percent of commercial organizations National Governments Belgium: On 23 June 2006, Belgium s Council of Ministers adopted a recommendation which would effectively introduce ODF as the preferred standard within its governmental agencies for the creation and exchange of text, spreadsheets, and presentations. 2 The guidelines state that all documents exchanged within the federal government must be in an open, standard format based on XML and implemented by more than one vendor. The Council is recommending a phased approach in which reading functionality would be implemented in the Belgian public administrations as of 1 September 2007, writing functionality by 1 September 2008, and document exchange in ODF by 1 October < iso_approval_of_oasis_ opendo_ pdf>. 2 < FEDICT_ OpenForumEurope_ pdf>. 3 < Open_Source_Software_Policy.pdf>. Author Marino Marcich is Managing Director of the OpenDocument Format (ODF) Alliance, a coalition of vendors, governments, universities, associations, libraries and archival organizations dedicated to educating policymakers, IT administrators and the public on the benefits and opportunities offered by ODF. Launched in March, 2006, the ODF Alliance has grown to more than 340 member organizations from 48 countries. To learn more about the Alliance, see < Croatia: As part of the Operational Plan for the Implementation of e-croatia Program for 2006, it is recommended that governments at all levels generate and archive digital content using open formats as part of the ecroatia plan. 4 Denmark: Denmark s Folketinget (parliament) unanimously decided on 2 June 2006 that by January 2008, all digital information exchanged between authorities and citizens, companies and institutions, should be available in formats based on open standards. 4 Also, all development and purchase of software for use in the public sector should at the latest by 1 January 2008 be based on open standards. The roadmap for implementing the decision is expected to be considered later this year. France: France s Direction Générale de la Modernisation de l État (DGME) specifically refers to ODF in its draft Référentiel Général d Interopérabilité (RGI), or Interoperability Guidelines. Under the RGI, which are generally followed by public administrations throughout France, it is required to be able to accept all documents in ODF, recommended to use ODF for office applications (text, charts, presentations), and prohibited to migrate to a format currently used by only one organization. 5 4 < 5 < domaines_d_expertise/architecture_ fonctio/public/rgi/ folder_contents>. Novática UPGRADE Vol. VII, No. 6, December

49 Malaysia: Malaysia s standards body voted to propose ODF as a country standard, following the recognition in May 2006 by the International Organization For Standardization (ISO) of ODF as an international standard. 6 After a public comment period in September, Malaysia s Ministry of Science, Technology and Innovation is expected to formally endorse ODF by year s end, recommending the format for use by the public sector. Norway: June 2005 policy paper "enorway 2009 the digital leap" states that by 2009 all new IT and information systems be based on open standards, and that by 2006 a set of administration standards for data and document exchange be established. 7 The Ministry of Government Administration and Reform has created a panel to establish standards for electronic information in the public sector. 1.2 Regions, States, and Municipalities Extremadura, Spain: By 25 July 2007, the government approved a motion that all public administrations must use ODF for document exchange and PDF/A "when guaranteed unalterable visualization is required." 8 Extremadura decided in 2002 to migrate 70,000 desktops to a local version of free, open source Debian software, called gnulinex. Massachusetts, USA: The Commonwealth of Massachusetts Enterprise Technical Reference Model of September 2005 states that ODF must be used for documents such as text, presentations, and spreadsheets. 9 It plans to implement ODF in a group of early adopter agencies, including the Massachusetts Office on Disability, by January 1, Thereafter, it plans to migrate all Executive Department agencies to compliance with ODF, in phases, by June of Vaud, Switzerland: The Swiss Canton (administrative district) of Vaud is switching to ODF. Its mayor-elect reports that the commitment to open software standards is spreading throughout Switzerland Government Agencies and Other Public Entities Several large government agencies and public entities deploying ODF for document exchange internally and with the public. Australia: The National Archives of Australia uses an open source-licensed program that converts other office file 6 < _a_ malays.html>. 7 < pdf>. 8 < consejo_gobierno _english.pdf>. 9 < 4&L0= Home&L1=Policies%2c+Standards+%26 +Guidance&L2 =Enterprise+Architecture&L3= Enterprise+ Technical+ Reference+Model+-+ServiceOriented+Architecture+ (ETRM+ v3.5)&sid=aitd&b=terminalcontent&f= policies_standards_ ETRMVersion3.5InformationDomain &csid=aitd> 10 < lechunk &&L= 3&L0=Home&L1=Open+Initiatives&L2= OpenDocument& sid=aitd&b=terminalcontent&f=accessibility_odf_ accessibility_midyear_ltr&csid=aitd>. 11 < formats to ODF for archival purposes. 12 India: India s Election Commission has adopted ODF nationally. Other government agencies include the Allahabad Courts, the Government of Delhi, the Department of Tax, Delhi, the Life Insurance Corporation (a government entity), and the Indian Army. 2 Government Procurement Support for ODF Governments influence the market not just through policy but through their purchasing decisions. Over 50 national, regional, and municipal government entities across the globe are using office productivity applications that support ODF. 13 The largest of these migrations involve applications that support ODF as the default-save format, which is important, as many solutions claiming to be "open" require user override intervention to save the file in any but the default "closed" formats. Designed to bias toward proprietary formats by saving to them (including "auto save"), this hidden default creates increased user dependence on a limited choice of vendors. Austria: The City of Vienna is migrating about 18,000 seats to OpenOffice.org and StarOffice, which also supports ODF as the default-save format. 14 Brazil: The Brazilian government is migrating 300,000 desktops to Linux and OpenOffice.org, which supports ODF as the default-save format. The Brazilian postal service has installed OpenOffice.org on 14,000 computers and intends to migrate some 32,000 around the country. France: More than 300,000 desktops in various government departments have been migrated to OpenOffice.org. They include the Gendarmerie Nacionale and the Direction Generale des Impots, and the ministries of Finance, Interior, Equipment, Justice, Agriculture, and Culture. 15 Germany: More than 50,000 government desktops have been migrated to OpenOffice.org or StarOffice at various levels of the German public adminstration. They include the Regional Tax Offices for Lower Saxony and Baden- Wurttemberg, and the cities of Treuchtlingen, Munich, Swaebisch Hall, Leonberg, and Isernhagen. The German Monopoly Commission has migrated to StarOffice. 16 Singapore: The Ministry of Defense has migrated 5,000 seats to OpenOffice.org, which supports the OpenDocument standard, and plans to migrate 15,000 more desktops. 3 Applications Support for ODF Software developers are responding to the growing demand from governments and other customer and implement- 12 < vation/digital/ xml_data_formats.html>. 13 < 14 < oss.htm> 15 < Share_ Analysis>. 16 < Share_ Analysis>. 48 UPGRADE Vol. VII, No. 6, December 2006 Novática

50 ing ODF in their products. A variety of applications are in the market today supporting ODF, ranging from open source solutions like OpenOffice.org and Koffice to commercial software solutions like Sun s StarOffice and IBM s Workplace. 17 The list includes not just major office productivity suites, but web-based applications such as Google s Writely and spreadsheet and software from document collaboration companies. The growing support from developers is an indication of confidence in the emergence of ODF as the document format of choice for governments. 4 Why Governments Are Moving to ODF Documents are the life blood of modern governments and their citizens. Governments use documents to capture knowledge, store critical information, coordinate activities, measure results, and communicate across departments and with businesses and citizens. Increasingly documents are moving from paper to electronic form. To adapt to everchanging technology and business processes, governments need assurance that they can access, retrieve and use critical records, now and in the future. Access. Alarmingly, governments today may no longer truly own their documents; they may lose the ability to access, modify and save archived documents at some future date, or struggling to open older documents. Even if they can be opened, the documents are sometimes completely indecipherable because the technical specifications used to produce them are not available. ODF, as an open, durable standard, can ensure that a government document saved today will not be technologically locked or abandoned tomorrow. Government want to avoid dependence on a single vendor s technology toprovide access to its own information. Interoperability. As an open standard, ODF allows governments to provide citizens greater choice (a variety of technologies to access, provide and use government information and services) independent of hardware, operating system or application. ODF accomplishes this by helping separate the document (information) from the application that created it. This document can then be processed by other applications seamlessly and with fidelity, without interference of any proprietary code or any other restrictions. Choice. Governments are often tied to the upgrade paths, strategies and pricing decisions of a single supplier, sometimes without reasonable access to viable alternatives. Because ODF is a truly open standard, it levels the playing field for multiple software providers to compete on functionality and price, providing governments with greater choice due to competition among vendors, including both proprietary and open source applications. 17 < applications_ supporting_opendocument>. 18 < CostAssess pdf>. 19 < _Exec Summary.pdf>. Lower Cost. ODF is cost effective because competition among applications that implement ODF will provide a variety of solutions (including open source solutions) at competitive prices. the results of several recent studies clearly indicate that governments are also gaining considerable costs savings when migrating to these ODF-supporting applications. 18 This will also help citizens who will not have to buy a specific application to access government information indeed, free-of-charge solutions are already available. Innovation. ODF provides a platform-independent format on which any company can build and distribute new applications and services. By providing a base-line open standard, documents will remain accessible even after innovations have been added. Preservation of Cultural Heritage. As more and more documents of potentially historical significance are being created and stored in digital form, it is essential that governments retain the ability to keep these documents and files free and accessible not only today but for future generations. ODF is the only open, XML-based document file format currently on the market that satisfies this basic test of public service. Emergency Management. The need for open standards is also compelling in the context of emergency preparedness. When the tsunami struck Thailand, its government and responding domestic and international agencies were unable to share and secure access to information essential to the relief effort because each used different data and document formats. Governments need to ensure that public access to essential government services, in emergency situations or otherwise, should never be restricted to users of one brand of software. 5 The Effect of Government Adoption of ODF on the Broader Market Governments and government entities are strategic adopters. Their purchasing power alone can exert major influence on the market. Government is now the second largest downstream IT purchaser with $552 billion in 2006, only slightly less than consumers ($700 billion) and ahead of all other industry segments, including finance, manufacturing, and services. 19 In the United States alone, local, state, and federal governments are anticipated to spend over $150 billion dollars in 2006 on technology products and services. Governments will also exert a major influence on technology choices more broadly through their electronic interaction with citizens. ODF gives citizens a choice from a variety of technologies to access, provide and use government information and services, which are increasingly moving online. Mobility is now at the core of strategic IT agendas. As a portable document format not linked to any one application or platform, ODF can be an essential component of a service-oriented architecture that governments are striving to develop. Novática UPGRADE Vol. VII, No. 6, December

51 Promotion of the Use of Open Document Formats by the IDA and IDABC Programmes Miguel A. Amutio-Gómez This article looks at the IDA (Interchange of Data Between Administrations) and IDABC (Interoperable Delivery of Pan- European egovernment Services to Public Administrations, Business and Citizens) Community programmes on the promotion of open formats for the exchange and storage of documents, whether editable or non-editable. On May 25, 2004 the steering committee of the IDA Programme gave their backing to recommendations relating to the promotion of the use of open document formats. On December 6, 2006, the steering committee of the IDABC Programme endorsed the conclusions and recommendations on open document formats that update and reaffirm the recommendations issued by IDA in the light of the current situation. Miguel A. Amutio Gómez, This article is distributed under the "Creative Commons Attribution-ShareAlike 2.5 Spain License", available at Keywords: IDA, IDABC, Open Document Formats. 1 The IDA Programme The IDA Programme (Interchange of Data Between Administrations), developed between 1999 and 2004, was aimed at establishing trans-european services between administrations in support of the application of Community policies and measures, inter-institutional communication within the European Union, and the Community decisionmaking process, together with the establishment of any horizontal actions and measures required to facilitate the deployment of those services. In May 2003, IDA embarked upon a plan of action aimed at promoting the use of open formats for document exchange. The decision to take action in this area was made for two basic reasons: Firstly, at that time there was little interoperability between office automation applications, a situation which was holding back the development of e-government And there was also a lack of support for open standard digital formats. Secondly, when IDA Programme experts examined the state of affairs at that time, they concluded that documents exchanged between public administrations and citizens should be in a format that did not oblige citizens to use specific software products and which ensured that they would be permanently accessible. In January 2004, IDA approved the comparative analysis of the available document format standards and, in particular, of the existing or emerging open document format standards, and the possible future market trend in this area. This analysis, known as the Valoris Report [4], made some valuable contributions, among the most important of which was the fact that it pinpointed eight qualities that the ideal Author Miguel A. Amutio-Gómez is a graduate in Computing from the Universidad de Deusto (1988). He joined the General State Administration in He has been a member of the Spanish delegation on the steering committees of the EC programmes IDA ( ) and IDABC (since 2005). Among other activities and projects, he leads the project "Criteria for security, standardization and preservation of information of applications used by the Administration for the exercise of its competences". <miguel.amutio@map.es>. document format should have It stated that the format should be open, non-binary, and modifiable, and should be characterized by presentation fidelity, multiplatform interoperability, the ability to support existing word processors,? the ability to support emerging requirements, and should enjoy widespread adoption Later, in March 2004, the major market players (Microsoft and SUN) were invited to comment on the Valoris Report and to present and argue their respective positions, and to debate the issue with the IDA Programme experts. Finally, on May 25, 2004, the Telematics between Administrations Committee (TAC), which guides the Community programme for the IDA Programme, endorsed the steering committee s expert recommendations regarding the promotion of the use of open document formats [2]. These recommendations recognized "the special responsibility of the European public sector to ensure the accessibility of its information, with a view to rationalizing and improving the interactions with citizens and enterprises, and taking into account the importance of the public sector as buyer of IT services and products". The aforementioned analysis also referred to the steps 50 UPGRADE Vol. VII, No. 6, December 2006 Novática

52 taken by the industry to address the matter, making particular reference to the publication of the open formats Office.org and WordML. Among the conclusions reached was that it was not necessary for all documents to be editable and that, if documents needed to be edited, XML was the best option with regard to the separation of content, structure, semantics, and presentation, and that the public sector should not oblige anyone to use a specific product. Consequently, the study recommended the promotion of a format that could be implemented on various platforms, that did not discriminate against any market players, and that ensured equal opportunity of implementation. Finally it welcomed the standardization of OASIS s OpenOffice.org format. These recommendations and their subsequent limitations must be considered in the context of the limitations that existed at the date of their issue in terms of the document formats that existed at that time, and bearing in mind that they were intended for the influential players of the day Consequently, the following was recommended: The OASIS Technical Committee considers whether there is a need and opportunity for extending the emerging OASIS Open Document Format to allow for custom-defined schemas; Industry actors not currently involved with the OASIS Open Document Format consider participating in the standardisation process in order to encourage a wider industry consensus around the format; Submission of the emerging OASIS Open Document Format to an official standardisation organisation such as ISO is considered; Microsoft considers issuing a public commitment to publish and provide non-discriminatory access to future versions of its WordML specifications; Microsoft should consider the merits of submitting XML formats to an international standards body of their choice; Microsoft assesses the possibility of excluding non- XML formatted components from WordML documents; Industry is encouraged to provide filters that allow documents based on the WordML specifications and the emerging OASIS Open Document Format to be read and written to other applications whilst maintaining a maximum degree of faithfulness to content, structure and presentation. These filters should be made available for all products; Industry is encouraged to provide the appropriate tools and services to allow the public sector to consider feasibility and costs of a transformation of its documents to XMLbased formats; The public sector is encouraged to provide its information through several formats. Where by choice or circumstance only a single revisable document format can be used this should be for a format around which there is industry consensus, as demonstrated by the format s adoption as a standard. After these recommendations were issued, the Directorate General for Enterprise and Industry (DG ENTR) of the European Commission invited the main software producers to work towards a greater interoperability in document formats. In response to this call, IBM, Microsoft and SUN expressed their commitment to move towards that goal [5]. 2 The IDABC Programme The scope of the new IDABC Programme (Interoperable Delivery of Pan-European egovernment Services to Public Administrations, Business and Citizens), to be developed between 2005 and 2009 as a successor to the IDA Programme, takes into consideration citizens and companies and focuses on the identification, promotion, and development of services supporting the application of Community measures and policies, inter-institutional communication within the European Union, and the decision making process, as well as the horizontal measures that enable these services to be deployed[6] [8]. In its work programme, IDABC has also included the promotion of open formats for document exchange at a pan- European level as, although considerable progress has been made thanks to the efforts of the IDA Programme, interoperability issues were thought to persist [3] [7]. The idea is that member states and industry players get involved in the debate and in the promotion of public sector awareness regarding the adoption of open document formats. Thus IDABC s conclusions and recommendations regarding open document formats have served to update and reinforce the conclusions of the IDA Programme in the light of the current situation. On December 6, 2006 the steering committee of the IDABC Programme endorsed the conclusions and recommendations on open document formats provided by their own expert group [1]. These conclusions and recommendations recognize the special responsibility of the European public sector to ensure the accessibility of its information, with a view to rationalizing and facilitating interactions with business and citizens, with the intention of unlocking the information contained in public sector documents, and taking into account the importance of the public sector as a buyer of IT services and products In its five pages the document covers the background, objective, progress made in standardization, progress made in public administrations, and problems to be solved. The following aspects are of particular interest: There is a common understanding among public administrations in Europe that electronic document exchange and storage should rely on open document formats. Such formats are to be defined in a process open to all interested parties and to be available for all interested and competent actors to implement without restrictions. It is supposed that such conditions will stimulate the competition and the innovation in the area of document management. Preferably, these open document exchange and storage formats would be subject to formal standardization via Novática UPGRADE Vol. VII, No. 6, December

53 international standardisation procedures. With regard to the progress made by public administrations, it should be noted that administrations in a number of member states (e.g. Belgium, Denmark, France, Spain (Extremadura), Italy) have adopted strategies involving the introduction of open document formats. The document refers to recent developments in matters of the standardization of the handling and storage of documents. In particular, it refers to the publication of the ISO/IEC standard (International Organization for Standardization/International Electrotechnical Commission) "Open Document Format for Office Applications", based on the OASIS ODF specification; the approval by the General Assembly of the ECMA ((European Computer Manufacturers Association) of the Microsoft Office Open XML format as ECMA 376, and the announcement of the intention to have the specification approved as an ISO standard; the publication of ISO standard "Document management Electronic document file format for long term conservation Part 1, Use of PDF 1.4 (PDF/A1)". It underscores the fact that, in spite of these recent developments, there are still compatibility issues between the new ISO/IEC standard and the formats used by the dominant office automation products. The document expresses the opinion that the prospect of a second ISO standard will do nothing to alleviate these problems and that filters, plug-ins, and converters are not the universal panacea. After which the following recommendations were put forward: "6. RECOMMENDATIONS In view of the present situation, public administrations are invited: 6.1. To make maximal use of internationally standardized open document exchange and storage formats for internal and external communication; 6.2. To use only formats that can be handled by a variety of products, avoiding in this way to force the use of specific products on their correspondents. When the usage of proprietary formats is unavoidable, alternative, internationally standardized open formats shall be provided in addition to proprietary formats; 6.3. To adapt, where appropriate, national guidelines and regulations, taking into account the arrival of international standards in this area; 6.4. To consider the definition of minimum requirements in regard to the functionalities of open document exchange formats in view of pursuing the compatibility of applications; 6.5. To create guidelines for the use of revisable and non-revisable document exchange and storage formats for different purposes To work together towards one international open document standard, acceptable to all, for revisable and non-revisable documents respectively; 6.7. To develop, applications that can handle all relevant international standards, leaving the choice to their customers as to what format will be used "by default"; 6.8. To avoid invalidating the purpose of open document exchange and storage formats by offering extensions to the relevant international standards as default formats To make proposals for conformance testing and to develop adequate tools in order to safeguard interoperability between applications; To continue to improve the existing standards, also taking into account additional needs such as electronically signed documents". Translation by Steve Turpin References [1] European Commision. PEGSCO approval on conclusions and recommendations on open document formats. < 3428/5644>, < servlets/doc?id=26971>. [2] European Commision. TAC approval on conclusions and recommendations on open document formats. < 5585#recommendations>. [3] European Commision. IDABC Promotion of Open Document Exchange Format. < idabc/en/document/3428/5644>. [4] Valoris. Comparative Assessment of Open Documents Formats Market Overview. < europa.eu/idabc/en/document/3439/5585#odf>. [5] European Commision. Responses from IBM, Microsoft and SUN to the TAC recommendations- Sept./Nov < document/3439/5585#responses>. [6] European Commision. The Programme IDABC. < [7] European Commision. IDABC work programme < 5101/3>. [8] Ministry of Public Administrations, Spain. The Construction of Paneuropean e-government Services (La construcción de los servicios pan-europeos de Administración electrónica.) < es/csi/pg3315.htm>. Industry, industry consortia and international standardisation bodies are invited: 52 UPGRADE Vol. VII, No. 6, December 2006 Novática

54 A Brief History of Open Standards in Denmark John Gøtze This article discusses current and recent developments in Denmark, where open standards have become a central policy issue. Although Denmark is known for leading the way in true, large-scale openization, a full-blown ef fort towards these ends is highly unlikely. Keywords: Denmark, E-government, Interoperability, OpenDocument, Openization, Open Standards. 1 Introduction In the e-government field, Denmark is almost always seen as a leader when compared to other nations. We re considered the e-readiest society with some of the most e- literate citizens. We have PKI and digital signatures, we have e-procurement and e-invoicing, we have digital registries and databases en masse, and so on. Denmark is also often regarded as the ultimate "Microsoft country". Denmark not only hosts the largest Microsoft development division in Europe, but Microsoft has also a de-facto monopoly in Danish government and society at large. Open standards have been on the political agenda in Denmark for several years. This was partly prompted by the situation with Microsoft and other monopolies, but also as a widely supported openization process. I use the concept of openization as an overall term for a deliberate transformation strategy, where open standardization plays a central role and where the ultimate goal is to create open ICT ecosystems that are healthy and sustainable, innovative and creative, inclusive and empowering. The concept openization became a shared concept for the Open epolicy Group when we made the Roadmap for Open ICT Ecosystems 1. In the Roadmap, we describe the openization process as a three-fold strategy, where the three arms are open standards, open source and service-oriented architecture (SOA). 2 The Danish Case Two years ago, the Danish government began requiring all companies selling goods or services to the state to submit their invoices electronically. This e-invoicing project picked up an EU award for innovation at a ministerial conference, not for anything technically clever but for the resolution with which it was made compulsory. EU-ministers were impressed by the direct link with efficiency savings - something that has so far eluded most European e-projects, as they reportedly admitted , ,00.html As a matter of fact, there were indeed also technically clever parts to the Danish e-invoicing project. The Danish government adopted the international UBL (Universal Business Language) standard from OASIS and by doing so, not only helped UBL reach critical acceptance, but also made the Danish solution ready for wider international adoption. Today several other countries, especially in the EU, are adopting UBL too. In the process, Denmark learned that compulsion will not be universally popular, but also that there are huge efficiency gains and potential service improvements by making standards compulsory. As a British reporter said 2, "The Viking streak always wins: it s more efficient". 3 Common and Open It is of course an important point to stress that UBL is an open standard. Unfortunately, one might say, the importance of this has not been completely clear in the Danish process, where the government s main interest was that it is a compulsory common standard. But it became obvious to policy makers that common is not enough, and that standards should be both common and open. This however led to many and too-often confusing debates about definitions: What is an open standard? It didn t help that Microsoft chose Denmark for the initial phase of opening up their document formats - the XML Schemas for WordML and such were published in the Danish government s XML repository. Did that make them an open standard? 4 The Interoperability Agenda The Danish debates about open standards were coupled to a trend in Europe, where interoperability became a hot IT-policy area in the years after Y2K. The UK created their e-gif (e-government Interoperability Framework) and other countries were working with similar programmes. Author John Gøtze has worked with government technology, enterprise architecture and open standards for more than 15 years. Today he is an independent consultant and writer, a non-tenured associate professor at Copenhagen Business School and a staff member of OASIS, the Organization for the Advancement of Structured Information Standards. He is also a member of the Open epolicy Group. <john@gotzespace.dk>. Novática UPGRADE Vol. VII, No. 6, December

55 At the pan-european level, the European Commission s Interchange of Data Between Administrations (IDA) programme in 2002 took the initiative to establish a pan-european Interoperability Framework 3. This was defined as a policy document providing recommendations and defining generic standards with regard to organizational, semantic and technical aspects of interoperability, thus offering a comprehensive set of principles for European cooperation in e-government. In the eeurope Action Plan 2005, adopted by the European heads of state at the Seville summit in June 2002, it stated that the European Interoperability Framework (EIF) would be "based on open standards and encourage the use of open source software". I had the pleasure of being involved in creating the European Interoperability Framework, serving as Denmark s representative in the EU working group in IDA. After a long and open process, the European Interoperability Framework for Pan-European E-Government Services was approved by the Telematics Between Administrations Committee (TAC) Steering Committee in November There they released a Final Version 1.0. EIF was instrumental in clarifying the meaning of interoperability, by explaining that there are three forms of interoperability, namely technical, semantic and organizational interoperability. Technical interoperability covers the technical issues of linking computer systems and services. It includes key aspects such as open interfaces, interconnection services, data integration and middleware, data presentation and exchange, accessibility and security services. Semantic interoperability is concerned with ensuring that the precise meaning of exchanged information is understandable by any other application which was not initially developed for this purpose. Also it enables systems to combine received information with other information resources and to process it in a meaningful manner. Organizational interoperability is concerned with defining business goals, modelling business processes and bringing about the collaboration of administrations that wish to exchange information and may have different internal structures and processes. It also aims at addressing the requirements of the user community by making services available, easily identifiable, accessible and user-oriented. The rationale for the support of open source software is articulated in the EIF document as: "Open Source Software (OSS) tends to use and help define open standards and publicly available specifications. OSS products are, by their nature, publicly available specifications and the availability of their source code promotes open, democratic debate around the specifications, making them both more robust and interoperable. As such, OSS corresponds to the objectives of this Framework and should be assessed and considered favourably alongside proprietary alternatives." While the EIF document does not deny the legitimate role of proprietary software alternatives in the IDABC Programme, it does present a strong statement about RAND (royalty-free standards) that are viewed to be acceptably defined as "open" standards. The principle of "open standards" is formulated in EIF (section 1.3, "Underlying principles") derived from the eeurope Action Plan 2005 as well as from Decisions of the European Parliament, where the Council and the Commission "have adopted and promote a set of general principles which should be respected for any egovernment services set up at a pan-european level" including "Use of Open Standards". "To attain interoperability in the context of pan-european egovernment services, guidance needs to focus on open standards. The following are the minimal characteristics that a specification and its attendant documents must have in order to be considered an open standard: The standard is adopted and will be maintained by a not-for-profit organisation and its ongoing development occurs on the basis of an open decision-making procedure available to all interested parties (consensus or majority decision etc.). The standard has been published and the standard specification document is available either freely or at a nominal charge. It must be permissible to all to copy, distribute and use it for no fee or at a nominal fee. The intellectual property - i.e. patents possibly present - of (parts of) the standard is made irrevocably available on a royalty-free basis. There are no constraints on the re-use of the standard.". I see the definition offered by EIF as a pragmatic, yet principled, statement. Although the definition became a central issue in the media after the launch of EIF, it was actually not the purpose of EIF to define "open standard". The main idea of EIF was to get member states to launch national interoperability frameworks and to offer guidance for how these could work together and be comparable. I was responsible for implementing the Danish Interoperability Framework, the Reference Profile (now named "OIO Catalogue") which was launched in parallel to the EIF. The OIO Catalogue 4 has since had a central position in the Danish e-government policy and has been an inspiration for other countries, for example Norway, in their work with interoperability. 5 Open Standards Politics In Denmark the "open standards"-issue became a "real" political issue in 2004, when MP Morten Helveg Petersen, from a small opposition party, raised the issue in Parliament by make a proposal about the use of open standards in government. This sparked a long debate. The result: on 2 nd June 2006 the Danish Parliament (Folketinget) unanimously passed a Parliamentary Resolution on Open Standards (B103) saying (in my translation): 54 UPGRADE Vol. VII, No. 6, December 2006 Novática

56 Parliament imposes on the government a duty to ensure that the public sector s use of IT, including the use of software, is based on open standards. And it goes on to specify that: The Government should adopt and maintain a set of open standards by 1 st January 2008, or as soon as technically possible, which can serve as an inspiration for the rest of the public sector. Open standards should be part of public IT and software procurement with the object of promoting competition. In the comments, this is suggested to be a "comply or explain"-model. But overall, this is the rule: The Government should ensure that all digital information and data that the public sector exchanges with citizens, companies and institutions, are available in open standards-based formats. It should be noted however, that the Government opposed the resolution until the last minute, but swung around some say because they realised they were outvoted, others argue that it was part of a wider tactical ploy. Regardless, the resolution was passed, setting a radical agenda for open standards in the Danish government. The responsible minister Helge Sander (the Minister of Science, Technology and Innovation) is yet to present an implementation plan. 6 Comply or Explain The ministry has on occasions acknowledged that the "comply or explain"-principle will be used, but has not yet publicly specified how this will work. Today there is no obvious place for an agency to go to and "explain" how things are, so some institutional arrangements will be needed if the principle is to be implemented. It could also be seen that this principle should be applied to contracts, CFPs (Call for Proposals) and procurements. But there are challenges here as well: what should the contractor comply with? The point should, of course, be that compliance must be achieved where it matters. In many situations, this means compliance with certain specific standards. If used at such level, many vendors will have a lot of explaining to do! Looking at the Danish e-invoicing case, compliance was the only option. In order to make that a realistic and operational solution, government chose to fund a "middle man", a conversion service through so-called "read-in bureaus" (scanning agencies), so that compliance was ensured through a middle-man solution. The Danish example shows that it is possible to enforce the use of standards on a large scale. By and large, the vendor market has also reacted positively and sees compliance in their products as a necessity "Unifier or Divider" is the theme of a forthcoming issue of Standards Edge, see standards_series.html 7 Governments Leading the Way? Much good can be said about having governments lead the way in standards adoption, but we also know there are inherent dangers. Governments are not always the fastest movers and once things are settled, it can be almost impossible to make changes and adapt to new circumstances. History shows this is a real challenge: for example, the now ancient mail protocol X.400 was sustained in many governments long after its life ended in industry. The Danish public administration has a long history of standards adoption and creation. During the first big wave of digitization in the 1970s and 1980s, the Administrative Department rolled out document management systems, data registries and weren t shy of mandating solutions and standards. That changed during the second wave in the 1990s, where market forces took over more and more. 8 The ODF Situation Proponents of ODF have established the ODF Alliance 5, whose mission is to get governments to adopt ODF. Although their mission has my full sympathy, I do want to stress that it is critically important that we get wide market adoption of standards such as ODF. If ODF is "only" a government standard it will fail. It needs to be adopted in society at large. During the B103-process, the Minister of Science announced that his ministry and a few other ministries would initiate a pilot project with ODF. The project should ensure that published documents on their websites would become available in ODF in addition to other formats (as they often published only Word-documents). Soon after, the ministry launched a rebuilt website with very few documents and appears to have learned not to publish Word-documents. 9 Unify or Divide? Although it would make a lot of sense to stress the idea that standards could be unifiers, something we agree upon and use universally, the reality is much too often the contrary. The major ramifications of standardization are shown in the current development of and around open standards for office document formats: it is used to both unify and divide 6 people, markets, geopolitical regions and technologies. Unify? Yes, Sun Microsystems, IBM and many others have joined up, unified, around the OpenDocument Format (ODF), which is an official ISO-standard and actively maintained by OASIS. Divide? Yes, because there is more than one standard. By now, as we know, Microsoft s new document format for their Office packages, Office Open XML, is an official standard published by Ecma, the European Computer Manufacturer s Association (which turned into a standardization consortium). Also the Chinese have their own UOF-standard, but work is going on aligning ODF and UOF, which are rather similar. CompTIA s president and CEO John Venator commented Novática UPGRADE Vol. VII, No. 6, December

57 in a letter 7 to Ecma: "Competition among multiple open document standards will enhance innovation in document formats and increase flexibility and interoperability, all to the benefit of software consumers." This view is clearly an argument by vendor interests (of which CompTIA is clearly a representative). An alternative view is that competition is good, but that it is only healthy when occurring between products, rarely between standards. In contrast, there is often a good business case in selecting if not mandating certain standards. 10 Show me the Costs! A report about the costs related to switching to open standards for document formats in the Danish government was made by Rambøll Management, a Danish consultancy, on behalf of The Danish Open Source Business Association (OSL). The report establishes three scenarios for the development: Scenario 1: Microsoft Office and ECMA Office Open XML - would cost 380 million kroner over 5 years with migration to MS Office 2007; 105 million kroner if using current versions with plug-in. Scenario 2: OpenOffice.org and ODF - would cost 255 million kroner over 5 years, covering all migrations costs plus already existing MS licence costs (until phased out). Scenario 3: Microsoft Office (with plug-in) and ODF - would have only marginally higher costs than scenario 1. The Open Source Business Association estimates that the whole of government (including local government) could save 550 million kroner by migrating to OpenOffice.org and ODF. 11 Open Schmopen In December 2006, IDC Nordic presented the results of a survey about document formats they had carried out among Nordic companies and governments. Per Andersen, Managing Director for IDC Nordic, said: "We believe that multiple open document standards are going to exist in the market just as the case would be with proprietary standards and each will mirror certain needs in the market." The $4200 report by IDC was made freely available on Microsoft s OpenXMLDeveloper.org site 8. Presumably, Microsoft wants the world to know the findings, which IDC summarises as:"companies generally do not consider ODF to be more open than Open XML or vice versa. Generally, companies are rating Open XML of higher importance to them when purchasing software than ODF. " IDC however also notes that ODF has its strongest adoption and ratings among public organisations and they believe this reflects the current positioning of ODF as ensuring the "free communication between public sector and citizens". The conclusion that IDC makes is that: "We do not believe there are, per se, any problems with the co-existence of two document standards". This development accentuates the importance of being clear about criteria for deciding what is and what is not an open standard. There are certain key "process" criteria: process open to anyone to join in development and evolution of the standard standard not effectively controlled by any one entity with a commercial interest due process and transparency in decision making The Open epolicy Group s Roadmap considers a standard to be open when it complies with all these elements: cannot be controlled by any single person or entity with any vested interests; evolved and managed in a transparent process open to all interested parties; platform independent, vendor neutral and usable for multiple implementations; openly published (including availability of specifications and supporting material); available royalty free or at minimal cost, with other restrictions (such as field of use and defensive suspension) offered on reasonable and non-discriminatory terms; and approved through due process by rough consensus among participants. At a minimum, open standards must allow all possible competitors to operate on a basis of equal access to implementing the standard. They should not drive others to follow any specific proprietary path or effectively preclude any software development model. While ODF passes as a truly open standard, Ecma Office Open XML does not. It is effectively and practically controlled by Microsoft, which has huge commercial interests in it. 12 My Conclusion Although Denmark is known for leading the way in true, large-scale openization, a full-blown effort towards these ends is highly unlikely. The likely development will be a pragmatic government policy which is more or less aligned with Microsoft s own on-going attempts at openizing themselves. On the other hand, there is a good and solid business case in ODF and the Ministry of Finance is out looking for good business cases, so anything can happen. 7 articleid= Open_Document_Standards.aspx 56 UPGRADE Vol. VII, No. 6, December 2006 Novática

58 Standard Open Formats and Libre Software in the Extremadura Public Administration Luis Millán Vázquez de Miguel The agreement of the Government Council of the Extremadura Regional Government of July 25, 2006, established the OASIS ODF and PDF A formats as the official formats within the Public Administration. This agreement, which created a major media impact at home and abroad, forms part of a global strategy to promote the information and knowledge society. Perhaps the greatest exponent of this strategy is the libre software distribution, gnulinex, which has a wide range of possible applications. For this reason, as part of the same agreement regarding document formats, the decision was taken to migrate to libre software on all the Extremadura Regional Government s PCs within a year at the latest. Keywords: Extremadura Regional Government, gnulinex, Interoperability, OpenDocument, Open Standards. On July 25, 2006, the Regional Government of Extremadura was instructed by the Government Council to adopt the OASIS ODF and PDF A formats as the official formats within the Public Administration. This agreement, which created a major media impact at home and abroad, did not come about by accident. It is another step forward along the road towards our region s incorporation into the Information and Knowledge Society and, to be more precise, it is the most important step towards improving public services through the use of the new technologies: the use of standard and open formats. The use of a common language. But to fully understand this Government Council agreement we need to go back in time. Since the year 2000, the Extremadura Regional Government has been intent on not only benefiting from the advantages of the digital revolution, but also being in command of its own destiny in that respect We are participating in the revolution of the new technologies, opening doors that no one has opened before, blazing a trail that will lead us to our goal. Since 1998 the Autonomous Government has been implementing a Regional Strategic Plan for the Development of the Information Society in every socio-economic sector of Extremadura. The purpose of the Technological Literacy and Libre Software Plan (hereinafter PAT, from its Spanish acronym) is to bring the Information Society to those sectors of the population that, for social, economic, or geographical reasons, may be excluded from the technological revolution. Which is why the PAT has tried to bring awareness of ICT and libre software to those citizens through the creation of 33 "New Knowledge Centres" spread over the entire region, especially in rural areas. "Vivernet", a project aimed at bringing the Information Author Luis Millán Vázquez de Miguel is a graduate in Chemical Sciences and a doctor of Sciences with an Outstanding Doctoral Award. He is an Associate Professor of Organic Chemistry at the Universidad de Extremadura. From 1985 to 1987 he was a visiting scholar and researcher at the University of Florida (Gainesville, USA). He is a member of the Federal Committee of the PSOE and since January 2003 he has been sectoral coordinator for the Information Society. He has been the Chairman of the Board of Governors of FUNDECYT (Foundation for the Development of Science and Technology in Extremadura) since its creation in He has held a number of responsibilities in the Extremadura Regional Government, all related with education. He was Regional Minister for Education and Youth from 1995 to 1999 and has held the post of Regional Minister for Education, Science and Technology from 1999 to the present day. <consejero.idt@juntaextremadura.net>. Society into the world of business, supports the creation of technology-based enterprises and the incorporation of the new technologies and libre software in the sector. Meanwhile, the purpose of the Centre for the Promotion of New Initiatives is to tailor the Extremaduran Information Society strategy to the ever changing circumstances of each moment, and to provide technical coordination for the transversal gnulinex (libre software) project. GnuLinEx libre software, which lies at the heart of Extremadura s Global IS Strategy, provides a secure, reliable, virus-free, open code platform for the Educational Technology Network, the Extremaduran health system (project JARA), and the administrative departments of the Extremadura Regional Government. This software is already a reality in the Educational System, where it has proven its soundness in critical situations such as the gnulinex version migration on some 45,000 computers in a month without any problems to mention. And the system has been totally and absolutely virus-free, Novática UPGRADE Vol. VII, No. 6, December

59 which substantially improves the productivity and profitability of both the installation and the human resources devoted to its maintenance. The process of installing gnulinex has already been initiated in the public libraries of Extremadura, which report to the Regional Ministry of Culture. At this point in time, gnulinex is being used by approximately 2,500 civil servants of the Extremadura Regional Government, not including teachers. There are also departments which are planning to increase their level of integration, all under the auspices of the Modernization, Simplification and Quality Plan for the Administration of the Autonomous Community of Extremadura ( ). These figures speak for themselves. Similarly, some time a go a number of regional ministries introduced internal management services using gnulinex-based servers. At a private level, the region s computer integrators and hardware wholesalers are collaborating by offering gnulinex-compatible machines and peripherals. Finally, it should be mentioned that a number of the largest hardware manufacturers are in conversations with the Extremadura Regional Government or have already signed collaboration agreements: Oki, Kyoscera, El Corte Inglés, Intel, etc. This, then, is the context in which the Government Council agreement of July 25 that we mentioned earlier should be understood. To achieve a greater integration and penetration of the Information Society in Extremadura it was necessary to take another step forward: to decide on what measures were required to create a nexus between all the various actions in progress. What element did they all have in common? The citizens, always the users of the technology - whether as pupils, healthcare workers, pensioners, etc. and always users of the Public Administration. It was necessary to increase the permeability of information between the various departments of the Extremadura Regional Government, the provincial authorities, local agencies, and the citizens. And this is where it is an advantage to have been one of the first. Our experience showed us that there is only one way to achieve real interoperability, while ensuring the validity of the data formats over time: to use standards. And for those standards to be open, so that, as well as bringing efficient government closer to its citizens, they would not have to invest in any kind of computer program in order to exchange information with the Public Administration. The Extremadura Regional Government has given official status to the most commonly used file formats by declaring that all electronically generated or exchanged information in various bodies making up the Extremadura Regional Government must use one of the following standard information storage formats: Open Document Format for Office Applications (OA- SIS Open Document Format, ISO/IEC standard DIS 26300), for information being created and undergoing administrative processes. Document Exchange Format PDF/A (Portable Document Format ISO :2005), for information where guaranteed unalterable visualization is required. In accordance with the above, all civil servants working in the Extremadura Regional Government will work with a set of personal productivity tools comprising office automation implementations that must natively support the abovementioned standards. These tools will be inventoried within the framework of the COMTIC (Inter-departmental Committee for the Coordination of IT Affairs), and are to be immediately installed at all workstations. From what we have seen so far, it is clear that if we are to progress in bringing the Information Society to modern Public Administration and our global society, we must ensure that the appropriate management and control mechanisms are in place to guarantee such important issues as technological independence, interoperability between platforms, homogeneity of information systems, security of information systems, real technological innovation, and compliance with open and libre computing standards It has been proven beyond any doubt that the above conditions can only be achieved by establishing a platform of open-standard and open-source systems and applications. In this framework, and given that all workstations will eventually be running OpenOffice 2.0 as their office application suite, since it meets all the standards adopted by the Extremadura Regional Government perfectly, it is also planned that candidates for any jobs falling vacant in the Regional Government offices should be familiar with OpenOffice 2.0 as opposed to the proprietary software that had been required up until now. Finally, it should be noted that, in spite of being a pioneer in the matter, the Extremadura Regional Government was aware of some very significant references that endorsed their commitment to open standards, such as: The Proposal for recommendations to the General State Administration on the use of free and open source software, which was drawn up by the libre software group of the General State Administration. This group was created by the Superior Council of Informatics and for the Promotion of e-government and mandated by the General State Administration to develop a set of recommendations regarding the use of libre and open source software. This working group is actually coordinated by the Extremadura Regional Government. The Proposal for recommendations was adopted by Superior Council of Informatics and for the Promotion of e-government on May 19, 2005, by the Sectoral Committee for e-government (AGE- CCAA) on May 11, 2005, and by the Plenary of CIABSI on April 21, The purpose of these recommendations is to optimize the procurement, development, maintenance, and operation of software, and to promote freedom of choice, protection of the investment, and to monitor price/performance and interoperability, while ensuring the Administration technological independence from specific providers. Further abroad, in Europe, in addition to particular ini- 58 UPGRADE Vol. VII, No. 6, December 2006 Novática

60 tiatives of not inconsiderable importance due to their instigators (such as the migration to libre software initiated by Munich City Council), there have been a great many studies in the last four years aimed at either providing in-depth explanations of libre software and open standard formats, and to explore the products available, or obtaining quantitative data about their level of use or dissemination in various areas, in both the public and private sectors. Examples of the above are: IDA (Interchange of Data between Administrations) Guidelines for migration to open source software. This guidelines were produced by the Community Programme, IDA, and are intended for managers and IT professionals working in Public Administrations. Their aim is to help them decide whether migration to open source software is advisable, and they describe how such a migration should be performed. The IDA Programme Study on the use of open source software in the Public Sector analyses various aspects relating to the use of open source software by Public Administrations. The European Interoperability Framework provides a set of recommendations and guidelines for e-government services. Translation by Steve Turpin Novática UPGRADE Vol. VII, No. 6, December

61 UPENET This section includes articles published by the journals that make part of UPENET. For further information see < Programming Languages Programming Languages: An Introduction Carlo Ghezzi Mondo Digitale, 2006 This paper was first published, in its original Italian version, under the title "I linguaggi per la programmazione", by Mondo Digitale (issue no. 17, March 2006, pp , available at < Mondo Digitale, a founding member of UPENET, is the digital journal of the CEPIS Italian society AICA (Associazione Italiana per l Informatica ed il Calcolo Automatico, < This paper provides a brief introduction to programming languages. It outlines the historical evolution of the field and its main achievements. To help understand programming languages, the paper proposes different classification schemes. In particular languages are characterized in terms of the programming paradigm they enforce. Keywords: Computation, Computer Architecture, Programming Languages. 1 Introduction Languages are the primary tools used by people to communicate. Organized societies exist only when there is a language to support oral and written communication. Without languages, interaction and cooperation among individuals would not be possible, let alone the accumulation and transmission of knowledge. Hence the study of languages has always been intertwined with social studies, anthropology, and philosophy. The development of computer science in the second half of the last century gave birth to a new class of languages: programming languages, used for communication between humans and computers. Indeed, man-machine communication is a more general and even earlier phenomenon that is not restricted to computers. However, it is only because of computers that manmachine communication became so sophisticated that the need arose for languages whose complexity can be compared to the languages used for human communication. In this paper, we focus primarily on programming languages; other typical linguistic forms of computer science (such as data management languages, specification languages, etc.) will be ignored for reasons of space. 2 Why Programming Languages? Computers can execute programs written in their own machine language: they are interpreters of their own language. By writing programs, one can specialize the machine to executing specific tasks. Once programs are translated into binary form and stored in the main memory of the computer, they can be executed by the CPU. Since computers were born, people realized that programming them Author Carlo Ghezzi is Full Professor of Software Engineering and member of the Academic Senate of the Politecnico di Milano, Italy, and Chief Scientific Officer of CEFRIEL (ICT Center of Excellence For Research, Innovation, Education and industrial Labs partnership) of the same University. He is a Fellow of the ACM (Association for Computing Machinery) and the IEEE (Institute of Electrical and Electronics Engineers), and Editor in Chief of the ACM Transactions on Software Engineering and Methodology. He has authored many papers in the fields of Programming Languages and Software Engineering, regarding specially theoretical, methodological and technological foundations of networked distributed applications. <carlo.ghezzi@polimi.it>. 60 UPGRADE Vol. VII, No. 6, December 2006 CEPIS

62 UPENET directly in their machine language would have been awkward. It was simply unfeasible for even small simple programs. Representing the program in machine language, in fact, implies decomposing the problem to solve into an algorithm expressed via very elementary, binary coded instructions, such as: data transfer from and to memory; simple arithmetic and logic expressions; explicit control transfer via modification of the program counter. The abstraction level of this language was simply inadequate for the programmer, who was required to express his or her intent in a very unnatural and low-level manner. A first solution consisted of making the language symbolic instead of binary. The resulting languages were called assembly languages. By using such languages, the programmer could use mnemonic names for the elementary operations performed by the computer and to denote the memory cells that contain data or instructions. These languages, however, kept a one-to-one correspondence between assembly instructions and machine instructions; they simply became more easily usable and understandable by humans. At that stage, the history of programming languages was still in its infancy. Since the 1950 s there has been a continuous flow of new language proposals and implementations, aiming at facilitating the task of programming computers. These languages may collectively be called high-level languages, to underscore the fact that their expressive level abstracts away from the minutiae of the operations supported by the circuitry of digital computers, to get closer to the expressive needs of humans. Without these FORTRAN ALGOL 60 COBOL APL LISP BASIC SIMULA 67 ALGOL 68 Pascal Smalltalk Modula-2 PROLOG Scheme languages, computers would have never become so pervasive in modern society, because the complex applications that run on them would have never been developed. The main driving force behind the continuous evolution of programming languages has been the need for computers to be equipped with software that would satisfy increasingly complex user requirements and support the development of more reliable software. Software applications are embedded in every sector of the modern society; from enterprise management to scientific computing, from videogames to document management, from real-time control of industrial plants to the distributed management of networked services. The requirements imposed on programming languages may differ in all these cases. At the same time, the requirements for reliability, easy modification, fast development became more and more important. It became clear that not only did the software have to be correct, but also other qualities had to be met. New programming languages were designed to respond to these challenges, and this is likely to continue to happen in the future. Table 1 provides a concise view of the historical developments of programming languages, by focusing on a limited portion of the existing tower of Babel, which corresponds to the most prominent practical languages. The table indicates the time period in which the languages were first defined, without showing the versions that denote their subsequent evolution. In what follows, we will not focus on the historical developments of programming languages, which would be an impossible task for a succinct survey of this kind, but rather we will sketch the fundamental concepts that can be Ada C++ Eiffel Common LISP CLOS Table 1: The Main Milestones of Programming Language Evolution today Java C# PERL Python used to understand and classify the languages. 3 Classes of Languages and Programming Paradigms Programming languages may be classified according to the programming style they implicitly enforce, also called programming paradigm. A paradigm characterizes the way the program looks, how computations are expressed, and how a program is organized into various parts. Different styles can be found in all linguistic forms: from poetry to music to architecture. In architecture, for example, one may talk about romanesque or gothic style, or art nouveau, or Bauhaus. Each style responds to some construction and aesthetic principle. The analogy applies to programming languages, even though the aesthetic motivation, which while it has some role, is not as important. What is special in the case of programming languages is the relationship that often exists between style of the language and quality of the software developed in that language. Programming language paradigms may be classified into two main areas: computational and structural paradigms. An understanding of paradigms and a classification of languages according to paradigms may help in better understanding both the languages and the applications developed in a language. A computational paradigm characterizes the computational model underlying the language. The prevalent computational paradigm is the so-called Von Neumann model. This term denotes computations described in a way that directly reflect the behaviour of the underlying hardware machine. It has been adopted because hardware machines still reflect the behaviour of the very early computer conceived by John Von Neumann. In other words, imperative languages are abstractions of the Von Neumann machine, whose computational model consists of a sequential, step-by-step execution of instructions that modify the state of the memory. They are abstractions because they hide the low level details, while they preserve the fundamental flavour of the abstracted behaviour. CEPIS UPGRADE Vol. VII, No. 6, December

63 UPENET Imperative languages operate on variables, which are abstractions of the memory cells that are modified during execution, and provide instructions for conditional execution (like if then else) and iterations (like while do ). This class includes older languages, such as FORTRAN and COBOL, as well as more modern languages, like C, C++, Java and C#. It is important to observe that the initial Von Neumann computational model has been evolving over the years. In particular, the notion of multiprogramming was developed, whereby a collection of Von Neumann machines may operate in parallel. This became necessary because programming languages provided an abstraction in which several computations could logically proceed in parallel, even if a single underlying physical machine actually switches execution from one program to another to create the illusion of parallelism. It also became necessary because in many cases the underlying hardware was indeed physically parallel (as in the case of multiprocessor systems, or distributed systems where several computers are interconnected through a network). Modern languages, like Java or C#, provide support for concurrent and distributed programming: they support both the view of several computations that evolve concurrently within a single machine, and computations that cross the network boundaries to coordinate physically parallel machines. Although concurrency and distribution make the overall computational model more complex, each computation continues to follow the Von Neumann style. Profoundly different computational paradigms are offered by two other paradigms: functional languages and logic (or declarative) languages. Both paradigms differ from imperative languages because they provide higher levels of abstraction, but unfortunately lower execution efficiency. A functional paradigm describes programs as mathematical functions. Its conceptual foundations can thus be found in mathematics, and in particular in the theory of recursive functions. Functional languages allow problem solutions to be expressed in a concise and elegant manner. The ancestor of functional languages is LISP, which was extensively used in the past in artificial intelligence applications. Other known languages of this class are ML, Scheme 1, and Haskell. The computational model of logic (declarative) languages is grounded on logic instead; more precisely, it is based on a subset of first-order predicate calculus for program descriptions and on logic inference for their evaluation. The best known language of this family is Prolog. Let us now turn to structural paradigms, which provide a classification of languages in terms of their support to structuring programs by decomposing them into units. We can distinguish between two main paradigms: procedural and object-oriented. Both of them support program decomposition into largely autonomous parts which can be developed separately. Both support a divide-and-conquer strategy that achieves program modularization. By decomposing a system into largely independent modules, the overall program is easier to read and write: both its initial development and subsequent maintenance are facilitated. Since modules may be developed separately, development time can be reduced. However, the two paradigms differ in the kinds of abstractions supported by the modularization mechanisms. The procedural paradigm supports a decomposition into separate functions, whereas the object-oriented paradigm supports a decomposition into objects. A procedural paradigm decomposes a program into subprograms. Typical examples are the SUBROUTINE and FUNCTION in Fortran, procedure in 1 Many universities adopted Scheme for introductory courses on programming, because it is generally felt that the abstractions provided by functional languages (and especially, recursion) have an important educational role. There are excellent textbooks that support introductory programming courses using Scheme. Pascal, function in C. A subprogram allows one to define and name a complex operation: for example, a function sort to order a set of values, a function solve to solve a system of equations, or a function draw_map to draw a map. Once the function is defined, it can be used many times by simply executing an operation that calls the subprogram. With regard to the kind of modularization, there is a clear distinction between the parts of the program that define a complex operation as a subprogram, from those that use such an operation. If a change occurs in the definition (i.e. internally to the subprogram), it does not affect the parts of the program that use it. The object-oriented paradigm decomposes a program into classes of objects. A class is defined by the set operations through which the objects of the class can be manipulated. As an example, consider a class that defines regular_polygons. The operations would provide say the perimeter, the area, and would draw the polygons; they would support different operations to create new polygons. Another example would be a class fax, which provides operations to send and receive documents. Yet another example is a banking application, where classes are used to define checking accounts (on which we can define operations to deposit and withdraw money, and to query the current balance), customers (who can open and close checking accounts, request a loan, and so on). These examples show that objects are characterized by a state, which stores modifiable data, and operations. Object-oriented languages provide a special construct the class through which objects can be defined and then generated. A class groups together a data structure and subprograms that implement the operations. Furthermore, a class behaves like a type in conventional languages. In FORTRAN or C, it is possible to instantiate several variables of type integer, and manipulate them via the arithmetic operations provided by the language. Likewise, in an object-oriented language one can define and instantiate several checking accounts or bank customers and manipulate them via the operations defined in the respective classes. We 62 UPGRADE Vol. VII, No. 6, December 2006 CEPIS

64 UPENET can thus say that classes support abstract data types. Object-oriented languages provide better structuring mechanisms and support better modularization than procedural languages. Not only do they allow operations to be grouped with the data structure they use, but they also allow such data structure to be hidden to the other parts of a program. These parts may act as clients of the module, but cannot modify directly the data protected by the module itself. This makes future program changes easier: if the data structure is modified but operations do not change, clients remain unaffected. Object-oriented languages also support a subclass mechanism. The construct supports the definition of new classes that specialize the behaviour of existing classes. As an example, having defined a class fax, one may define a subclass fax_with_phone, which supports communication of vocal messages, not just documents. A subclass defines only the newly added functions provided by the objects of the subclass; all other operations are inherited from the original class. Furthermore, the subclass may redefine previous functions. The subclass construct is a powerful linguistic mechanism supporting software evolution. In the example, the change that adds to the fax a new functionality (sending vocal messages) is obtained without modifying the existing software, but simply adding a subclass that implements the new functionality. All existing programs continue to operate correctly, even though the objects they now operate on are not plain faxes but faxes with phone, since these also support the operations to send and receive faxes 2. The object-oriented paradigm was first introduced by the SIMULA 67 language; it was later rediscovered by Smalltalk and finally popularized, in a hybrid form, by C++. C++ made object-oriented design popular by extending the C language, which only supported traditional procedural programming. The new paradigm could therefore be adopted more gradually by practitioners. The more modern, and already widely adopted, Java e C# support object-oriented programming in a more coherent manner. 4 Further Classifications and Conclusions Computational and structural paradigms classify programming languages in a way that enlightens their main distinctive features. Different classifications are also possible, and sometimes useful. One may classify languages according to the definition style: for example, textual versus graphical (or visual). One may also distinguish between compiled and interpreted languages, or between statically typed and dynamically typed languages. There are, in fact, languages for which the prevalent implementations are based on a translation of the source code into machine language (or a language close to it); others are instead directly interpreted in a high-level form 3. LISP and the now popular scripting languages (like Perl, Python, or Ruby) are examples of interpreted languages. The Java language adopts a hybrid solution: the source code is translated into an intermediate language (called Bytecode), which is then interpreted. The latter classification distinguishes between languages where every variable has a statically associated type, which characterizes the legal operations that may be applied to them, and languages where the association may change dynamically. In the former case, correctness of data manipulation can be checked before execution; in the latter, this may only be achieved by imposing specific constraints. To conclude this brief overview of programming languages, it is important to stress that software professionals need to master the concepts that underlie programming languages. Knowing how to use a language is simply not enough. Programming languages are the main tools for software development, and software is at the heart of every automated system people use daily in every sector of life. The quality of such systems and, ultimately, their dependability strongly depends on the programming language used for their development. Understanding programming languages means not only using them better in software development, but ultimately being able to develop quality products. Translation by the author Bibliography For reasons of space it is impossible here to provide references to all practically used languages. We simply provide a reference to a specialized book, which provides an in-depth analysis of programming language concepts and references to individual programming languages and to research contributions. C. Ghezzi, M. Jazayeri, Programming Language Concepts (III Edition), J. Wiley and Sons, Technically, this important result is due to two main features of object-oriented languages: dynamic binding and polymorphism. 3 One might say that we are actually distinguishing between two ways of implementing languages. However, it can be argued that there are language characteristics that make one solution better than the other. CEPIS UPGRADE Vol. VII, No. 6, December

65 UPENET E-Commerce Organization and Economics of Entertainment Services Networks Exchanging Virtual Goods Andrzej P. Urbañski Pro Dialog, 2006 This paper was first published, in English, by Pro Dialog (issue no. 21, 2006, pp ). Pro Dialog, < prodialen.html>, a founding member of UPENET, is a biannual journal published jointly, in English or Polish, by the Polish CEPIS society PTI-PIPS (Polskie Towarzystwo Informatyczne Polish Information Processing Society) and the Poznan University of Technology, Institute of Computing Science. In this paper we present how we organize in practice the exchange of virtual goods between services. We or our partners have some independent services on individual domains. We can exchange both points, user data and virtual goods. We have also developed quite a sophisticated credit system, which is used in our game zone. We relate our approach with points in the rapidly developing LETS (Local Exchange Trade System) and loyalty points in other companies, especially airlines. Keywords: e-commerce, Internet services, loyalty point, virtual goods. 1 Introduction The idea of a portal [1] started only ten years ago, but it has changed the world in a way which people did not expect. The most significant change is to make a large part of people s activity concentrate on virtual instead of real life. Many people spend hours sitting in front of their computers working, learning or even having fun. However, it is difficult to force these people to pay for services via the Internet and it is difficult to develop a profitable business. Many people started with their own service, which in some cases became a great portal, but mostly it is difficult. The power of a great portal lies in its versatility. Such a portal could serve each basic need, which is necessary for an Internet user. Small services are rather much more specialized and cannot offer many services needed by Internet users. However, if we look at the whole area of services offered, even small services can serve each need, but separately. And the problem is that sometimes we need to do many things in the same environment. In this paper we try to develop a unified framework which enables small services to efficiently cooperate. First, we present the idea of a local currencybased economy in a broader and narrow sense. Next we discuss how this currency units could be efficiently and safely exchanged between cooperating services. And than we discuss the exchange of other virtual goods and user oriented data. 2 Local Currency-Based Economies 2.1 LETS Local Exchange Trading Systems Local Exchange Trading Systems (LETS) are trading groups organized locally to provide their members with a system which facilitates the exchange of goods and services. LETS members [6] exchange so called favors and the system plays the role of a bookkeeper for their transactions. Each member has an account, where all transactions are recorded if the member receives a favor, LETS debits his/her account. If the member gives a favor, his/her account is credited. The system is quite simple and does not introduce any additional bonus points (even for being an active member whose account is in constant credit). Nor does it charge any interest to accounts in debit. The system does not try to control its members nor does it impose anything on them. The main principles of LETS are: LETS should not be perceived as a financial system. The LETS points are favors, not dollars, LETS is not a profit motivated system the members should be thought of as an active community. Let us notice that the points at our portal are not of LETS type because Author Andrzej P. Urbañski graduated in Computer Science from the Poznañ University of Technology, Poland, and obtained his PhD in the same field from the Polish Academy of Science. Actually he is a Researcher and Lecturer in the Institute of Computer Science of the Poznañ University of Technology. His research includes the fields of Computer- Aided Design, Artificial Intelligence, and Electronic Commerce. He has been publishing and presenting papers at many European scientific conferences and has also published articles and books popularizing Computer Science, especially for children, with some literary setting. Some of his computer programs have been marketed and where reviewed in popular media, among others by Polish TV networks, Deutsche Rundfunk, BBC, DIE WELT, New York Times, and Canadian CRBC. <andrzej.urbanski@cs.put.poznan.pl> 64 UPGRADE Vol. VII, No. 6, December 2006 CEPIS

66 UPENET Figure 1: Sources and Destinations of User Points at FantasyWorld.pl they are used in transactions between the portal and its users, and not between users. The only exception we have is a possibility for the users to trade avatars on a very simple virtual stock exchange. However, our credits at game zone (Labiryntus.pl) have many points in common with credits in LETS. Moreover, it could be very interesting to experiment with full LETS capabilities at a portal for some interest groups. 2.2 Loyalty Points in Other Commercial Companies Loyaltyware [2, 7, 9] is a type of adware. A company tries to gain clients "loyalty" by introducing a system for collecting extra cash, points, airline miles etc. while spending money for the company s products. Probably the best known examples of loyaltyware are frequent flyer programs (FFP), which are systems offered by many airlines. Customers who take part in the program earn bonus points (aka frequent flyer miles) for flying with the airline. The number of points is proportional to the flown distance or fixed (depending on the airline and the system) and can be additionally increased or decreased depending on other conditions (flying first class vs. flying on discounted tickets). Loyalty points can later be exchanged for free airline tickets, other free services and goods, or for obtaining a higher status level (which enables priority bookings or access to VIP airport lounges). Traditional FFPs have been systematically extended and now the user can often gain points not only by flying with a certain airline (or even with one of the airlines belonging to an alliance) but also by purchasing goods and services from other companies taking part in the program (hotels, car-renting companies, credit card companies, restaurants etc.). Depending on the program, the points can be valid only for a certain period of time (a fixed period of time for each earned amount of points or a period of inactivity counted for the whole account). Some programs offer additional bonuses for people who fly very often. They are often awarded a higher status level and gain access to more benefits, e.g.: access to business and first class lounges with an economy ticket, access to other airlines lounges, increased mileage accumulation (such as doubling or tripling), Date FW points Initial gaming credit Credit deadline reserving an unoccupied adjacent seat, the ability to reserve specific seats, such as exit row seats with more leg room, free or discounted upgrades to a higher travel class etc. Some airlines allow their VIP clients to book tickets for sold-out flights, thus bumping ordinary passengers. In order to gain more loyal customers, many airlines organize special promotions and allow for accelerated admissions to VIP programs. Conditions of promotions vary, but can e.g. require a customer to fly miles in one month. Since many customers never exchange their points, the programs are for many airlines a good source of profits. The profits are guaranteed not only by loyal customers but also by other companies which take part in the program and pay to be included on the partners list. Since the 1990s, U.S. airlines have sold billions of miles to partners such as credit cards, hotel chains, and car rental agencies, who try to attract customers by offering special promotions, goods and services which can be obtained with the airline loyalty points. Problems can arise if an airline company is sold or liquidates. For many sold companies, the new owners honor the points held in the customer accounts. Many of them offer the customers to convert to new loyalty programs. But if an airline liquidates, the customers lose their points and privileges. Compared with the traditional idea of loyalty points represented by FFPs, points at our portal introduce a significant Buying points transactions New gaming credit New credit deadline (2,44 PLN) Final upload at deadline (4,88 PLN) (7,32 PLN) (2,44 PLN) (2,44 PLN) (7,32 PLN) Table 1: Some Examples of Our Idea for Cumulating Credits on Labiryntus.pl CEPIS UPGRADE Vol. VII, No. 6, December

67 UPENET Figure 2: Income from Special SMS Payments at FantasyWorld.pl innovation which allows the users to earn loyalty points without having to pay anything. Cooperation between the main portal FantasyWorld.pl and the new (introduced in April 2005) game site Labiryntus.pl has many points in common with the idea of companies accepting loyalty points of other companies, incorporated originally in the frequent flyer program. expenses. Real life transactions are: buying points with national currency and purchasing real goods. Points are necessary for the user to take part in the games, which are offered by or connected with Fantasy World.pl. Users can both spend points on game expenses and earn points from game prizes. Such games have their own separate point zones, where points from FantasyWorld.pl can be uploaded and downloaded. It prevents FantasyWorld.pl users from ad hoc losing too many points. For example, at the Labiryntus.pl game zone, when the user uploads points to his/her account with the use of a cellular phone, he/she purchases a 2000 points credit for three days, which enables gaming even below zero points, but points from credit cannot be transferred to FantasyWorld.pl. When the credit time finishes, the user obtains 600 points extra on Labiryntus.pl. Moreover, many separate transactions cumulate, i.e. the available credit is a sum of all transactions, credit period is also appropriately extended and the additional points upload also cumulates as a sum, but is transferred at the end of the whole credit period see Table 1. Every user can earn extra points even from the fact that he/she is with FantasyWorld.pl for some period of time. As for now, each user obtains at least 10 points per week for staying with FW (even without any activity!). This idea works very well if a person registers and forgets about FW for some time. After returning some time later he or she is really pleasantly surprised that her or his user account has increased. If such a person additionally performs some logins, the number of 2.3 Points at FantasyWorld On the basis of our five-year experience with the development of Fantasy World.pl [3, 4, 8] we can show how loyalty points became a legal tender, which is convenient for both the service administrator and the users. Figure 1 presents the basic components of our idea for mixing virtual and real life profits and costs. Each user of FantasyWorld.pl can earn and spend points at our portal. He or she can earn points from: game prizes, pure loyalty to the site, buying points with the use of national currency. He or she can also spend points on: playing games, virtual properties and real goods. Each user functions between real and virtual life. Virtual life transactions are: games prizes and expenses, loyalty earnings, and virtual properties Figure 3: Exchanging Currency Units (Points) between Two Services on Different Servers. 66 UPGRADE Vol. VII, No. 6, December 2006 CEPIS

68 UPENET Figure 4: Exchanging Transactions, Their Total Balance ad Total Transfer Rate. points increases by 5 for every logging, but it counts at most once a day. This is a prize for a very simple activity, but if a person logs in every day, he or she will sooner or later do something positive for the portal. A bit more exacting activity is clicking on ranking links and voting for FantasyWorld.pl, which increases our site s position on the list of the most interesting web pages. This sort of activity brings small, but important benefit for the site. Yet another prize is for each message on a forum and each comment to an article. Buying virtual products seems to be unattractive at a glance, but young users find many of such properties worth investing. At the first place are avatars graphics or photos not necessarily of a human being, which are useful as an easy recognizable sign of a person. One has to pay about 500 points for a unique avatar. The benefit of having an avatar is getting double loyalty points, i.e. instead of 10 per week, one obtains 20. So after a year of staying with FW one can have an avatar for free (buying an avatar doubles loyalty points also backwards), and can earn points two times faster from this moment. Users love such bonuses. The logo at the top of the page changes randomly, but if a person wants to have a specific logo he or she has to pay 300 points for a permanent logo. A user can collect more than one logo and switch between them. Another virtual property is a magic ring, which can increase the user s power in earning points. Rings costs range from 2000 points to points and depending on its type a ring can increase the income from clicking links, adding messages at forums, adding comments, etc. and combinations of these possibilities. Possibilities of spending points even on virtual properties became so attractive that many users complained that there were not enough possibilities to earn them. Therefore, it was quite natural to introduce a possibility of buying points with the use of real life money. In Poland, like in many other countries, there is a relative lack of payment cards (especially credit cards), which enable easy and cost effective payments on the Internet. Hopefully, mobile phones have recently become inseparable gadgets of young people, and telecommunication operators have created unified platforms for remote payments. It is also easy to develop even a small service which requires people to pay for using it. However, in many countries the problem is the greed of mobile phone operators and intermediary companies, which can take even 60% of the users payment, leaving only 40% to finance the whole service. It makes simple businesses like e.g. selling bus tickets via SMS not profitable in any circumstances. When a user wants to buy points he/she is charged PLN 2+VAT (approx. 0.7 EURO), for 600 points. Relatively early, FantasyWorld.pl introduced competitions thanks to sponsors, who found them to be an efficient way of promoting their products. Such a producer offers some products connected with the FantasyWorld.pl theme for a competition and the portal organizes a competition among the registered users. Sometimes, in order to take part in such a com-petition, the user has to buy a Figure 5: Exporting Data to Another Service in One Phase, but with Data Security Lost Danger. CEPIS UPGRADE Vol. VII, No. 6, December