Metadata standards for book trade communication. Graham Bell EDItEUR

Transcription

1 Metadata standards for book trade communication Graham Bell EDItEUR Vilnius 17 th November 2016

2 About me 20 years experience at the point where book publishing and technology meet formerly senior manager in IT department for HarperCollins UK led development of bibliographic, editorial and digital asset management systems involved in e-book, e-audio, print-on-demand and online projects joined EDItEUR in mid-2010, primarily responsible for ONIX development

3 About EDItEUR not-for-profit membership organisation develops, supports and promotes metadata and identification standards for the book, e-book and serials supply chains best known for ONIX, EDItX and Thema based in London, but a global membership of publishers, distributors, wholesalers, subscription agents, retailers, libraries, system vendors, rights organisations and trade associations EDItEUR is 25 years old it has around 110 members in 25 countries, and they represent the whole of the supply chain, for both books and serials (academic journals)

4 About EDItEUR also provides turnkey management services to International ISBN and ISNI Agencies, and limited services to IDF and LCC acknowledged centre of expertise on metadata standards and identifiers also work closely with other standards organisations (including BISG and BIC) took part in ARROW and ARROW Plus projects, designed data flows and formats to support books-in-print and orphan work systems we do not issue ISBNs or ISNIs ourselves. The International Agencies provide coordination with 151 national ISBN agencies and a dozen ISNI agencies around the globe, to promote smooth running of the systems, consistent policies and to ensure uniqueness of identifiers EDItEUR is extremely well connected, to its members, to other trade associations and to other standards organisations I am going to talk about what metadata and identifiers are, and about two key standards that have proved valuable to European book trade ONIX and ISNI. My colleague Azar will talk about Thema

5 What is metadata?

6 What is metadata? often thought of as bibliographic data too narrow a definition all the product management information that s needed to make the industry work to describe the books that are published, so booksellers, librarians, readers can discover the books they need to support processes like planning, production, distribution, merchandising, promotion, sales data modelling and communication aspects metadata comprises both what a librarian would call bibliographic data AND essential commercial data and collateral (marketing material) this data needs to be shared internally within a publishing organisation, and externally with business partners like distributors and retailers

7 And what are identifiers?

8 Identifiers unique labels for things (in IT systems) to address resources such as books or people to disambiguate and collocate and thus allow automated communication ideally, persistent, meaningless, managed provide a link to a specific set of metadata identification is contextual identifiers have specific functions or roles, a scope (types of thing can be identified), and must demonstrate a clear business benefit unambiguous communication allows automation, which speeds things up, reduces costs, and reduces errors and customer service issues - identifiers cannot easily be free of charge (even if the identifier itself is free to use, managing it can be costly), so they have to have a clear function and a clear business benefit. For example, ISBN s function is to enable the book trade supply chain. They are about reducing friction and increasing automation

9 Identifiers and metadata inextricably linked. Each type of identifier has a minimum set of metadata attributes, whereby any change in the metadata implies a change in the identifier (and vice versa) metadata attributes define what is unique about the identified thing an identifier can be thought of as shorthand for, or link to, one particular set of attribute values metadata registries (eg books-in-print) hold both the identifiers and the metadata minimum metadata set defines what we mean by the same and different in the particular context of the identifier may miss this slide out

10 Metadata and identifier standards critical enablers for the book business metadata and identifiers are often criticised as complex and expensive not the most glamorous job in publishing. No-one becomes a publisher because they like ISBNs but to a large extent, it is what makes publishing work, and the evidence is that investment in metadata pays off handsomely through increased sales, lower customer service issues and a lower cost to serve (last year I presented some evidence for this, and a little later this year I will send an updated study from the UK and US that confirms very strong correlation between improved metadata and improved in sales

11 ONIX for Books

12 What is ONIX? ONIX for Books is a standard data format based on XML, used to communicate a rich range of information about book and bookrelated products between computer systems in the book and e-book supply chains primarily publisher to retailer direct, or via intermediaries such as wholesalers and distributors, data aggregation services, books in print registries

13 Typical use cases a publisher needs to provide information about its catalogue of products to a distributor, wholesaler, retailer or other supply chain partner includes both current and forthcoming products may cover basic product information and a wide range of commercial information and collateral material scope extends over the full lifecycle for book, e-book and other products ie includes postpublication updates to price and availability

14 you can just do this using a spreadsheet of your own design but if you publish more than say 25 books a year, or you have more than one person this very quickly gets to be difficult and error prone

15 database database ONIX for Books is a standardised message specification, not a database but what you can deliver in your ONIX is dependent on the design of your in-house database ONIX data model often used to guide design of internal applications ONIX is about communication mostly communication between computers and between databases. It is not intended to be human-readable (at least not easily) the point about communicating between computers is that you need to be unambiguous and precise

16 Roots of ONIX 1997 EPICS and BIC Basic 1998 <indecs> project 1998 W3C XML specification 1999 Online Information Exchange initiative from AAP Digital Issues working party ONIX developed by EDItEUR, originally in collaboration with BISG (USA) and BIC (UK) March 2000 ONIX International v1.0 ONIX as we would recognise it really started with the American Association of Publishers and its Online Information Exchange project, which combined the ideas and data elements from BIC Basic and EPICS with the XML syntax developed and standardised by the World Wide Web Consortium, and it published ONIX in January of 2000 within a few months EDItEUR further developed the AAP s guidelines into ONIX INTERNATIONAL version 1.1, the first implementable version of the ONIX message format it s been developed for a decade and a half by EDItEUR, initially with direct support from the Book Industry Study Group and Book Industry Communication

17 Roots of ONIX 1997 EPICS and BIC Basic 1998 <indecs> project 1998 W3C XML specification 1999 Online Information Exchange initiative from AAP Digital Issues working party ONIX developed by EDItEUR, originally in collaboration with BISG (USA) and BIC (UK) March 2000 ONIX International v1.0 ONIX as we would recognise it really started with the American Association of Publishers and its Online Information Exchange project, which combined the ideas and data elements from BIC Basic and EPICS with the XML syntax developed and standardised by the World Wide Web Consortium, and it published ONIX in January of 2000 within a few months EDItEUR further developed the AAP s guidelines into ONIX INTERNATIONAL version 1.1, the first implementable version of the ONIX message format it s been developed for a decade and a half by EDItEUR, initially with direct support from the Book Industry Study Group and Book Industry Communication

18 current status Roots of ONIX support for 2.1 was reduced at end of 2014 managed by EDItEUR and international steering committee June 2003 ONIX v2.1 most widely deployed April 2009 ONIX v3.0 growing in importance widely used in North America, Western Europe, Japan, Russia, parts of Eastern Europe, Korea, growing in China used by small and large organisations alike early versions completely obsolete version 2.1 (from 2003 to 2006 era) still in use version 3 is the latest version, slowly growing in importance it s a hugely comprehensive standard, but not so complex as to be beyond the scope of a single developer working in a small publisher larger publishers are likely to use commercial off-the-shelf systems which incorporate ONIX messaging into a bigger IT system

19 Roots of ONIX current status managed by EDItEUR and international steering committee June 2003 ONIX v2.1 most widely deployed April 2009 ONIX v3.0 growing in importance widely used in North America, Western Europe, Japan, Russia, parts of Eastern Europe, Korea, growing in China used by small and large organisations alike early versions completely obsolete version 2.1 (from 2003 to 2006 era) still in use version 3 is the latest version, slowly growing in importance it s a hugely comprehensive standard, but not so complex as to be beyond the scope of a single developer working in a small publisher larger publishers are likely to use commercial off-the-shelf systems which incorporate ONIX messaging into a bigger IT system

20 ONIX business benefits standard is free of charge to use for publishers enables supply of rich metadata in a single, standard format, for all downstream needs for distributors, metadata aggregators and retailers efficient, timely delivery and aggregation of data from multiple publishers a shared language enables unambiguous electronic communication

21 ONIX 3.0 data elements message details identity and authority record details product identifiers 1. descriptive details product form special features packaging physical size DRM, usage constraints trade classification product parts collection titles titles contributors event edition language extent subject audience so these are the areas that ONIX covers there are 7 blocks of data this slide shows the first two blocks

22 ONIX 3.0 data elements 2. collateral details supporting text cited material supporting resources prizes 3. content detail 4. publishing details imprint and publisher contact details lifecycle dates and status copyright details territorial sales rights 5. related material related works related products 6. supply details market-specific details suppliers discounts prices and tax here s the general outline of the other five blocks of an ONIX message note old ONIX 2.1 does not use these blocks, but carries broadly similar range of data EXCEPT that it is not as good for e- books and complicated international trading note if you are a librarian, Block 1 and Block 4 contain information that would also exist in a MARC record, whereas Blocks 2, probably 5 and certainly 6 contain information that is not in MARC note that these aren t the actual data elements: those are much more granular. For example take the <Price> composite. A product can have many prices, in different currencies, with different tax breakdowns, specific to different countries or even to different classes of customer or different business models, with different discounts etc. That requires around 40 different data elements covered under the heading of price and tax

23 <Contributor> 3.0 <SequenceNumber>1</SequenceNumber> <ContributorRole>A01</ContributorRole> <NameIdentifier> <NameIDType>16</NameIDType> <IDValue> X</IDValue> </NameIdentifier> <NamesBeforeKey>Kristina</NamesBeforeKey> <KeyNames>Sabaliauskaitė</KeyNames> <BiographicalNote textformat="05"><p>kristina Sabaliauskaitė is a Lithuanian art historian and historical novelist. She lives in London.</p> </BiographicalNote> </Contributor> a sample of ONIX XML A01 is part of that shared language or controlled vocabulary or codelist (means author ) code 16 means ISNI, and the 16 digit number is an identifier for this particular author note that some textual data can include HTML markup

24 <Contributor> 3.0 <SequenceNumber>1</SequenceNumber> <ContributorRole>A01</ContributorRole> <NameIdentifier> <NameIDType>16</NameIDType> <IDValue> X</IDValue> </NameIdentifier> <NamesBeforeKey>Kristina</NamesBeforeKey> <KeyNames>Sabaliauskaitė</KeyNames> <BiographicalNote textformat="05"><p>kristina Sabaliauskaitė yra Lietuvos meno istorikas ir istorinė beletristas. Ji gyvena Londone.</p> </BiographicalNote> </Contributor> a sample of ONIX XML

25 a database designed for HarperCollins in the UK brought together product metadata from multiple different departments of the company, and then exported it in ONIX format use an application like this to manage the data, mostly for your own internal purposes, then the ONIX data becomes a simple step, almost a by-product of good business practice

26 or if you don t have your own system, some data aggregators use online forms that you fill in.

27 * An offer that matches the scale of your ambition. Some applications make sense only for the largest indie publishers, while others work well in companies where the staff can all sit on a single sofa. This is in part about functionality and pricing, but it is also about the level of support you get, and the choice between a traditional software purchase and an online service. Implementing ONIX apparently complex, but in fact modular and consistent not too large for a single developer with simple software such as Filemaker or Ruby on Rails implemented in many off-the-shelf solutions installable applications or cloud-based what should you be looking for? best practices from EDItEUR What should you look for? Something forward-looking, which implements ONIX 3.0, ISNI and maybe even ISTC, as well as the usual ISBN and your internal identifiers or project numbers * An approach that treats books, e-books and maybe even mobile apps together, rather than making e-publishing a semi-detached afterthought * A database which makes it simple to manage data once, for all products based on a particular work. If you are publishing the hardback, the paperback and three different varieties of e-book, you don t want to do every task five times * A system that allows you to manage a far richer set of data than you currently aim for. * This richness might extend to the use of neatly formatted descriptive text and a full set of foreign characters. * An application that supports other parts of your business process or workflow.

28

29 ISNI public identity identifier So let s look at a couple of relatively unfamiliar identifiers, both of which can be thought of as bridging identifiers

30 ISNI International Standard Name Identifier ISO Standard for identification of public identities of parties in creative industries parties may be people, organisations or even fictional (for pseudonyms, characters in novels) typical use case identify an author, establish difference from another author of the same name, or establish same persona as musician or actor (possibly of different name ) The identification of the identities used publicly by parties involved throughout the media content industries in the creation, production, management, and content distribution chains across multiple fields of creative activity Mickey Mouse has an ISNI as well as the usual collocation and disambiguation within a sector (eg See other books by the same author ) The ISNI is not intended to provide direct access to comprehensive information about a public identity but can provide links to other systems where such information is held a bridge identifier bridging id is an identifier whose minimum referent metadata is (or could be) composed entirely of other identifiers...

31 Richard Holmes

32 Richard Holmes

33

34 What does it look like? for display purposes only check digit may be X ISNI decimal digits for persona spaces are for display purposes only NB I have published the check-digit algorithms in the Schematron

35 Implementing ISNI strictly, does not identify a person or a name person (or party) persona in ONIX, a contributor is a persona personality (or presentation) ISNIs identify personas, or public identities Фёдор Достоевский = Fyodor Dostoyevski Ruth Rendell Barbara Vine (but well-known pseudonyms can be linked) Sue Welfare *** ***** (allows for anonymity) allows for alternative spellings, transliterations, different languages (these are multiple presentations of the same persona) creates links to other creative sectors one party can have multiple personas minimum referent metadata name Julian Cope date and place of birth, death 1957, Wales reference resource Kilimanjaro work class and roles Music/Songwriter + URI

36 Current status 2010 ISNI International Agency set up as UK not-for-profit, now operated by EDItEUR partners include CISAC, OCLC, IFFRO, various National Libraries, SCAPR, Numerical Gurus standard published by ISO, March 2012 central registration system developed and operated by OCLC in the Netherlands first ten million ISNIs assigned, mostly based on pre-existing national library data IPDA = International Performer s Database Association CISAC = International Confederation of Authors and Composers Societies CENL = Centre for European National Libraries (48 national libraries, represented by BNF and BL) IFRRO = International Federation of Reproduction Rights Organisations current data comes primarily from national library catalogues (authority files) QA team working on improving data quality first-time authors will not yet have an ISNI

37 <Contributor> 3.0 <SequenceNumber>1</SequenceNumber> <ContributorRole>A01</ContributorRole> <NameIdentifier> <NameIDType>16</NameIDType> <IDValue> X</IDValue> </NameIdentifier> <NamesBeforeKey>Kristina</NamesBeforeKey> <KeyNames>Sabaliauskaitė</KeyNames> <BiographicalNote textformat="05"><p>kristina Sabaliauskaitė is a Lithuanian art historian and historical novelist. She lives in London.</p> </BiographicalNote> </Contributor> a sample of ONIX XML A01 is part of that shared language or controlled vocabulary or codelist (means author ) code 16 means ISNI, and the 16 digit number is an identifier for this particular author note that some textual data can include HTML markup

38 ISNIs already assigned Mikalojus Dauk a Jonas Mekas Antanas kėma Wilhelm Storost-Vydūnas Laurynas Gucevičius Violeta Urmanavičiūtė these are unique and persistent, with public metadata at Laurynas Gucevičius - architect of Vilnius Cathedral Violeta Urmanavičiūtė opera soprano

39 Benefits of ISNI identifier resolves uncertainty over identity other books by same author rights and royalties cross-publisher and cross-sector links to ORCID / ResearcherID / VIAF in future, will increasingly also be used to identify names of publishers and imprints ORCID: Open Researcher and Contributor ID (all academic contributors) ResearcherID: Thomson Reuters proprietary ID for scientific contributors VIAF: (virtual international authority file)

40