Europeana Food and Drink

Transcription

1 Grant Agreement Europeana Food and Drink Technical Demonstrator Deliverable number D3.5 Dissemination level PU Delivery date Month 28 Status Author(s) Final Lise Schauer (SHIFT), Andrey Tagarev (ONTO), Maros Strmensky (EEA), Miroslav Valuš (EEA), Sašo Zagoranski (SEM) This project is funded by the European Commission under the ICT Policy Support Programme part of the Competitiveness and Innovation Framework Programme Page 1

2 Revision History Revision Date Author Organisation Description V April 2016 Lise Schauer SHIFT First draft V April 2016 Breandán Knowlton SHIFT Comments on first draft V April 2016 Andrey Tagarev Maros Strmensky Miroslav Valuš Sašo Zagoranski ONTO EEA EEA SEM Second draft V April 2016 Angelika Leitner ONB First review V April 2016 Chris Vastenhoud KMKG Second review V April 2016 Lise Schauer SHIFT Incorporating comments and finalising V April 2016 Susie Slattery CT Final review Statement of originality: This deliverable contains original unpublished work except where clearly indicated otherwise. Acknowledgement of previously published material and of the work of others has been made through appropriate citation, quotation or both. Page 2

3 Content 1 Introduction Approach Scope and aims Audience Value proposition Specifications Dataflow 1 - Data from Europeana to Historypin (EEA) Harvesting Converting Pushing to HP Configuring the job Dataflow 2 - Semantic tagging service (ONTO) Configuring the job Semantic enrichment service Example Call Dataflow 3 - Internal Historypin enrichments (HP) and Storytelling app (SEM) Metadata Crowdsourcing on Historypin Storytelling App Dataflow 4 - Annotations from Historypin to Europeana (EEA) Configuring the job Dataflow 5 - New content from Historypin to Europeana (EEA) Harvesting Saving to MINT Configuring the job Data in MINT Dataflow 6 - Tagging game (EEA) Game rules Marketing activities Success metrics Sustainability Conclusion and next steps Page 3

4 1 Introduction This document is the D3.5 Technical Demonstrator product report, which describes the scope and development of the Technical Demonstrator product. During the technical review of the Europeana Food and Drink project in December 2015, the European Commission approved the merging of technical development activities associated with tasks T5.2 and T3.4 of the project to better align the technical strength of the consortium around the common goal of facilitating the development and enrichment of the Europeana Food and Drink Content Base. The work under Task 3.4 as listed in the Description of Work on page 18 is outlined below: Specification of a technical Research & Development product based on the theme of AMBROSIA: Europeana Food and Drink Design, build and testing of a technical platform or demonstrator based on this theme Product launch and promotion The work under Task 5.2 as listed in the Description of Work on page 29 is outlined below: WAWWD will scope and implement an online platform for community engagement, crowdsourcing and social tagging of the AMBROSIA Content Base (D5.2) to support the objectives of the Community Engagement Strategy and facilitate the capture of new knowledge and content from the Community Engagement Pilot events. This platform will also include the facility for communities to set up and manage their own local projects that re-uses and adds to the AMBROSIA content base These two tasks will be integrated by focusing the work of the Technical R&D (Demonstrator) product stream on the needs of the Content Base contributors in the Europeana Food and Drink project (and by extension, other heritage institutions), creating a seamless workflow for the upload of content sets to a crowdsourcing platform (Historypin.org) using a combination of mapped data and machine tags, and then the export of those enriched content sets to metadata federator platforms such as Europeana. This meets the original goals and objectives for the technical demonstration within the project and offers a route for large-scale metadata enrichment and annotation while simultaneously meeting the strong requirements for integration with Europeana expressed in the mid-project review. In this deliverable, the scope of the Technical Demonstrator is described, as well as the approach to development, the product cluster has taken. Screenshots and descriptions will show the development and how the demonstrator functions. A section at the end of the deliverable will detail possible future iterations and use. Page 4

5 2 Approach The original Technical Demonstrator scope as described in Task 3.4 in the Europeana Food and Drink DOW on page 18 is as follows: EEA will lead the specification of a technical R&D project making use of the AMBROSIA Content Base to demonstrate innovative technical applications of cultural content EEA will design, build and test a Technical Demonstrator called ACCURATOR (D3.5) showcasing the application of content and metadata relating to European Food and Drink EEA will launch and promote the Technical Demonstrator, with support from ONB (under WP 6) Rather than build the envisaged Accurator platform which significantly duplicates functionality of the existing crowdsourcing platform already in use within the project, the project s management board has decided that the technical work of the project can be best coordinated by consolidating the effort and budgets of these two tasks into a single, more refined workflow. As said in the introduction, this has been approved by the reviewers during the technical review of the Europeana Food and Drink project in December The benefits of the technical demonstrator as proposed and subsequently developed are as follows: In order to meet the requirement of social tagging of the AMBROSIA content base, it is necessary that seed content being contributed to the Content Base can be easily mapped for upload to the crowdsourcing platform. Therefore, the technical efforts should focus on the mapping of metadata from the formats and platforms required for participation in the Food & Drink Content Base into the formats needed by the crowdsourcing platform. In addition, this transformation and upload offers a good opportunity for the deployment of intelligent machine-tagging of metadata which can improve the discoverability of this content. The crowdsourcing platform is also being used to address the requirement to add to the Content Base. As all new content coming from the Europeana Food and Drink project is required to be added to Europeana, there is a necessity for exports and mappings that map from the crowdsourcing platform into a format that can be ingested directly by metadata federators such as Europeana. Existing infrastructure will be used and reused to create more value by connecting them and, within them, creating a new set of activities that center around the discovery and enrichment of Europeana Food and Drink content base The Technical Demonstrator product cluster is made up of all technical partners within the Europeana Food and Drink project who have hours against either Task 3.4 or 5.2, or both. Each partner has a specific responsibility within the development of the Technical Demonstrator. The partners within the Technical Demonstrator product cluster are: EEA Semantika National Technical University of Athens Ontotext Stichting Europeana Shift (formerly WAWWD) Page 5

6 3 Scope and aims The aims of the Technical Demonstrator are to build a roundtripping infrastructure to serve Europeana-derived content to new audiences on a crowdsourcing platform, where these audiences can engage with the content and improve its metadata, as well as upload new material that relates to this content. The user-generated enrichments and the ones done semi-automatically will then be available to be pulled back into Europeana and can be used to improve the metadata there. The technical demonstrator as it has been developed is mainly a roundtripping mechanism between Europeana and Historypin, with a user interface for the collection holders. The technical demonstrator includes 2 additional user interfaces, which are targeted to the general public and aim to increase engagement with the Europeana content, while also serving as ways to improve the objects metadata. Figure 1: An illustration of the various data flows within the Technical Demonstrator product Page 6

7 The functionality of the Technical Demonstrator is divided into various dataflows, which can be seen in the figure above and are described below. They are described in more detail later in this Deliverable. Dataflow 1 takes content from Europeana through the Europeana Search API and connects to Historypin.org through its Write API to publish this content in a Collection in the form of pins (Historypin s term for an object). Dataflow 2 takes the above Collection from Historypin through its Read API and adds it to the Semantic Tagging Service, which infers better locations, dates, food and drink related topics and links to concepts on Wiki. These enrichments will be added to the pins in that Collection on Historypin.org, as tags. Dataflow 3 centers around Historypin users enriching the Europeana content within the Historypin.org platform, using the Comment, Suggest better location, Suggest better date and Suggest tag functionalities. Within Dataflow 3, users can also remix Europeana and Historypin content through the Story Hub, a storytelling app which allows users to pull in objects from Europeana.eu and Historypin.org through their respective APIs. The user then creates a story with the objects, putting them in order and adding captions. These stories are then imported into Semantika s platform, Museums.eu, as virtual exhibitions. In this way, they can then be linked to directly from anywhere on the internet, for example from the Europeana Food and Drink channel on Europeana.eu. Dataflow 4 takes the annotations that have been created in dataflows 2 and 3, and serves them through Historypin s Annotations API to the Europeana s Annotations API, where they can be added to the Europeana objects. Dataflow 5 takes new objects that have been contributed by Historypin users to the Europeana collections on the Historypin.org platform and sends them to MINT via Historypin s Read API. In MINT, they are mapped to Europeana s Data Model and pushed to Europeana through their Unified Ingestion Manager. As these new objects have been contributed by Historypin users, they will be presented as coming from the institution Historypin. Lastly, Dataflow 6 is a Tagging Game, which is a further way through which Europeanaderived objects can be enriched with user-generated tags. The objects will be taken from Europeana s Search API, and any tags that are generated through the game will be fed back into Europeana through their Annotations API. As the Historypin crowdsourcing platform works best with photographs and objects which have strong location metadata, the Tagging Game enables engagement with Europeana Food and Drink content that is more transient, such as paintings and museum objects. 4 Audience As the Technical Demonstrator is a technical R&D product, the main audience for the Technical Demonstrator is the collective Europeana Food and Drink consortium, and especially the content providers. The data that they have contributed to Europeana is being ingested into Historypin.org, which allows them to set up campaigns to reach out to their own audiences to ask them to help tag the objects and thus improve the metadata.. Page 7

8 The Technical Demonstrator is a technical solution connecting the rich Europeana content to a successful crowdsourcing platform with tools for enrichment. It will make it easier for content providers to crosscut and showcase their data in different ways, and will also help increase visibility of the Europeana Food and Drink content. The Storytelling App developed by Semantika will allow users to not only annotate the Europeana content, but also to remix and reuse it to illustrate their own stories. The Technical Demonstrator s interface, also called the Orchestrator, allows the Europeana Food and Drink partners to choose which Europeana collections to showcase on the Historypin crowdsourcing platform, as well as which annotations and new objects related to these collections to publish back to Europeana. As this is quite a technical solution, the Technical Demonstrator will be tested thoroughly with Europeana Food and Drink s data providers in May 2016 to make sure it is usable by partners with an average level of technical understanding. EEA and Historypin are committed to provide help and troubleshooting for those partners using the Technical Demonstrator. The Technical Demonstrator can also be used by other content providers that have published their material to Europeana (outside of the Europeana Food and Drink project) to push their material to the Historypin crowdsourcing platform and to get these assets enriched through the Technical Demonstrator. 5 Value proposition The Technical Demonstrator as documented in this deliverable will create value for the Europeana Food and Drink project specifically and for the Europeana ecosystem in general as follows: It creates an easy way for the Europeana Food and Drink project to showcase its data on a crowdsourcing platform in an engaging way, based on location; It creates an easy way for the Food and Drink project to have a presence on Museums.eu (Semantika s platform); It attracts digital community engagement with the Food and Drink data through Historypin.org s online engagement tools, which will increase awareness of and engagement with the Europeana Food and Drink project and Europeana; It attracts crowdsourced enrichments of the Europeana Food and Drink data through Historypin.org s online enrichment tools, improving the metadata and thus the quality of the record, and pushes these enrichments and annotations to Europeana through Historypin.org s API; It creates a simple on-ramp for data partners to publish interesting content sets to the general public, solicit enrichments and then provide data to a metadata federator such as Europeana, enhancing discoverability and potential for creative re-use of content. Page 8

9 6 Specifications The section below outlines the Technical Demonstrator specification and functionality, divided up into dataflows. Each subsection explains what the dataflow does, as well as which product cluster partner built it. The word Orchestrator as mentioned in the documentation below refers to the user interface that allows the Technical Demonstrator users to pull in data from Europeana map it to Historypin and push selected annotations and new objects back to Europeana. The Orchestrator user interface can be found at The open-source code for the Orchestrator interface can be found at The open-source code for the Tagging Game can be found at Dataflow 1 - Data from Europeana to Historypin (EEA) The purpose of this dataflow is to retrieve Europeana objects through the Europeana REST API and send them to Historypin so they will be displayed on the Historypin.org platform. Figure 2: The steps needed to harvest data from Europeana and import it into Historypin. The three components of the Technical Demonstrator shown above are described below: Harvester - this component harvests EDM from Europeana using OAI-PMH or REST API and stores harvested data on a disk Convertor - this converts the EDM, which is in xml format, into JSON pin objects so it is compatible with Historypin s data model HP saver - this creates a collection of pins from the JSON and adds these converted pins to a Collection on Historypin, where it is now visible for users. The three processes are described in more detail below Harvesting To harvest data from Europeana, the user (here referring to a Europeana Food and Drink project partner or another content partner who has contributed data to Europeana) uses Europeana s REST API. When the harvesting has finished, the user is notified and instructed to go to the next step. If the harvesting fails for any reason, the user is also notified and will have to start the harvesting again. Page 9

10 6.1.2 Converting Now the data has been imported, it needs to be transformed from Europeana s data model (EDM) to Historypin s data model. This means a transformation from EDM xml to Historypin s JSON. Every individual object is converted into a single JSON file. Below is some information on how the EDM has been mapped to Historypin s data model. EDM items title dcdescription edmisshownby id guid type year rights country edmplacelatitude edmplacelongitude HP records pin caption description content remoteid link type (see Mapping of type 1 ) date License (see Mapping of license 2 ) location (see Mapping of location 3 ) lat lng range 1. Mapping of type Europeana TEXT IMAGE SOUND VIDEO HP PHOTO MEDIA 2. Mapping of license Europeana rr-p/ out-of-copyright-noncommercial/ nses/by-nd/3.0/ rr-f/ HP copyright no-copyright open-government Page 10

11 icdomain/mark/1.0/ icdomain/zero/1.0/ nses/by/3.0/ nses/by-nc/3.0/ nses/by-sa/3.0/ nses/by-nc-nd/3.0/ nses/by-nc-sa/3.0/ Any other not recognized license public-domain <string of license from EDM> 3. Mapping of location If there is longitude and latitude in EDM this will be used for lat and lng with range 0. If not, the HP API will be queried using country as location Pushing to HP When the conversion is complete, the user is prompted to create a Collection on Historypin.org, setting the Collection name and location, as well as the pin defaults (in case of incomplete metadata) of location, date and tags. Once these are set, the HP saver starts to add pins to the Collection Configuring the job In order to get data from Europeana to Historypin, the user has to take the following steps in the Orchestrator interface: 1. Go to Orchestrator administration console instance and use provided username and password to log in. Page 11

12 Figure 3: The login screen of the Orchestrator 2. After successful login, the user lands on the Create task screen. In this screen, you can create tasks and review previously created ones. Figure 4: the Create task screen in the Orchestrator. 3. To create a task the user must fill all the required fields in form. The form will dynamically change, depending on the selected options. a. Give the task a name, e.g.: 'Europeana to HP task'. b. Select 'Europeana' as the source of harvesting. c. Select 'REST' as the type of harvesting. d. Choose 'Historypin' in Destination part. Page 12

13 e. Fill out Historypin API credentials (API key, API secret) for storing data f. Fill out the Collection name that will display on Historypin. This collection will be created during the process. E.g. 'My new collection'. g. Choose default location for objects without location data. Use the map to choose a default location, e.g. Budapest, or Hungary. h. Add default tags which will be added to every pin i. Add Lucene query for additional filtering of Europeana data j. Fill out Metadata prefix, e.g. "edm". Figure 5: The active and planned harvesting tasks in the Orchestrator 4. Click on Create task button. After that task will appear in list below the form. The table will show the ID of the task, and its status. Refresh the page to see the changes in the table. Page 13

14 Figure 6: the Orchestrator s interface 5. After the task creation, the Orchestrator picks the task and schedules its run. Users can see more detailed information about the task in Logs view. Click on the "Logs" in right navigation menu. 6. The table contains information about the last task run and also shows detailed error messages, warnings, as well as time information about the processing steps. 7. When the task is executed correctly, the results can be verified on Historypin.org Figure 7: A test collection on Historypin.org, imported with the technical demonstrator Page 14

15 Figure 8: a test pin on Historypin.org with the link to the original Europeana object in the metadata 6.2 Dataflow 2 - Semantic tagging service (ONTO) The Orchestrator connects Historypin.org to the Semantic Tagging Service provided by Ontotext. The flow of this enrichment happens as follows: Figure 9: the process for the semantic enrichments of Europeana-derived Historypin collections The components of the process above are explained in more detail below: Harvester - harvests collection defined by project_slug and store data to disk Enricher Page 15

16 o Extracts Name, Description, Location from HP pin and sends it to the Semantic Enrichment Service o Receives enriched content, writes it into HP pin json Place tags - written into pin/places F&D Topics - written into pin/tags Approver o Approver sends an to the user who owns the enrichment job with a link to the Graphical User Interface (GUI) so they can approve the enrichment o GUI shows data from enrichment process and allows the user to remove nonrelevant content o The Approver then saves approved content into HP pin JSON HP saver - sends enriched HP json data to HP Configuring the job To send a HP collection to the Semantic Tagging Service, the user needs to: 1. Go to Orchestrator administration console instance and use provided username and password to log in. Figure 10: the Orchestrator login screen 2. After successful login, landing view will be "Create task" screen. a. Fill out name of a job e.g. 'Semantic demonstrator enrichment' b. Set 'Harvesting' to 'Historypin' c. Set destination to 'Semantic demonstrator' d. Set 'Project slug' from Historypin e.g. 'project_1'. This can be found in the Collection link. Page 16

17 Figure 11: the Create Task screen where the semantic tagging task can be created 3. Click on Create task button. After that task appears in list below the form. The table shows the ID of the task, and its status. Refresh the page to see the changes in the table. Figure 12: The Orchestrator interface showing the planned semantic enrichment task 4. After the task creation, Orchestrator will pick the task, and schedule its run. The rest of the process is identical to the harvesting process described above. Page 17

18 6.2.2 Semantic enrichment service The semantic enrichment service creates two kinds of enrichments that are offered as suggestions to the end-user: Places F&D Topics from the F&D Classification (see D2.2) It was developed by ONTO based on technology created for the Semantic Demonstrator (D3.20). The enrichment service works with free text typically collected from the title and description of objects (although several other fields are good candidates to be added to the data as well). The aim of the enrichment process is to identify mentions of food and drink-related terms and locations within the texts. At present the service only works with English text but work is ongoing to extend it to French as well. The extracted enrichments are presented by dbpedia URLs that identify them uniquely, give further context and allow for hierarchical organization and searching. There is also an additional step that connects identified locations to the geonames database which puts them in a geographical hierarchy and provides reliable coordinates. This hierarchy is used to remove unnecessary enrichments (e.g. with and the latter will be removed because London is a more specific location contained within the United Kingdom) and also add parent locations with coordinates when coordinates are not available for a certain location (e.g. has no coordinates so its parent is added along with its coordinates allowing the object to be displayed on a map). The final step is to transform the discovered enrichments into the appropriate format. The enrichments are returned as a collection of Linked Data triples. Each triple's subject is the URI of the enriched objects provided in the uri parameter of the initial request (e.g. and there are a few kinds of triples according to the enrichment they provide: Food and Drink topics have the dct:subject predicate and the object is the dbpedia URL of their article. Place enrichments have the dct:spatial predicate and the object is either the dbpedia URL of their article or the GeoNames URI in case no dbpedia article exists for this location. Coordinates are two separate triples with the wgs:lat and wgs:long predicates and their objects are float literals. Unlike the other kinds, this triple has the place URI as subject. Page 18

19 Below is an example of the information the service provides: { "@context" : "efd-context.jsonld", "@graph" : [ { "@id" : "dbr:city_of_westminster", "lat" : " ", "long" : " " }, { "@id" : "dbr:palazzo_rosso_(genoa)", "lat" : " ", "long" : " " }, { "@id" : " "spatial" : [ "dbr:city_of_westminster", "dbr:fleet_street", "dbr:palazzo_rosso_(genoa)" ], "subject" : "dbr:tea" } ] } The JSONLD format was selected as the output format, since it is both JSON and RDF, therefore is simple to consume by web applications, yet is Linked Data. It depends on the following JSONLD Context efd-context.jsonld: { "@context": { "long" : { "@id" : " "@type" : " }, "lat" : { "@id" : " "@type" : " }, "spatial" : { "@id" : " "@type" : "@id" }, "subject" : { "@id" : " "@type" : "@id" }, "wgs" : " "dct" : " "dbr" : " "edm" : " } } Note: The enrichment service does not guarantee that any enrichments will be discovered in the text or that only one location with coordinates will be returned. Page 19

20 6.2.3 Example Call Below, an example call to the semantic enrichment service is described. curl -X POST -H "Content-Type: text/plainxml" " " TopFotoSample.txt contains the body of the POST request which is simply free text to be enriched. The identifier of the object is passed in the uri parameter. The response received from the service is in the specified JSONLD format and looks like this: [ { "@id" : " "dct:spatial" : [ { "@id" : "dbr:seattle" }, { "@id" : "dbr:london" } ], "dct:subject" : [ { "@id" : "dbr:chess_piece" }, { "@id" : "dbr:cooking" }, { "@id" : "dbr:sponge_cake" }, { "@id" : "dbr:blue" }, { "@id" : "dbr:strawberry" }, { "@id" : "dbr:cake" }, { "@id" : "dbr:plate_tectonics" }, { "@id" : "dbr:coffee" }, { "@id" : "dbr:black_people" }, { "@id" : "dbr:sugar" }, { "@id" : "dbr:food" } ] } ] The context of the response is external and is available here: Page 20

21 6.3 Dataflow 3 - Internal Historypin enrichments (HP) and Storytelling app (SEM) Metadata Crowdsourcing on Historypin As part of the technical demonstrator, Historypin has developed Metadata Crowdsourcing on the Historypin.org platform, which enables Historypin users to: Suggest a better date for a pin Suggest a better location for a pin This functionality is added to the already existing comment functionality, where a user can leave a comment on a pin, and the suggest a tag functionality, where the user can suggest a subject tag. The process for a Historypin user to suggest a better date or location for a pin is detailed below: 1. The user logs into Historypin.org and navigates to the pin they want to improve Figure 13: the Historypin.org pin interface 2. The user scrolls down and clicks Suggest a better date on the right hand side of the screen. Page 21

22 Figure 14: The suggest a better date option on Historypin.org 3. The user enters their suggestion in the fields and presses Save. Figure 15: the Suggest a better date interface Page 22

23 4. The date suggestion now shows up as a comment below the pin. Figure 16: the newly suggested date as it is visible on Historypin.org 5. The location suggestion functionality follows the same steps. Figure 17: the Suggest a better location functionality on Historypin.org Page 23

24 Figure 18: the newly suggested location visible under the pin 6. Both suggestions are now visible as comments under the pin. 7. The date and location are changed immediately to match the most recent suggestion. The owner of the pin gets an notification about the changes to their object. All suggestions can be rejected by the owner of the pin by clicking the Reject button on the suggestion when logged in. The location and date of the pin then revert to the previous suggestion or the original location and date. Page 24

25 Figure 19: The date and location suggestions with their Reject buttons Storytelling App The purpose of the Storytelling application is to demonstrate that the rich content from Europeana together with data from Historypin can be made more accessible by enabling the end users to create stories from existing content. The app promotes user and community engagement by allowing everyone to enrich the content by adding their own captions, fitting into the overall story they are creating. The user added content could potentially be pushed back to the original data source further improving the data already in Europeana and Historypin. This Storytelling app is mobile and portable. Any stories that are created by its users are imported to the Museums.eu platform, where they are presented as virtual exhibitions. This makes the reach of the app wider, as this means users without a device that compatible with the app can still browse and engage with the stories created by other users. They can also use the Museums.eu website to create a virtual exhibition themselves, pulling in objects and data from Europeana and Historypin. The virtual exhibitions on Museums.eu are permanent and can be linked to from other websites, such as the Europeana Food and Drink channel on Europeana.eu. Page 25

26 Figure 20: the landing page of a Story on the Storytelling app Figure 21: the search functionality of the Storytelling app Users can create stories by adding data that can be found on Europeana using the search tool built into the story editor. Figure 22: The item view of the Storytelling app Page 26

27 After selecting an object, details of the object are displayed and user is given the option to add it to the story line. Figure 23: The functionality to add a caption to an item After adding the item the user has the option to add a personal caption to the item. Figure 24: The virtual exhibition functionality within the Museu.ms CMS From within the Museu.ms CMS the user can create a new virtual exhibition by selecting Add new. The user-friendly editor allows simple editing of content. Page 27

28 Figure 25: The virtual exhibition editing view Figure 26: The virtual exhibition editing view The CMS editor allows for selection of the slide template for different scenarios. After the story is saved it can be immediately viewed both on the web and in the Storytelling app. The title slide allows the user to enter the full-screen playback mode. Page 28

29 Figure 27: the chapter title slide in the Storytelling app Figure 28: Display of a single item together with the caption selected by the user. Page 29

30 6.4 Dataflow 4 - Annotations from Historypin to Europeana (EEA) Rather than updating Europeana objects with metadata derived from Historypin, Europeana objects are annotated with enrichments from Historypin. This way, existing metadata cannot be overwritten, but the enrichments will still make the Europeana object s metadata richer. Figure 29: The process for pushing Historypin annotations to Europeana The components of the above process are explained in more detail below: Harvester - harvests HP annotations for recently updated pins. HP will provide only updated content Converter - the updated content is converted to JSON-LD format accepted by Europeana Annotation AP EU saver - this part pushes annotations to Europeana through the Europeana Annotation API Configuring the job To push annotations from HP into Europeana, configuration of job is needed. This will require following steps. 1. The user needs to log into the Orchestrator as described in the processes above. 2. The user fills out the required fields in the form, which changes dynamically. a. Fill in the name of the task. E.g.: 'HP annotation harvest'. b. Select 'HistoryPin'' as the source of harvesting. c. Select 'REST Annotation API'' as the type of harvesting. d. There will be only 'Europeana Annotation API' as a target e. (optional) Date from - from which date data should be collected. Formatted in Europeana specified date format. E.g. " T17:12:06Z". f. (optional) Date to - until which date data should be collected. Formatted in Europeana specified date format. E.g. " T17:12:07Z". Page 30

31 Figure 30: The Annotation task in the Orchestrator interface. 3. The user clicks on the Create a task button and the Orchestrator will schedule the task. This follows the same process as the other types of tasks described above. 6.5 Dataflow 5 - New content from Historypin to Europeana (EEA) New data which originates in Historypin and is not yet found in Europeana (i.e. it does not have a Europeana ID) is uploaded to Europeana. This is done via MINT software as shown in the diagram below. Figure 31: the process of pushing new content from Historypin to Europeana The components of the process above are described in more detail below: Harvester - loads newly created pins from HP and stores them on disk MINT saver o Uploads JSON pin data into MINT o Calls defineitems method of MINT API for uploaded dataset o Calls transform method of MINT API to transform data to EDM Page 31

32 o Calls publish method of MINT API to publish transformed dataset on Europeana Harvesting Harvester reads a collection, defined with project_slug identifier, from HP and stores data (json files) of pins belonging to this collection on disk for MINT saver Saving to MINT When saving to MINT, MINT saver first compresses all pins obtained from HP into a.zip file. Then this file is uploaded to MINT and the MINT saver calls a procedure for publication on the MINT provider interface Configuring the job For this to achieve user has to do following steps. 1. The user logs into the Orchestrator as described above. 2. User chooses destination Mint and provides project_slug for the collection that needs to be uploaded into MINT. The user then clicks on Create task Figure 32: the Orchestrator interface with the Publish to MINT task 3. After the creation of this task, it can be seen in Task list, ready for scheduled execution. Page 32

33 Figure 33: the Orchestrator interface showing the task list 4. At the scheduled time (as defined in the configuration of the Orchestrator), the Orchestrator will harvest data from Historypin and store the JSON object of pins into MINT for processing Data in MINT In MINT the data are automatically transformed to the EDM schema and then published to NTUA s OAI PMH server from which Europeana harvests for publishing on its portal. This process happens automatically and does not require any action from the user of the Orchestrator. This process is described below: 1. Upload of a JSON file that contains metadata of PINs made in Historypin, via the Orchestrator. The structure of the JSON file corresponds to API response of Historypin for a retrieving a PIN. Once the upload is done the dataset appears under the Historypin organization in MINT (as shown in the figure below) Page 33

34 Figure 34: the Historypin organisation workspace in MINT 2. The next step is the definition of items. MINT uses XML technologies so the JSON import is automatically converted to an XML document. However, since XML is a tree structured representation of data MINT needs to know the element that acts as a parent and under which all the information about a record is held. Therefore by this step the root element for the import is set. This is done automatically by setting as the root element /json/results, as the label item i.e. the element that holds the title of the record /json/results/caption and /json/results/id as the identifier that needs to be unique. Figure 35: Defining the items in MINT Page 34

35 Once this information is set the itemization process i.e. the process the separates a batch of records into individual records starts. When this process is over a view of the imported records is possible. Figure 36: a view of the individual imported Historypin records Figure 37: a view of the imported Historypin items 3. The step that follows is the transformation of the imported records to EDM. A mapping has been made in MINT that exploits the expressiveness of PINs in creating a semantically aligned version in EDM. This mapping maps all the necessary information for EDM (i.e. rights, type, isshownat, isshownby) together with the important geographical information provided by PINs and it is used automatically by the creation of an EDM dataset as shown below. Note that once a dataset is transformed to EDM Europeana previews appear when Show all items option is followed and then an item is clicked that present the item as it will appear when published on Europeana. Page 35

36 Figure 38: the transformation of Historypin data to EDM Figure 38: the transformation of Historypin data to EDM Figure 39: the preview of a Historypin record in MINT, as it would look on Europeana Page 36

37 4. Finally the last step is the actual publication - datasets published on MINT appear with a globe under the workspace, as seen in the figure below. It is important to mention at this point that datasets are not directly sent to Europeana but to NTUA s OAI-PMH server from which Europeana harvests. The server is available at The figure below presents the records published in EDM (rdf namespace) in OAI_DC that is the default namespace and is produced from the EDM file by keeping only the dc and dcterms elements. Duplicates are considered as conflicts. In other words a conflict is created when the same record is re-sent to the OAI-PMH server. Figure 40: The publication history screen in MINT 6.6 Dataflow 6 - Tagging game (EEA) The tagging game Art Tag is built as a web application and is connected to Europeana digital content. The cultural content it uses is selected by the content provider that wants to use the game to ask the public to enrich their content, from predefined datasets of artworks from the Food and Drink project or from another domain. This Tagging Game is particularly useful for engaging the general public with objects that are not easily displayed on the Historypin crowdsourcing platform, as they lack good location or date metadata. The Europeana Food and Drink project has gathered many objects that are related to food and drink but are not easy to locate on a map, such as vessels and other utensils, or paintings of food and feasting. The purpose of the Art Tag game therefore is to provide ways of enriching this particular type of content. Page 37

38 Among other things, the game will help the users to explore and get to know new objects and artworks while the artwork itself will acquire unique descriptions that cannot be obtained from computer tagging or picture recognition. The curators will acquire these descriptions from the general public which can then be approved and visualized when displaying these cultural heritage objects in Europeana Game rules 1. Players will join the game by subscribing via social media or by creating a new identity in the game via a simple registration process. 2. The first player creates a new game for 4-6 players and the others can join the existing game. At the beginning, each player has 6 cards available, displayed on the screen - each card is a work of art from a predefined Europeana dataset. 3. There are two categories of players in every game - one narrator and the rest are listeners. The Players take turns being the narrator in each round. 4. Each player starts the game with six random artworks. The player whose turn it is to be a narrator looks at the six images on the screen. This narrator chooses one of these images and describes it with a written sentence or a phrase that best fits the image. This description is recorded. 5. The phrase describing the artwork image appears to the other players who are then asked to select from their own sets of artworks the image that most closely matches the description given by the narrator. 6. These selected images, one from each player, are displayed to everyone in the game and the players who were listeners in the game (so all players except the narrator) have to guess which of these artworks the narrator s selected work is. 7. In case either no one or everyone finds the correct image, the narrator scores 0 points, and each of the other players scores 2 points. Otherwise the narrator together with all players who had the correct answer will receive 3 points. The listeners also score 1 point for each vote that their own image receives from other listeners. 8. After the game, the artworks receive the sentence that the narrator used as a description which can be used for tagging, search purposes and overall better identification. Page 38

39 Figure 41: the Art Tag home page Figure 42: The screen where you can choose to join an existing game or create a new game Page 39

40 Figure 43: the Waiting screen. As a player you might have to wait until there are other players available. In the meantime, you can browse art from a predefined Europeana dataset. Figure 44: the narrator s screen with the six cards and the box to enter a description Page 40

41 Figure 45: A card s detail. The narrator sees this screen when choosing a card to describe. Each player, while waiting for the narrator s description, can also click cards to see their metadata. The Art Tag game is still in development, but will be available for Europeana Food and Drink partners to test with their collections before the end of the Europeana Food and Drink project in June Marketing activities As WP5 lead, Shift will firstly encourage all Europeana Food and Drink partners to use the Technical Demonstrator to import their collections into Historypin.org and to set up crowdsourcing and engagement campaigns around the content as part of Work Package 5, focused on directing their audience to Historypin.org s user interface and Semantika s Storytelling App. These audiences will be asked to add comments, location information, date information and subject tags to the Europeana objects in the Historypin interface, and to create stories or exhibitions using the Storytelling app. They will also be asked to suggest tags for objects using EEA s Art-Tag game. As the primary users of the Technical Demonstrator interface are institutions and organisations with assets in the Europeana database, the Technical Demonstrator development cluster will also showcase the product to other Europeana projects that will benefit from being able to import large batches of Europeana objects into Historypin to have them enriched, both automatically and by end-users. The user-generated stories that will be made using Semantika s app will be showcased on the Europeana Food and Drink channel on the Europeana.eu website and through social media marketing. Page 41

42 8 Success metrics As the technical demonstrator is a R&D product, there were no specific success metrics mentioned in the Europeana Food and Drink Description of Work. No specific indicators have been provided against these two activities because they are exploratory in nature and it is not clear exactly how the outputs will be used. We anticipate developing indicators for these deliverables as part of their specification and development. Europeana Food and Drink DOW Part B, page 108 Because of the limited time remaining in the project, we expect to be able to meet the following success indicators by the end of the project in June 2016: 3 Europeana Food and Drink partners have imported their content from Europeana into Historypin and have run a small online crowdsourcing campaign around it All Europeana collections imported to Historypin will be enriched using the Tagging Machine developed by Ontotext (where possible) 50 objects originating from Europeana have been manually enriched by audiences, through the Historypin.org user interface 10 new objects have been added to Historypin as a result of the online crowdsourcing campaigns, which have appeared in Europeana or are scheduled to appear in Europeana 5 user-generated stories using Europeana Food and Drink content have been created using the mobile storytelling application developed by Semantika 40 tags have been suggested by users through EEA s Art-Tag game 9 Sustainability The Technical Demonstrator user interface is ready for use by Europeana Food and Drink project partners at the time of writing, bringing Europeana collections into the Historypin platform and pushing annotations and new objects back to Europeana. This roundtripping infrastructure to and from Europeana will be maintained as it is an excellent tool to ingest Europeana collections and showcase them in a different place to attract new and more varied audiences, which will increase the cultural and commercial value of these collections and meet the objectives of the various Europeana-based projects that are currently running. The respective technical partners each commit to maintaining their parts of the Technical Demonstrator beyond the lifetime of the Europeana Food and Drink project. Page 42

43 10 Conclusion and next steps This Deliverable describes the Technical Demonstrator product, which is a technical R&D product that enables the roundtripping of Europeana objects to the Historypin crowdsourcing platform, where they can be enriched both automatically and by users with comments, tags, location suggestions and date suggestions. These enrichments are then sent back to Europeana, so the assets in the Europeana database will also be enriched. The Technical Demonstrator also has the capability of sending to Europeana new objects that are contributed via the Historypin crowdsourcing platform, which broadens Europeana s collections. The next steps for the Technical Demonstrator are to encourage Europeana Food and Drink partners to use the roundtripping infrastructure. This will lead to improvements in the user interface as partners are feeding back their experiences. EEA will maintain the infrastructure and has committed to making improvements to the user interface based on partner feedback. Page 43