Homer: a case study of federation among open data portals. Nives Alciato - CSI Piemonte

Transcription

1 Homer: a case study of federation among open data portals Nives Alciato - CSI Piemonte nives.alciato@csi.it

2

3

4 Legal framework Regional Law n. 24 dated 23/12/2011 First regional law in Italy on Open Data Guidelines for reuse Basic principle: Data belong to people Cornerstones of reusability of data: Diffusion without restriction and in open and standard digital formats Use of standard legal tools Creative Common Licences Re-use and re-distribution of data is free of charge

5 Organizational framework Regional level An initiative whith ANCI Piemonte (association of municipalities): dati.piemonte.it is the infrastructure for all the regional territory (120 Municipalities and other bodies like ARPA Piemonte and Unioncamere) National level Re-use of the platform and joint project with Emilia Romagna Region and Milano Municipality European level HOMER project to transfer methodological /technical standards and increase circulation and re-use of public data OPENDAI project to improve a new architectural model to increase digital services and business opportunities

6 Technological framework: a permanent beta Portal Harmonize policies and licenses for the re-use of Si riesce Search a trasformare queste scatole data in una grafica più carina? Federation of Open Data Portal New platform DATA from Open data Open data silos PA Cloud architecture Open data Services Operational data bases of PAs to Data Services and a Federated Search Engine

7

8 HOMER is the acronym of Harmonising Open data in the MEditerranean through better access and Reuse of public sector information It is a project within the MED Programme financed by the EU Commission Implementation Starting date 01/04/2012 Implementation End date 31/03/2015

9 HOMER is A tool to unlock the full potential of Public Sector Information in the Mediterranean space A federation among open data portals and people Hundreds of public datasets opened enhancing digital heritage related to MED strategic domains (agriculture, culture, energy, environment, tourism) Promotion of digital transparency and open data culture

10 Who are the Homer s Partners 13 Partners as territorial government and 6 Partners as technological support Country Partner Mission Spain SARGA - Agencia de Gestion Agraria y Pesquera de Andalucia Territorial Gov. AGAPA - Sociedad Aragonesa de Gestión Agroambiental Territorial Gov. FUNDITEC Foundation for Development, Innovation and Technology Technical Support France Région Provence-Alpes-Côte d'azur, Territorial Gov. Région Corse Territorial Gov. AVITEM Agency for sustainable Mediterranean cities and territories Technical Support FING Fondation Internet Nouvelle Generation Technical Support Italy Piedmont Region Project Leader Sardinia, Emilia-Romagna and Veneto Regions Territorial Gov. CSI Piemonte Technical Support Slovenia Geodetic Institute Territorial Gov. Montenegro Mediterranean University of Montenegro Territorial Gov. Greece GFOSS The Greek Free Open Source Software Society Technical Support Crete Decentralized Administration of Crete Territorial Gov. University of Crete Technical Support Cyprus Sewerage Board of Limassol Amathus Territorial Gov. Malta Local Council Ass. of Malta Gozo Territorial Gov.

11 CSI Piemonte s responsabilities in HOMER In Work Package 3 «Capitalisation and long lasting effects» is the Task Leader of «Task 3.2 Federation development» it is the developer of a Federation of Open Data Portals among partners providing ICT and legal support and it is the promoter of the reuse of the technological solutions underlying the portal, developed in the context of the project

12 What we intend for federation of open data portals? Federation means the virtual system composed by a software able to collect and retrieve the metadata of published data derived from the 5 categories (agriculture, culture, energy, environment, tourism) exposed and searched by Open Data Partners Portals Look at this symbol: it represents the metadata catalogue

13 Legal framework Memorandum of Understanding Partners have been involved upon signing a Memorandum of Understanding where technological, organizational and legal boundaries have been defined as common understanding for everybody and referring to the Directive 2013/37/EU It is indicated that all technological components of the solution for the Federation (Index, Semantic Search Engine, Translator) are provided and managed under the conditions and the coordination of CSI Piemonte that releases them on the basis of an open source philosophy

14 Data framework the metadata structure Each Open Data Portals share metadata common fields: this structure builds the Federated Index CKAN title description url metadata source package_id topics language tags geographic bounding box refresh date creation date spatial scale resolution license id owner Inspire DCAT Dublin Core Intersecting the Protocols and Directives in the schema, it has been identified the minimun common set of fields for the definition of a metadata structure and to federate, indipendent from the type of dataset geographical or alphanumerical

15 Data framework the use of EuroVoc (1) Homer, now, speaks 7 languages (spanish, french, italian, slovenian, serbian-montenegrin, greek and english) with 4 different alphabets and we must share a dictionary to communicate iso code to identify the language title description url metadata source package_id topics language tags geographic bounding box refresh date creation date spatial scale resolution license id owner

16 Data framework the use of EuroVoc (2) EuroVoc is a multilingual, multidisciplinary thesaurus of the EU conformant to W3C recommendations and in it a specific concept of the 5 categories involved has the same classification and meaning in the domains and languages title description url metadata source package_id topics language tags geographic bounding box refresh date creation date spatial scale resolution license id owner Homer s categories = EuroVoc domains iso code to identify the language WATER νερό VODA вода AGUA EAU ACQUA Each ODP inserts tags in the metadata cards in its own language without the burden of translation The same concept is identified in all languages

17 Data framework The cross lingual search The semantic search multi language engine needs a specific common structure to index and retrieve the metadata of all metadata catalogues of the Homer s Partners Open Data Portals. The search engine is like a librarian who finds books only if the request form is filled out in a specific way Field_0

18 Technological framework: the federated search multi language engine The technological solution for indexing and searching among all the federated open data portals has 4 components: 1. Fed-Index Homer: the federated index file component containing the complete list of metadata 2. Fed-Translator: the component that translates every tags of the datasets via EuroVoc 3. Fed-Searcher: the centralized semantic search engine component 4. Fed-Loader API: the loader that calls the API o Webservices exposed by each Open Data Portal to create the federated Index Based on the open source project Apache Sorl Released open source on sourceforge

19 Technological framework: the indexing scenario each ODP exposes url1 that returns the list of the data id: Package List url2 that returns the attributes for the single data: Package Dataset Open Data Portals Voda stand alone nightly process scheduled Scheduled Eau Water Search Engine Agua

20 Technological framework: the indexing scenario 3 ways supported to expose the metadata: API CKAN compliant: Package List >url1 that returns a xml file1 with the list of the data id Package Dataset > url2 that returns a xml file2 with the attributes for the single data Web services dati.piemonte.it compliant: Package List >url1 that returns a xml file1 with the list of the data id =xml&layout=xml Package Dataset > url2 that returns a xml file2 with the attributes for the single data =xml&layout=xml&itemid=1083 API Catalogue Service for the Web compliant: Package List > url1 that returns a csw file1 with the list of the data id Package Dataset > url2 that returns a csw file2 with the attributes for the single data

21 Technological framework: the indexing scenario 3 ways supported to expose the metadata: Field_0 API CKAN compliant Web services dati.piemonte.it compliant API CSW compliant

22 Technological framework: the searching scenario Each Open Data Portal can use this page to test the search engine, the datasets and verify its situation inside the Federated Index:

23 Results and ongoing activities The Homer project hackathon, May The hackathon will take place simultaneously in 7 Mediterranean regions (Andalusia, Corsica, Région Provence-Alpes-Côte d'azur, Piedmont, Veneto, Emilia-Romagna, Crete, Montenegro), which will provide more than 800 datasets free (open data constantly increasing!)

24 Results and ongoing activities The Federation in terms of: shared knowledge, experiences and relationships among the Partners open hundreds of public datasets enhancing digital heritage transparency and promoting open data culture across the Mediterranean looking for new stakeholders as it is possible to configure new categories and new languages

25 A video to promote Homer

26 Thank you! Nives Alciato CSI Piemonte