Fi-domain names OData service description Version: 2.1 Page 1(18)
1 Introduction... 3 2 What is OData?... 4 2.1 OData: technical description... 4 3 OData service of fi-domain names... 6 3.1 Rights of use and identification... 6 3.2 Data structures of the service... 7 4 Using OData interface in Excel... 10 4.1 Installation of PowerPivot for Excel 2010... 10 4.2 Enabling of PowerPivot in Excel 2013... 10 4.3 Opening OData material in Excel... 12 4.4 Using the data... 14 5 OData for application developers... 18 Page 2(18)
1 Introduction FICORA has provided a technical interface for the domain name system managing Finnish fi-domain names. Users of the service may use the interface for obtaining information about domain names registered by organisations and associations. Information about domain names held by private persons is not published in the OData service. This document contains a description on the OData service data content and instructions for using the service. The service is available on any platform supporting the OData protocol. The instructions mainly cover using the interface with Microsoft Excel. In addition, the document briefly discusses the special features of programming relating to OData interfaces. You can use the data of the domain name system in your own services. Any use of the data content is subject to the laws of Finland. Page 3(18)
2 What is OData? Open Data Protocol, or OData, is a protocol for sharing open data. OData was originally developed by Microsoft but currently the protocol is standardised at the OASIS consortium. OData is designed for sharing structural file-like data in a web environment. The protocol may be used both for communication between ready-made applications and as an interface for custom designed solutions. The most popular desktop application supporting OData is Microsoft Excel (version 2010 and newer). PowerPivot, which is a free add-in for Microsoft Excel, enables importing atom formatted OData material directly into a worksheet and then editing the material. This means that it is possible to create an Excel file which automatically updates a list of domain names registered by a certain company, for instance. For more examples of using Excel, see Using OData interface in Excel below. FICORA implements OData versions 3 and 4. 2.1 OData: technical description OData is a protocol designed for web environment that relies heavily on other existing open protocols. It is based on the Hypertext Transfer Protocol (HTTP) which performs all data searches. On top of this, OData builds a few layers which are described briefly below. Firstly, OData describes the structure of the data with a metadata document. Due to this Entity Data Model (EDM), the structure of the OData interface is similar to a relational database: sets of concepts may have, e.g. typed fields, references and keys. The metadata is sufficient for using the interface description to automatically generate a data cube for analysis purposes or a programming language model, for example. Secondly, OData determines a query language for data searches. The query language is built on http requests using the REST mechanism. FICORA's OData service, for instance, is available at https://odata.domain.fi/opendomaindata.svc. The Domains data set is one part of the entire database in FICORA's OData service which may be used at https://odata.domain.fi/opendomaindata/domains(). One of the key benefits of OData is that it directly supports data searches in parts to avoid transferring large data sets unnecessarily. For example, query https://odata.domain.fi/ OpenDomainData.svc/Domains()?$orderby=GrantDate desc&$top=100 searches for a hundred latest domain names granted. Similarly, filter $filter=postalarea eq 'HELSINKI' returns domain names registered only in Helsinki. Thirdly, OData determines the data transfer format. By default, OData material is presented using the AtomPub protocol which is an established format for publishing data sets. Alternatively, the data may be requested in a more lightweight JSON format. OData V4 returns data only in JSON format. XML/Atom support will be added to V4 interface after OASIS releases specifications for XML representation. Both versions return the same material and most queries will work with both standards. Changes in OData V4 can be found in this OASIS consortiums technical document. Page 4(18)
OData also supports editing data but this feature is not enabled in the interface in FICORA's domain name service. For more information about OData, visit www.odata.org. Page 5(18)
3 OData service of fi-domain names The OData interface in the domain name system provides information about domain names registered by organisations and associations (Domains). It also contains name service data (NameServers) and contact details of the domain name administrators. Below are examples of queries that can be executed for this set of data: Domain names registered by the Ministry of Finance Domain names expiring within a month Domain names containing the word telephone Recently granted domain names Please note that in principle, the interface does not provide information about domain names registered by private persons. Therefore, the interface cannot be used to check whether a certain domain name is available. This restriction should be noted in other queries, too. However, in practice, the data is relatively comprehensive. Roughly 80% of domain names are used by organisations of associations. The data provided by the OData service is updated once a day during the night. 3.1 Rights of use and identification OData service does not require identification. OData service is available at two locations, one for both versions: V3: https://odata.domain.fi/opendomaindata.svc V4: https://odata.domain.fi/v4/odata Please note that in practice, the service cannot be used on a normal web browser because browsers cannot present large data sets in a readable format. Page 6(18)
3.2 Data structures of the service The data provided by the OData service is presented in the scheme below. The scheme also describes the structure of the data provided. Domains Name State GrantDate LastValidityDate IsDNSSecInUse Holder OrganizationID Address PostalCode PostalArea PhoneNumber AssociationType DepartmentOrContactPerson Country Registrar Nameserver1 Nameserver2 Nameserver3 Nameserver4 Nameserver5 Nameserver6 Nameserver7 Nameserver8 Nameserver9 Nameserver10 The Domains list contains details of the domain name and its holder, and specifies the registrar and the name servers of the domain name. Page 7(18)
Domains Data Description Data format Name State The domain name without the TLD.fi (e.g. ficora ). This field is a record key specifying the record within the set. State of the domain name. Possible states are: Registered Temporarily removed from fi-root Transfer denied In grace period Validity expired GrantDate When the domain name was registered Date and time LastValidityDate When the domain name expires, if not renewed IsDNSSecInUse Indication whether DNSSEC is in use Yes/no Holder OrganizationID Address Name of the organisation or association using the domain name Business ID of the organisation or registration number of the association or political party using the domain name The association registration number contains no points (. ). Postal address of the domain name holder Date and time PostalCode Postal code of the domain name holder PostalArea Postal area of the domain name holder PhoneNumber AssociationType DepartmentOrContactPer son Telephone number of the domain name holder Type of the domain name holder Company Association Foundation Municipality State Public Sector Community Political Party 'Private Person Please note: Domain names held by private persons appear only if the holder's information has been specifically marked as public. Contact organisation or contact person of the domain name Country Home country of the user Page 8(18)
Registrar NameServer1 Nameserver10 Domain name registrar If the registrar field reads '- --', there is no registrar for the domain name. Name servers of the domain name. Name servers are not required. The maximum number of name servers is ten. Please note that the capacity of the interface only allows returning a maximum of 100 records at the time. At the end of each batch, there is a continuation link for the following records in accordance with the OData standard. Some of OData query tools follow such links automatically or at least provide an easy user interface for page navigation. Page 9(18)
4 Using OData interface in Excel One of the easiest ways to use OData services is the Microsoft Excel spreadsheet. Excel does not directly support OData but the PowerPivot add-in for Excel 2010 and newer versions imports the data directly to a workbook from OData sources. PowerPivot does not support JSON formatted data, so the add-in can only be used for OData V3 interface. This chapter contains instructions for installing the PowerPivot add-in and describes different ways of processing the data in Excel. 4.1 Installation of PowerPivot for Excel 2010 The PowerPivot add-in must be downloaded and installed separately. The add-in may be downloaded at http://www.microsoft.com/en-us/download/details.aspx?id=29074. Please note that the add-in does not work in versions preceding Excel 2010. The add-in requires.net Framework 4.0. The installation of PowerPivot usually requires administrative privileges on the computer. After installing PowerPivot, a PowerPivot tab appears in the Excel ribbon. After this, please follow the instructions in chapter 4.3. 4.2 Enabling of PowerPivot in Excel 2013 The PowerPivot add-in is built into Excel 2013 but by default, it is not enabled. To enable PowerPivot, follow these steps: Go to File > Options > Add-Ins. Page 10(18)
In the Manage drop-down menu at the bottom, select COM Add-ins and click Go. Check the Microsoft Office Power Pivot for Excel 2013 box, and then click OK. The ribbon at the top of the Excel window now has a PowerPivot tab. You can now use PowerPivot for opening OData sources, for example. Page 11(18)
4.3 Opening OData material in Excel Below are instructions for connecting PowerPivot to the OData service in FICORA s domain name system. The screenshots are from PowerPivot for Excel 2013 but the features are essentially the same in Excel 2010. Go to the PowerPivot tab in the ribbon at the top of the Excel window and click Manage: The PowerPivot add-in creates a PowerPivot window in Excel where you can process data from OData sources, for instance. You can also copy such data to ordinary worksheets. Connect Excel to the domain name system by clicking Get External Data > From Data Service > From OData Data Feed at the top of the PowerPivot window. As the Table Import Wizard opens, type the address https://odata.domain.fi/opendomaindata/opendomaindata.svc/ in the Data Feed Url field. If you Page 12(18)
wish, you can name the connection by typing a descriptive name, such as Domains, in the Friendly connection name field. Click Advanced to enter the required user IDs. Click Next. In the following window, you can import data to Excel from the OData interface. Check the data elements boxes you want to import. In version 2 of the fi-domain name OData interface, the only data element available is Domains. Continue and wait. There are more than 328,000 domain name records which means that the import may take a while. After the import is completed, you have all the data of the OData interface in the domain name service. Page 13(18)
4.4 Using the data Once you have imported the data, you can browse it in the PowerPivot window. If you save the Excel file, the data is also saved. You can also update the data by clicking the Refresh button at the top of the PowerPivot window. The PowerPivot window is designed for browsing, filtering and summarising data. If you want to process the data using formulas, the easiest way is to copy and paste it to Excel. The PowerPivot window is workbook-specific and you can always open it by clicking the Manage button on the PowerPivot tab of the Excel ribbon. To filter the data, you can use the usual column filters in Excel which are available in the PowerPivot window: To see the sorting and filtering features, click the down arrow at the right of the column heading. In the example in the screenshot, the column to be filtered is the AssociateType and the criteria are domain names registered by the State. You can export data to normal Excel view using either the copy and paste technique or the PivotTable tool in the PowerPivot window. The benefit of PivotTable is that it remains linked to the original data. As you edit the PowerPivot view, the data copied into ordinary Excel worksheet is also updated. Page 14(18)
In the following screenshot, PivotTable has been selected on the PowerPivot tab. After that, a PivotTable Fields list appears on the right side of the page. In this example, the columns Name and Holder have been checked by dragging Holder to the Rows area and Name to the Values area (see bottom right-hand corner of the screenshot). Finally, the list has been sorted in reverse order using the Count of Name field. In this case, the result is a list of parties that have registered the most domain names in Finland. Page 15(18)
You can also create charts with the same tool. Creating charts follows the same logic. Drag the columns to areas that suit your purposes (Rows, Values, etc.) and create charts that break down the set of domain names as you wish. First, go to PivotTable > PivotChart. Page 16(18)
In the following screenshot, the AssociationType field has been selected as the Axis and the Name field (domain name) as the Value. The result is a column chart representing the distribution of domain names by registering parties. However, it would be more convenient to view the same data as a pie chart. Right-click the chart, select Change Chart Type and select the pie chart. You can edit the charts by removing or modifying automatic texts ('Total', 'Count of Name'). The charts are mainly normal Excel charts and you can usually edit them like any other Excel charts. However, charts created by PowerPivot tools remain linked to the original source. If you update the domain data to match FICORA s service by clicking Refresh, all your charts will also be updated. Read more about PowerPivot: https://support.office.com/en-us/article/getstarted-with-power-pivot-in-microsoft-excelfdfcf944-7876-424a-8437-1a6c1043a80b Page 17(18)
5 OData for application developers The chapters above describe the structure of the OData service and its use in desktop applications. However, you can also use OData in your own applications. There is a comprehensive library support for the protocol. For more technical information about OData, visit www.odata.org. Development tools for different environments are available at http://www.odata.org/libraries. Client libraries which may be connected to this OData service, for example, are available for Microsoft development platforms and also for all key programming environments (Android, ios, Java, PHP, Ruby etc.). As the OData interface is based on Atom or JSON, you do not necessarily need a library to use the data source. An XML/JSON converter and a web interface capable of HTTP Basic authentication are sufficient. Please note that the JSON format may only be enabled by HTTP Content negotiation feature (http header Accept: text/json ). FICORA does not recommend building applications which reveal the username and password for the connection (e.g. by providing OData endpoint contact details to a browser making JSON requests to the interface). It is better to collect the necessary information on your own server and then provide them further. For practical instructions on using OData interfaces in different languages, see Microsoft's developer documentation: http://msdn.microsoft.com/en-us/data/ee720180.aspx Page 18(18)