User Guidance Note GE-04

Transcription

1 Accessing SASPAC s Web Services Wizard This note describes how to utilise the proof of concept Web Services Wizard (WSW) Introduction: tool that queries data from the ONS NeSS Data Exchange and released with v9.50. Introduction: SASPAC v9.50 is the first full release of the software to query and interrogate data from a live online source. In this case the online source is the Neighbourhood Statistics (NeSS) Data Exchange (NDE) service. The NDE is a web service with an Application Programming Interface (API) that allows software developers to build applications that directly query the NeSS datastore. In SASPAC s case the Web Services Wizard allows users to search, browse, select and download any of the 300+ datasets available from the NeSS website. The advantage of this technical approach is that SASPAC will always query the NeSS central database, meaning that the data is automatically kept up to date and the user accesses the latest available. The WSW also builds additional functionality to that available from the NeSS website, for example, it is possible to retrieve multiple topics of data for multiple years in one single query something not possible from the website. This is a proof of concept development by the SASPAC team and is the first step towards a service that will ultimately deliver the 2011 Census datasets (and others) to users in the future. Therefore, the SASPAC helpdesk (info@saspac.org or ) would like to hear your feedback on the application and any comments you may have on its future development. The WSW application requires Java virtual machine (JVM) software to be present on the user s PC/network/thin-client. There are no licence restrictions or costs associated with this application and it is freely available from: Running the Web Services Wizard: During the application's development some test sites experienced security issues when attempting to run the WSW. Investigations indicated that these were caused by the organisation's firewall blocking the software's attempt to communicate with the NDE website. The WSW client is a Java program deployed as *.JAR (SasWebServices.jar) file and installed in the route of the SASPAC directory. In some instances an organisation's firewall/proxy server may restrict the TCP request from getting through to the ONS NDE endpoint (IP address: ). If this occurs it is likely that the software will hang when attempting to initially 'update cache' or when the 'check NDE Service status' button is used - the latter will return a message stating 'It was not possible to determine the status of the NDE v2 Service'. To resolve this issue the following two solutions are recommended: Information for system administrators. Page 1 of 11 January 2012

2 1) If a proxy server is in operation add the host and port settings to the 'Tools Configuration Webservices' menu (below), save changes and then attempt to run the WSW again. Web Services Wizard 2) Alternatively, grant permission for the firewall to allow the user's PC or the SASPAC clients (WSAS32.EXE/SasWebServices.jar) to access the IP address: directly (e.g. adding this site to a white list of accessible sites or similar). Both of these solutions will require support from your organisation's system administrator. If these do not resolve the problem, please contact the SASPAC helpdesk on or info@saspac.org. See UGN INI-01 for more information on how to install v9.50 Launching SASPAC s Web Services Wizard: Having completed the v9.50 installation process the software may be launched through the Start All Programs menu and selecting SASPAC for Windows. Once open, the text SASPAC for Windows v9.50 should appear in the software s title bar. The Web Services Wizard may now be launched using the File Run Web Services menu or CTRL-W keyboard shortcut. This opens the Web Services landing page below and displays a number of images - representing potential future data sources - but the user will notice that only one of these is currently available: the NeSS Data Exchange. January 2012 Page 2 of 11

3 Web Services Wizard To run the NDE wizard the NeSS Data Exchange button is simply selected followed by Explore. When the WSW is run for the first time - and on occasions depending on the cache refresh interval setting the following Information window will appear: User Guidance Note GE-04 SASPAC stores, or caches, some data locally to help improve the performance of the application. Selecting Yes (recommended and required when running the application for the first time) will create/update these local files and ensure SASPAC is accessing the latest NeSS datasets. The default update interval is set to weekly but can be changed to daily or monthly through SASPAC s Tools Configuration Web Services tab. In the first instance the cache update may take a few minutes to complete but following this, the Introduction tab of the wizard will be presented: See UGN SP-03 for more information on the configuration menu. This tab provides a brief overview of the service and includes links to more information from the ONS and SASPAC websites. As the Wizard is reliant on the NDE service being available to SASPAC, the Check NDE service status button allows the user to ascertain if their PC has access to the internet and that the NDE site is live and functioning correctly. If the NDE services are available then the following window will appear: If these services are not available it may be that the ONS NDE service is experiencing issues and if this persists please contact the SASPAC helpdesk. Please note that it is not necessary to run this check each time the Wizard is accessed but it may be useful if you are experiencing any problems with the application. Page 3 of 11 January 2012

4 Four further tabs entitled: Explore Datasets, Explore Geography, Get Data and View Data are now available. Both the Explore Datasets and Geography tabs allow the user to browse, search and select variables and geographical zones respectively - these tasks may be undertaken in any order. Once the data and geography tabs have been completed the Get Data tab must be selected and allows the user to specify the download format of the data request, whilst View Data enables these files to be viewed in applications such as Excel. Each tab is described in more detail below. Web Services Wizard Explore Datasets: The Explore Data tab provides the ability to browse and select single or multiple variables from NeSS. The first step in this window is to select a dataset from the Select a dataset drop-down list, this replicates the Topic list available on the current NeSS website. Having selected a dataset, e.g. Crime and Safety, the second drop-down list provides a list of all the topics available under this chosen dataset. After selecting the required topic, e.g. Notifiable Offences Recorded by the Police, the date ranges for this topic can be seen in the Available date(s) drop-down list. Dates are selected by picking the required entry and using the Select button, the date will now appear under the Selected date(s) window. Some or all dates may be selected or removed using the appropriate Select All and Remove buttons. Having made the date selections the user now selects the Query NeSS button, and returns a list of variables that meet the chosen criteria. [NB. If multiple ranges of dates are selected the software will make several enquires to check that all the required variables are available for each date selected.] Now the variables textual description and unit are provided and single or multiple items can be selected. After selecting the variables of interest, it is now possible to move to the Explore Geography screen using the button at the bottom of the page or selecting the next tab at the top of the window. If at any stage the user wishes to restart their data selections, the Reset Page button at the bottom of this window will clear all current selections within this tab and allow a new query to begin. It is also possible to return to this tab following the selection of geographies via the Explore Geography tab or button. January 2012 Page 4 of 11

5 Web Services Wizard User Guidance Note GE-04 Explore Geography: This window allows the user to search, browse and select geographical areas by hierarchy or unit. The Select hierarchy box allows the user to browse geographies by selecting a hierarchy from the drop-down list, once selected, the Browse button becomes available and presents the top level of this hierarchy in the Explore & Select Zone(s) window. To expand these geographies below this level left-click the zone it will be highlighted in blue - and select the Expand selection button: the next level in the hierarchy may now be selected/expanded. This process may be repeated to expand down the hierarchy until the lowest geographical level is displayed: when this is reached the following message will be shown: Having expanding the hierarchy to the required level, zones may now be selected. This is undertaken by left-clicking a zone a blue highlight will appear and the Select highlighted button is chosen. Once a zone(s) is selected it will appear in Page 5 of 11 January 2012

6 the Selected Zone(s) drop-down box below this window. If a user wishes to select multiple zones, e.g. all the Parliamentary Constituencies in London, they can highlight the zone immediately above the level required in the hierarchy (e.g. London in the 2010 Parliamentary Constituencies hierarchy (GOR)) and choose the Select highlighted button, this will then present the following multiple area selection window: Web Services Wizard Here the window prompts the user to choose between selecting just London or all the 74 Constituencies in London. Having selected the required tick boxes and Continue, the Wizard returns to the Explore Geography tab with these selections added to the Selected Zone(s) box. It is also possible to remove some or all of these zones using the buttons to the right of this box. Alternatively, within a geographical hierarchy the user may select the Advanced Selection button to carry out a bulk selection of zones. For example, with London selected from the Ness Geographical Hierarchy the Advanced Selection button presents a window to the user of all the hierarchies available below that level: To select all zones from a given hierarchy the appropriate radio button is chosen followed by Select. As this will potentially allow the user to select a large number of zones (e.g. all 24,000 Output Areas in London) in one operation, the application may take some time to return large zone selections. Once complete, the Advanced Area Selection window will close and the requested zones presented in the Selected Zone(s) drop-down box. January 2012 Page 6 of 11

7 Web Services Wizard An alternative method to browsing zones is carrying out a search of a specific geographical unit, using a postcode or place name. For example, selecting the Geographical Unit radio button then Local Authority from the drop-down list, allows the user to search for all the zones containing a specific text string. Simply choose the hierarchy or unit to search, input a postcode or place name and select the Query NeSS button. Again, the results of this search are presented in the Explore & Select Zone(s) box to the right of the window and these can be expanded these as necessary: User Guidance Note GE-04 In some cases, if the searched for text string is found to occur across a number of hierarchies, the results for all of these are presented. Zones may now be expanded and selected in the same method described above. Finally, for convenience, it is also possible to save and retrieve previous zone selections. Once zones have been identified and added to the Selected Zones drop-down list the Save Zones button can be used to save these as an external file (*.ZCSV format). This a native file format based on CSV but we do not recommend users edit this file as it contains details specific to the data query mechanism. The user may choose a suitable file name and location to save this file. Any previously saved files may be retrieved using the Import saved zones button. This is particularly useful for saving frequently requested or large numbers of zones. Once the geographical zones have been selected the next tab is progressed to by selecting the Get Data button or the tab at the top of the Wizard. Again, the Reset Page button is available to restart the geography selection at any time by clearing all current zones and allow a new query to be undertaken. Get Data: This tab allows the user to choose the format(s) they wish to download the data in. However, before it is possible to make these specifications, the Validate selections button must be selected. This allows SASPAC to check if the combination of variable and geographical selections are available from the NeSS database. If these are available the following window will appear: Page 7 of 11 January 2012

8 Web Services Wizard If no variables are available for the selected geography, or vica-versa, (this may occur when a dataset has not been published for a specific geography), the following window will appear: and the user must then return to the Explore Datasets or Explore Geography tabs and alter their variable or geographical selections. Having confirmed the data is available, the data format tick boxes are no longer greyed-out and the boxes may be ticked for the required formats below: When a certain file format is selected default file names and locations are automatically provided by the software. However, these can be easily altered through the use of the Browse button and a suitable file name and path selected. In the case of the System file metadata, the user may left-click and edit these fields directly. SASPAC will require these fields to be complete before a System file can be created. January 2012 Page 8 of 11

9 Web Services Wizard User Guidance Note GE-04 Data can be downloaded in one, or all, of the following formats: 1) Comma Seperated Values file (CSV) - suitable for viewing in spreadsheet packages like Excel or linking to a Geographical Information System 2) XML file (extensible Markup Language) - a bespoke NeSS XML format 1 3) SASPAC System file (.SYS) - native SASPAC format that can be used like any other System file within the software The user now selects the Download data from NeSS button to retrieve their data from the NeSS database. Once completed the following message will appear: The Continue button is now selected to continue to the final View Data tab. As with previous tabs, the Reset Page button is available, this will clear any selections in this tab and require the user to Validate selections to start again. It is also possible to alter variable or geography selections by returning to the previous tabs if necessary. View Data This is the final tab within the Wizard and provides the ability to view the files containing the data retrieved from NeSS. Data file formats will only appear here if they were choosen in the previous Get Data screen. To alter these formats users will need to return to the Get Data tab and use the Reset Page function. 1 Page 9 of 11 January 2012

10 Selecting the View in Excel button will open the data in Excel: Web Services Wizard Here the geographies, complete with zone identifiers ( zoneids ) and labels ( zlabel ), are presented as rows and the variables, along with metadata descriptions, as columns. Each variable is also provided with a unique identifier, e.g. NESS6175, which is also used by SASPAC to build the System file if/when requested. Choosing to Make System File will minimise the Wizard and present a completed command file task window similar to below: This has the effect of creating a new System file and saving this to the location previously specificed. Simply select the Close button to shut this window. The NeSS data in the System file can now be utilised like any other.sys in SASPAC, e.g. creating a thematic map using Mapshore or publishing data via an interactive map in InstantAtlas. Finally, choosing to View XML File will open the xml data file in the user s default viewer this may well be Internet Explorer, as shown below. Here the data is presented in a be-spoke format - defined by the NDE in collaboration with the Communites and Local Government (CLG) - called LGDX (Local Government Data exchange). January 2012 Page 10 of 11

11 Web Services Wizard User Guidance Note GE-04 On the right of this tab the View buttons provide links to a number of additional metadata files with further details of the query just undertaken. These are for information only and may be viewed in a text viewer such as Notepad. Finally, the Start a new NDE query button at the bottom of this tab allows the user to return to the start of the Wizard again and begin a new query. Alternatively, the Close button can be used to return to SASPAC. Page 11 of 11 January 2012