EUROPA Search project

Transcription

1 EUROPA Search project - Functional Specifications - Version /07/ of 19

2 Table of contents 1. Purpose and scope Background Purpose EUROPA Global Search overview User Interface (UI) Web application Search link Restricted search Backend search engine Autonomy/IDOL Overview Autonomy/IDOL as used by the EUROPA global search Front end application detail General overview Common EUROPA features The search bar Free text search box Basic text search options Autocorrection (Spellcheck feature) Access button to display the more options Reset button The More options bar Date filtering Language filtering File formats Terms appearing 15 2 of 19

3 3.5 The tab interface The results view Results pane Sort pane Filter pane Pagination Other features of the application Language processing Caching Configuration RSS feed of 19

4 1. PURPOSE AND SCOPE 1.1 Background The EUROPA Global Search is the replacement search capability for the existing search facility based on the Verity engine. Autonomy s IDOL (Intelligent Data Operating Layer) engine was chosen to replace the existing verity search. The new search engine is to simplify the advanced search whilst allowing searches to be tailored to the needs of individual webmasters. 1.2 Purpose The purpose of this document is to detail the functional specification of the new EUROPA Global Search. It covers such features as: The web interface Advanced search options Results overview Results filtering Search customization Language processing These are dealt with from an end user perspective with a view to outlining how the front end is experienced by such an individual. Technical information on how the various aspects of the search function and how they can be configured can be found in the Technical Specifications Document (TSD) which covers this in more detail. 4 of 19

5 2. EUROPA GLOBAL SEARCH OVERVIEW 2.1 User Interface (UI) The Europa Global Search application exists in 4 different guises: 1) A central web based application (using both EC and EUROPA web templates) 2) As a link from existing EUROPA site searches which currently link to the existing verity search. 3) As a restricted search interface where webmasters can create customized versions of the search application based on their web sites look and feel Web application The central web application will be reached through the standard search links of the existing EUROPA environment ( ) and use the Inter Institutional template. The application will also be accessible under the EC search ( using the relevant EC banner. Figure 1 - EC template 5 of 19

6 Figure 2 - EUROPA template Search link Many sites under the EUROPA / EC domains already contain search boxes which link to the existing verity search engine. In this case the links will be intercepted and directed to the new search engine thanks to an internal Compatibility module developed for such scope. Webmasters are not required to change their code to link to the new search. The information they normally send to Verity is intercepted and mapped to the new IDOL based search. The results are then displayed by the new application as opposed to the existing verity application. Figure 3 - Existing result display 6 of 19

7 Figure 4 - New result display 7 of 19

8 2.1.3 Restricted search The new search has also been designed to allow webmasters to create a restricted search based on their own customised index. This index is essentially a subset of the main Europa index where the content is of particular interest to the users of a particular sub site or DG such as Agriculture. For example, the webmaster of the DG TRADE website may wish to offer users the opportunity to search also DG TRADE related content. In this case the user is presented with search DG TRADE specific search results using DG TRADE specific templates. Users will have always the possibility to search the same terms under EUROPA thanks to the second tab (EUROPA) available under each restricted search. Figure 5 - DG TRADE restricted search 8 of 19

9 2.2 Backend search engine The backend search engine is beyond the scope of this document but for completeness of the application overview it is described here at a very high level, Autonomy/IDOL Overview The back end search engine of the Europa Global Search application is based upon the Autonomy IDOL7 architecture. The architecture provides a means to: 1) Locate and pull down content from various sources (web, database, filesystem etc.). 2) Process the content to standardize it and create new meaningful data based on the content (titles, dates etc). 3) Index the content into a form whereby it can be retrieved quickly (based on application specific configuration) from the front end application. 4) Query data Figure 6 - Indexing data into IDOL 9 of 19

10 2.2.2 Autonomy/IDOL as used by the EUROPA global search Locating content All content in the EUROPA index is retrieved from the web (or other data repository) using Autonomy s specifically developed connectors. The connectors are configured to retrieve data based on sources identified by DG COMM. Processing Data Prior to indexing the retrieved content, the content is cleaned and standardised. This process creates the appropriate meta data (language, dates etc) required to all the front end search options to find the correct documents. Indexing data The indexes are populated through the use of the HTTP Connector that is configured to index pre-defined URLs. The Connector collects the information and transforms it into a format that is compatible with the indexer. The indexer then populates the defined IDOL databases to be used by the search. The initial index is performed across all data indexing every page defined, this is performed periodically. Incremental indexing, whereby only undated content is indexed, ensures that this process remains efficient whilst the indexed content remains up to date. Details on how the indexing is configured can be found in the TSD. Query data The front end UI described previously is responsible for building the appropriate queries which are sent to IDOL for interpretation. The general process involves translating the values entered by the user in a query string built using the Autonomy query language. The query string is sent to an IDOL module where it is directed to a single instance of the IDOL server (multiple instances exist for failover reasons). The query is interpreted by the content engines and the results are returned back through the Autonomy distributer module where they are routed back to the UI and displayed to the user. 10 of 19

11 3. FRONT END APPLICATION DETAIL 3.1 General overview The web interface is designed around the guidelines provided in the IPG and has been structured in the following manner: The standard EUROPA header (Or the standard EC Header) The search bar The More options bar (advanced search drop down) The tab interface The results view Filter options Pagination The standard EUROPA footer Figure 7 - Search options (with advanced dropdown displayed) 11 of 19

12 3.2 Common EUROPA features The common EUROPA header and EUROPA footer are derived from the guidelines provided in the IPG. The only feature that directly concerns the search application is the language drop down menu on the top right of the header which controls the language the web interface will be displayed in. This language control is a standard feature of the EC/EUROPA templates and is integrated into the web interface as defined in the IPG. 3.3 The search bar The search bar is the section of the web interface first presented to the user. Figure 8 - The search bar It is made up of the 4 components 1) free text search box 2) basic text search options (drop down) 3) access button to display the more options bar 4) Reset button Free text search box The free text search box is a standard search box that users enter their search terms. Clicking the blue button marked with the magnifying glass logo executes the search. 12 of 19

13 3.3.2 Basic text search options Search options can be found in the drop down selection box that provides the following search options: Containing all the words: which returns all results which contain all the inputted words in any order. Containing the exact phrase: which returns all results which contain the words all together in the order they appear in the text box. Containing any of the words: which returns all results which contain any of the included words Autocorrection (Spellcheck feature) Search terms might be automatically corrected to provide a better match of what users are looking for. By clicking on the suggested terms a new query will be ran and other results will be shown Access button to display the more options Clicking the button results in its closure. caused additional more advanced search options to be dropped down. Clicking again Reset button The reset button their default values. restores all search values of the interface (including those selected through more options ) to 13 of 19

14 3.4 The More options bar The more options bar contains additional means to narrow the scope of results. Figure 9 - The search bar It is made up of the 4 filters 1) Date filtering 2) Language Filtering 3) File format filtering 4) Term appearing filters Date filtering The user can select a date range for which results must not be outside of Language filtering Results can be selected to appear in another language. The language option Other shows no- EU language documents File formats This provides the option of restricted the search results to a particular format (HTML, PDF, WORD, POWERPOINT and EXCEL.). 14 of 19

15 3.4.4 Terms appearing The user can select where within a document the search terms must exist. The options available are, o o o Anywhere in the page: this is the default search filter and will return results in which the search terms appear anywhere within the page. In the title of the page: this option will return only those results where the search term appears in the title of the page. In the url of the page: this option will return only those results where the search term appears only in the url of the page. 3.5 The tab interface Figure 10 - The Tab interface In restricted search mode the tab interface will display two tabs. Each tab contains the result sets of both the restricted search and following this the results set based on the full Europa index. Users select the desired result set by clicking the appropriate tab. In other modes only the Europa tab is displayed. 15 of 19

16 3.6 The results view The results view is divided into three main areas: 1. The results pane to the right 2. The sort pane to the left (top) 3. The filter pane to the left (bottom) 4. Pagination (top and bottom) Figure 11 - The results view Results are generated after the search is executed based upon the terms and filters provided. The search results view follows a specific format, although there are slight variances depending upon the search engine called. The main features of this format include: Results display: provides a summary of the number of results and search terms and filters applied. 16 of 19

17 Sort by: allows the user to select whether they wish to view the results by search engine determined relevance or by the most recently added page. Search results: this is the list of results ordered as defined by the sort by drop down selection box Results pane The main results (10 per page) are displayed in this pain. Each result consists of the following components, Document type icon (HTML, PDF etc.) Document title Summary description A generated summary description Document URL An external link to the document. The last update date of the document Sort pane The sort pane provides two means to sort results. Date Relevance ( based on the relevance value generated by IDOL for each result) Search results: this is the list of results ordered as defined by the sort by drop down selection box Filter pane Filters displayed may have sub levels. Clicking on a filter will display the sublevels available. Clicking on a sublevel will display only the results for that filter. The pagination will adjust accordingly. The total results for each filter takes into account results which exist at that filter level in addition to the total results of all sub levels. 17 of 19

18 3.6.4 Pagination Figure 12 - The pagination bar The pagination bar is in three parts, Previous This is not clickable where the current page displayed is page 1. Page numbers A max of 10 Next This is greyed if less than 2 pages of results are available or the last page has been reached. The user can select any of the above by clicking on them. Where a page number is greater than 10, selecting page 10 will allow further pages to be displayed. The selecting page is highlighted through bolding of the page number. 18 of 19

19 4. OTHER FEATURES OF THE APPLICATION 4.1 Language processing The search is assumed to be in the language of the interface unless otherwise specified through the appropriate language filter. 4.2 Caching For performance reasons the application stores the results of user queries for a period of minutes. During this time the same query is not sent again to the IDOL engines in the backend. When the caching period has timed out, queries are sent again to the IDOL engines. 4.3 Configuration The application is designed so that many of the values of the application can be altered through configuration files. The associated TSD (technical specification) contains more information on this. 4.4 RSS feed Clicking the icon let users to set up a feed of frequently changing content on a specific search query. 19 of 19