Intelligence Community and Department of Defense Content Discovery & Retrieval Integrated Project Team (CDR IPT)

Similar documents
Intelligence Community and Department of Defense Content Discovery & Retrieval Integrated Project Team (CDR IPT)

Intelligence Community and Department of Defense Content Discovery & Retrieval Integrated Project Team

Federated Search Developer Guide

Network Working Group Internet-Draft January 25, 2006 Expires: July 29, Feed Rank draft-snell-atompub-feed-index-05.txt. Status of this Memo

Network Working Group Internet-Draft August 2005 Expires: February 2, Atom Link No Follow draft-snell-atompub-feed-nofollow-03.

Earth Data discovery OpenSearch Best Practices

Network Working Group Internet-Draft August 2005 Expires: February 2, Atom Link No Follow draft-snell-atompub-feed-nofollow-00.

CEOS OpenSearch. Best Practice Document. Version 1.2. CEOS Document [CEOS-OPENSEARCH-BP-V1.2]

Network Working Group Internet-Draft October 27, 2007 Intended status: Experimental Expires: April 29, 2008

Linking and Finding Earth Observation (EO) Data on the Web

Search Web Services - searchretrieve Operation: Binding for OpenSearch Version 1.0

Implementing the Army Net Centric Data Strategy in a Service Oriented Environment

Open Geospatial Consortium Inc.

CWIC Data Partner s Guide (OpenSearch) Approval Date:

Glossary of Exchange Network Related Groups

OpenSearch GeoSpatial-Time and! Earth Observation Extensions!

Automatic Test Markup Language <ATML/> Sept 28, 2004

RealMe. SAML v2.0 Messaging Introduction. Richard Bergquist Datacom Systems (Wellington) Ltd. Date: 15 November 2012

Naming & Design Requirements (NDR)

IVI. Interchangeable Virtual Instruments. IVI-3.10: Measurement and Stimulus Subsystems (IVI-MSS) Specification. Page 1

Digital Imaging and Communications in Medicine (DICOM) Supplement 194: RESTful Services for Non-Patient Instances

The TAXII XML Message Binding Specification

Web Services Description Language (WSDL) Version 1.2

Testbed-12 CITE User Guide - Profiles

Web Services Description Language (WSDL) Version 1.2

Request for Comments: 3989 Category: Informational T. Taylor Nortel February Middlebox Communications (MIDCOM) Protocol Semantics

The European Commission s science and knowledge service. Joint Research Centre

ONVIF OSD Client Test Specification

Open Archives Initiative Object Reuse & Exchange. Resource Map Discovery

ONVIF Real Time Streaming using Media2 Device Test Specification

Open Command and Control (OpenC2) Language Specification. Version 0.0.2

Recommended Practice for Software Requirements Specifications (IEEE)

Abstract Code-Signing Profile of the OASIS Digital Signature Services

DICOM Correction Proposal

WHY WE NEED AN XML STANDARD FOR REPRESENTING BUSINESS RULES. Introduction. Production rules. Christian de Sainte Marie ILOG

Search Web Services - searchretrieve Operation: Abstract Protocol Definition Version 1.0

Name type specification definitions part 1 basic name

Network Working Group. November 1999

Draft Recommendation X.sdnsec-3 Security guideline of Service Function Chain based on software defined network

GEOFidelis SDSFIE Implementation Roles and Responsibilities Guide

CDM Implementation Requirements

Conformance Requirements Guideline Version 0.1

MarkLogic Server. REST Application Developer s Guide. MarkLogic 9 May, Copyright 2017 MarkLogic Corporation. All rights reserved.

IEEE LANGUAGE REFERENCE MANUAL Std P1076a /D3

Web Services Resource Metadata 1.0 (WS-ResourceMetadataDescriptor)

Search Operation. Using the Search Operation CHAPTER

XML ELECTRONIC SIGNATURES

BEAAquaLogic. Service Bus. Interoperability With EJB Transport

ONVIF Real Time Streaming using Media2 Device Test Specification

White paper Viz One APIs. Overview of our API Offering

Multi-Vendor Key Management with KMIP

Java EE 7: Back-end Server Application Development 4-2

Open ebook File Format 1.0. DRAFT VERSION 001 November 5, 1999

National Identity Exchange Federation. Web Services System- to- System Profile. Version 1.1

SWIM Standards Evolution Workshop

ETSI GS MEC 016 V1.1.1 ( )

Oracle Fusion Middleware

Extension of INSPIRE Download Services TG for Observation Data

Deployment Profile Template Version 1.0 for WS-Reliability 1.1

Enabler Release Definition for Application Layer Security Common Functions

GeoDCAT-AP Representing geographic metadata by using the "DCAT application profile for data portals in Europe"

Using Common Media Manifest with Interoperable Media Format (IMF)

Multicast Audio. .: :. Step by Step Guide. 20 April, Copyright 2016 Prolancer Pty Ltd, Sydney, Australia.

Internet Engineering Task Force (IETF) Juniper Networks K. Watsen Watsen Networks R. Wilton Cisco Systems March 2019

Internet Engineering Task Force (IETF) Request for Comments: ISSN: November 2013

Content Management for the Defense Intelligence Enterprise

INTERNATIONAL STANDARD

This Specification is provided for future development work within onem2m only. The Partners accept no liability for any use of this Specification.

edugain Policy Framework SAML Profile

Supplement 207 Conformity Assessment

Schema Document Has Different Target Namespace From The One Specified In Instance Document

RESTful API Design APIs your consumers will love

XML Information Set. Working Draft of May 17, 1999

NextData System of Systems Infrastructure (ND-SoS-Ina)

ONVIF Advanced Security Client Test Specification

ONVIF Event Handling Test Specification

Apache Wink User Guide

D. Crocker, Ed. Updates: RFC4871 June 10, 2009 (if approved) Intended status: Standards Track Expires: December 12, 2009

Open Geospatial Consortium Inc.

TAXII 2.0 Specification Pre Draft

ONVIF Analytics Engine Device Test Specification

Dictionary Driven Exchange Content Assembly Blueprints

NDEV Mobile HTTP Services for NDEV Mobile Clients

Test Assertions Part 1 - Test Assertions Model Version 1.0

The Modeling and Simulation Catalog for Discovery, Knowledge, and Reuse

Suggestions for writing Abstract Test Suites (ATS) for INSPIRE conformance testing for Metadata and Network Services

Proposed Visual Document Signatures Profile of OASIS DSS

Puma REST samples. Table of Contents

ONVIF Real Time Streaming using Media2 Device Test Specification

Search Operation. Using the Search Operation CHAPTER

WMS (Web Map Service) Reader

INFORMATION ASSURANCE DIRECTORATE

Search Operation. Using the Search Operation CHAPTER

STAR Naming and Design Rules. Version 1.0

ISO/IEC INTERNATIONAL STANDARD. Information technology Metadata registries (MDR) Part 3: Registry metamodel and basic attributes

Test Assertions for the SCA Web Service Binding Version 1.1 Specification

Category: Standards Track December 2006

NISO Next Generation Discovery

BEAAquaLogic. Service Bus. JPD Transport User Guide

ISO/IEC INTERNATIONAL STANDARD

Transcription:

Intelligence Community and Department of Defense Content Discovery & Retrieval Integrated Project Team (CDR IPT) IC/DoD REST Interface Encoding Specification for CDR Search, v1.1 12 May 2011

REVISION/HISTORY Doc Revision Revised By Revision Date Revisions 0.1 29 January 2010 Initial draft for subgroup review. 0.2 Dave Lemen, Dan Wigglesworth 1 February 2010 Second draft, limiting specification to formatagnostic OpenSearch implementation. 0.3 Dave Lemen 16 February 2010 0.4 Dave Lemen 23 February 2010 0.5 Dave Lemen 9 March 2010 0.5 Mary Farley 14 April 2010 Technical Edits ii

TABLE OF CONTENTS 1 Introduction 5 1.1 Service Overview 5 1.2 Relationship to Other CDR Architecture Elements.. 5 1.3 Notational Convention. 6 1.4 Conformance. 7 1.5 Namespaces 7 1.6 License. 7 2 Search Service Behavior. 8 2.1 Main Flow.. 8 2.2 Query. 9 2.3 Paging.. 9 2.4 Result Presentation.. 11 2.5 Relevance of Search Results.. 11 3 Search Service Interface.. 13 3.1 Input 13 3.1.1 Relation to Inputs Defined in the Specification Framework 13 3.1.2 OpenSearch Paging Parameters.. 14 3.1.3 OpenSearch Query Type Extensions 14 3.2 Behavior 15 3.3 Output 15 3.3.1 Relation to Outputs Defined in the Specification Framework. 15 3.3.2 Including the Search Request in the Response 16 3.3.3 Including Metadata in the Results.. 16 3.3.4 OpenSearch Paging Elements.. 18 3.4 Fault Conditions 20 4 Search Service Implementation.. 20 4.1 Policy. 20 4.2 Query Types 20 4.3 Result Types 21 4.4 Sorting of Search Results. 21 4.5 Security Considerations 21 5 Reference Documents 21 iii

LIST OF FIGURES Figure 1 - CDR Architecture Model.. 6 LIST OF TABLES Table 1 Referenced XML Namespaces 7 Table 2 Framework Input Variable Disposition 13 Table 3 Framework Output Variable Disposition. 16 Table 4 OpenSearch Output Extensions 18 Table 5 Fault Conditions and HTTP Responses 20 iv

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 1 Introduction 1.1 Service Overview The Search Component, as defined by the Intelligence Community/Department of Defense (IC/DoD) Content Discovery and Retrieval (CDR) Specification Framework [CDR-SF], serves as the primary content discovery mechanism to expose content collections for discovery and accessibility. This component provides a common interface and behavioral model for IC and DoD content collections, enabling content consumers to discover relevant content resources from disparate collections across the IC/DoD enterprise. This specification defines requirements and provides guidelines for the realization of the CDR Search Component as a RESTful, OpenSearch [OS] web service, hereafter termed a Search service in this document. It describes a Search service s behavior, interface and other aspects in detail, providing enough information for Search service providers and implementers to create CDR-compliant Search services. The Search service exposes a single Search operation that is responsible for three activities that underpin Content Discovery capabilities: search, result presentation, and results paging. As discussed in the CDR Specification Framework, a Search service s results are resource metadata rather than actual content resources. In the context of Search, resource metadata generally refers to a subset of a resource s available metadata, not the entire underlying record 1. Some of the information contained within each Search result may provide the information necessary for a consumer to retrieve or otherwise use a resource. 1.2 Relationship to Other CDR Architecture Elements The CDR Architecture prescribes an abstract-to-concrete model for the development of architecture elements and guidance for content discovery and retrieval. Each layer or tier of the model is intended to provide key aspects of the overall guidance to achieve the goals and objectives for joint DoD/IC content discovery and retrieval. The following graphic, discussed in detail within the CDR Reference Architecture [CDR-RA], illustrates this model. 1 The Search Component returns metadata about a resource, which may sometimes describe the underlying resource (e.g., an image), while at other times representing a sub-set of the data the makes up a resource (e.g., a collection of attributes). In some cases, the metadata returned from an instantiation of the Search function and the Retrieve function, which returns a resource itself, may happen to be the same, though this considered an edge condition. 5

33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 Figure 1 - CDR Architecture Model As illustrated in Figure 1, the Specification Framework derives from the Reference Architecture (RA) and can describe behavior in terms of the capabilities, components, and usage patterns defined in the RA. The Specification Framework allows multiple Service Specifications to provide consistent interfaces, both in terms of the structure and semantics of the exchanged information. This specification provides guidance for implementing the CDR Search Component using the RESTful OpenSearch [OS] standard. It is intended to provide minimal requirements for implementing OpenSearch. Additional sub-specifications will provide further guidance for implementation profiles that include specific query types and result formats. 1.3 Notational Convention The key words "MUST," "MUST NOT," "REQUIRED," "SHALL," "SHALL NOT," "SHOULD," "SHOULD NOT," "RECOMMENDED," "MAY," and "OPTIONAL" in this specification are to be interpreted as described in the IETF RFC 2119. When these words are not capitalized, they are meant in their natural-language sense. When describing concrete XML schemas and example XML documents, this specification uses XPath as the notational convention. Each member of an XML schema is described using an XPath notation (e.g., /x:rootelement/x:childelement/@attribute). 6

56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 To make the text easier to read and understand, some examples include data types that are derived in auxiliary specifications from abstract types defined in this specification. To distinguish these derived types from those defined as part of this Search Specification, they are presented in green. Examples in this text are distinguished by a black border. These are meant to be illustrative and only one way that the described syntax can be used. <atom:entry> <atom:title>this is an example.</atom:title> </atom:entry> 1.4 Conformance Search services must support OpenSearch 1.1 Draft 4 [OS]. This specification defines an interface to a Search service to which an implementation and a subsequent deployment MUST conform. A deployment is an instance of an implementation. For an implementation to conform to this Search specification, it MUST adhere to all mandatory aspects of the specification. 1.5 Namespaces Namespaces referenced in this document and the prefixes used to represent them are listed in the following table. Table 1 Referenced XML Namespaces Prefix URI Description opensearch http://a9.com/-/spec/opensearch/1.1/ OpenSearch 1.1 (Draft 4) 2 atom http://www.w3.org/2005/atom Atom 1.0 81 82 83 84 1.6 License This specification is licensed under the Creative Commons Attribution-ShareAlike 2.5 Generic License (http://creativecommons.org/licenses/by-sa/2.5/), because it builds on the OpenSearch [OS] standard, which is licensed with the share-alike clause. 2 The OpenSearch specification can be found at http://www.opensearch.org/specifications/opensearch/1.1/draft_4. 7

85 86 87 2 Search Service Behavior 2.1 Main Flow 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 1. The Consumer invokes the Search function on the Search Component with the parameters for a particular type of Search Request specified. The Search Request contains the query, paging information and any other information required by the particular query type being sent. 2. The Search Component leverages a set of security components to verify that the Consumer is authenticated and authorized to perform the search query. The DoD/IC Service-Oriented Architecture Security Reference Architecture [SEC- RA] defines the specific security components and interactions needed to perform this verification. See Section 4.5 of this document. 3. The Search Component checks the type of the incoming Search Request (e.g., keyword, temporal etc.) to verify that it supports that particular type and that its syntax is valid. 4. The Search Component checks the validity of the provided paging information and utilizes default values for any that are missing. 5. Because Search service result presentation is completely determined via the OpenSearch binding, this step of the main flow is not applicable to OpenSearchbased Search services. 8

108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 6. The Search Component executes the query against its Content Collection. 7. The Search Component receives the results for the query that was executed against its Content Collection. 8. The Search Component performs any necessary formatting of the results before returning a response to the Consumer. 9. The Search Component returns a formatted response back to the Consumer containing: A result set containing the resource metadata for each search result, presented in the format associated with the service s interface. A Timestamp containing the time that the search was executed. The Start Index value that was used. The Results Per Page value that was used. The Input Search Request that was executed. The Response Result Count containing the number of results within the current page. The Total Result Count containing the number of total results matching the Input Search Request that was executed, if available. 2.2 Query The OpenSearch specification does not define a syntax for its primary query parameter, searchterms, but it is generally used to support simple keyword queries. There are a number of OpenSearch extensions that support query types beyond simple keyword queries as well. These include the OpenSearch Geo Extension [OS-GEO] for geospatial query terms, and the OpenSearch Time Extension [OS-TIME] for temporal query terms. Use of these extensions is discussed in more detail in section 3.1.3. Additional query types may be developed by the CDR in the future. 2.3 Paging Paginated search results can be useful when the number of results is very large, infinite, or indeterminate. Service consumers can page through the result sets, accessing a subset of the overall result set as necessary. This capability will prevent search requests with very large result sets from overloading the server, network, or client. Search result pages may be traversed using the information from the original Search service request combined with the endpoint information provided by the OpenSearch Description (OSD) document describing the Search service from which the current result set was generated. The Search service OSD allows a service consumer to issue a search request for the next "page" of data. CDR implementations SHOULD support paging. Description of significant parameters which can be included in the URL template within an OSD document: /opensearch:count 9

149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 The desired number of results to return per page /opensearch:startindex* The starting index number of the returned results /opensearch:startpage* The starting page number of the returned results * A Search service SHOULD NOT use both startindex and startpage, since their functions overlap. Example: http://example.com/test?q={searchterms} &startindex={startindex?}&format=atom Description of significant elements that a Search service SHOULD return in the response: /opensearch:totalresults The total number of results available for the query /opensearch:startindex The start index of this result set /opensearch:itemsperpage The maximum number of items to be returned in a result response <atom:feed> <opensearch:totalresults>492420</opensearch:totalresults> <opensearch:startindex>1</opensearch:startindex> <opensearch:itemsperpage>10</opensearch:itemsperpage> </atom:feed> The response MAY include paging links to the previous and next results. In addition, the service MAY include the first results page, current results page, and last results page. For instance, if the response format is Atom, the paging links can be included as follows: <atom:feed> <atom:link rel="previous" href="http://example.com/test?q=tanks&page=2&format=atom" type="application/atom+xml"/> <atom:link rel="next" href="http://example.com/test?q=tanks&page=4&format=atom" type="application/atom+xml"/> <atom:link rel="self" href="http://example.com/test?q=tanks&page=3&format=atom" type="application/atom+xml"/> <atom:link rel="first" href="http://example.com/test?q=tanks&page=1&format=atom" type="application/atom+xml"/> <atom:link rel="last" href="http://example.com/test?q=tanks&page=447&format=atom" type="application/atom+xml"/> </atom:feed> The paging mechanism supported by the Search service does not guarantee continuity of search results while switching pages.content resources may be added, updated, or 10

197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 removed in the period of time between which the different pages of the result set are accessed -- without the consumer being aware of these changes. Therefore, service consumers SHOULD NOT present paged result sets as coherent or complete or make assumptions to that effect. 2.4 Result Presentation The CDR Specification Framework identifies additional result presentation behaviors: Result Sorting Order Result Metadata Style Support for sorting functionality is OPTIONAL, however Search services SHOULD provide results sorted by relevance by default, if possible. The base search request does not include any parameters to request or control sorting (i.e., the Result Sorting Order variable described by the CDR Specification framework is ignored). Instead, individual Query Type specifications MAY add sorting parameters and behavior requirements. There will be a different endpoint for each supported response type (e.g., HTML, Atom feed). 2.5 Relevance of Search Results The OpenSearch Relevance Extension [OS-RELV] provides a mechanism for communicating the relative importance of each result. Result relevance is generally a measure of how well a specific result matched the original query. Providing a result relevance measure allows better matched results to be prioritized relative to other results. A Search service implementation MAY provide relevance scores for individual search results with respect to the particular search with which it is identified. The OpenSearch Relevance extension defines a single element: /relevance:score The range of values allowed is any decimal between 0 to 1, inclusive, with 1 being the most relevant and 0 the least. Usage: <relevance:score>0.97</ relevance:score> This scheme does not define the mechanism by which the relevance score is determined. In addition, comparing scores calculated under this scheme by different Search service instances may not provide a true comparison of relevancy. An example Atom feed that includes opensearch:relevance elements in its individual results follows: 11

238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 <atom:feed> <atom:entry> result metadata <relevance:score>0.97</relevance:score> </atom:entry> <atom:entry> result metadata <relevance:score >0.42</ relevance:score> </atom:entry> </atom:feed> 12

253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 3 Search Service Interface 3.1 Input The URL template within the OSD document defines the syntax for how to call the Search service. The URL element requires two attributes to be present: /@type Contains the MIME type of the query result format /@template Contains the query URL to be processed according to the syntax rules The template syntax rules define how values can be placed into the URL to create a valid query. For example: <Url type="application/atom+xml" template="http://example.com/?q={searchterms}&pw={startpage?}&format=atom"/> The type parameter determines the return format of the query. Guidance on return formats is discussed in Section 4.3. Both {searchterms} and {startpage?} are variables that are to be replaced by a user or application using the URL. Variables without a question mark are required to be replaced with an actual value. Variables with question marks are required to be replaced by an empty string or by an actual value. An empty string indicates that the parameter should not be used for the query. For further guidance regarding the URL element and the URL template syntax, refer to the OpenSearch specification. 3.1.1 Relation to Inputs Defined in the Specification Framework The IC/DoD CDR Specification Framework defines a number of required (R) and optional (O) inputs to the Search operation. The following table relates the disposition of each variable defined in the Framework in this specification: Table 2 Framework Input Variable Disposition Activity Framework Input Variable Search Specification Search Query (R) The query is an aggregate of the parameters in the request (R). Query type (O) Implicit query type is implicit in the Open Search parameter. 13

280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 Results paging Results presentation Query metadata (O) Timeout (O) Results per page (O) Start index (O) Query identifier (O) Result metadata format (O) Result sorting order (O) 14 Specialized query type definitions MUST specify all possible information (including metadata) that can be used as input for its type. In essence, query metadata is moved from the common search request and down into the specific query types, to be defined and used as necessary on a per-type basis. Not supported opensearch:resultsperpage (O) opensearch:startindex (O) Not supported Implicit each Search service binding MUST be associated with a specific result metadata format. Therefore this input variable is not needed. Default sorting by relevancy is RECOMMENDED. Individual query types MAY define input variables to control custom sorting; otherwise sorting order input is not supported and no variable is defined. The CDR Specification Framework allows for Query Metadata as an optional Search input. This Search Service Specification does not enable input of metadata independent of a particular type of query. Individual query types MAY define metadata inputs and other kinds of inputs, as desired, as part of their search syntax. In the RESTful implementation specified by this document, this information is implicit in the type of parameter(s) sent in the Search Service request, therefore a query type input is not necessary. 3.1.2 OpenSearch Paging Parameters The Search service specification is REQUIRED to function as described by the Specification Framework with any input, behavior, output, and fault condition extensions listed below. /startindex This OPTIONAL element indicates the offset of the first result in a result set. Its value, if provided, MUST be an integer. The base value SHOULD be 1. /resultsperpage This OPTIONAL element describes the desired number of search results per page. Its value, if provided, MUST be greater than or equal to 1. The default value is 10. Support for this property is REQUIRED. 3.1.3 OpenSearch Query Type Extensions The OpenSearch specification defines one parameter, searchterms, for the query. OpenSearch does not specify a syntax for the contents of the searchterms parameter. Extensions to OpenSearch have been developed to support query types that go beyond keyword-style queries. These include:

305 306 307 308 309 310 311 312 313 314 315 OpenSearch Geo Extension [OS-GEO] Defines parameters for supporting bounding box, point-radius, polygon, and other geospatial query types. OpenSearch Time Extension [OS-TIME] Defines parameters for specifying a time range. This specification does not describe how to interpret the temporal query terms. For example, it does not specify that the temporal terms apply to the date a resource was created or posted, or that it may apply to a historical event described by the resource. Guidance related to interpretation can be found in implementation guidance, such as the DoD Discovery Metadata Specification Implementation Guide [DDMS]. 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 3.2 Behavior An implementation of the Search service MUST follow the behavior defined in the CDR Specification Framework. Additional requirements on behavior MAY be defined by query type specifications. Search services MUST implement the behaviors required by the query types they support. 3.3 Output In addition to the requirements imposed by the CDR Specification Framework, the Search service Search function output is additionally constrained by the requirements specified in the OpenSearch Specification. The following example illustrates the high level components of a response message (containing a result set of unspecified type) from a Search service: <result-set> <result></result> <result></result> </result-set> 3.3.1 Relation to Outputs Defined in the Specification Framework The IC/DoD CDR Specification Framework [CDR-SF] defines a number of required (R) and optional (O) outputs from the Search operation. The following table relates the disposition of each variable defined in the Framework in this specification: 15

341 Table 3 Framework Output Variable Disposition Activity Framework Output Variable Search Specification Search and Results Paging Result set (R) Results metadata (R) Result set retrieval properties (O) Result relevancy value (O) Result retrieval properties (O) Timestamp (O) Query identifier (O) Response result count (O) Total result count (O) A Search service MUST return a formatted set of results. Search service implementations SHOULD support HTML and Atom response formats. CDR Result Type specifications MAY require certain types of data to be returned as part of the result set or individual result entries. Those specifications MAY also allow other types of metadata to be included and describe the mechanism for doing so. A Search service that supports a particular result type MUST follow the syntax and processing rules defined by that type. (R) Not Supported. relevance:score in each result An element describing the linkage to the Retrieve service MUST be included in the results. (O) CDR Result Type may require timestamp element (ie feed/updated). Not Supported. opensearch:itemsperpage SHOULD be included in the response. opensearch:totalresults SHOULD be included in the response. 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 3.3.2 Including the Search Request in the Response To facilitate paging capabilities and to provide service consumers the ability to re-execute their queries, the search request that produced the output MAY be included in the results response. The mechanism for doing this will depend on the response format being returned. In the following example, the Search Request is inserted directly under the root of a notional response: <atom:feed> <atom:link rel="self" href="http://example.com/test?q=tanks&page=1&format=atom" type="application/atom+xml"/> </atom:feed> 3.3.3 Including Metadata in the Results Depending on the underlying data resources and the type of search request being executed, Search services MAY return metadata about each resource beyond that required by the Result Type specification. That specification controls the mechanism and 16

361 362 363 364 syntax for including any additional metadata and whether or not such inclusion is permitted. Search services that support a particular Result Type in its response MUST follow the requirements in the associated Result Type specification. 17

365 366 367 368 369 370 371 372 3.3.4 OpenSearch Paging Elements Search service implementations SHOULD use the elements and parameters from the OpenSearch specification for paging purposes. The exact mechanism and syntax for including the OpenSearch extensions are defined by the specification for the Result Type being used. The requirements for the inclusion of OpenSearch extensions are listed in the table below. Table 4 OpenSearch Output Extensions Element Description The actual or estimated number of resources that match the current query. opensearch:totalresults In the absence of a totalresults element or a next link (//link[@rel= next ]), the search client should consider the current page to be the last page of search results. Restrictions: The value must be a non-negative integer. Default: The default value is equal to the offset index of the last search result on the current page. Requirements: The element may appear zero or one time. Recommendation: The Search Service implementation SHOULD include the opensearch:totalresults in the response. Example: <opensearch:totalresults> 492420 </opensearch:totalresults> The offset of the first search result in the current set of search results. opensearch:startindex The OpenSearch specification allows for any integer offset for the first search result (0, 1, -5, etc.) The index of the first search result MAY be indicated in the OSD document s Url element, using the indexoffset attribute. If the startindex element does not appear on the page then the search client should consider the current page to be the first page of search results. Restrictions: The value MUST be an integer. 18

Requirements: The element MAY appear zero or one time. Recommendation: The Search Service implementation SHOULD include the opensearch:startindex in the response. A Search implementation SHOULD use an indexoffset of 1. Example: <opensearch:startindex> 11 </opensearch:startindex> The number of search results returned per page. opensearch:itemsperpage If the itemsperpage element does not appear on the page then the search client SHOULD use the number of items of the current page as the default page size. Restrictions: The value must a non-negative integer. Default: The default value is equal to the number of search results on the current page. Requirements: The element may appear zero or one time. Recommendation: The Search Service implementation SHOULD include the opensearch:itemsperpage in the response. Example: <opensearch:itemsperpage> 10 </opensearch:itemsperpage> 373 19

374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 The following XML illustrates one possible representation of these properties in a notional response (the Result Type specification will define the exact representation): <atom:feed> <opensearch:totalresults>12</opensearch:totalresults> <opensearch:startindex>11</opensearch:startindex> <opensearch:itemsperpage>10</opensearch:itemsperpage> </atom:feed> 3.4 Fault Conditions An implementation of the Search service MUST allow for the Fault Conditions defined in the CDR Specification Framework. Query type specifications MAY create additional Fault Conditions, as necessary. Any new fault types SHOULD derive from existing fault types, if possible. Table 5 maps the CDR Specification Framework fault conditions to the HTTP status that SHOULD be returned for each. Table 5 Fault Conditions and HTTP Responses CDR Framework Fault Condition HTTP Status HTTP Description Unauthorized Access 403 Forbidden Unsupported Query Type 400 Bad Request Unsupported Search Request Syntax 400 Bad Request Unsupported Search Element 400 Bad Request Invalid Paging Value 400 Bad Request Paging Value Out of Range 404 Not Found Service Execution Error 500 Internal Service Error 4 Search Service Implementation This section provides additional implementation guidance beyond the behavior and interface guidance provided in the previous sections. 4.1 Policy This specification defines the technical requirements and guidelines for implementing a Search service. Policy for Search service implementations is described in auxiliary documents. See the Reference Documents section for a listing of relevant policy documents. Implementers MUST follow the guidance in those policy documents. 4.2 Query Types The CDR Specification set includes a number of Query Type definitions that IC/DoD organizations can leverage in their Search service implementations, depending on the applicable policies and implementation profiles. 20

408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 4.3 Result Types The CDR Specification set includes a single predefined Result Type definition that IC/DoD organizations can leverage in their Search service implementations, the IC/DoD Content Discovery and Retrieval Atom 1.0 Result Set Specification [CDR-ATOM]. Implementers SHOULD consult appropriate policy and implementation guidance to determine requirements or recommendations concerning the use of particular Result Types. 4.4 Sorting of Search Results Sorting is OPTIONAL for Search services. Search services that do implement sorting SHOULD return results sorted by relevance. 4.5 Security Considerations Any resource may have associated policies for use, especially as applies to authentication and authorization. These policies may be asserted by both the resource owner and those responsible for governance and management of the enterprise. The implementation of policies related to security considerations SHOULD leverage the specific security components and interactions defined by the Joint IC/DoD Security Reference Architecture (SRA), and MUST be in compliance with requirements and guidance for security outcomes as specified in the SRA and its associated specifications. 5 Reference Documents The documents in this section provide the foundation for the Search service. Each document is assigned a reference identifier, which is cited when the document is referenced within this Search Service Specification. Ref. Title Version Date CDR-SF IC/DoD Content Discovery and Retrieval Specification Framework DRAFT 0.6.1 25 Jan 2010 CDR-RA IC/DoD Content Discovery and Retrieval Reference Architecture DRAFT 0.4 16 Dec 2009 ATOM CDR-ATOM OS OS-RELV The Atom Syndication Format http://www.ietf.org/rfc/rfc4287 IC/DoD Content Discovery and Retrieval Atom 1.0 Result Set Specification OpenSearch http://www.opensearch.org/specifications/opensearch/ 1.1/Draft_4 OpenSearch Relevance Extension http://www.opensearch.org/specifications/opensearch/ Extensions/Relevance/1.0 1.0 Dec 2005 1.0 March 2010 1.1, Draft 4 2009 1.0, Draft 1 2007 21

OS-GEO OS-TIME OpenSearch Geo Extension http://www.opensearch.org/specifications/opensearch/ Extensions/Geo/1.0/Draft_1 OpenSearch Time Extension http://www.opensearch.org/specifications/opensearch/ Extensions/Time/1.0/Draft_1 1.0, Draft 1 2009 1.0, Draft 1 2010 SDA Joint IC/DoD Service Discovery Architecture - 2010 SEC-RA DDMS DoD/IC Service-Oriented Architecture Security Reference Architecture DoD Discovery Metadata Specification Implementation Guide - 2010-2010 431 22