No Trade Secrets. Microsoft does not claim any trade secret rights in this documentation.

Transcription

1 [MS-FSFIXML]: Intellectual Property Rights Notice for Open Specifications Documentation Technical Documentation. Microsoft publishes Open Specifications documentation for protocols, file formats, languages, standards as well as overviews of the interaction among each of these technologies. Copyrights. This documentation is covered by Microsoft copyrights. Regardless of any other terms that are contained in the terms of use for the Microsoft website that hosts this documentation, you may make copies of it in order to develop implementations of the technologies described in the Open Specifications and may distribute portions of it in your implementations using these technologies or your documentation as necessary to properly document the implementation. You may also distribute in your implementation, with or without modification, any schema, IDL s, or code samples that are included in the documentation. This permission also applies to any documents that are referenced in the Open Specifications. No Trade Secrets. Microsoft does not claim any trade secret rights in this documentation. Patents. Microsoft has patents that may cover your implementations of the technologies described in the Open Specifications. Neither this notice nor Microsoft's delivery of the documentation grants any licenses under those or any other Microsoft patents. However, a given Open Specification may be covered by Microsoft Open Specification Promise or the Community Promise. If you would prefer a written license, or if the technologies described in the Open Specifications are not covered by the Open Specifications Promise or Community Promise, as applicable, patent licenses are available by contacting iplg@microsoft.com. Trademarks. The names of companies and products contained in this documentation may be covered by trademarks or similar intellectual property rights. This notice does not grant any licenses under those rights. Fictitious Names. The example companies, organizations, products, domain names, addresses, logos, people, places, and events depicted in this documentation are fictitious. No association with any real company, organization, product, domain name, address, logo, person, place, or event is intended or should be inferred. Reservation of Rights. All other rights are reserved, and this notice does not grant any rights other than specifically described above, whether by implication, estoppel, or otherwise. Tools. The Open Specifications do not require the use of Microsoft programming tools or programming environments in order for you to develop an implementation. If you have access to Microsoft programming tools and environments you are free to take advantage of them. Certain Open Specifications are intended for use in conjunction with publicly available standard specifications and network programming art, and assumes that the reader either is familiar with the aforementioned material or has immediate access to it. 1 / 31

2 Revision Summary Date Revision History Revision Class Comments 11/06/ Major Initial Availability 02/19/ Editorial Revised and edited the technical content 03/31/ Editorial Revised and edited the technical content 04/30/ Editorial Revised and edited the technical content 06/07/ Editorial Revised and edited the technical content 06/29/ Editorial Changed language and formatting in the technical content. 07/23/ No change No changes to the meaning, language, or formatting of the technical content. 09/27/ No change No changes to the meaning, language, or formatting of the technical content. 11/15/ No change No changes to the meaning, language, or formatting of the technical content. 12/17/ No change No changes to the meaning, language, or formatting of the technical content. 03/18/ No change No changes to the meaning, language, or formatting of the technical content. 06/10/ No change No changes to the meaning, language, or formatting of the technical content. 01/20/ Minor Clarified the meaning of the technical content. 04/11/ No change No changes to the meaning, language, or formatting of the technical content. 07/16/ No change No changes to the meaning, language, or formatting of the technical content. 2 / 31

3 Table of Contents 1 Introduction Glossary References Normative References Informative References Structure Overview (Synopsis) Relationship to Protocols and Other Structures Applicability Statement Versioning and Localization Vendor-Extensible Fields Structures Part Enumerations Extensions Global Elements document Global Attributes Complex Types CT_document Item Requirements on Managed Properties Document Vector Document-Level Security Properties Anchor Text Associated Queries Static Rank CT_catalog Context Catalog Types Synthetic Context Catalog (bt1) Numeric Context Catalog (bi1) Context Catalog for Item Metadata (meta) Context Catalog for Anchor Text Context Catalog for Associated Queries Context Catalogs for Full-Text Index Field CT_context Content Requirements for Property Context Rules for Item Processing FIXML Mapping Rules for Managed Properties Handling of Grammatical Variations CT_sep CT_rank CT_summary CT_sField Mapping Rules for Document Summary CT_attrVec Mapping Rules for Attribute Vector Simple Types Structure Examples FIXML Structure / 31

4 4 Security Considerations Appendix A: Full XML Schemas Appendix B: Product Behavior Change Tracking Index / 31

5 1 Introduction This document specifies the FAST Index Markup Language (FIXML) data object format. This format is used to transmit and store documents submitted for indexing. Sections 1.7 and 2 of this specification are normative and can contain the terms MAY, SHOULD, MUST, MUST NOT, and SHOULD NOT as defined in RFC All other sections and examples in this specification are informative. 1.1 Glossary The following terms are defined in [MS-GLOS]: UTF-8 The following terms are defined in [MS-OFCGLOS]: anchor text associated query attribute vector complete proximity boost content collection context catalog crawled property datetime deep refinement document identifier document summary document vector FAST Index Markup Language (FIXML) field importance level full-text index context full-text index field hit highlighted summary index schema input summary class internal property item item processing managed property occurrence boost phrase break property context refiner search clickthrough search index search service application static rank stemming by expansion stemming by reduction synthetic context catalog tokenization white space word stacking 5 / 31

6 The following terms are specific to this document: MAY, SHOULD, MUST, SHOULD NOT, MUST NOT: These terms (in all caps) are used as described in [RFC2119]. All statements of optional behavior use either MAY, SHOULD, or SHOULD NOT. 1.2 References References to Microsoft Open Specifications documentation do not include a publishing year because links are to the latest version of the technical documents, which are updated frequently. References to other documents include a publishing year when one is available Normative References We conduct frequent surveys of the normative references to assure their continued availability. If you have any issue with finding a normative reference, please contact dochelp@microsoft.com. We will assist you in finding the relevant information. Please check the archive site, as an additional source. [ISO-639] International Organization for Standardization, "Codes for the Representation of Names of Languages", ISO 639, [MS-FSCF] Microsoft Corporation, "Content Feeding Protocol Specification". [MS-FSDP] Microsoft Corporation, "Document Processing Protocol Specification". [MS-FSIN] Microsoft Corporation, "Input Normalization Data Structure". [MS-FSSCFG] Microsoft Corporation, "Search Configuration File Format Specification". [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997, Informative References [MS-FSCX] Microsoft Corporation, "Configuration (XML-RPC) Protocol Specification". [MS-FSIXDS] Microsoft Corporation, "Index Data Structures". [MS-FSO] Microsoft Corporation, "FAST Search System Overview". [MS-GLOS] Microsoft Corporation, "Windows Protocols Master Glossary". [MS-OFCGLOS] Microsoft Corporation, "Microsoft Office Master Glossary". 1.3 Structure Overview (Synopsis) This document specifies the format of the FIXML data structure. The data structure contains items prepared for indexing. One instance of the FIXML data structure represents the processed data of one item after item processing. The FIXML data structure represents the data payload of the protocol specified in [MS-FSDP]. The following figure provides a high-level overview of a search system that uses the FIXML data structure. 6 / 31

7 Figure 1: Overview The search system first retrieves content from content sources such as Web servers, relational databases, file servers, or document repositories. An item processing service performs required processing to prepare the content for indexing. Examples of item processing are mapping of crawled properties to managed properties, parsing of document formats, linguistic normalization of text, and normalization of numeric data. An [MS-FSDP] protocol client maps the processed managed properties into the appropriate FIXML data structures based on the mapping configuration, as described in the fixml_mappings.xml configuration file. The content of a managed property normally maps to more than one element in the FIXML data structure. For more information, refer to [MS-FSSCFG]. For more information about how the search system uses this data structure specification, refer to [MS-FSO]. The purpose of FIXML is to provide the data structure for a processed item and then to map the data to the search index structures that an index schema specifies. An entity that performs indexing consumes the FIXML data structure. The FIXML data structure maps to the index data structure via the following index concepts: The context catalog represents a particular view of the searchable content. A numeric context catalog contains all numeric managed properties. A synthetic context catalog contains all text managed properties that can be queried individually. A full-text index field context catalog contains the index structures for a set of managed properties that accept ranked queries. Special-purpose context catalogs for anchor text and associated query data also exist. For more information about context catalog types, refer to section The property context represents an individual managed property within the context catalog. The property context element in the FIXML contains the data of the managed property that will be indexed. The document summary elements in the FIXML are a textual representation of the managed property that can be returned with a query result. The attribute vector elements in the FIXML are used by the indexer to create data structures for deep refinement and full-text sorting. 7 / 31

8 1.4 Relationship to Protocols and Other Structures The following figure provides a high-level overview of the services and protocols related to the FIXML data structure. Figure 2: Service and protocol relationship The FIXML data structure represents the data payload in [MS-FSDP]. A single data structure represents one item to be submitted for indexing. The indexing service uses the FIXML data structure to create the index data structures, as specified in [MS-FSIXDS] section 2. The indexing service protocol client (described in [MS-FSDP]) creates the FIXML data structure by using the configuration information that is related to the index schema. The configuration information is in the fixml_mappings.xml configuration file. For more details, see [MS-FSSCFG]. The protocol client downloads the configuration file by using the protocol specified in [MS-FSCX]. 1.5 Applicability Statement The data structure represents items that are ready for indexing. The content of the data structure uses an XML document format and is represented as text. The document format applies for all fulltext search applications that use the feeding protocol described in [MS-FSDP]. The feeding protocol creates index data structures. 1.6 Versioning and Localization None. 1.7 Vendor-Extensible Fields None. 8 / 31

9 2 Structures 2.1 Part Enumerations None. 2.2 Extensions None. 2.3 Global Elements document The document element represents one item to be submitted for indexing. <xs:element name="document" type=" CT_document "/> 2.4 Global Attributes None. 2.5 Complex Types CT_document Referenced by: document A complex type that specifies one item to be submitted for indexing. Child elements: catalog: A CT_catalog element that specifies a context catalog. For a specification of context catalog types and required <catalog> elements, refer to section rank: A CT_rank element. summary: A CT_summary element that specifies the input summary class. attrvec: A CT_attrVec element that specifies a managed property formatted to facilitate deep refinement. The following table describes the attribute for this type. Name lang Description The language code of the item, as specified by [ISO-639]. <xs:complextype name="ct_document"> <xs:sequence> <xs:element name="catalog" type="ct_catalog" maxoccurs="unbounded"/> <xs:element name="rank" type="ct_rank"/> <xs:element name="attrvec" type="ct_attrvec" minoccurs="0" 9 / 31

10 maxoccurs="unbounded"/> <xs:element name="summary" type="ct_summary"/> </xs:sequence> <xs:attribute name="lang" type="xs:string"/> Item Requirements on Managed Properties The following subsections specify item formatting requirements on managed properties that are mapped to the FIXML data structure. The managed properties are mapped to one or more of the <catalog>, <summary>, and <attrvec> elements, depending on the index schema configuration. This section specifies requirements on the content of the managed properties before the mapping to the FIXML elements. For a specification of how the managed properties are mapped to these elements, refer to the corresponding sections Document Vector The "docvector" managed property contains the document vector. The property MUST contain zero or more name/value pairs. The name/value pairs are retrieved from the item during item processing. The syntax of the "docvector" managed property is a string of successive name/value pairs that are enclosed in square brackets. The square brackets are a required part of the following syntax. [<concept>, <weight>][<concept>, <weight>] [<concept>, <weight>] In the preceding syntax: <concept> is a single word or a string that specifies a concept in the document vector. <weight> is a floating point value between 0 and 1 that indicates the relevance of the concept. The following syntax is an example of a "docvector" string. [windows, 0.842][sharepoint, 0.736][microsoft office, ] Document-Level Security Properties The "docacl" and "docaclsystemid" managed properties MUST be formatted according to the following specification: If user access restrictions apply to the item, "docacl" and "docaclsystemid" managed properties MUST be formatted as specified in [MS-FSCF] section If the item has no access restrictions (all users have read access), "docacl" and "docaclsystemid" managed properties MUST be set to the reserved word "all" Anchor Text The "anchortext" managed property represents the anchor text information for the item. 10 / 31

11 The managed property MUST contain zero or more anchor text strings. Each anchor text string represents the anchor text associated with one hyperlink that refers to this item. The managed property is mapped to the context catalogs named "bt1" and "anchortext", and as specified in a <map> setting in the fixml_mappings.xml configuration file. For a specification of this configuration file, refer to [MS-FSSCFG] section For details about mapping to the "anchortext" catalog, refer to section Associated Queries The managed property named "assocqueries" represents the associated query information for the item. The managed property MUST contain zero or more associated query strings. Each associated query string represents a user query string that resulted in a search clickthrough leading to this item. The managed property is mapped to the context catalogs named "bt1" and "assocqueries", and as specified in a <map> setting in the fixml_mappings.xml configuration file. For a specification of this configuration file, refer to [MS-FSSCFG] section For details about mapping to the context catalog named "assocqueries", refer to section Static Rank Managed properties related to static rank SHOULD be set in the item processing as follows: "docrank": The quality of the document based on link analysis. "siterank": The quality of the document's corresponding site based on link analysis. "urldepthrank": Inverse to the depth of the URL of the document. If set, the value for each managed property MUST be an integer value that represents the rank contribution for this aspect of the static rank. The actual value range is implementation specific CT_catalog Referenced by: CT_document A complex type that specifies one context catalog in the search index. For further details, refer to [MS-FSSCFG] section Child element: context: A CT_context element that specifies one property context within a context catalog. The <map> element in the fixml_mappings.xml configuration file identifies the <context> element to include. For details about the configuration file, see [MS-FSSCFG] section The type attribute of the <map> element specifies the mapping. When type is set to the value "context", the managed property content that this <map> element specifies MUST be mapped to a <context> element. The following table describes the attribute for this type. Name name Description The name of the context catalog. This name MUST refer to a context catalog with the same name 11 / 31

12 Name Description in the indexconfig.xml file, as specified in [MS-FSSCFG] section <xs:complextype name="ct_catalog"> <xs:sequence> <xs:element name="context" type="ct_context" maxoccurs="unbounded"/> </xs:sequence> <xs:attribute name="name" type="xs:string"/> Context Catalog Types The context catalog names correspond to the value of the dstcatalog attribute in the fixml_mappings.xml <map> element when type is set to "context". For a specification of context catalogs (<catalog> elements) that MUST be present for a particular index schema configuration, see [MS-FSSCFG] section For further details about context catalog types, see [MS-FSSCFG] section Synthetic Context Catalog (bt1) A <catalog> element named "bt1" MUST be present in the FIXML. This represents a synthetic context catalog. The <context> entries inside this <catalog> element represent the non-numeric managed properties Numeric Context Catalog (bi1) A <catalog> element named "bi1" MUST be present in the FIXML, with <context> element child objects representing the numeric managed properties. Floating-point managed properties MUST use "." as the decimal separator. Datetime managed properties MUST be encoded in the following format: "[-][Y]YYYY-MM-DDThh:mm:ss[.d{1,7}]Z" The datetime format legend is as follows: The letter Y indicates one cipher in the year (four or five digits accepted). The year specification can be negative. The minimum year is "-29000". The maximum year is "29000". The letter M indicates one cipher in the month number. The letter D indicates one cipher in the day number within the month. The letter h indicates one cipher in the hour number. The letter m indicates one cipher in the minute number. The letter s indicates one cipher in the second number. Seconds can optionally be specified with up to seven digits of decimal precision. The letters T and Z are literal constants. 12 / 31

13 Context Catalog for Item Metadata (meta) A <catalog> element named "meta" MUST be present in the FIXML, with two <context> elements representing the following internal properties: collection: MUST contain the name of the content collection that this item is assigned to. contentid: MUST contain the document identifier(3) for the item. The context catalog does not have any corresponding <map> entries in the fixml_mappings.xml configuration file Context Catalog for Anchor Text The context catalog for anchor text contains a complete proximity boost representation of the "anchortext" managed property. For details about this managed property, refer to section The <context> elements within this <catalog> element MUST include the attribute xml:lang="complete". This setting indicates that all white space characters in each anchor text string in the managed property MUST be replaced with the UTF-8 character C2 BA (masculine ordinal indicator) in the <context> element. For details about white space characters, see [MS-FSIN] section 2.2. Each anchor text string MUST be separated with the phrase break character represented by the UTF-8 character c7 82 (alveolar click) in the <context> element. This character instructs the indexer to treat each anchor text string as one word in the search index. A <catalog> element named "anchortext" MUST be present. The <catalog> element MUST contain a <context> element named "canchortext" if any anchor text information is associated with the item. For details, see section A <catalog> element named "assocqueries" MUST be present. If any associated query information is associated with the item, the <catalog> element MUST contain a <context> element named "cassocqueries". For details, see section Context Catalog for Associated Queries The context catalog for associated queries contains a complete proximity boost representation of the "assocqueries" managed property. For details about this managed property, refer to section The <context> elements within this <catalog> element MUST include the attribute xml:lang="complete". This setting indicates that all white space characters in each associated query string in the managed property MUST be replaced with the UTF-8 character C2 BA (masculine ordinal indicator) in the <context> element. For details about white space characters, see [MS-FSIN] section 2.2. Each associated query string MUST be separated with the phrase break character represented by the UTF-8 character c7 82 (alveolar click) in the <context> element. This character instructs the indexer to treat each associated query string as one word in the search index. A <catalog> element named "assocqueries" MUST be present. If any associated query information is associated with the item, the <catalog> element MUST contain a <context> element named "cassocqueries". For details, see section / 31

14 Context Catalogs for Full-Text Index Field The FIXML MUST contain one full-text index field <catalog> element for each full-text index field context catalog defined in the index schema. For further details about context catalog structure, see [MS-FSSCFG] section The <map> elements in fixml_mappings.xml that have a context catalog name (dstcatalog attribute) that starts with "bcat" specify managed property mapping to a full-text index field. For details about fixml_mappings.xml syntax, see [MS-FSSCFG] section The destination full-text index context, specified by the <map> attribute dst, is one of eight reserved property context names: "bconf1", "bconf2", "bconf3", "bconf4", "bconf5", "bconf6", "bconf7", and "bconf8". These eight property contexts are associated with the field importance level for the managed property. Multiple managed properties can be mapped to the same full-text index context. In this case, the content for each managed property MUST be included in a separate <context> element within the <catalog> element CT_context Referenced by: CT_catalog A complex type that specifies one property context within a context catalog. For further details, see [MS-FSSCFG] section Child elements: sep: A CT_sep element that specifies an optional field separation within a <context> element. a: A simple element that specifies grammatical variations for a word. For details, see section The following table describes the attributes for this type. Name name numoccs xml:lang Description The name of the property context. The name MUST be equal to the value of a valid <map> attribute dst in fixml_mappings.xml. For further details about fixml_mappings.xml syntax, refer to [MS-FSSCFG] section For details about mapping from managed properties to property contexts, refer to section An optional occurrence count for the content of this <context> element. Setting the numoccs attribute to a value N for a context element that contains the word X has the same effect as repeating the word X N times inside the <context> element. This has a corresponding effect on the occurrence boost of the content inside the <context> element. Specifies whether the consumer of the FIXML data structure will perform tokenization. If the attribute is not set, tokenization is already performed for the managed property as part of item processing. The usage of this attribute deviates from the definition in the XML namespace. The attribute is used for a specific purpose that is specified later. This is a deviation from XML schema rules, and hence this attribute is not specified in the XML schema. Valid values for the attribute are as follows: "space": The item content passed in the FIXML <context> element is not tokenized. For 14 / 31

15 Name Description details, see section "substring": The item content passed in the FIXML <context> element is not tokenized. For details, see section "complete": Tokenization is already performed for the managed property as part of item processing. This setting MUST be used for the <context> elements named "canchortext" and "cassocqueries". For details, see sections and section If xml:lang is not present, this equals the setting xml:lang="space". <xs:complextype name="ct_context" mixed="true"> <xs:choice minoccurs="0" maxoccurs="unbounded"> <xs:element name="sep" type="ct_sep"/> <xs:element name="a"/> </xs:choice> <xs:attribute name="name" type="xs:string" use="required"/> <xs:attribute name="numoccs"> <xs:simpletype> <xs:restriction base="xs:unsignedint"> <xs:mininclusive value="1"/> </xs:restriction> </xs:simpletype> </xs:attribute> Content Requirements for Property Context The mapping configuration specified in the following subsections specifies how the content mapping to the FIXML <catalog> element MUST be performed based on configuration data in fixml_mappings.xml and FieldProperties.xml Rules for Item Processing The FieldProperties.xml configuration file is specified in [MS-FSSCFG] section 2.12 and contains item processing rules for managed properties. The following table specifies the rules for item processing that MUST be applied on managed property data, before the mapping of the data to FIXML <context> elements. The Parameter column indicates the element or in FieldProperties.xml that specifies the configuration parameter. The Item processing column specifies the corresponding item processing that MUST be performed. Parameter generic-tokenization substring-tokenization Item processing If this element is present, tokenization MUST NOT be applied to the managed property data before the mapping to the FIXML <context> element. For this tokenization mode, generic (language-independent) tokenization is performed inside the search service application. The consumer of the FIXML data structure MUST tokenize the content of the <context> element, as specified in [MS-FSIN] section 2.2. The <context> element MUST include the attribute xml:lang="space". If this element is present, tokenization MUST NOT be applied to the managed property data before the mapping to the FIXML <context> element. For this tokenization mode, substring tokenization is performed 15 / 31

16 Parameter Item processing inside the search service application. The consumer of the FIXML data structure MUST tokenize the content of the <context> element by using substring tokenization. The <substringtokenization> attribute N specifies the n-gram value. The <context> element MUST include the attribute xml:lang="substring". For further details, see [MS-FSSCFG] section language-tokenization languagetokenization/@lemmatization result/@type If this element is present, tokenization MUST be applied to the managed property data before the mapping to the FIXML <context> element. For a specification of required tokenization, see [MS-FSIN] section 2.1. The <context> element MUST NOT include the attribute xml:lang. For the "url" and "urls" managed properties, tokenization MUST NOT be applied as part of item processing, even if the <language-tokenization> element is set for the properties in FieldProperties.xml. The consumer of the FIXML data structure MUST tokenize the content of the <context> element, as specified in [MS-FSIN] section 2.1. If this element is present, stemming MUST be applied to the managed property data before the mapping to the FIXML <context> element. For a specification of required stemming, see section If type="static" or type="dynamic", the <summary> element MUST include an <sfield> element for this managed property that MUST be formatted as specified in section FIXML Mapping Rules for Managed Properties The fixml_mappings.xml configuration file is specified in [MS-FSSCFG] section 2.10 and contains FIXML mapping rules for managed properties. The following table specifies FIXML mapping rules based on particular values of the <map> attributes in fixml_mappings.xml. <map> attribute src maxsize phrasebreak fieldseparationlength Item mapping rule Specifies the name of the source managed property. For a specification of src naming rules, see [MS-FSSCFG] section Managed properties larger than this size limit MUST be truncated before the mapping to the corresponding FIXML <context> element. The truncation MUST be applied to a word boundary. phrasebreak="yes": Semantically separated regions in the managed property text MUST be separated by the UTF-8 character c7 82 (alveolar click) in the <context> element. This phrase break character indicates a phrase break in the text flow. The query evaluation can use this character to avoid phrase match across such boundaries. If this attribute is present and contains a nonzero value, a <sep> element with num="fieldseparationlength value" MUST be added to the end of the FIXML <context> element. For details, see section / 31

17 Handling of Grammatical Variations When a grammatical variation exists for a word, the <a> element MUST be included in the <context> element, replacing the word itself in the text flow. The <a> element indicates that all words inside the element are indexed through word stacking. All words inside the <a> element MUST have a single uppercase suffix character added to the word: T: The original word as it appears in the managed property. L: Grammatical variations or the base form. The use depends on the stemming mode. The L-suffix words added to the <a> element depend on the stemming mode specified in [MS-FSIN] section 2.3. Stemming MUST be applied according to the language of the textual content of the managed property for each word (noun, adjective, verb, and pronoun) that has one or more grammatical variations. Stemming form="variations" indicates stemming by expansion. All grammatical variations of the word MUST be added as L-suffix words, including the original word form as it appears in the managed property. Stemming form="base form" indicates stemming by reduction. The canonical base form of the word MUST be added as an L-suffix word. For a specification of the language-specific stemming mode to be applied, refer to [MS-FSIN] section 2.3. Stemming MUST be applied according to the language of the textual content of the managed property for each word (noun, adjective, verb, and pronoun) that has one or more grammatical variations CT_sep Referenced by: CT_context A complex type that specifies an optional field separation specification within a <context> element. The element instructs an indexer to insert num number of invisible words between the word before and after the <sep> element in the search index. The following table describes the attribute for this type. Name num Description The number of invisible words to insert. <xs:complextype name="ct_sep"> <xs:attribute default="1" name="num" type="xs:unsignedshort"/> CT_rank Referenced by: CT_document A complex type that specifies a <rank> element with a fixed value, as specified in the XML schema. The consumer of the FIXML data structure MUST ignore this element. 17 / 31

18 <xs:complextype name="ct_rank"> <xs:attribute name="class" use="required"> <xs:simpletype> <xs:restriction base="xs:string"> <xs:enumeration value="dummy"/> </xs:restriction> </xs:simpletype> </xs:attribute> CT_summary Referenced by: CT_document A complex type that specifies a set of document summary fields, as specified in fixml_mappings.xml. For a specification of this configuration file, see [MS-FSSCFG] section Child element: sfield: A CT_Sfield element that specifies a document summary for the managed property. The <map> elements in the fixml_mappings.xml configuration file indicate which <sfield> elements will be included. For a specification of this configuration file, see [MS-FSSCFG] section The type attribute of the <map> element specifies the mapping to <sfield> elements. The managed property content that this <map> element specifies MUST be mapped to an <sfield> element. The following table describes the attribute for this type. Name class Description The name of the input summary class. It MUST be set to "content". <xs:complextype name="ct_summary"> <xs:sequence> <xs:element name="sfield" type="ct_sfield" minoccurs="1" maxoccurs="unbounded"/> </xs:sequence> <xs:attribute name="class" use="required"> <xs:simpletype> <xs:restriction base="xs:string"> <xs:enumeration value="content"/> </xs:restriction> </xs:simpletype> </xs:attribute> CT_sField Referenced by: CT_summary A complex type that specifies a document summary for the managed property, as specified by the name attribute. The following table describes the attribute for this type. 18 / 31

19 Name name Description The name of the document summary. It MUST be equal to the value of the dst attribute to the corresponding <map> element of fixml_mappings.xml. <xs:complextype name="ct_sfield"> <xs:attribute name="name" type="xs:string" use="required"/> Mapping Rules for Document Summary The <map> elements in the fixml_mappings.xml configuration file that have property context names that start with "bsum" or "bsrc" specify a managed property mapping to an <sfield> element. For a specification of this configuration file, refer to [MS-FSSCFG] section For details about the naming of index field prefixes, see [MS-FSSCFG] section The document summary <sfield> elements that have names that start with "bsrc" represent input to creation of a hit highlighted summary. The <sfield> element MUST include the following additional formatting markup: The UTF-8 character c7 82 (alveolar click) MUST separate semantically separated regions in the text. This character indicates a phrase break in the text flow. The hit highlighted summary feature can use it to avoid phrase match across such boundaries CT_attrVec Referenced by: CT_document A complex type that specifies a set of attribute vector data (<avfield> elements). Child element: avfield: An xs:string element that specifies one attribute value. If no attribute value exists for the managed property in this item, an empty <avfield> element MUST be present. The <map> elements in the fixml_mappings.xml configuration file identifies the <attrvec> elements to include. For a specification of this configuration file, see [MS-FSSCFG] section The type attribute of the <map> element specifies the mapping. When type is set to "attributevector", the managed property content that this <map> element specifies MUST be mapped to an <attrvec> element. The following table describes the attribute for this type. Name name Description The name of the attribute vector. It MUST be equal to the value of the dst attribute to the corresponding <map> element of fixml_mappings.xml. For a specification of attribute vector types and associated naming rules, refer to [MS-FSSCFG] section dst="batvmanagedpropertyname" for a managed property named managedpropertyname that have full-text sort enabled. dst="bavnmanagedpropertyname" for a managed property managedpropertyname that have an associated refiner. <xs:complextype name="ct_attrvec"> <xs:sequence> 19 / 31

20 <xs:element maxoccurs="unbounded" name="avfield" type="xs:string"/> </xs:sequence> <xs:attribute name="name" type="xs:string" use="required"/> Mapping Rules for Attribute Vector The following table specifies FIXML mapping rules based on particular values of the <map> attributes in fixml_mappings.xml. For a specification of this configuration file, see [MS-FSSCFG] section <map> attribute src multi defaultvalue Item mapping rule Specifies the name of the source managed property. For a specification of src naming rules, see [MS-FSSCFG] section If this attribute is set to "yes", the source managed property supports multiple (text or numeric) values for attribute vectors. Each value MUST be inserted as individual attrvec/avfield elements in the FIXML. If this attribute is set and not empty, and the source managed property is empty, the associated attrvec/avfield element MUST be set to the value of defaultvalue. 2.6 Simple Types None. 20 / 31

21 3 Structure Examples 3.1 FIXML Structure The following example illustrates one FIXML <document> element that represents one item that is sent for indexing. Content for indexing in the full-text index field named "content". In this example, "body" is mapped to "bconf1", and "urls" is mapped to "bconf4". Note the special handling of "urls", as specified in section The following examples indicate the phrase break UTF-8 character c7 82 (alveolar click) as "ǂ ". Stemming is shown via the <a> element. <document lang="en"> <catalog name="bcatcontent"> <context name="bconf1"> ǂ <a>welcomet welcomel welcomesl</a> to the Official contoso <a>pharmaceuticalst pharmaceuticall pharmaceuticalsl</a> <a>websitet websitel websitesl</a> the best <a>choicet choicel choices</a> in <a>pharmaceuticalst pharmaceuticall pharmaceuticalsl</a> ǂ <sep num="256"/></context> <context name="bconf4"> <![CDATA[ ǂ ǂ ]]><sep num="256"/></context> <context name="bconf7"> ǂ <a>contosot contosol</a> ǂ <sep num="256"/></context> </catalog> The following example shows content for indexing in the synthetic context catalog named "bt1". This contains all non-numeric managed properties that are configured as individually searchable. In this example, "docacl" and "docaclsystemid" are set to "all". This indicates that no restrictions on access rights exist on the item. <catalog name="bt1"> <context name="bcontitle"> ǂ <a>contosot contosol</a> ǂ </context> <context name="bconanchortext"><![cdata[]]></context> <context name="bconassocqueries"><![cdata[]]></context> <context name="bconcontenttype"><![cdata[ ǂ text/html ǂ ]]></context> <context name="bconformat"><![cdata[ ǂ Web Page ǂ ]]></context> <context name="bconlanguage"><![cdata[ ǂ en ǂ ]]></context> <context name="bconlanguages"><![cdata[ ǂ en ǂ ]]></context> <context name="bconcharset"><![cdata[ ǂ utf-8 ǂ ]]></context> <context name="bconurls"> <![CDATA[ ǂ ǂ ]]></context> <context name="bcondocacl"><![cdata[ ǂ all ǂ ]]></context> <context name="bcondocaclsystemid"><![cdata[ ǂ all ǂ ]]></context> <context name="bconcrawledpropertiescontent"></context> <context name="bconcrawledpropertynames"><![cdata[]]></context> <context name="bconcompanies"> <![CDATA[ ǂ Fabrikam ǂ Woodgrove Bank ǂ Contoso Pharmaceuticals ǂ ]]></context> <context name="bconlocations"><![cdata[ ǂ Contosoville ǂ ]]></context> <context name="bconpersonnames"><![cdata[]]></context> <context name="bcon s"><![cdata[]]></context> <context name="bcondates"><![cdata[ ǂ ǂ ]]></context> <context name="bcontimes"><![cdata[]]></context> <context name="bconprices"><![cdata[]]></context> 21 / 31

22 <context name="bconextractedurls"> <![CDATA[ ǂ ǂ ǂ ]]></context> </catalog> The special context catalog named "assocqueries" contains query terms that have led to a clickthrough to this item from a query hit list, as shown in the following example. <catalog name="assocqueries"> <context name="cassocqueries" xml:lang="complete" numoccs="35"> <![CDATA[contoso]]></context> </catalog> The special context catalog named "anchortext" contains anchor texts that reference this item, as shown in the following example. <catalog name="anchortext"> <context name="canchortext" xml:lang="complete" numoccs="2"> <![CDATA[contoso]]></context> </catalog> The following example shows content for indexing in the numeric context catalog named "bi1". This contains all numeric managed properties. <catalog name="bi1"> <context name="bconprocessingtime"> <![CDATA[ T16:44:02Z]]></context> <context name="bcondocdatetime"><![cdata[ t15:19:23z]]></context> <context name="bconsize"><![cdata[28601]]></context> <context name="bcondocrank"><![cdata[0]]></context> <context name="bconsiterank"><![cdata[0]]></context> <context name="bconurldepthrank"><![cdata[630]]></context> </catalog> The following example shows item metadata ("contentid" and "collection"). <catalog name="meta"> <context name="contentid"> <![CDATA[ ǂ ǂ ]]><sep num="256"/></context> <context name="collection" xml:lang="space"> <![CDATA[test1]]><sep num="256"/></context> </catalog> The following example shows the dummy element that MUST be present. <rank class="dummy"> </rank> The following example shows attribute vectors for managed properties that have full-text sorting enabled. <attrvec name="batvtitle"> 22 / 31

23 <avfield><![cdata[contoso Pharmaceuticals: <attrvec name="batvprocessingtime"> <avfield><![cdata[ t16:44:02z]]></avfield> <attrvec name="batvdocdatetime"> <avfield><![cdata[ t15:19:23z]]></avfield> <attrvec name="batvsize"> <avfield><![cdata[28601]]></avfield> <attrvec name="batvdocrank"> <avfield><![cdata[0]]></avfield> <attrvec name="batvsiterank"> <avfield><![cdata[0]]></avfield> <attrvec name="batvurldepthrank"> <avfield><![cdata[630]]></avfield> The following example shows attribute vectors for managed properties that have an associated refiner. <attrvec name="bavnlanguages"> <avfield><![cdata[en]]></avfield> <attrvec name="bavncompanies"> <avfield><![cdata[fabrikam]]></avfield> <avfield><![cdata[woodgrove Bank]]></avField> <avfield><![cdata[contoso Pharmaceuticals]]></avField> <attrvec name="bavnlocations"> <avfield><![cdata[contosoville]]></avfield> <attrvec name="bavnpersonnames"> <avfield><![cdata[]]></avfield> <attrvec name="bavn s"> <avfield><![cdata[]]></avfield> <attrvec name="bavndates"> <avfield><![cdata[ ]]></avfield> <attrvec name="bavntimes"> <avfield><![cdata[]]></avfield> <attrvec name="bavnextractedurls"> <avfield><![cdata[ <avfield><![cdata[ <attrvec name="bavnprices"> <avfield><![cdata[]]></avfield> <attrvec name="bavnformat"> <avfield><![cdata[web Page]]></avField> 23 / 31

24 <attrvec name="bavncrawledpropertynames"> <attrvec name="bavnsize"> <avfield><![cdata[28601]]></avfield> <attrvec name="bavndocdatetime"> <avfield><![cdata[ t15:19:23z]]></avfield> The following example shows content for document summaries. <summary class="content"> <sfield name="internalid"> <![CDATA[7894d0895b21e07148fbcfe1471d83ca_test1]]></sField> <sfield name="contentid"> <![CDATA[ <sfield name="collection"><![cdata[test1]]></sfield> <sfield name="contentids"> <![CDATA[ <sfield name="bsrctitle"> <![CDATA[Contoso Pharmaceuticals: www. contoso. com]]></sfield> <sfield name="bsrcbody"> <![CDATA[Welcome to the Official Contoso Pharmaceuticals Website. The best choice in pharmaceuticals.]]></sfield> <sfield name="bsumteaser"> <![CDATA[The best choice in pharmaceuticals]]></sfield> <sfield name="bsumcontenttype"><![cdata[text/html]]></sfield> <sfield name="bsumformat"><![cdata[web Page]]></sField> <sfield name="bsumlanguage"><![cdata[en]]></sfield> <sfield name="bsumlanguages"><![cdata[en]]></sfield> <sfield name="bsumcharset"><![cdata[utf-8]]></sfield> <sfield name="bsumurls"> <![CDATA[ <sfield name="bsumurl"> <![CDATA[ <sfield name="bsumprocessingtime"><![cdata[ t16:44:02z]]></sfield> <sfield name="bsumdocdatetime"><![cdata[ t15:19:23z]]></sfield> <sfield name="bsumsize">28601</sfield> <sfield name="bsumdocrank">0</sfield> <sfield name="bsumsiterank">0</sfield> <sfield name="bsumurldepthrank">630</sfield> <sfield name="bsumdocacl"><![cdata[all]]></sfield> <sfield name="bsumdocaclsystemid"><![cdata[all]]></sfield> <sfield name="bsumcrawledpropertynames"><![cdata[]]></sfield> <sfield name="bsumcompanies"> <![CDATA[Fabrikam; Woodgrove Bank; Contoso Pharmaceuticals]]></sField> <sfield name="bsumlocations"><![cdata[carlsberg]]></sfield> <sfield name="bsumpersonnames"><![cdata[]]></sfield> <sfield name="bsum s"><![cdata[]]></sfield> <sfield name="bsumdates"><![cdata[ ]]></sfield> <sfield name="bsumtimes"><![cdata[]]></sfield> <sfield name="bsumprices"><![cdata[]]></sfield> <sfield name="bsumextractedurls"> <![CDATA[ </summary> </document> 24 / 31

25 4 Security Considerations None. 25 / 31

26 5 Appendix A: Full XML Schemas For ease of implementation, this section provides the full World Wide Web Consortium (W3C) XML schemas for the elements, attributes, complex types, and simple types described in the preceding sections. <?xml version="1.0" encoding="utf-8"?> <xs:schema xmlns:xs=" <!-- *********************** Global elements ********************* --> <xs:element name="document" type="ct_document"/> <!-- ************************ Complex types ********************** --> <xs:complextype name="ct_document"> <xs:sequence> <xs:element name="catalog" type="ct_catalog" maxoccurs="unbounded"/> <xs:element name="rank" type="ct_rank"/> <xs:element name="attrvec" type="ct_attrvec" minoccurs="0" maxoccurs="unbounded"/> <xs:element name="summary" type="ct_summary"/> </xs:sequence> <xs:attribute name="lang" type="xs:string"/> <xs:complextype name="ct_catalog"> <xs:sequence> <xs:element name="context" type="ct_context" maxoccurs="unbounded"/> </xs:sequence> <xs:attribute name="name" type="xs:string"/> <xs:complextype name="ct_context" mixed="true"> <xs:choice minoccurs="0" maxoccurs="unbounded"> <xs:element name="sep" type="ct_sep"/> <xs:element name="a"/> </xs:choice> <xs:attribute name="name" type="xs:string" use="required"/> <xs:attribute name="numoccs"> <xs:simpletype> <xs:restriction base="xs:unsignedint"> <xs:mininclusive value="1"/> </xs:restriction> </xs:simpletype> </xs:attribute> <!-- Note that the xml:lang attribute is used for the context element but --> <!-- is not used according to the standard XML 1.0 namespace definition. --> <xs:complextype name="ct_sep"> <xs:attribute default="1" name="num" type="xs:unsignedshort"/> <xs:complextype name="ct_rank"> <xs:attribute name="class" use="required"> <xs:simpletype> 26 / 31

27 <xs:restriction base="xs:string"> <xs:enumeration value="dummy"/> </xs:restriction> </xs:simpletype> </xs:attribute> <xs:complextype name="ct_summary"> <xs:sequence> <xs:element name="sfield" type="ct_sfield" minoccurs="1" maxoccurs="unbounded"/> </xs:sequence> <xs:attribute name="class" use="required"> <xs:simpletype> <xs:restriction base="xs:string"> <xs:enumeration value="content"/> </xs:restriction> </xs:simpletype> </xs:attribute> <xs:complextype name="ct_sfield"> <xs:attribute name="name" type="xs:string" use="required"/> <xs:complextype name="ct_attrvec"> <xs:sequence> <xs:element maxoccurs="unbounded" name="avfield" type="xs:string"/> </xs:sequence> <xs:attribute name="name" type="xs:string" use="required"/> </xs:schema> 27 / 31

28 6 Appendix B: Product Behavior The information in this specification is applicable to the following Microsoft products or supplemental software. References to product versions include released service packs: Microsoft FAST Search Server 2010 Exceptions, if any, are noted below. If a service pack or Quick Fix Engineering (QFE) number appears with the product version, behavior changed in that service pack or QFE. The new behavior also applies to subsequent service packs of the product unless otherwise specified. If a product edition appears with the product version, behavior is different in that product edition. Unless otherwise specified, any statement of optional behavior in this specification that is prescribed using the terms SHOULD or SHOULD NOT implies product behavior in accordance with the SHOULD or SHOULD NOT prescription. Unless otherwise specified, the term MAY implies that the product does not follow the prescription. 28 / 31