Unica Web Analytics Data Conduit for Vignette

Transcription

1 Unica Web Analytics Data Conduit for Vignette

2 Copyright Copyright, Unica Corporation All rights reserved. Unica Corporation Reservoir Place North 170 Tracer Lane Waltham, MA s and Data All software and related documentation is subject to restrictions on use and disclosure as set forth in the applicable Unica Software License and Services Agreement or End User License Agreement, with restricted rights for U.S. government users and applicable export regulations. Companies, names, and data used in examples herein are fictitious unless otherwise noted. Trademarks and Patents Unica, the Unica logo, Affinium, and NetTracker are registered trademarks of Unica Corporation with the U.S. Patent and Trademark Office. Affinium Campaign, Affinium Campaign Collaborate, Affinium Campaign emessage, Affinium Campaign Interact, Affinium Campaign Optimize, Affinium Detect, Affinium NetInsight, Affinium Leads, Affinium Leads Referral, Affinium Leads Contact, Affinium Model, Affinium Plan, Affinium Mart, Universal Dynamic Interconnect, Interconnect Modules, Marketic, Powering Marketing Success, Customer Value Maximization, MAP, One-to-One Optimization, and Right-time Marketing are trademarks of Unica Corporation. All other trademarks are the property of their respective owners. Portions of the software described in this document are covered by U.S. Patent Numbers: 6,317,752, 6,269,32, 6,542,894, and 6,782,390. NOTICE: This document contains confidential and proprietary information of Unica Corporation ( Unica ). Use, duplication, or disclosure without the express written consent of Unica Corporation is prohibited. 2

3 Introduction The Vignette Content Management Server generates Custom URLs (CURLs) with the following format: /templatepath/cacheval,templateid,objectid,variation.ext?querystring The following table explains the different parts of the CURL. Placeholder cacheval templateid objectid variation Description Indicates whether a cached or freshly generated version of the requested page should be delivered to the client. A value of 0 indicates that a cached version should be sent, while a value of 1 indicates that the page should be regenerated before delivery. ID of the requested page s user-defined template. The user can define any number of templates to allow for delivery of pages similar in structure but with variable content. ID of the requested page s content. The content identified by this portion of the CURL is formatted using the template defined in the templateid portion of the CURL. Identifier of the variation of the given template to be used when generating the requested page. For example, different variations of the same template can be built to take advantage of the varying functionality offered by different Web browsers. The default value of 00 indicates that the standard variation should be used. The resulting CURL will look similar to this: /products/netinsight/0,1234,5678,00.html Log-file analysis applications can encounter three difficulties with Vignette CURLs: Page duplication CURLs pose problems from a log analysis point of view because multiple distinct CURLs may in fact all generate the same page. For example, consider the following four CURLs: /products/netinsight/0,1234,5678,00.html /products/netinsight/1,1234,5678,00.html /products/netinsight/0,1234,5678,0.html /products/netinsight/0,1234,5678,cc.html Each of these four CURLs generates a page providing the user with the same content (because they all have the same templateid and objectid), and from a log analysis point of view, should therefore be considered a single page. 1

4 Base64-encoded objectids In some situations, the value in the objectid field is base64-encoded, obscuring the true value of the field. For example, consider the following CURL: /products/netinsight/0,1234,zw5nbglzafy0,00.html The value of the objectid field in the CURL above is useless, even to those who may be familiar with the values in the other portions of the CURL. When decoded, however, the value of this field can be meaningful. For example, the above CURL, after base64-decoding the objectid field, is: /products/netinsight/0,1234,englishv4,00.html Multi-part objectids In addition to base64-encoded objectids, it is also possible to have objectid values that are comprised of multiple values joined with a delimiter character (often a dash). In these situations, understanding the true value of the objectid value requires knowledge about all the individual pieces of the objectid. You can use the NetInsight URL Search and Replace feature to address some of the problems posed by CURLs. In particular, URL Search and Replace can be used to eliminate differences in the CURL due to the cacheval and variation fields. For example, to modify a CURL such that all requests have a cacheval of 0, the following URL Search and Replace rule could be defined: Description: Remove cacheval Search rule: \/(\d),\s+?,\s*?,\s+?\.\s* Replacement value: 0 Replacement scope: Only first match Type of rule: Page Similarly, the following URL Search and Replace rule could be used to modify CURLs so that all requests have a variation of 00: Description: Remove variation Search rule: \/\d,\s+?,\s*?,(\s+?)\.\s* Replacement value: 00 Replacement scope: Only first match Type of rule: Page These rules allow NetInsight to identify CURLs based strictly on their templateid and objectid values, allowing for more accurate and meaningful statistics. However, even after the application of these URL Search and Replace rules, the pages displayed in the NetInsight reports remain cryptic and uninformative. The Unica Web Analytics Data Conduit for Vignette will allow assignment of a descriptive title to, or further modification of, a CURL during the NetInsight import process, making the reports generated by NetInsight more meaningful and easier to understand. 2

5 To use the Unica Web Analytics Data Conduit for Vignette, you must complete the following: 1. Edit the data conduit's configuration file to indicate which CURLs should be modified and how they should be modified. 2. Test the configuration. 3. Tell NetInsight to use the data conduit. Note: The Unica Web Analytics Data Conduit for Vignette currently supports only Microsoft SQL Server and Oracle databases. Installing the Unica Web Analytics Data Conduit for Vignette To install the data conduit on a Windows system: 1. Double-click ntdivign-{release Number}.exe. 2. Click Next. 3. Specify the temporary directory in which you want to place the data conduit files. Make note of this directory, as you will need to move the files. Click Next. 4. Open the directory that contains the data conduit files. 5. Copy vigntest.exe, ntdivign.conf, and ntdivign.dll to the NetInsight program directory. The default directory is C:\Program Files\NetInsight. To install the data conduit on a UNIX system: 1. Unpack the archive into a temporary directory. gunzip ntdivign-{release Number}.tar.gz tar xvf ntdivign-{release Number}.tar 2. Change directories into the unpacked directory. The configuration file is in that directory, but vigntest and ntdivign.so are in a subdirectory specific for your platform. 3. Copy vigntest, ntdivign.conf, and ntdivign.so to the NetInsight program directory. The default directory is /usr/local/netinsight. Note: On HP-UX, make sure ntdivign.so is executable by the NetInsight user. Editing the Data Conduit s Configuration File The data conduit has a configuration file that tells it which CURLs to modify. The configuration file also contains the information that the data conduit needs to connect to the Vignette database. The configuration file, which is written in XML, is located in the same directory in which you installed the data conduit above. (The default directory is the NetInsight program directory, which by default is C:\Program Files\NetInsight for Windows and /usr/local/netinsight for UNIX.) The first element is <ntdivignette>. This is an overall container element. All other entries in the file must be after <ntdivignette> and before </ntdivignette>. This 3

6 element tells you that this is the configuration file for the Unica Web Analytics Data Conduit for Vignette. The following two lists contain all the elements in the configuration file. The first list contains the elements needed for creating action definitions. The second list contains miscellaneous elements. Action Definition Elements actions The <actions> element contains a list of action definitions. This element has no attributes and no value. Use the child element <action> to specify each action definition. The <action> element has four attributes and no value. Attribute Description Possible values Default decodeobjectids Base64 decode object IDs before processing true/false (inherited from global <decodeobjectids> fallbacktohttp splitchar* templates NetInsight should resolve page titles via HTTP for this template if the action fails Character to be used as a delimiter to split object IDs associated with this action A comma-delimited list of template IDs for which to perform this action true/false Any commadelimited list Or default+ element) (inherited from global <fallbacktohttp> element * If this attribute is not specified, object IDs associated with this action will not be split into multiple parts. + If default is used, the action will be performed for any template ID that has no action explicitly defined for it. <actions> <action templates= 1234,5678 > <query> SELECT Title, ISBN FROM BookTitles WHERE AuthorID = %TEMPLATEID% AND BookID = %OBJECTID% </query> <format>book: %VALUE1% (%VALUE2%)</format> <searchrules> <searchrule column= 2 replacewith= >^(ISBN:)</searchrule> </searchrules> 4

7 </action> </actions> format The <format> element contains strings to be used by the data conduit to format the value retrieved from the database. Within the value specified for this element, the following tokens will be recognized and replaced with the given value. Token Value %EXTENSION% File extension parsed from the CURL %OBJECTID% Object ID parsed from the CURL %OBJECTIDn% Part n of the object ID parsed from the CURL (n >=1). Use only if using the splitchar attribute of the <action> element %TEMPLATEID% Template ID parsed from the CURL %VALUEn% The nth value returned from the database by the query specified in the query element (n >=1) %VARIATION% Variation value parsed from the CURL The <format> element has no attributes and no child elements. It must be contained within the <action> element. <format>book: %VALUE1% (%VALUE2%)</format> query The <query> element contains the SQL query to be executed by the data conduit to retrieve information for this action from the database. Within the value specified for this element, the following tokens will be recognized and replaced with the given value. Token Value %EXTENSION% File extension parsed from the CURL %OBJECTID% Object ID parsed from the CURL %OBJECTIDn% Part n of the object ID parsed from the CURL (n >=1). Use only if using the splitchar attribute of the <action> element %TEMPLATEID% Template ID parsed from the CURL %VARIATION% Variation value parsed from the CURL The <query> element has no attributes and no child elements. It must be contained within the <action> element. 5

8 <query> SELECT Title, ISBN FROM BookTitles WHERE AuthorID = %TEMPLATEID% AND BookID = %OBJECTID% </query> searchrules The <searchrules> element contains a list of search rules. The <searchrules> element has no attributes and no value. Use the child element <searchrule> to define each search rule. The <searchrule> element has two attributes. Attribute Description Possible Values Default column Number of the column returned by Any number >=1 the query executed for this action to which this search rule should be applied replacewith String to be substituted for the portion of the value matching the regular expression specified in the searchrule element that is contained in parenthesis* Any string * If this attribute is blank or not specified, the portion of the value matching the regular expression that is contained in parenthesis will be removed. The <searchrule> element has no child elements. The <searchrules> element must be contained within the <action> element. <searchrules> <searchrule column= 2 replacewith= >^(ISBN:)</searchrule> </searchrules> Miscellaneous Elements curlregex The <curlregex> element specifies the regular expressions used to determine if a page is a CURL. The expression must contain four parenthesized subexpressions. The values inside the parentheses should be the four fields extracted from a CURL. The four values are extracted from the CURL in the following order: templateid, objectid, variation, extension. The <curlregex> element has no attributes and no child elements. For a brief tutorial on using regular expressions, see the NetInsight manual. <curlregex>\/\d,(\s+?),(\s*?),(\s+?)\.(\s*)</curlregex> 6

9 dbconnect The <dbconnect> element specifies the database connection string. For SQL Server databases, this should be a complete ODBC connect string. For Oracle databases, this should be a semi-colon-delimited string containing the username, password, and service name of the database to which to connect. This element has no attributes and no child elements. For Oracle: <dbconnect>vignuser;vignpw;vigndb</dbconnect> For SQL Server (all one line): <dbconnect>driver={sql Server};UID=vignuser;PWD=vignpwd; SERVER=vignserver;DATABASE=vigndb</dbconnect> decodeobjectids The <decodeobjectids/> element indicates that the data conduit should base64- decode all object IDs before attempting database lookups. This element can be overridden for individual actions. If the element is not present, the IDs will not be decoded. This element has no attributes, no child elements, and no value. <decodeobjectids/> fallbacktohttp The <fallbacktohttp/> element indicates that the data conduit should instruct NetInsight to resolve page titles via HTTP if the page titles cannot be resolved by the data conduit. This element can be overridden for individual actions. If the element is not present, NetInsight will not resolve page titles that cannot be resolved by the data conduit. This element has no attributes, no child elements, and no value. <fallbacktohttp/> logging The <logging> element specifies the full path and filename of the file to which performance and debug logging should be written. This element has one attribute. Attribute Description level The type of logging to be performed by the data conduit Possible Values Value Description No logging Performance logging only Debug logging only Performance and debug logging Default 0 The <logging> element has no child elements. 7

10 <logging level= 1 >c:\temp\ntdivign.log</logging> maxcached The <maxcached> element specifies the maximum number of lookups to be cached in memory. The minimum acceptable value for this element is 2. The default value is This element has no attributes and no child elements. <maxcached>15013</maxcached> nettrackerdir The <nettrackerdir> element specifies the location of the NetInsight installation that is using the data conduit. The default value is the current directory. This element has no attributes and no child elements. For Windows: <nettrackerdir>c:\program Files\NetInsight</nettrackerdir> For UNIX: <nettrackerdir>/usr/local/netinsight</nettrackerdir> unloadlibrary The <unloadlibrary/> element indicates that the data conduit should instruct NetInsight to manually unload it after processing is complete. If the element is missing, NetInsight will not manually unload the data conduit. This element has no value, no attributes, and no child elements. <unloadlibrary/> Testing the Configuration The Unica Web Analytics Data Conduit for Vignette Test Application enables you to ensure that you have the data conduit properly installed and configured before you use it on your NetInsight data. Since the data conduit can modify the NetInsight database, it is essential that you test the configuration before you use it with your data. The test application is a command-line program that loads the data conduit, calls the operation within the data conduit, displays any results or error messages, and unloads the data conduit. The test application is vigntest.exe (for Windows) or vigntest (for UNIX). Command-line Arguments -config <filename> 8

11 Specifies the filename of the configuration file to be passed to the data conduit. If this argument is not used, the test application uses the default value, which is ntdivign.conf. -input <filename> Specifies the name of the input file to be used by the test application. If this argument is not used, the test application uses the default value, which is ntdivign.in. -lib <filename> Specifies the full path and filename of the data conduit. If this argument is not specified, the test application uses the default value, which is./ntdivign.so (UNIX) or ntdivign.dll (Windows). -op <rewrite [title]> Indicates the function in the data conduit to be called by the test application. A value of rewrite will instruct the test application to rewrite pages. A value of title will instruct the test application to look up page titles. If this argument is not specified, the test app uses the default value, which is title. -output <filename> Specifies the name of the output file to which the test application will write the results returned by the data conduit. If this argument is not used, the test application will write all output to standard out. Input File Format The input file provided to the test application must contain a single page or CURL on each line. All lines in the input file will be read sequentially and individually passed to the data conduit as input. The page or CURL contained on each line should appear as it would in the request portion of a Web server log file, i.e. /path/to/file.html or /path/to/curl/0,12,34,00.html. Output File Format The output produced by the test application will contain the following three values per line, in a comma-quote delimited format: 1. The input page or CURL, as read from the input file 2. The output page or page title, as returned by the data conduit 3. The status code returned by the data conduit for the given page or CURL In addition to producing the output described above, the test application will write any errors returned by the data conduit to standard error. Telling NetInsight to Use the Data Conduit You tell NetInsight to use the data conduit by setting the Data Conduits option in the profile options for the profile with which you want to use the data conduit. 9

12 To tell NetInsight to use the data conduit: 1. View the profile with which you want to use the data conduit. 2. Click the Options tab. 3. In the left-hand panel, click Data Conduits under Dynamic Content. 4. Click Add. 5. In the Name of data conduit box, enter the name you want used for the data conduit in the list of data conduits. 6. In the Full path to conduit box, enter the full path (including the filename) to ntdivign.dll (Windows) or ntdivign.so (UNIX). 7. In the Full path to conduit config box, enter the full path (including the filename) to ntdivign.conf. 8. In the Conduit tasks box, select the task for which you are using the data conduit. Select Page Rewrite to modify the value of the CURL that is stored in the NetInsight database. Select Page Title Lookup to store a page title in the NetInsight database in addition to the CURL. The page title will be displayed in NetInsight reports instead of the CURL. 9. Click Save Options and then click Continue. If Vignette is using an Oracle database, you may need to tell NetInsight to set the required environment variables. You do this by specifying environment variable options in the administrative options in NetInsight. The options define the environment variables that should be set when NetInsight is started. If you are not using the Oracle Instant Client, you may need to set ORACLE_HOME and NLS_LANG. Note that if you are using the Unica Web Analytics Data Conduit for Vignette with NetInsight for Oracle, the ORACLE_HOME and NLS_LANG variables are already set for you. If you are using the Oracle Instant Client, you need to set the environment variable that applies to your operating system. Operating system Solaris HP/UX Linux AIX 5 Environment variable LD_LIBRARY_PATH SHLIB_PATH LD_LIBRARY_PATH LIBPATH On Windows systems, ensure that the Oracle Instant Client is in your path. To specify an environment variable: 1. In NetInsight, display the Profile Manager. 2. Click the Options tab. 3. In the left-hand panel, click Main and then click Environment Variables. 4. Click Add. 10

13 5. In the Name of variable box, enter the name of the environment variable you want to set. 6. In the Value of variable box, enter the value to which the environment variable should be set. 7. Click Save Options. Name of variable ORACLE_HOME NLS_LANG Value of variable /home/oracle/u01/app/oracle/product/ AMERICAN_AMERICA.WE8ISO8859P1 Sample Configuration For example, suppose your Vignette Content Management Server generates the following Custom URLs (CURLs): /samplesite/0,1000,1111,00.html /samplesite/0,1000,2222,00.html /samplesite/0,2000,mtexmq%3d%3d,00.html /samplesite/0,2000,mjiymg%3d%3d,00.html /samplesite/0,3000, ,00.html /samplesite/0,3000, ,00.html /samplesite/0,4000,mtiznc01njc4,00.html /samplesite/0,4000,mtexms0ymjiy,00.html /samplesite/0,13,2,00.html /samplesite/0,1111,12,00.html Suppose you also have a database table that stores the titles and ISBNs of books: 11

14 Books Title ISBN AuthorID Category1 Category2 Small Things ISBN: Butterfly Stories ISBN: Willows ISBN: Wild Flowers ISBN: Animal Kingdom ISBN: Wild Animals ISBN: Beautiful Skies ISBN: Little Worries ISBN: Sea Creatures ISBN: Scorpions ISBN: To store the book titles and ISBNs in the NetInsight database in addition to the CURLs, the configuration file for the Unica Web Analytics Data Conduit for Vignette should look like the following: <?xml version="1.0" encoding="iso "?> <ntdivignette> <dbconnect>user;pass;service</dbconnect> <curlregex>\/\d,(\s*?),(\s*?),(\s+?)\.(\s*)</curlregex> <logging level="3">ntdivign.log</logging> <maxcached>15013</maxcached> <nettrackerdir>.</nettrackerdir> <actions> <action templates="1000" decodeobjectids="false"> <query> SELECT Title, ISBN FROM Books WHERE AuthorID = %TEMPLATEID% AND Category1 = %OBJECTID% </query> <format>book: %VALUE1% (%VALUE2%)</format> <searchrules> <searchrule column="2" replacewith="">^(isbn: )</searchrule> </searchrules> </action> <action templates="2000" decodeobjectids="true"> <query> SELECT Title, ISBN FROM Books WHERE AuthorID = %TEMPLATEID% AND Category1 = %OBJECTID% </query> <format>book: %VALUE1% (%VALUE2%)</format> </action> <action templates="3000" splitchar="-" decodeobjectids="false"> <query> SELECT Title, ISBN FROM Books WHERE 12

15 AuthorID = %TEMPLATEID% AND Category1 = %OBJECTID1% AND Category2 = %OBJECTID2% </query> <format>book: %VALUE1% (%VALUE2%)</format> </action> <action templates="4000" splitchar="-" decodeobjectids="true"> <query> SELECT Title, ISBN FROM Books WHERE AuthorID = %TEMPLATEID% AND Category1 = %OBJECTID1% AND Category2 = %OBJECTID2% </query> <format>book: %VALUE1% (%VALUE2%)</format> </action> <action templates="default" fallbacktohttp="true"> <query> SELECT Title FROM Books WHERE AuthorID = %TEMPLATEID% </query> <format>first Book: %VALUE1%</format> </action> </actions> </ntdivignette> You would then add the following data conduit in the Data Conduits option for the NetInsight profile to tell NetInsight to use the data conduit to store page titles: Name of data conduit: Vignette Page Titles Full path to data conduit: C:\Program Files\NetInsight\ntdivign.dll Full path to conduit config: C:\Program Files\NetInsight\ntdivign.conf Conduit tasks: Page Title Lookup This would result in the following output from the test application: "/samplesite/0,1000,1111,00.html","book: Small Things (ISBN: )","MATCH" "/samplesite/0,1000,2222,00.html","book: Butterfly Stories (ISBN: )","MATCH" "/samplesite/0,2000,mtexmq%3d%3d,00.html","book: Willows (ISBN: )","MATCH" "/samplesite/0,2000,mjiymg%3d%3d,00.html","book: Wild Flowers (ISBN: )","MATCH" "/samplesite/0,3000, ,00.html","book: Animal Kingdom (ISBN: )","MATCH" "/samplesite/0,3000, ,00.html","book: Wild Animals (ISBN: )","MATCH" "/samplesite/0,4000,mtiznc01njc4,00.html","book: Beautiful Skies (ISBN: ISBN: )","MATCH" "/samplesite/0,4000,mtexms0ymjiy,00.html","book: Little Worries (ISBN: ISBN: )","MATCH" "/samplesite/0,13,2,00.html","first Book: Sea Creatures","MATCH" "/samplesite/0,1111,12,00.html","first Book: Scorpions","MATCH" The actual NetInsight reports would list the following in the Page column: 13

16 Book: Small Things (ISBN: ) Book: Butterfly Stories (ISBN: ) Book: Willows (ISBN: ) Book: Wild Flowers (ISBN: ) Book: Animal Kingdom (ISBN: ) Book: Wild Animals (ISBN: ) Book: Beautiful Skies (ISBN: ) Book: Little Worries (ISBN: ) First Book: Sea Creatures First Book: Scorpions 14