TRANSCODER SDK CERTIFICATION PROCESS VENDORS Instructions to Validate the Correct Implementation of the Nielsen ID3 Software Development Kit
Page 2 Copyright 2016 The Nielsen Company (US) LLC. All rights reserved. Nielsen and the Nielsen Logo are trademarks or registered trademarks of CZT/ACN Trademarks, L.L.C. Other company names, products and services may be trademarks or registered trademarks of their respective companies. This documentation contains the intellectual property and proprietary information of The Nielsen Company (US) LLC. Publication, disclosure, copying, or distribution of this document or any of its contents is prohibited. Revision History Revision Date Description A 2015-09-25 Initial version B 2015-11-06 Revised forms C 2016-03-03 Clarified content to be sent to Nielsen
Page 3 Contents Purpose... 4 Terminology... 4 Related Documents... 4 Nielsen PCM-to-ID3 Overview... 4 Licensing Agreement Requirements for Validation... 5 Required Items... 5 Required Tasks... 5 Validation Tests... 6 Approach... 6 Scope... 6 Validation Elements... 7 Content You Send Nielsen... 7 Validation Test #1 Normal Content... 9 Summary... 9 Execution Steps... 9 Validation Test #2 Boundary Content... 10 Summary... 10 Execution Steps... 10 Validation Test #3 Stress Content... 11 Summary... 11 Execution Steps... 11 Validation Test #4 120 Minutes of Live Content... 12 Summary... 12 Execution Steps... 12 Appendix A Audio Profile Test Input and Output Forms... 13 Live Audio Test Input... 13 Audio Profile Output... 14 Appendix B Inventory of Content Provided to Nielsen... 16
Page 4 Purpose Terminology The purpose of this certification process is to create transcoder output data that allows Nielsen to validate the proper implementation of the Nielsen PCM-to-ID3 software development kit (SDK) into the code base of a vendor s transcoder application. This certification process requires collaborative efforts between you and Nielsen to be successful. If you have questions, errors, or challenges at any point, do not hesitate to contact your Nielsen Engineering representative. This document uses the following terms: Device refers to your equipment and/or software that is being tested for certification as a Nielsen ID3 Tag Inserter. You refers to the vendor. Nielsen audio watermark refers to Nielsen Audio Encoding System 2 (NAES 2) audio code and Nielsen Watermark audio code. Watermark, as a noun, refers to these same audio codes. Watermark, as a verb, refers to the insertion of Nielsen audio watermarks into content. Related Documents For more information on the integration of the Nielsen PCM-to-ID3 SDK into your software and for a list of the breakout codes, see the Nielsen PCM-to-ID3 SDK Developer s Guide dated 2015-01-26 or later. For instructions on using the Nielsen ID3 Tag Validator and analyzing the output, see the Nielsen ID3 Tag Validator User Guide. If you need a copy of either of these documents, contact your Nielsen Engineering representative to request a copy. Nielsen PCM-to-ID3 Overview The Nielsen mobile and computer television audience measurement service is based on the identification of Nielsen audio watermarks in the broadcast content. For the watermark to pass through a mobile device or a computer to the Nielsen data collection system, the watermark must be packaged into an ID3 metadata container stored in the broadcast content. During the transcoding of the content stream, the PCM-to-ID3 SDK translates the watermark into an ID3v2 tag. A vendor integrates the SDK into their transcoder application code base and then collaborates with the Nielsen Engineering team to validate the SDK integration through the transport stream.
Page 5 Licensing Agreement Requirements for Validation Required Items To perform the tests, you must have the following from Nielsen: Required Tasks Nielsen ID3 SDK Validator application (command-line tool): You need this application to decode the transport stream output of your device and send the decoding results to Nielsen for the validation process. If you do not already have the Nielsen ID3 SDK Validator tool, contact your Nielsen Engineering representative to obtain it. Test files: VendorCert-NORMAL-v01.ts file, VendorCert-BOUNDARY-v01.ts file, and VendorCert-STRESS-v01.ts file As part of the agreement for integrating the Nielsen PCM-to-ID3 SDK, you must do all of the following: 1. Verify you have the latest versions of the ID3 SDK Validator and PCM-to-ID3 SDK a. Make a note of the versions you have on your hard drive. b. Go to the website at https://engineeringforum.nielsen.com/encoders/id3.php. c. Compare your version against the list of versions on the right side of the page. d. If you do not have the latest version, click any of the links and complete the request. Nielsen approves your request and sends you an email with a link to the download page. Important Your email application may send the email from encodersuppoort@nielsenengineering.com to your Junk or Spam folder so check there if you do not see the email within 24 hours. We recommend that you add encodersuppoort@nielsenengineering.com to the contact list in your email application. e. Click the link in the email from Nielsen Engineering. 2. Certify the device before the product can be deployed with the Nielsen ID3 Tag Inserter feature enabled for commercial or production use. 3. Use your device or system to execute the tests outlined in this document, and send the output of the tests to Nielsen for verification. 4. To help ensure you cover all of the audio profiles that your device supports, complete the forms in Appendix A Audio Profile Test Input and Output and attach them to an email to your Nielsen Engineering representative. 5. Run the four required tests outlined in this document for a single audio profile that your device supports.
Page 6 Validation Tests Important For help with test device configurations, the execution of the validation tests, or questions about ID3 Tag Validator output, contact your Nielsen Engineering Representative for assistance. Approach Scope The approach for these procedures uses the black-box software quality assurance methodology. Our process examines the functionality of your device as it applies to Nielsen ID3 data without understanding internal code structures or functionality of your device. You feed to your device known, measurable content that has Nielsen audio watermarks embedded in the broadcast audio. Nielsen provides you with test files that you pass through the Nielsen PCM-toID3 SDK software installed on your device. Once the Nielsen-supplied content is transcoded and tagged, you capture the transportstream output and/or the HLS output of your device. You may use the Nielsen ID3 Tag Validator command-line diagnostic tool to decode the transport-stream output. You send the captured transport stream and/or the zipped HLS content to Nielsen Engineering, where we compare your results to a set of expected results. You also feed to your device live broadcast content that has Nielsen audio watermarks embedded in the audio. You may select any live broadcast feed from a national cable network as your content. Once the live content is transcoded and tagged, you capture the transport-stream output and/or the HLS output of your device. You may use the Nielsen ID3 Tag Validator command-line diagnostic tool to decode the transport-stream output. You send the captured transport stream and/or the zipped HLS output to Nielsen Engineering. 1. All tests are required, and the results of the tests must be validated before you deploy your device with the Nielsen ID3 Tag Inserter feature enabled. 2. Except for your live content testing, tests are limited to the test files developed by Nielsen Engineering, available to you through your Nielsen Engineering representative. 3. Except for your live content testing, all test files must include the following three input types. All devices must accept these input types and produce output content that facilitates accurate audience measurement by Nielsen. Normal condition content that represents programming that is correctly Nielsenwatermarked (VendorCert-NORMAL-v01.ts file) Boundary condition content includes pre-existing Nielsen ID3 tags that your software must remove and replace with newly generated tags. (VendorCert-BOUNDARYv01.ts file) Stress condition content that represents programming that is incorrectly Nielsenwatermarked. The primary audio stream in this content switches periodically from stereo to 5.1-channel to stereo (VendorCert-STRESS-v01.ts file)
Page 7 Validation Elements Nielsen analyzes the transport stream and/or HLS output of your device for the elements listed below. For more information about these elements, refer to the guides listed in the Related Documents. PCM audio channel selection AC-3 channel switching Decoder fidelity ID3 Tag cadence, consistency, synchronization, and overwrite Nielsen watermarks fidelity within the ID3 Tag Non-Nielsen-watermarked ID3 Tag fidelity within the ID3 Tag Nielsen live/video-on-demand flag fidelity within the ID3 Tag Nielsen InfoTag content within the ID3 Tag Nielsen ad-supported breakout code fidelity within the ID3 Tag Note For a list of the breakout codes, see the Nielsen PCM-to-ID3 SDK Developer s Guide dated 2015-01-26 or later. HLS output profiles Standard interfaces capabilities Error logging Content You Send Nielsen 1. For each test that you run, Nielsen requires that you provide the following for analysis: Tagged output: for Tests 1, 2, and 3, the tagged output must represent the same content as the original Nielsen test stream. It is acceptable for the content to include a pre-roll, trailing data, or both, but the original Nielsen test content must be complete and contiguous. For Tests 1, 2, 3, and 4, the output must be in one or both of the following formats, depending on the capabilities of your system: Captured transport stream Captured HLS output, zipped Screenshots showing: Note Your user interface with any user-configurable features that relate to the generation of Nielsen ID3 tags. For example, if several breakout codes appear on a drop-down menu, capture the screen with the menu open. The product name and version in a screenshot, if applicable. Name the screenshot file to align with its contents. Complete instructions on how to access the output and screenshots.
Page 8 2. Complete the following 3 forms and attach them to an email. Send the email to your Nielsen Engineering representative. If you provide these forms before you begin the test process, Nielsen will confirm that all required tests are represented. You only need to send these forms one time. Two forms in Appendix A Audio Profile Test Input and Output Forms. One form in Appendix B Inventory of Content Provided
Page 9 Validation Test #1 Normal Content Summary Execution Steps The purpose of this test is to validate the ability of the device to generate Nielsen ID3 Tags during the transcoding of normal condition broadcast content (correctly Nielsen audio watermarked). 1. Go to the website at https://engineeringforum.nielsen.com/encoders/id3.php. 2. Download the 3 test files. 3. Configure your device for the upcoming test. Make certain that all configurable fields are set. Make note of the breakout code that you have selected. Note For a list of the breakout codes, see the Nielsen PCM-to-ID3 SDK Developer s Guide dated 2015-01-26 or later. 4. Stream the content in VendorCert-NORMAL-v01.ts to your device in a way that simulates the method and rate of content delivery when your device is installed in the field. 5. Into a file, capture the transport-stream output of your device after the Nielsen ID3 tags have been multiplexed into the transport stream and prior to HLS segmentation. 6. Use the Nielsen ID3 Tag Validator application output to analyze the ID3 transport stream. Correct any errors that you find. Important For help with test device configurations, the execution of the validation tests, or questions about ID3 Tag Validator output, contact your Nielsen Engineering Representative for assistance. See the Nielsen ID3 Tag Validator Application User Guide for information on analysis. 7. Capture the m3u8 files and HLS transport-stream segments from your device and zip these into a file. 8. Send the items listed in Content You Send Nielsen on page 7 to Nielsen for evaluation and certification:
Page 10 Validation Test #2 Boundary Content Summary Execution Steps The purpose of this test is to validate the ability of the device to generate Nielsen ID3 Tags during the transcoding of challenged Nielsen audio-watermarked broadcast content 1. If you have not already done so, do the following: a. Go to the website at https://engineeringforum.nielsen.com/encoders/id3.php. b. Download the 3 test files. 2. Configure your device for the upcoming test. Make certain that all configurable fields are set. Select a breakout code setting that you have not used in a previous test. Note For a list of the breakout codes, see the Nielsen PCM-to-ID3 SDK Developer s Guide dated 2015-01-26 or later. Make note of the breakout code that you have selected. 3. Stream the supplied Nielsen content in VendorCert-BOUNDARY-v01.ts to your device in a way that simulates the method and rate of content-delivery when your device is installed in the field. 4. Into a file, capture the transport-stream output of your device after the Nielsen ID3 tags have been multiplexed into the transport stream and prior to HLS segmentation. 5. Use the Nielsen ID3 Tag Validator application output to analyze the ID3 transport stream. Correct any errors that you find. For this test, do not be alarmed if the Validator CID SID test reports Fail. Important For help with test device configurations, the execution of the validation tests, or questions about ID3 Tag Validator output, contact your Nielsen Engineering Representative for assistance. See the Nielsen ID3 Tag Validator Application User Guide for information on analysis. 6. Capture the m3u8 files and HLS transport-stream segments from your device and zip these into a file. 7. Send the items listed in Content You Send Nielsen on page 7 to Nielsen for evaluation and certification:
Page 11 Note In the Microsoft Windows operating system, open the drop-down menu in your transcoder interface, press <Print Screen>, and paste the screenshot into Paint. In an ios operating system, open the drop-down menu, press <Cmd + Shift + 4>, which automatically saves the screenshot to a file in your default location. Validation Test #3 Stress Content Summary Execution Steps The purpose of this test is to validate the ability of the device to generate Nielsen ID3 Tags during the transcoding of stressed broadcast content (incorrectly Nielsen-watermarked content where the audio switches periodically from stereo to 5.1 to stereo). 1. If you have not already done so, do the following: a. Go to the website at https://engineeringforum.nielsen.com/encoders/id3.php. b. Download the 3 test files. 2. Configure your device for the upcoming test. Make certain that all configurable fields are set. Select a breakout code setting that you have not used in a previous test. Note For a list of the breakout codes, see the Nielsen PCM-to-ID3 SDK Developer s Guide dated 2015-01-26 or later. Make note of the breakout code that you have selected. 3. Stream the supplied Nielsen content VendorCert-STRESS-v01.ts to the device in a way that simulates the method and rate of content-delivery when your device is installed in the field. 4. Into a file, capture the transport-stream output of your device after the Nielsen ID3 tags have been multiplexed into the transport stream and prior to HLS segmentation. 5. Use the Nielsen ID3 Tag Validator application output to analyze the ID3 transport stream. Correct any errors that you find. For this test, do not be alarmed if the Watermark Timecode Test reports Warn. Important For help with test device configurations, the execution of the validation tests, or questions about ID3 Tag Validator output, contact your Nielsen Engineering Representative for assistance. See the Nielsen ID3 Tag Validator Application User Guide for information on analysis. 6. Capture the m3u8 files and HLS transport-stream segments from your device and zip these into a file. 7. Send the items listed in Content You Send Nielsen on page 7 to Nielsen for evaluation and certification:
Page 12 Validation Test #4 120 Minutes of Live Content Summary Execution Steps The purpose of this test is to validate the ability of the device to generate Nielsen ID3 Tags during the transcoding of live Nielsen Audio Watermarked broadcast content. 1. Configure your device for the upcoming test. Make certain that all configurable fields are set. Select a breakout code setting that you have not used in a previous test. Make note of the breakout code that you have selected. Note For a list of the breakout codes, see the Nielsen PCM-to-ID3 SDK Developer s Guide dated 2015-01-26 or later. 2. Stream the live broadcast content the device in a way that simulates the method and rate of content-delivery when your device is installed in the field. Please note the national broadcast network that was used and the date and time of the live broadcast and include it in the information you send to Nielsen. 3. Allow the device to run in normal operation for a minimum of 120 minutes. 4. After 120 minutes of normal operation, into a file, capture 15 minutes of the transportstream output of your device after the Nielsen ID3 tags have been multiplexed into the transport stream and prior to HLS segmentation. 5. Use the Nielsen ID3 Tag Validator application output to analyze the ID3 transport stream. Correct any errors that you find. Important For help with test device configurations, the execution of the validation tests, or questions about ID3 Tag Validator output, contact your Nielsen Engineering Representative for assistance. See the Nielsen ID3 Tag Validator Application User Guide for information on analysis. 6. Capture the m3u8 files and HLS transport-stream segments from your device and zip these into a file. 7. Send the items listed in Content You Send Nielsen on page 7 to Nielsen for evaluation and certification.
Page 13 Appendix A Audio Profile Test Input and Output Forms To help ensure you cover all of the audio profiles that your device supports, complete the forms below and attach them to an email to your Nielsen Engineering contact. The three test files (Validation Tests #1, 2, and 3) are MPEG-2 transport streams with one or two tracks of AC-3 audio. Each track alternates periodically between stereo and 5.1 channels. In processing this content, your device must correctly handle the switch between stereo and 5.1. Where applicable, your device must select only the primary audio stream for processing. Live Audio Test Input For the transport stream that you deliver for the live test, select the highest bitrate audio output that your device supports. For this test only, Dolby AC-3 stereo is preferred. For the live test, the Nielsen analysis is best performed on audio that has not been highly compressed because Nielsen uses this test to evaluate the synchronization of ID3 tags to the corresponding audio. For live content (Validation Test #4), check the box for the audio input that represents the input that your device typically processes in the field. Table 1 Live Audio Input (Validation Test #4) Form Content Bitrate Input MPEG-1 Layer II Stereo (2.0) 32 384 Kbps AAC Stereo (2.0) 32 384 Kbps AAC Multichannel (5.1) 224 640 Kbps HE-AAC v1 Stereo (2.0) 32 128 Kbps HE-AAC v1 Multichannel (5.1) 96 192 Kbps HE-AAC v2 Stereo (2.0) 32 64 Kbps AC-3 Stereo (2.0) 96 640 Kbps AC-3 Multichannel (5.1) 32 640 Kbps E-AC-3 Stereo/Multichannel 32 1024 Kbps
Audio Profile Output For the output of each validation test, select one audio profile. Be sure to select profiles that best represent configurations for your device in the field. Table 2 Audio Profile Outputs Validation Test Stream Audio Profile Output #1 Transport Select one: MPEG-1 Layer II Stereo (2.0) AAC Stereo (2.0) AAC Multichannel (5.1) HE-AAC v1 Stereo (2.0) HE-AAC v1 Multichannel (5.1) HE-AAC v2 Stereo (2.0) AC-3 Stereo (2.0) AC-3 Multichannel (5.1) E-AC-3 Stereo / Multichannel HLS Select one: MPEG-1 Layer II Stereo (2.0) AAC Stereo (2.0) AAC Multichannel (5.1) HE-AAC v1 Stereo (2.0) HE-AAC v1 Multichannel (5.1) HE-AAC v2 Stereo (2.0) AC-3 Stereo (2.0) AC-3 Multichannel (5.1) E-AC-3 Stereo / Multichannel #2 Transport Select one: MPEG-1 Layer II Stereo (2.0) AAC Stereo (2.0) AAC Multichannel (5.1) HE-AAC v1 Stereo (2.0) HE-AAC v1 Multichannel (5.1) HE-AAC v2 Stereo (2.0) AC-3 Stereo (2.0) AC-3 Multichannel (5.1) E-AC-3 Stereo / Multichannel HLS Select one: MPEG-1 Layer II Stereo (2.0) AAC Stereo (2.0) AAC Multichannel (5.1) HE-AAC v1 Stereo (2.0) HE-AAC v1 Multichannel (5.1) HE-AAC v2 Stereo (2.0) AC-3 Stereo (2.0) AC-3 Multichannel (5.1) E-AC-3 Stereo / Multichannel
Validation Test Stream Audio Profile Output #3 Transport Select one: MPEG-1 Layer II Stereo (2.0) AAC Stereo (2.0) AAC Multichannel (5.1) HE-AAC v1 Stereo (2.0) HE-AAC v1 Multichannel (5.1) HE-AAC v2 Stereo (2.0) AC-3 Stereo (2.0) AC-3 Multichannel (5.1) E-AC-3 Stereo / Multichannel HLS Select one: MPEG-1 Layer II Stereo (2.0) AAC Stereo (2.0) AAC Multichannel (5.1) HE-AAC v1 Stereo (2.0) HE-AAC v1 Multichannel (5.1) HE-AAC v2 Stereo (2.0) AC-3 Stereo (2.0) AC-3 Multichannel (5.1) E-AC-3 Stereo / Multichannel #4 Transport AC-3 Stereo 192 Kbps or higher HLS Select one: MPEG-1 Layer II Stereo (2.0) AAC Stereo (2.0) AAC Multichannel (5.1) HE-AAC v1 Stereo (2.0) HE-AAC v1 Multichannel (5.1) HE-AAC v2 Stereo (2.0) AC-3 Stereo (2.0) AC-3 Multichannel (5.1) E-AC-3 Stereo / Multichannel
Page 16 Appendix B Inventory of Content Provided to Nielsen 1. Complete Table 3 to identify the pieces of content that you are providing to Nielsen. In the Test File column, indicate whether the test file is NORMAL, BOUNDARY, STESS, or LIVE. If, for some reason, you have run multiple tests using one of the test files, distinguish the file names so we know why you ran multiple tests with the same input. Note For a list of the breakout codes, see the Nielsen PCM-to-ID3 SDK Developer s Guide dated 2015-01-26 or later. 2. If you have not already done so, include your Audio Profile Test Input and Output Forms from Appendix A. 3. Send proof that you are logging errors. Nielsen requires this so we can assist you with troubleshooting. 4. Send these items in an email to your Nielsen Engineering representative. Table 3 Content Sent to Nielsen Test File Full Path and File Name Breakout Code Other Pertinent Configuration Parameters