Release Notes for Cisco TV CDS 2.5.3

Transcription

1 These release notes cover Cisco TV CDS Release Revised: January 2012 Contents The following information is in the release notes: Enhancements, page 1 Supported Environments, page 2 System Requirements, page 2 Open Caveats, page 3 Resolved Caveats, page 8 Upgrading to Release 2.5.3, page 10 Related Documentation, page 13 Obtaining Documentation and Submitting a Service Request, page 13 Enhancements The following enhancements have been added in Release 2.5.3: Added asset filename (mname) to archived stream reports (Report > Archived Data > Stream Reports). (CSCtn11549) In RTSP environment, support more than four playlist items if master Streamer fails over. (CSCtu34010) Americas Headquarters: Cisco Systems, Inc., 170 West Tasman Drive, San Jose, CA USA 2012 Cisco Systems, Inc. All rights reserved.

2 Supported Environments Supported Environments Release of the Cisco TV CDS supports both the ISA and RTSP environments. Some RTSP environment features are only applicable to certain RTSP Deployment Types. System Requirements The Cisco TV CDS Release runs on the CDE110, CDE220, CDE250, and CDE420. See the Cisco Content Delivery Engine110 Hardware Installation Guide, and the Cisco Content Delivery Engine 205/220/250/420 Hardware Installation Guide. The Cisco TV CDS Release also runs on the CDE100, CDE200, CDE300, and CDE400 hardware models that use the Lindenhurst chipset. See the Cisco Content Delivery Engine CDE100/200/300/400 Hardware Installation Guide for set up and installation procedures. Release does not support the CDEs with the ServerWorks chipset. All CDEs with the ServerWorks chipset need to be replaced with the CDEs with the Lindenhurst chipset or the Next Generation Appliances (CDE110, CDE220, CDE250, andcde420) before upgrading to Release Note If you are using Internet Explorer 8 to access the CDSM GUI, for the online help to display correctly, you need to turn on Compatibility View. To turn on Compatibility View in Internet Explorer 8, choose Tools > Compatibility View. Limitations and Restrictions This release contains the following limitations and restrictions: RTSP Redirect Server Limitations The TV CDS Redirect Server provides a central entry-point for stream requests, which are then redirected based on internal load balancing policies to other Setup servers in the CDS deployment for session processing. The load balancing policies require that the CDS server running the Redirect Server have access to the database of each potential Setup server. The policies use key pieces of information from the database, such as the RTSP server status, the server load, and the Setup IP address to management IP address mappings. If the Redirect Server does not have access to this data, it cannot properly redirect the setup request to the "best" Setup server. In the case of multiple Stream Groups, with the Redirect Server configured to run on Streamers within one of the Stream Groups, it is required that the database replication partners for the Streamers contain all of the Streamers from all Stream Groups, which basically constitutes a full-mesh database replication approach among all of the Streamers If a full-mesh database replication for the Streamers is not desired, the Redirect Server can operate on the Vaults, which already have the necessary database replication partners defined to support required data. 2

3 Open Caveats Open Caveats This release contains the following open caveats: CServer CSCtx54504 Server may errantly (and very temporarily) report system pressure. When the system is running at full capacity and resources are very low. CSCtt22964 Transmit unit hang detected on e1000 interfaces during a server bring-up soon after upgrading from Release Happens only after upgrading the software from Release during server bring-up on servers with e1000 adapters in quad formation and specific adapters on that quad configuration. Rebooting a second time fixes the issue. This is a know issue with Intel drivers. Following is the excerpt from the driver README: Detected Tx Unit Hang in Quad Port Adapters In some cases ports 3 and 4 don't pass traffic and report 'Detected Tx Unit Hang' followed by 'NETDEV WATCHDOG: ethx: transmit timed out' errors. Ports 1 and 2 don't show any errors and will pass traffic. This issue MAY be resolved by updating to the latest kernel and BIOS. The user is encouraged to run an OS that fully supports MSI interrupts. CSCtr46736 Primary-backup status conflict in some servers. Network partition. None. CSCtt02130 Server goes into kernel debugger (KDB) mode after a service network restart is issued or an individual adapter is restarted. It only happens when the adapters are shared with the Linux operating system. 3

4 Open Caveats None. CSCtr15443 Evaluators shown as continuously active; they never seem to complete. A disk drive that is bad or failing, yet not sufficiently bad to be declared sick by the disk driver continues to go in and out of repair mode, causing the evaluators to constantly restart and inhibiting the evaluators from completing. In addition, the evaluators need to have a significant amount of local work that needs to be done (such as defragging or smoothing). If the amount of work is small, this symptom may not be exhibited because the evaluators are able to complete the task during the small windows of time that the disk drive is not in repair mode. Remove and replace the bad disk drive. CSCtu19076 System KDB may occur. Removing a front-mounted device before the logical preparations for removal are complete. Before removing a functioning storage device from the front of the server, ensure that it has first been logically removed by echoing the device name to /proc/cds/cdd/remove_device. For example, echo csd3 > /proc/cds/cdd/remove_device prepares the device csd3 for removal from the server. Ensure that the message csdx can now be safely removed (where X is the device number) appears on the console before pulling the device. Also, the red LED on the device will be flashing after it has been prepared to be removed. Following these recommendations will avoid a possible system KDB event. CSCts35941 During a failure of the Streamer hosting the primary Control server, occasionally part of the playing streams could be lost. This loss happens because of communication problems between the Streamers, causing confusion over which Streamer was the primary Control server, and also in some cases preventing the backup server from receiving the current state for all playing streams. Loss of streams during a primary Control server failure are most likely to occur when a new Streamer has been brought online shortly before the primary Control server failed. Code that should have been sending switch forwarding probes on the ports for the new server to determine if the router forwarding table was sending and receiving traffic on the ports, before using the ports for live video and control traffic, was incorrectly making the ports available for use before the switch forwarding table was properly setup for sending and receiving traffic on these ports. This caused a very high percentage packet loss for communication traffic to the new server. The new server could then mistakenly believe there were no other primary Control server and could temporarily take over the 4

5 Open Caveats role of primary Control server with no stream state. While in this mode, the new Streamer would issue commands to tear down all the streams that were playing on Streamer that it could communicate with. A second problem could occur when the new Streamer did see the current primary server and the new server was the second Streamer to come up. There is a time window, during which the new Streamer has not finished bringing all of its ports online and had only one or two active ports, where the primary server could decide to use this new Streamer as the backup server. The primary would then send the new Streamer stream state information in parallel from multiple ports on the primary server exceeding the capacity to deliver the information being sent from many ports to the one or two active ports on the receiving Streamer. This would result in a high level of packet loss, and retransmitting the packets would continue to fail until the Streamers declared each other unreachable. The backup server would then see the primary as unreachable, but would only have state for part of the playing streams. If the primary server really failed so that the primary control IP address was no longer reachable over the management network, the backup server could become primary without having state for many of the stream. These problems are much more likely to occur when the switch being used has a significant delay when a new port comes online, before the switch forwarding table is setup to forward traffic to and from the new port. The problem is also dependent on the timing between when a new Streamer comes online and when the existing primary server fails. When possible, avoid shutting down the primary server if another Streamer has just come online. Wait for a couple of minutes after bringing the new Streamer online to make sure that it has had time for the switch to enable communication on all of the ports. Some switches have configuration settings that can be modified to reduce the time delay between when a new port first detects link state and when the switch sets up the forwarding table information to send and receive traffic on the port. Reducing this time delay will help minimize the window in which this problem can occur. CSCtt22591 Usually the first symptom is the presence of "WARNING: Stream transmit..." errors in the protocol timing log. This indicates that the LAN ports are getting behind. Next some streams are dropped to reduce the load on the server, and some streams may end up getting changed to a NULL state see the Nulls: part of the Active streams: lines in the protocol timing log to track this. This appears to happen when the LAN channel has a total combined bandwidth (transmit and receive) of 41 to 46 Gbps. There currently is no workaround other than to avoid having the server support both a maximum fill-bandwidth load and a maximum transmit-bandwidth load at the same time. It takes a well balanced load to do this; usually a server reaches a maximum on one or the other instead of both at the same time; so it is not likely you will actually encounter this problem. 5

6 Open Caveats Release has the following LAN transmit and receive maximum limits: Maximum transmit-bandwidth for a server with 4 10-gigabit Ethernet adapters is around 37.5 Gbps. Maximum fill receive-bandwidth for a server with 4 10-gigabit Ethernet adapters is around 9.45 Gbps. CSCtn28329 In the CDSM (or VVIM) GUI, on the Server Setup page (Configure > Server > Server Setup), when specifying an interface for the FTP Out Interface field under the FTP Out Setting section, the configuration setting does not take affect and the specified interface is not used for FTP out operations. Using the CDSM (or VVIM) GUI, specify an FTP out interface and submit the Server Setup page. The setupfile on the server will be updated to reflect this configuration, as in following example: ftpout if eth0 max utilization mbps 0 max sessions 0 The IP address of the interface to be used in FTP out operations can be specified on the ftp command line. Configuration CSCti78219 Not all packages are found in the drop-down list on the Monitor > System Level > Package Monitor page. When the number of packages in the system reaches a level greater than 70,000. If the number of packages in the system grows much beyond 70,000, then the drop-down list may not be able to display them all. The solution is to delete packages that are no longer needed. The maximum number of packages that will display in the drop-down list is variable, and is dependent on the system that is accessing the web page. 6

7 Open Caveats FSI CSCtt46144 For a duplicated live future request, the state field in response XML body is empty instead of having the correct state of the scheduled live recording. The client sends a request to schedule a live content with the same assetid, providerid, capturestart and captureend as an already scheduled future recording on the same channel. None. Monitoring CSCtu23071 The following symptoms have been observed: 1. Monitor > Server Level > Disk Monitor shows a different failed disk than the Disk Activity graph. 2. When using some web browsers, the Disk Activity graph may reload repeatedly. 3. The Disk Activity Graph may show zero read/write activity if the disk activity is greater than100 Mbps. Any CDE250. For item one, there is no workaround. For item 2, using an older browser (IE6) should allow the user to see the Disk Activity graph. For item 3, when viewing the graph, the user can click on the icon to switch between the graph and a table display of the values to see if any disks are working at greater than 100 Mbps. Statsd CSCto43335 Caching Node remote servers file contains all servers from old group. Moving Caching Node from group A to group B. Restart statsd on the Caching Node that was moved and log in to the CDSM GUI and submit the Server Setup page for that Caching Node. 7

8 Resolved Caveats Resolved Caveats This section contains he resolved caveats in Cisco TV CDS Release Not all resolved issues are mentioned here. The following list highlights resolved caveats associated with customer deployment scenarios. CServer CSCtx27620 In an group of streams, nearly all the streams get assigned to be streamed on one Play server. When a playlist is used and there are very short bumpers or advertisements put at the beginning of the playlist and before the main feature; the algorithm to assign where a playlist play uses the first element in the playlist. The result of this is that with the short bumpers or advertisements, the whole playlist is assigned to the same Streamer. This causes an imbalance of the assignment of the streams. CSCtw89998 The following assert is seen: CALYPSO ASSERT: "!getdisklookahead()->cangroupwithinreadaheadlimit(earlier AdjacentDiskIo, diskio)" failed in file"/build/cdsbuild/workspace/cdstv_release_builds/ cdstv_2.4.2-b39/cds/cserver/lom/lookaheadelevator.cpp", line 64 Entering kdb current=0xffff8100a6ec1040, pid 4647) on processor 2 due to KDB_ENTER() If the tunable echo x > /proc/calypso/internal/restart_adapter_index is executed with an adapter number (the value located in x) outside the bounds of the legal adapters in the system, it causes a CServer crash. CSCtu45475 Both primary and backup Streamers crash simultaneously in selectstreamplayer(). The Setup server is sending three successive play requests just as the first segment of a multi-segment play list is about to end (within the last two seconds of the end). See the description about replicating this issue below. Replicating the issue: 1. Start a stream with a playlist with at least two segments. 2. Send three play requests successively that are timed to play just as the first segment is about to complete and while it is preparing to transition to the next play list segment. 3. The first play request for 1X should be sent about 1.3 seconds before the segment transition (before the end of the first segment). 4. Last two requests should be sent back-to-back ms after the first request is processed and the play-response is sent back to the application. These last two requests are queued and delayed to run 500 ms after the first request, placing them within in the last 800ms of the first segment. 8

9 Resolved Caveats The second play request should be a pause, followed by a 1X play request. The delay in the processing of successive play requests is determined by the tunable cm_minplaytime, which has a default of 500ms. Placing the processing of the last two play requests within the last 800 ms of the first segment is important, because this is when the play transition starts and the first segment's active play state changes from PLAYING to COMPLETING. This really confuses CServer, getting three play requests just as it is preparing to transition to a new segment. CSCtu02586 System asserted. Only occurs on assert and debug code, and the content being streamed live has to be deleted. CSCtt99042 Streams stop unexpectedly after sending 16 GB of data, without playing all the way to the actual end point. This issue can happen intermittently when the content used is larger than 16 GB and the content is live (specifically, the content is streamed as it is ingested). CSCtu37690 System crashes. Longevity test. CSCtu12699 When the primary Control server loses link state on all of the fill ports, but does not lose link state on the management network port, it will continue to hold the virtual IP address on the management network for the control service. This will allow other servers that may still have fill link-state to become the primary server because the virtual IP address for the control service is no longer in use by the original primary. After another server becomes primary it will make sure that all streams are playing on servers which still have fill link-state. Later when the original primary server gets link state again, the two primary servers detect that there is a conflict and the primary server that has most recently communicated with the backoffice will win and the other server will stop being primary. If the new primary server never communicated with the back end, then the original primary server will take over as primary again. However, because it does not know about the stream state changes that happened while it did not have fill link-state, some of the streams can be lost when the original primary takes over again. The primary control server must lose link state on all of its fill ports, but not lose link state on its management network port. Another server must become primary, but never communicate with the back office. When the original primary gets fill link state again it will take over as the primary again. 9

10 Upgrading to Release RTSP CSCtu12643 Streaming when some Streamers are running Release and some Streamers are running Release or Release causes assert "LARGE_POOL_MAX_SIZE." Streaming content when some Streamers are running Release or Release and some Streamers are running Release CSCtu90061 RTSP server crashes sometimes during processing of a message that contains an RTSP ANNOUNCE response and a TEARDOWN request in the same packet. The RTSP message should contain both an RTSP ANNOUCE response and a TEARDOWN request. RTSP/ OK Session: CSeq: Date: :24:31 M TEARDOWN * RTSP/1.0 Session: M CSeq: 32 User-Agent: Coship MAP-C Date: :24:31 Other CSCtw45759 The ImporterServer stops working. Socket() call fails, cannot check if it is master, and cannot work anymore. Happens once a month. Upgrading to Release The supported upgrade path is from Release to Release If the CDS is running an earlier software release, you must first upgrade to Release before upgrading to Release For software upgrade procedures, see the Cisco TV CDS 2.5 Installation, Upgrade, and Maintenance Guide. 10

11 Upgrading to Release Upgrading from Release to Release For the upgrade sequence of the CDS devices in your system (CDSM/VVIM and CDS servers [Vault, Streamer, Caching Node, ISV]), see the Overview chapter in the Cisco TV CDS 2.5 Installation, Upgrade, and Maintenance Guide. To upgrade a CDS device, do the following: Step 1 Step 2 Step 3 Step 4 Download the ISO image file (CDS-TV iso) and the cdsinstall script from the Cisco Software Download website. For more information on downloading the software files, see the Overview chapter in the Cisco TV CDS 2.5 Installation, Upgrade, and Maintenance Guide. For the CDS servers, offload the device before upgrading the software. a. From the CDSM GUI, choose Maintain > Servers > Server Offload. The Server Offload page is displayed. b. From the Server IP drop-down list, select the IP address or nickname of the sever and click Display. c. Select Enable and click Submit. d. Ensure the CDS server has been offloaded by checking the protocoltiming log. Log in to the CDS device as user root. For CDS servers, before installing the image file, stop the database with the following command: # db_shutdown Wait until the following command returns no output: # netstat -an grep 9999 and #pgrep avsdb Step 5 Copy the ISO image file and the cdsinstall script to the CDS server. # scp -p <user>@<remote_ip_address>:cds-tv iso / # scp -p <user>@<remote_ip_address>:cdsinstall / Step 6 Run the cdsinstall script to upgrade the ISO image to Release # cd /root #./cdsinstall /CDS-TV iso Step 7 Step 8 Step 9 For the CDSM or VVIM, when the script has completed, reboot the server. For CDS servers, make sure that the database is not running and reboot the server. # pgrep avsdb # reboot For CDS servers, after the CDS server has been verified as reachable, disable the Server offload. a. Choose Maintain > Servers > Server Offload. The Server Offload page is displayed. b. From the Server IP dropdown list, select the IP address or nickname of the server and click Display. c. Choose Disable and click Submit. d. Verify the CDS server is online by viewing the status on the Monitor > System Level > System Health page. 11

12 Upgrading to Release Downgrading from Release to Release For the downgrade sequence of the CDS devices in your system (CDSM/VVIM and CDS servers [Vault, Streamer, Caching Node, ISV]), see the Overview chapter in the Cisco TV CDS 2.5 Installation, Upgrade, and Maintenance Guide. To downgrade a CDS device, do the following: Step 1 Download the ISO image file (CDS-TV iso) and the cdsinstall script for Release from the Cisco software download website. For more information on downloading the software files, see the Overview chapter in the Cisco TV CDS 2.5 Installation, Upgrade, and Maintenance Guide. Step 2 For CDS servers, offload the server (see Step 2 in Upgrading from Release to Release ). Step 3 Step 4 Log in to the CDS device as user root. For CDS servers, before installing the image file, stop the database with the following command: # db_shutdown Wait until the following command returns no output: # netstat -an grep 9999 and #pgrep avsdb Step 5 Step 6 Step 7 Step 8 Copy the ISO image file and the cdsinstall script to the /root directory and run the cdsinstall script to download the software. # scp -p <user>@<remote_ip_address>:cds-tv iso / # scp -p <user>@<remote_ip_address>:cdsinstall / # cd /root #./cdsinstall /CDS-TV iso For the CDSM or VVIM, when the script has completed, reboot the server. For CDS servers, make sure that the database is not running and reboot the server. # pgrep avsdb # reboot For CDS servers, after the CDS server has been verified as reachable, disable the Server offload. a. Choose Maintain > Servers > Server Offload. The Server Offload page is displayed. b. From the Server IP dropdown list, select the IP address or nickname of the server and click Display. c. Choose Disable and click Submit. d. Verify the CDS server is online by viewing the status on the Monitor > System Level > System Health page. (see Step 7 in Upgrading from Release to Release procedure). 12

13 Related Documentation Related Documentation Refer to the following documents for additional information about the Cisco TV CDS 2.5: Cisco TV CDS 2.5 ISA Software Configuration Guide onfig.html Cisco TV CDS 2.5 RTSP Software Configuration Guide p_config.html Cisco TV CDS 2.5 API Guide ml Cisco TV CDS 2.5 Installation, Upgrade, and Maintenance Guide e.html Cisco Content Delivery System 2.x Documentation Roadmap Cisco Content Delivery Engine 205/220/250/420 Hardware Installation Guide 420_hig.html Cisco Content Delivery Engine 110 Hardware Installation Guide Cisco Content Delivery Engine 100/200/300/400 Hardware Installation Guide Regulatory Compliance and Safety Information for Cisco Content Delivery Engines Open Source Used in TV CDS The entire CDS software documentation suite is available on Cisco.com at: The entire CDS hardware documentation suite is available on Cisco.com at: Obtaining Documentation and Submitting a Service Request For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What s New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at: 13

14 Obtaining Documentation and Submitting a Service Request Subscribe to the What s New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0. Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R) This product contains watermarking technology that is licensed from Verimatrix, Inc., and such functionality should not be used or distributed further by you without any additional license(s) required from Verimatrix, Inc. Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental Cisco Systems, Inc. All rights reserved. 14