Consolidated hardware infrastructure for IP video solutions

Transcription

1 Consolidated hardware infrastructure for IP video solutions Using Milestone XProtect Corporate and IBM Pure Flex System in a VMware environment David L Taylor IBM Systems and Technology Group ISV Enablement July 2013 Stay connected with IBM System Storage Twitter LinkedIn Facebook YouTube RSS feeds Forum Community More communities Copyright IBM Corporation, 2013

2 Table of contents Abstract...1 Introduction...1 Equipm ent used during the test process About IBM PureFlex System IBM Flex System x240 Compute Node... 3 Key features... 4 Abou t the IBM Storwize family IBM Storwize highlights... 7 IBM Flex System V3700 Storage Node IBM Storwize V5000 highlights IBM Storwize V5000 storage system External storage Test strategy used to ensure compliance...12 Results of the test...13 VMware aspects of the test...14 Choices for long-term archive storage...15 Recommended reference architecture Reference information...18 Summary...23 Resources...24 About the author...24 Trademarks and special notices...25

3 Abstract The objective of this paper is to discuss the hardware considerations to assist users in properly sizing computing resources within the IBM PureFlex System, allowing it to operate in the Milestone XProtect Corporate and expert digital video surveillance solutions. This paper discusses the opportunity to use VMware vmotion to offer high availability features while allowing migration of the virtual machines (VMs) between server platforms while the VMs are still active. This paper covers the test strategy used and the results of the testing. Additionally, considerations for archiving content and long-term retention requirements are outlined and suggestions regarding the techniques for retention are covered. This paper can guide architects to make appropriate hardware and networking choices to complete an effective and efficient digital video surveillance solution. Introduction Gaining the best performance from your video surveillance system involves more than selecting best-of- to run on an open-architecture software platform. You must know how to configure the breed components system for optimal use of all components involved. Milestone XProtect video management software is a powerful surveillance platform that is easy to manage. The configuration capabilities of the software design enable you to customize the solution to fit your exact needs. Milestone and IBM system engineers thoroughly tested the solution using the system parameters outlined in this paper. This technical white paper covers the test results that describe the performance of the Milestone XProtect Corporate Recording Server running on IBM PureFlex System. It illustrates the hardware impact (on the processor, storage, and network) when changing the individual parameters of the video surveillance system, and provides recommendations on how to configure the surveillance system for optimal performance. Equipment used during the test process Recording and management server (system under test) PureFlex System Chassis is configured as follows: Dual CMM EN Gb Ethernet switches FC Gb Fibre Channel (FC) switches IBM Flex System x240 Compute Nodes are configured as follows: Dual 8C 2.90 GHz Intel Xeon processor E GB RAM - Dual 300GB 10K SAS HDDs Emulex 10Gb CNA LOM - two ports Emulex 8Gb FC Adapter (FC3052) Microsoft Windows 2008 Server SP1 1

4 Camera simulation, data archive, and Milestone client server IBM Flex System x240 Compute Nodes configured as follows: Dual 8C 2.90 GHz Intel Xeon E processor 262 GB RAM Dual 300GB 10K SAS HDDs Emulex 10Gb CNA LOM - two ports Emulex 8Gb FC Adapter (FC3052) Microsoft Windows 2008 Server SP1 About IBM PureFlex System Figure 1: The IBM Flex System Enterprise Chassis To meet today s complex and ever-changing business demands, you need a solid foundation of server, storage, networking, and software resources that are simple to deploy and that can quickly and automatically adapt to changing conditions. You also need access to, and the ability to take advantage of, broad expertise and proven best practices in systems management, applications, hardware maintenance, and more. IBM PureFlex System is a comprehensive infrastructure system that provides an expert integrated computing system, combining servers, enterprise storage, networking, virtualization, and management into 2

5 a single structure. Its built-in expertise enables organizations to simply manage and flexibly deploy integrated patterns of virtual and hardware resources through unified management. These systems are ideally suited for customers interested in a system that delivers the simplicity of an integrated solution but who also want control over tuning middleware and the run-time environment. IBM PureFlex System recommends workload placement based on virtual machine compatibility and resource availability. Using built-in virtualization across servers, storage, and networking, the infrastructure system enables automated scaling of resources and true workload mobility. IBM PureFlex System undergoes significant testing and experimentation, so it can mitigate IT complexity without compromising the flexibility to tune systems to the tasks that businesses demand. By providing both flexibility and simplicity, IBM PureFlex System can provide extraordinary levels of IT control, efficiency, and operating agility that enable businesses to rapidly deploy IT services at a reduced cost. Moreover, the system is built on decades of expertise, enabling deep integration and central management of the comprehensive, open-choice infrastructure system and dramatically cutting down on the skills and training required for managing and deploying the system. IBM PureFlex System combines advanced IBM hardware and software along with patterns of expertise and integrates them into three optimized configurations that are simple to acquire and deploy, and thus enabling you to get fast time to value for your solution. The three PureFlex System configurations are: IBM PureFlex System Express: Is designed for small and medium businesses, and is the most affordable entry point for PureFlex System. IBM PureFlex System Standard: Is optimized for application servers with supporting storage and networking, and is designed to support your key ISV solutions. IBM PureFlex System Enterprise: Is optimized for transactional and database systems and has built-in redundancy for highly reliable and resilient operation to support your most critical workloads. IBM Flex System x240 Compute Node The IBM Flex System x240 Compute Node is a high-performance server that offers outstanding performance for virtualization with new levels of processor performance and memory capacity, and flexible configuration options. The Flex System x240 Compute Node is an efficient server designed to run a broad range of workloads, armed with advanced management capabilities allowing you to manage your physical and virtual IT resources from a single pane of glass. Suggested uses: Database, virtualization, enterprise applications, collaboration/ , streaming media, web, high-performance computing (HPC), Microsoft RemoteFX, and cloud applications. 3

6 Figure 2: The IBM Flex System x240 Compute Node Key features The IBM Flex System x240 Compute Node is a high-availability, scalable compute node optimized to support the next-generation microprocessor technology and is ideally suited for medium and large businesses. This section describes the key features of the server. Scalability and performance The Flex System x240 Compute Node offers numerous features to boost performance, improve scalability, and reduce costs: The Intel Xeon processor E product family improves productivity by offering superior system performance with up to 8-core processors and up to 3.3 GHz core speeds depending on the processor s number of cores, up to 20 MB of L3 cache, and QuickPath Interconnect (QPI) links of up to 8 GTps. The Intel Xeon processor E provides up to 80% performance boost over the previous generation, the Intel Xeon processor 5600 (Westmere EP). Up to two processors, 16 cores, and 32 threads maximize the concurrent execution of multi-threaded applications. Intelligent and adaptive system performance with Intel Turbo Boost Technology 2.0 allows processor cores to run at maximum speeds during peak workloads by temporarily going beyond processor thermal design power (TDP). Intel Hyper-Threading Technology boosts performance for multithreaded applications by enabling simultaneous multithreading within each processor core, up to two threads per core. Intel Virtualization Technology integrates hardware-level virtualization hooks that allow operating system vendors to better utilize the hardware for virtualization workloads. 4

7 Intel Advanced Vector Extensions (AVX) significantly improve floating point performance for compute-intensive technical and scientific applications compared with Intel Xeon processor 5600 series. Up to 24 DDR3 ECC memory RDIMMs provide speeds up to 1600 MHz and a memory capacity of up to 384 GB. Load-reduced DIMMs (LRDIMMs) are supported with a maximum capacity of 768 GB. The theoretical maximum memory bandwidth of the Intel Xeon processor E5 family is 51.6 GBps, which is 60% more than in the previous generation of Intel Xeon processors. The use of solid-state drives (SSDs) instead of or along with traditional spinning drives [hard disk drives (HDDs)] can significantly improve I/O performance. An SSD can support up to 100 times more I/O operations per second (IOPS) than a typical HDD. Supports the storage expansion node providing an additional 12 hot-swap 2.5-inch drive bays for local storage. Up to 16 virtual I/O ports per compute node with integrated 10 Gb Ethernet ports, offering the choice of Ethernet, Fibre Channel, iscsi, or FCoE connectivity. The Flex System x240 Compute Node offers PCI Express (PCIe) 3.0 I/O expansion capabilities that improve the theoretical maximum bandwidth by 60% (8 GT/s per link), compared with the previous generation of PCI Express 2.0. With Intel Integrated I/O Technology, the PCI Express 3.0 controller is integrated into the Intel Xeon processor E5 family. This helps to dramatically reduce I/O latency and increase overall system performance. The Flex System x240 Compute Node provides support for high-bandwidth I/O adapters, up to two in each x240 Compute Node. It also provides support for 10 Gb Ethernet, 16 Gb Fibre Channel, and FDR InfiniBand. The Flex System x240 Compute Node supports the PCIe expansion node to provide support for up to six additional I/O adapters. Availability and serviceability The Flex System x240 Compute Node provides the following features to simplify serviceability and increase system uptime: Chipkill, memory mirroring, and memory rank sparing for redundancy in the event of a non-correctable memory failure Tool-less cover removal provides easy access to upgrades and serviceable parts, such as processor, memory, and adapter cards. Hot-swap drives supporting integrated RAID 1 redundancy for data protection and greater system uptime. A light path diagnostics panel and individual light path light-emitting diodes (LEDs) to quickly lead the technician to failed (or failing) components. This simplifies servicing, speeds up problem resolution, and helps improve system availability. Predictive Failure Analysis (PFA), which detects when system components (such as processors, memory, and hard disk drives) operate outside of standard thresholds and generates proactive alerts in advance of possible failure, therefore increasing uptime. SSDs, which offer significantly better reliability than traditional and mechanical HDDs for greater uptime. 5

8 Built-in Integrated Management Module II (IMM2) continuously monitors system parameters, triggers alerts, and performs recovering actions in case of failures to minimize downtime. Built-in diagnostics using Dynamic Systems Analysis (DSA) Preboot speeds up troubleshooting tasks to reduce service time. Three-year customer replaceable unit and onsite limited warranty, next business day. Also, optional service upgrades are available. Manageability and security Powerful systems management features simplify local and remote management of the Flex System x240 Compute Node: The Flex System x240 Compute Node includes an Integrated Management Module II to monitor server availability and perform remote management. Integrated industry-standard Unified Extensible Firmware Interface (UEFI) enables improved setup, configuration, and updates, and simplifies error handling. Integrated Trusted Platform Module (TPM) 1.2 support enables advanced cryptographic functionality, such as digital signatures and remote attestation. Industry-standard AES-NI support for faster, stronger encryption. Integrates with the IBM Flex System Manager (FSM) for proactive systems management. It offers comprehensive systems management for the entire IBM Flex System platform that helps to increase uptime, reduce costs, and improve productivity through advanced server management capabilities. IBM Fabric Manager simplifies deployment of infrastructure connections by managing network and storage address assignments. Intel Execute Disable Bit functionality can help prevent certain classes of malicious buffer overflow attacks when combined with a supporting operating system. Intel Trusted Execution Technology provides enhanced security through hardware-based resistance to malicious software attacks, allowing an application to run in its own isolated space protected from all other software running on a system. Energy efficiency The Flex System x240 Compute Node offers the following energy-efficiency features to save energy, reduce operational costs, increase energy availability, and contribute to the green environment: Component-sharing design of the IBM Flex System chassis provides ultimate power and cooling savings. The Intel Xeon processor E product family offers significantly better performance over the previous generation while fitting into the same TDP limits. Intel Intelligent Power Technology powers individual processor elements on and off as needed, to reduce power draw. Low-voltage Intel Xeon processors draw less energy to satisfy demands of power and thermally constrained data centers and telecommunication environments. 6

9 Low-voltage 1.35 V DDR3 memory RDIMMs consume 15% less energy than 1.5 V DDR3 RDIMMs. SSDs consume as much as 80% less power than traditional spinning 2.5-inch HDDs. The Flex System x240 Compute Node uses hexagonal ventilation holes, a part of IBM Calibrated Vectored Cooling technology. Hexagonal holes can be grouped more densely than round holes, providing more efficient airflow through the system. IBM Systems Director Active Energy Manager provides advanced power management features with actual real-time energy monitoring, reporting, and capping features. About the IBM Storwize family The IBM Storwize family of products includes the IBM Storwize V7000 system, IBM Storwize V7000 Unified system, the IBM Storwize V5000 disk system, and the IBM Flex System V3700 Storage Node. IBM Storwize highlights The IBM Storwize system was introduced in the fall of 2010 to provide a cost-effective, flexible storage platform designed to help users reduce the costs and complexity associated with storage, and improve efficiency in mid-range storage environments. Similar to the earlier IBM System Storage SAN Volume Controller product, the IBM Storwize system is capable of pooling storage volumes from other IBM and non-ibm storage systems into reservoirs of capacity for centralized management. One of the key differentiators between the Storwize system and the IBM SAN Volume Controller is that the IBM Storwize system includes standard serial-attached SCSI (SAS) or SSDs, or a combination of each in the primary storage engine, and can be scaled further in size using expansion enclosures containing additional drives. The IBM Storwize disk system is designed to: Provide mid-range customers with a cost-effective, scalable storage platform that can provide advanced features typically only available to more expensive enterprise-class products. Deliver a new, easy-to-use graphical interface to access all the features of the platform. Provide automated tiering capabilities with the IBM System Storage Easy Tier function, which when enabled can reallocate the busiest disk extents to faster SSDs, maximizing the utilization of storage resources. Provide multiprotocol support for Fibre Channel and Internet Small Computer Systems Interface (iscsi) attachment. Deliver the same key virtualization features made popular with SAN Volume Controller. 7

10 The IBM Storwize family can also: Combine storage capacity from multiple disk systems into a single reservoir of capacity that can be better managed as a business resource and not as separate boxes. Help increase storage utilization by providing host applications with more flexible access to capacity. Provide thin-provisioned and real-time compressed volumes. Help improve productivity of storage administrators by enabling management of heterogeneous storage systems using a common interface. Support improved application availability by insulating host applications from changes to the physical storage infrastructure. Enable a tiered storage environment in which the cost of storage can be better matched to the value of data. Support advanced copy services from higher to lower cost devices and across storage systems from multiple vendors. IBM Flex System V3700 Storage Node Figure 3: The IBM Flex System V3700 Storage Node IBM Flex System V3700 Storage Node is a powerful block storage system designed to integrate into the IBM PureFlex System infrastructure for extremely rapid storage deployment and breakthrough management simplicity. By enabling virtualization, consolidation, and tiering in businesses of all sizes, it is designed to improve application availability and resource utilization. Flex System V3700 offers easy-to-use, efficient, and cost-effective management capabilities for both new and existing storage resources in the IT infrastructure. This new class of storage system combines no compromise design along with virtualization, efficiency, and performance capabilities. IBM Flex System V3700 Storage Node is built on the industry-leading storage virtualization and efficiency capabilities of IBM Storwize family products. As an integrated component in the PureFlex System, storage management and automated deployment is possible using IBM Flex System Manager. 8

11 The following functions are available with the IBM Flex System V3700 Storage Node: Thin provisioning (no license required) Volume mirroring (no license required) IBM FlashCopy (included with the base IBM Flex System V3700 Storage Node license) IBM Flex System V3700 Storage Node includes the following FlashCopy functions: Full / Incremental copy Multitarget FlashCopy Cascaded FlashCopy Reverse FlashCopy FlashCopy nocopy with thin provisioning Space-efficient source and target with background copy Space-efficient target with no background copy Consistency groups Remote Copy feature Metro Mirror Global Mirror Data migration function that can be used to import external storage systems into the IBM Flex System V3700 Storage Node system IBM Storwize V5000 highlights The IBM Storwize V5000 system was introduced in the summer of 2013 to provide a cost-effective, flexible storage platform designed to help users reduce the costs and complexity associated with storage, and improve efficiency in mid-range storage environments. Similar to the earlier IBM System Storage SAN Volume Controller product, the IBM Storwize V5000 system is capable of pooling storage volumes from other IBM and non-ibm storage systems into reservoirs of capacity for centralized management. One of the key differentiators between the Storwize V5000 system and the IBM SAN Volume Controller is that the IBM Storwize V5000 system includes standard serial-attached SCSI (SAS) or SSDs, or a combination of each in the primary storage engine, and can be scaled further in size using expansion enclosures containing additional drives. The IBM Storwize V5000 disk system is designed to: Provide mid-range customers with a cost-effective, scalable storage platform that can provide advanced features typically only available to more expensive enterprise-class products. Deliver a new, easy-to-use graphical interface to access all the features of the platform. Provide automated tiering capabilities with the IBM System Storage Easy Tier function, which when enabled can reallocate the busiest disk extents to faster SSDs, maximizing the utilization of storage resources. Provide multiprotocol support for Fibre Channel (FC) and Internet Small Computer Systems Interface (iscsi) attachment. Deliver the same key virtualization features made popular with SAN Volume Controller. 9

12 The IBM Storwize V5000 system can also: Combine storage capacity from multiple disk systems into a single reservoir of capacity that can be better managed as a business resource and not as separate boxes. Help increase storage utilization by providing host applications with more flexible access to capacity. Provide thin-provisioned and real-time compressed volumes. Help improve productivity of storage administrators by enabling management of heterogeneous storage systems using a common interface. Support improved application availability by insulating host applications from changes to the physical storage infrastructure. Enable a tiered storage environment in which the cost of storage can be better matched to the value of data. Support advanced copy services from higher to lower cost devices and across storage systems from multiple vendors. IBM Storwize V5000 storage system Figure 4: The IBM Storwize V5000 storage system IBM Storwize V5000 storage system is a powerful block storage system designed to integrate into any infrastructure for extremely rapid storage deployment and breakthrough management simplicity. By enabling virtualization, consolidation, and tiering in businesses of all sizes, it is designed to improve application availability and resource utilization. Storwize V5000 offers easy-to-use, efficient, and costeffective management capabilities for both new and existing storage resources in the IT infrastructure. This new class of storage system combines no compromise design along with virtualization, efficiency, and performance capabilities. IBM Storwize V5000 storage system is built on the industry-leading storage virtualization and efficiency capabilities of IBM Storwize family products. 10

13 The following functions are available with the IBM Storwize V5000 storage system: Thin provisioning (no license required) Volume mirroring (no license required) IBM FlashCopy IBM Storwize V5000 storage system includes the following FlashCopy functions: Full / Incremental copy Multitarget FlashCopy Cascaded FlashCopy Reverse FlashCopy FlashCopy nocopy with thin provisioning Space-efficient source and target with background copy Space-efficient target with no background copy Consistency groups Remote Copy feature Metro Mirror Global Mirror Data migration function that can be used to import external storage systems into the IBM External storage There are several options for attaching storage systems to the Flex System Enterprise Chassis: Storage area networks (SANs) based on FC technologies SANs based on iscsi Converged networks that are based on 10 Gb Converged Enhanced Ethernet (CEE) The following IBM storage products that are supported by the Flex System Enterprise Chassis are addressed: IBM Storwize V7000 IBM Storwize V5000 IBM XIV Storage System series IBM System Storage DS8000 series IBM System Storage DS5000 series IBM System Storage DS3000 series IBM System Storage N series IBM System Storage TS3500 Tape Library IBM System Storage TS3310 Tape Library IBM System Storage TS3100 Tape Library 11

14 Test strategy used to ensure compliance The process used for testing the environment, controlled and changed camera settings and quantity of simulated cameras while monitoring key performance metrics for write latency, processor consumption, network utilization, and video frame loss. To verify the solution running in a VMware environment, two Identical VMs were created, each supporting 300 cameras. The StableFPS utility was used to simulate the cameras. As sustained testing of a given benchmark value is preferred, it is important to test only one camera combination at a time and for a given period that spans multiple archive periods. Tests were also performed after the system had run for a period of four days with the initial benchmark value and some amount of data had accumulated in the archive database, in order to simulate a production environment. Through the StableFPS simulation utility, the camera count was set at 300 and ran for four days, without loss of frame data. As the full performance limit testing for IBM and Milestone was completed on discrete servers and documented on a separate paper, the testing on the PureFlex System did not complete a full 1200-camera simulation, but verified functionality with 300 cameras per server instance. Test settings: In the management client, select the StableFPS node under the recording server and click the Settings tab to make the following changes: - Set Codec to H264 - Set Compression to 60 - Set motion duration to 0 - Set FPS to 15 - Set motion duration HMS to seconds. - Set motion interval to 0. - Set motion interval HMS to seconds. - Set motion mode to Always. - Set the remote path to that of the management server where the Stable_FPS data resides. For example, \\managementserverip\e (where e represents the shared RAM disk drive) Testing with client load: - After running the test ran for a period of one hour, the test team logged into XProtect Smart Client from both XProtect Smart Client systems. - Open a 24-camera view on each and click the Playback tab. - Start video loop playback. Use the timeline selector to select the last hour of video. Increase the playback speed by eight times. Click Play. - The video playback time must span between data that resides in the live and archive database and must be adjusted every hour to ensure that this is the case. For the last four hours of testing, it will only be necessary to playback a single loop with starts in the live database (DB) volume, and eventually spans into the archive database. Target start and benchmark values are provided in Table 1 (note that this does not cover all possible combinations): 12

15 h.264 Target benchmark values Resolution Compression Frame Pe rcentage Test start Benchmark rate motion target camera target camera count count 640x % x % x % x % MJPEG Target benchmark values Resolution Compression Frame rate Percentage motion Test start target camera count 640x % x % Table 1: Target start and benchmark values Benchmark target camera count During the 4-day test, 150 cameras were simulated with h.264, 10 frames per second (FPS), 60% compression, video motion detection (VMD) on and simulation of 30% motion in the environment in all cases exceeding the benchmark. Results of the test The testing team feels that the PureFlex System had comparable results to the previously tested discrete system. The test team also demonstrated that the the IBM Flex System V7000 Storage Node has sufficient bandwidth and performance to support multiple Flex System x240 Compute Node recording servers. This creates a unified homogeneous solution suggesting that the PureFlex System Enterprise Chassis is ideal for environments where more than four recording servers will be needed in a single location. In this test, the same server instance contained all Milestone Corporate modules. In an actual multi-server environment these functions might provide better performance if separated on to different nodes. The PureFlex System environment is also perfect for applications that are virtualization aware. As Milestone Corporate is virtualization-aware,it is being now included in the reference architecture. For this test, the primary storage was the IBM Storwize V5000 Storage Node and the IBM Flex System V3700 Storage Node. An external Storwize system was used for the archive storage due to spindle availability in the lab. Tape tiering was not tested due to library availability. For this testing, all content storage used external disk controllers (block) with both spindle and cache benefits. If the camera counts are reduced, the possibility of using internal storage for primary content storage is acceptable. However, as it is expected that long-term retention of the content is preferred, it is 13

16 recommended that an external storage is included in each high camera count design, due to its capacity and performance advantages. All video content was received through the 10G interface, and both recording servers had a unique address on that adapter, as outlined in the Milestone recording server documentation. During the performance run, the 10G adapter operated at a very low utilization. It was attempted to run 600 cameras over the standard unbound 1G adapter. But this was unsuccessful, possibly due to the processor overhead running a non TCP offload engine (TOE) adapter and the data load of 600 cameras. In a moderate camera count environment, binding multiple 1G adapters might be effective, but this was not tested. All storage content was connected using dual 8GB FC. Subsystem Device Driver (SDD) was the multi- No optimization of pathing software used. All drives selected were chosen by the respective controllers. drive placement was considered. No SSDs or IBM Easy Tier optimization was considered. This is because the data footprint is a write-once, read-never scenario and does not receive any benefit from SSD architecture. The random data access write model did however, direct the test team to a RAID 10 storage architecture for the first tier of storage. As it is unclear what the write order to the hundreds of cameras simultaneously sending data, the test team chose RAID 10 access times. As there were fewer threads active when writing the archive, the t disk and tape as archive tiers. disk would be with to provide the best eam used both RAID 5 Sending data to an archive tier connected to a separate server using a Common Internet File System (CIFS) mount was also tested. In this test, the data was again sent over the 10 Gb link to the CIFS mount on the archive server. The archive process used the tiering software clustore from Tiger Technology. The clustore product includes a disk tier to migrate the data to an IBM Linear Tape File System (LTFS) tape tier using a policy-based strategy. During this testing, the policy-based movement was not used, and the files and directories were manually selected, and the running of the migration was initiated through the user interface. One consideration when using Linear Tape File System Library Extension (LTFS LE) as an archive tier is that because content is stored on an LTFS media in a journaling style with recovery after delete capability, users need to remember that if content ages off, the tape media will still consume that space on the tape. Long-term archive users might need to implement a policy that can reformat the tapes after all data has been expired. As most of the long-term archives have a multiyear policy for expiration, it would be useful to create separate LTFS pools or logical libraries based on the retention requirements, so that all data on a tape would expire at nearly the same rate. This can ensure the best reclamation value when data starts to expire. VMware aspects of the test The testing was completed on a PureFlex System Enterprise Chassis with a full compliment of server and storage node modules. The system was configured in a way that each recording server had access to all hyperthreaded processor cores. The testing scenario being reviewed was that the servers would be configured to use all resources within the blade. An additional fully loaded server instance would then be VMotioned onto the first blade and the results would be reviewed to determine impacts. In all testing, the VMware server indicated a processor alarm, but there was no noticeable impact to the collection of data. During the migration process between the server node blades, there was a loss of approximately 1 14

17 second s worth of camera data out of the 600 cameras being simulated. As soon as the migration was complete, however, there were no more frame losses. In conversations with Milestone field operations, they felt that these results were acceptable. If customers cannot tolerate a 1-second loss of data for any cameras, there are other high availability techniques that could be chosen to provide mirroring of camera data to multiple recording servers. Using the vmotion technique allows an affordable way to perform maintenance or upgrades to infrastructure with minimum impact to the collected data. This test also proved that a hot stand-by blade is not a requirement, but some customers might choose to add one to ensure that no processes would be throttled in the event of maintenance. The testing, however, demonstrates that the Milestone Corporate product works well in a VMware environment, and this in addition to now being part of the reference architecture, is a solution that provides good high availability features and allows for seamless maintenance when required or if an upgrade is planned. Additionally, with the virtualization capabilities of Storwize V5000, even the storage can be upgraded or maintained without user impact. Choices for long-term archive storage At first glance, it is often considered that the long-term retention policy for video archive data is a spinning disk. What is often however overlooked about the retrieval process and retention process is that after seven days, the opportunity that the content might be needed for review is quite remote. The cost of keeping this data available, without latency in retrieval on spinning disk, from a power and heat perspective is quite high. The potential for data corruption is always a factor when the archive is kept in a live and modifiable state. When creating tiers of storage for long-term data access, it is recommended that the retrieval characteristics of the content be highly scrutinized and considered. If limited access to the data is expected, the value from placing this data on a tape tier is very good. After the data is written to tape, it can be protected from erasure, copied to an off-site location, copied for transportation for evidentiary use, or put on a shelf allowing an almost limitless repository that is both green and secure. IBM s use of the LTFS LE technology allows the Milestone XProtect video content to be placed on a self- the appropriate tape describing tape medium that functions similar to a flash drive allowing anyone with drive and LTFS Single Drive Edition to access this content for use. If security is required, the LTFS cartridges can be encrypted without any loss of performance when being written or read. This green technology allows an expanding depth of archive capability with the cost of the cartridge media, the only incremental expense. It is noteworthy that tape is a sequential technology and must be treated a little differently than a standard disk environment. Tape does not perform well when content is written to tape in a multithreaded way. The content will be written safely, but the files will be interleaved on the tape, possibly causing latency when the files are read later. To mitigate this situation, there are a few solution options that can be considered in a Milestone XProtect Corporate environment. First, if the archive pool thread count is left at one, the system behaves as expected and the tapes will not be fragmented. However, if the video content is first archived to a disk and then tiered separately to a tape that is external to the Milestone archive process, there are a few advantages as well as the ability to hide the single cartridge size limitation. Tools, such as Tiger Technology clustore and StorageQuest Archive Manager control the write process to tape, and therefore, ensure that the data is stored on the cartridge in the most efficient manner. From Milestone s perspective, the mount appears to be a large disk, while in reality, the archive application manages data 15

18 placement and the complete write commitment process. The consideration to employ LTFS provides safety for the data, because the metadata describing the content of the tape is stored on the tape. This mitigates any corruption of that database when the content is not locally stored. Additionally, as Milestone XProtect stores the video content in many tiny files, the descriptors and metadata are stored locally in an easy-to-read manner, when LTFS is used. Recommended reference architecture Figure 5: An illustration of the recommended reference architecture When choosing the hardware for a traditional Digital Video Surveillance solution, the following criteria need to be evaluated. Number of cameras per server Frame rate and resolution of the cameras Amount of average change within the field of view Physical location of the servers, cameras, and network infrastructure Bandwidth capabilities of the network Remote cameras or wide-area networking for remote locations Whether the solution can be attached to a larger federated Digital Video Surveillance network When choosing the management, database, and recording servers, the testing team recommends servers of the IBM System x 3550 or System x3650 class for discrete servers and the Flex System x240 Compute Node for consolidated environments. These two socket servers provide excellent value and performance. During the testing, the Intel Xeon processor E was chosen because it would showcase the performance potential of the Milestone. The moderately priced Intel Xeon processor E5-16

19 2690 did perform well, if the expected workload is light, four- or six-core processors are good value choices. The archive and LTFS storage servers have been found to be less processor- and memory-intensive, and allow either a single socket server, such as Flex System x220 Compute Node, System x3250, or a System x3550 / System x3650 server with only one socket populated for use. The XProtect Smart Wall and XProtect Smart Client servers also are reasonable in respect to resources, and are much more tightly coupled to the video card chosen. Again, the Flex System x220 Compute Node or System x3250 class server is a good choice or a similar classed desk side server. When choosing the server, the consideration for model is always of interest. Often, the deciding factor is based on the adapters in use and the number of local spindles needed for each server. Due to the significant random write requirements of a high camera Digital Video Surveillance solution, the team recommends that an external controller-based architecture be selected for all central storage solutions. The testing showed that the IBM Storwize V5000 Storage Node system is an excellent balance between performance and value. If existing legacy storage is to be considered as a long-term strategy, the IBM Storwize system is a great performer and supports the virtualization of the legacy storage. Clearly for the first tier of storage, a large number of small spindles can provide the best solution for running RAID 10. For the secondary and archive tiers, large nearline SAS drives can provide the longest retention and value. Requiring these two disparate storage needs, the low spindle count provided by local server storage prevents a reasonable scalable solution. When considering islands of cameras as outlined in Figure 5 and you have central storage, 1U servers can be used and this storage can be shared by multiple operating components of the solution. As you move away from the central location, you have the option of a small storage array tied to a few local recording servers. In the most remote locations, you can have a server with local tier 1 storage, archiving to a local tape subsystem. As you move away from the core location, network bandwidth becomes a significant factor and techniques, such as data pruning, might be needed to send the key data to the management and client sites. Figure 5 shows a recommended solution including: Management, recording, and client servers. One additional server type not included in the figure is the video analytics server. IBM offers the IBM Smart Surveillance analytics package as part of the IBM Smart Surveillance solution offering. This software package currently operates with a maximum of 32 GB of RAM and four cores of processing power per instance. To make best use this solution, it is recommended that multiple instances of this package be installed on a single high-performance server using a virtualization package, such as VMware or kernelbased virtual machine (KVM). In this scenario, each Smart Surveillance System instance should be configured with four cores and 32 GB of RAM. One of these server instances can support 16 cameras simultaneously. A server of the Flex System x240 Compute Node, System x3550, or System x3650 class can simultaneously manage and operate four server instances per physical server, providing 64 cameras worth of analytics per physical server. It is understood that not every camera in a Digital Video Surveillance solution will need analytics, and the physical server count can be adjusted to match the cameras requiring this analytics feature. 17

20 Reference information In this section, screen captures for the test environment are provided. Figure 6: Summary review of 300 cameras with primary and archive tiers Figure 7: Milestone recording server resource summary 18

21 Figure 8: Storage layout of IBM Storwize V5000 Figure 9: RAID configuration for a 300-camera solution showing both RAID 10 and RAID 5 storage pools 19

22 Figure 10: Spindle pool selection and member drives for RAID 10 pools Figure 11: Spindle pool selection and member drives for RAID 5 pools 20

23 Figure 12: vsphere client showing recording server VMs on different nodes Figure 13: Two recording server instances running on a single node CPU alarm, without any loss of data 21

24 Figure 14: vsphere client showing recording server VMs on the same nodes Figure 15: Milestone corporate error log showing the 1-second of loss for 50 cameras 22

25 Summary Milestone Systems and IBM in partnership Innovative Milestone Systems is internationally recognized as an innovator and thought leader in open platform Internet Protocol (IP) video management software. Milestone XProtect products operate as the core of surveillance systems for connecting, sharing, and managing all devices through a single interface that is easy to learn and operate. Easy to use The XProtect platform is easy to use, proven in operation, and scales to support unlimited devices. XProtect products can support a wide choice of network video hardware and are designed with an application programming interface (API) that integrates seamlessly with IBM hardware platform. Best in class Using XProtect, you can build scalable, best-in-class solutions to video-enable your business, reduce costs, optimize processes, protect assets, and ultimately increase value in your organization s products and services. Solution partnership By combining the innovative software and ease of use of Milestone XProtect Corporate suite with IBM leadership servers, storage, and networking, customers can be comfortable that any solution created can provide the best performance and value. The partnership also ensures that a solution investment can provide value for many years to come with the opportunity to upgrade as required while protecting the legacy investment in technology. Figure 16: Total IBM and Milestone solution 23

26 Resources The following websites provide useful references to supplement the information contained in this paper: IBM Systems on PartnerWorld ibm.com/partnerworld/systems IBM Redbooks ibm.com/redbooks IBM Publications Center About the author Dave Taylor is a consultant and an architect in IBM Systems and Technology Group ISV Enablement Organization. He has more than 20 years of experience working with the IBM System Storage and Server platforms. You can reach Dave at dltaylor@us.ibm.com. Stay connected with IBM System Storage Twitter LinkedIn Facebook YouTube RSS feeds Forum Community More communities 24

27 Trademarks and special notices Copyright IBM Corporation References in this document to IBM products or services do not imply that available in every country. IBM intends to make them IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at legal/copytrade.shtml. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel Inside (logos), MMX, and Pentium are trademarks of Intel Corporation in the United States, other countries, or both. INFINIBAND, InfiniBand Trade Association and the INFINIBAND design marks are trademarks and/or service marks of the INFINIBAND Trade Association. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-ibm products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-ibm list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-ibm products. Questions on the capability of non-ibm products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. 25

28 Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Photographs shown are of engineering prototypes. Changes may be incorporated in production models. Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. 26