Long-term archive storage for video surveillance environments

Transcription

1 Long-term archive storage for video surveillance environments Using Milestone XProtect Corporate and IBM Spectrum Archive David L Taylor IBM Systems and Technology Group ISV Enablement March 2015 Copyright IBM Corporation, 2015

2 Table of contents Abstract... 1 Introduction... 1 Equipment used during the test process... 1 Test strategy used to ensure compliance... 2 IBM Spectrum Scale... 2 IBM Spectrum Archive... 4 Operational storage...6 Active archive...7 IBM Archive Server configuration requirements... 7 Example of a cron job that migrates files to the IBM Spectrum Archive pool...8 Milestone Corporate server configuration to allow access and control of the archive server Milestone Corporate batch recall function Solution overview IBM Tape Prefetch Service Remarks Smart Client plug-in IBM Tape Prefetch Service Milestone prefetch licensing Considerations when using the tape as an archive tier Recommended reference architecture Reference information Summary Resources About the author Trademarks and special notices... 24

3 Abstract The objective of this white paper is to discuss the considerations to assist the user in properly sizing storage to operate in Milestone XProtect Corporate digital video surveillance solution when using IBM Spectrum Archive tiering to manage a deep archive that includes both tape and disk storage. This paper covers a scenario that can be used as a template to manage the movement of video content between tiers while maintaining an acceptable user experience. When accessing content from the tape tier specific licensed functions of XProtect Corporate will be used, and these will be outlined. All the functions related to the XProtect batch recall function will work with both XProtect Corporate and XProtect Expert. Using this paper as a guide, an architect should be able to make appropriate hardware and networking choices to complete an effective and efficient digital video surveillance solution. Introduction Gaining the best performance from your video surveillance system requires more than selecting best-ofbreed components to run on an open architecture software platform. You must know how to configure the system for optimal use of all the components involved. Milestone XProtect video management software is a powerful surveillance platform that is easy to manage. One reason for this is due to its design with just such configuration capabilities that let you customize your solution to fit your exact needs. Milestone and IBM system engineers thoroughly tested the solution using the system parameters outlined in this paper. One consideration regarding surveillance video content is that a great deal of data is collected, any content collected traditionally will remain idle unless the content is retrieved for forensic study or for evidentiary use. As a result of this traditional approaches for storing this data would cause the content to remain on spinning disk for the entire retention period. This paper outlines the use of IBM Spectrum Scale as a disk tier connected to IBM Spectrum Archive as a tape tier. IBM Spectrum Scale and IBM Spectrum Archive are members of the IBM Spectrum Storage family. The solution provides unlimited retention of content with fast recall and a very cost effective method to protect the content for its complete retention lifecycle. This technical white paper shares an approach template allowing a skilled architect to create a fully tiered hybrid storage environment. It illustrates the benefits of using a deep archive using tape and how the Milestone Corporate software deals with the latency characteristics of tape. This paper does not cover IBM Spectrum Scale installation or Milestone Corporate installation. Refer to the appropriate IBM Redbooks and similar documentation for the assistance in those areas. Equipment used during the test process Recording and management servers Intel dual socket servers: Processor: Dual Intel Xeon E GHz 8 core processor Memory: 96 GB RAM (system use never exceeded 40 GB) 1

4 Disk: Four 900 GB 2.5 inch hot swap drives RAID 5 (system boot C:) Host bus adapter (HBA): Brocade 8 Gb Fibre Channel (FC) dual-port HBA HBAs: QLogic 10 Gb dual-port CNA HBAs: 6 Gb SAS HBA controller Microsoft Windows 2008 Server SP L4U IBM System Storage TS3200 Tape Library: 8245 Ultrium 5 serial-attached SCSI (SAS) 8245 Ultrium 5 SAS Test strategy used to ensure compliance The process used for testing the environment simulated a number of high-resolution cameras. These cameras had their video content stored on IBM Spectrum Scale running IBM Spectrum Archive as the archive engine. The policy driven tiering of IBM Spectrum Archive was set to allow immediate commitment of data to tape and then deletion of the content from disk after two hours so that the testers could retrieve content both from disk or tape. In addition to using timings to verify that content was only on tape. Looking at the directory in question would also display that the content was only on tape as displayed as part of the extended attributes shown when the director was displayed. The testing procedure was designed specifically to test the batch recall function interface from Milestone. Multiple scenarios were tried to exercise different aspects or use cases where content would require recall from the tape. The test environment includes two recording servers running Milestone Corporate 2014, each with up to 32 cameras simultaneously writing content to two separate data pools on the archive server. Each recording server stored the first hour s data locally and then archived the content to the IBM Spectrum Archive server. IBM Spectrum Scale IBM Spectrum Scale is an extremely powerful file system that is based on the IBM General Parallel File System (IBM GPFS ) technology, which is a proven, scalable, high-performance data and file management solution, and it is also enabled for technical computing, big data and analytics, and cloud. Spectrum Scale is being used extensively across multiple industries worldwide to provide simplified data management and integrated information lifecycle tools capable of managing petabytes of data and billions of files, in order to arrest the growing cost of managing ever-growing amount of data. Today, data growth is challenging traditional storage and data management solutions. Limited data access, good performance, and reliability are required for IT environments. Also, the application performance is affected by data access issues that delay schedules and waste expensive resources. Workloads are scaled up to large numbers of application nodes and disks, and because not all components are working correctly at all times, IT environments are required to handle component failures and continue the operation. 2

5 Spectrum Scale can enable enterprise clients to use a highly available clustered file system. IBM and independent software vendor (ISV) solutions can provide higher value for clients by using the following Spectrum Scale functionality: A highly available cluster architecture: Spectrum Scale can improve data availability through data access even when the cluster experiences storage or node malfunctions. Capabilities for high-performance parallel workloads: Concurrent high-speed, reliable file access from multiple nodes in the cluster environment. Smooth, nondisruptive capacity expansion and reduction are possible. Services are available to effectively manage the large and growing quantities of data. Spectrum Scale is designed to provide high availability through advanced clustering technologies, dynamic file system management, and data replication. Spectrum Scale can continue to provide data access even when the cluster experiences storage or node malfunctions. Spectrum Scale scalability and performance are designed to meet the needs of data-intensive applications. 3

6 IBM Spectrum Archive IBM Spectrum Archive, based on IBM Linear Tape File System (LTFS), provides a seamless integration of LTFS with IBM GPFS by creating a LTFS tape tier. You can run any application that is designed for disk files on tape by using IBM Spectrum Archive. It can play a major role in reducing the cost of storage for data that does not need the access performance of primary disk. With IBM Spectrum Archive, you can enable the use of LTFS for the policy management of tape as a storage tier in a GPFS environment and use tape as a critical tier in the storage environment. Spectrum supports IBM Linear Tape-Open (LTO) Ultrium 6 and 5 tape drives, and IBM System Storage TS1140 TS1150 tape drives that are installed in TS4500 and TS3500 tape libraries or LTO Ultrium 6 and 5 tape drives that are installed in the TS3310 tape libraries. The use of IBM Spectrum Archive to replace disks with tape in tier 2 and tier 3 storage can improve data access over other storage solutions because it improves efficiency and streamlines management for files on the tape. IBM Spectrum Archive simplifies the use of tape by making it not apparent to the user and manageable by the administrator under a single infrastructure. User / Application Global Namespace GP FS Disk Tier File Migration File Recall Tape Tier Cartridge Export Cartridge Import Tape Figure 1: High-level overview of IBM Spectrum Archive solution 4

7 The IBM Spectrum Archive uses an enhanced version of the LTFS Library Edition, which is referred to as the LTFS LE+ component, for the movement of files to and from tape devices. The scale-out architecture of IBM Spectrum Archive provides the ability to add nodes and tape devices as required to meet the bandwidth requirements between GPFS and the LTFS tape tier. Low-cost storage tier, data migration, and archive needs that are described in the following use cases can benefit from IBM Spectrum Archive: Operational storage Active archive (provides a low-cost, scalable tape storage tier) A local or remote LTFS Library Edition node serves as a migration target for GPFS that transparently archives data to tape that is based on policies set by the user. The following IBM Spectrum Archive characteristics cover a broad base of integrated storage management software with leading tape technology and the highly scalable IBM tape libraries: Includes the standard LTFS Library Edition with its supported tape. Integrates with GPFS similar to IBM Tivoli Storage Manager hierarchical storage management (HSM) by supporting file-level migration and recall with an innovative database-less storage of metadata. Provides a scale out architecture that supports multiple IBM Spectrum Archive nodes that share the same tape inventory with load balancing over multiple tape drives and nodes. Enables tape cartridge pooling and data exchange for LTFS tape tier management: Tape cartridge pooling allows the user to group data on sets of tape cartridges. Multiple copies of files can be written on different tape cartridge pools. It supports tape cartridge export with and without removal of file metadata from GPFS. It supports tape cartridge import with pre-population of file metadata in GPFS. IBM Spectrum Archive provides the following benefits: Is a low-cost storage tier in a GPFS environment Is an active archive or big data repository for long-term storage of data that requires the file system access to that content Provides a file-based storage in the LTFS tape format that is open, self-describing, portable, and interchangeable across platforms Lowers capital expenditure and operational expenditure costs by using cost-effective and energy-efficient tape media without dependencies on external server hardware or software Allows the retention of data on tape media for long-term preservation (more than 10 years) Provides the portability of large amounts of data by bulk transfer of tape cartridges between sites for disaster recovery and the initial synchronization of two GPFS sites by using open-format, portable, self-describing tapes 5

8 Enables migration of data to newer tape and or newer technology that is managed by GPFS Provides ease of management for operational and active archive storage. Note: For a no-cost trial version of the IBM Spectrum Archive, contact your local IBM sales representative Operational storage This section describes how IBM Spectrum Archive is used as a storage tier in a GPFS environment. The use of an LTFS tape tier as operational storage is useful when a significant portion of files on a network storage system is static, that is, the data is not changing. In this case, as shown in the following figure, it is optimal to move the content to a lower-cost storage tier. The files that are migrated to the LTFS tape tier remain online, that is, they are accessible from GPFS under the GPFS namespace. Tape cartridge pools can also be used for backup. NFS, CIFS, HTTP GPFS node LTFS GPFS node Gold Silver Bronze Figure 2: Tiered operational storage with IBM Spectrum Archive managing the LTFS tape tier 6

9 With IBM Spectrum Archive, the user specifies files to be migrated to the LTFS tape tier using standard GPFS scan policies. IBM Spectrum Archive then manages the movement of GPFS file data to the LTFS tape. With IBM Spectrum Archive, the user specifies files to be migrated to the LTFS tape tier using standard GPFS scan policies. IBM Spectrum Archive then manages the movement of GPFS file data to LTFS tape cartridges and edits the metadata of the GPFS files to point to the content on the LTFS tape tier. Access to the migrated files through the GPFS file system remains unchanged with the file data provided at the data rate and access times of the underlying tape technology. The GPFS namespace is unchanged after migration, making the placement of files in the LTFS tape tier not apparent to users and applications. Active archive This section describes how IBM Spectrum Archive is used as an active archive in a GPFS environment. The use of an LTFS tape tier as an active archive is useful when there is a need for a low-cost, long-term archive for data that is maintained and accessed for reference. LTFS meets the needs of this type of archiving by using open-format, portable, self-describing tapes. In an active archive, the LTFS file system is the main store for the data while the GPFS file system, with its limited disk capacity, is used as a staging area, or cache, in front of LTFS. GPFS policies are used to stage and de-stage data from the GPFS disks to the LTFS tape. The tapes from the archive can be exported for vaulting or for moving data to another location. Because the exported data is in the LTFS format, it can be read on any LTFS compatible system. IBM Archive Server configuration requirements The following components are required to create a hybrid storage environment using IBM Spectrum Archive to provide a disk and tape tiered solution. Load Red Hat Enterprise Linux (RHEL) 6.4 or similar Linux distribution on the archive server Load GPFS / IBM Spectrum Scale server release Load IBM Spectrum Scale or later Create a mixed Windows / UNIX GPFS / IBM Spectrum Archive cluster Create the GPFS / IBM Spectrum Archive file systems (with possibly different performance characteristics) Create the GPFS policy files Create the migration cron jobs 7

10 Example of a cron job that migrates files to the IBM Spectrum Archive pool The migration of files to IBM Spectrum Archive is performed by GPFS policy rules to migrate to external pools and is called by the cron job. The jobs run once an hour and the output is redirected to /gpfs1/testout(2). You can find the cron jobs in /etc/cron.hourly. The cron jobs call the GPFS mmapplypolicy command to apply the policy rule. The output is redirected to /gpfs1/testout(2) for diagnostic purposes. /etc/cron.hourly/migrate_recorder1: mmapplypolicy /dev/gpfs1 P /home/milestone/desktop/recorder1 >>/var/log/testout /etc/cron.houtly/migrate_recorder2: mmapplypolicy /dev/gpfs1 P /home/milestone/desktop/recorder2 >>/var/log/testout2 The policy rule migrates files that have not been accessed in 4 hours that have not already migrated or pre-migrated, and do not have the file extensions.ini,.xml, or.idx. File data is stored in IBM Spectrum Archive tape pool recorder1 for migrate_recorder1 and recorder2 for migrate_recorder2. Refer to the following example of the GPFS policy, /home/milestone/desktop/recorder1(2): /* gpfs manual policy migration for directory /gpfs1/recorder1 to EE pool recorder1*/ /* exclude_list excludes all GPFS system files as well as the EE work directories other recorder directories and file names ending in.ini,.idx, and.xml */ /* gpfs manual policy migration for directory /gpfs1/recorder2 to EE pool recorder2*/ /* exclude_list excludes all GPFS system files as well as the EE work directories other recorder directories and file names ending in.ini,.idx, and.xml */ define( exclude_list, (PATH_NAME LIKE '%/.SpaceMan/%' OR PATH_NAME LIKE '%/.ctdb/%' OR PATH_NAME LIKE '/gpfs1/00l4u78p7203_ll1/%' OR PATH_NAME LIKE '/gpfs1/00l4u78p7203_ll0/%' OR PATH_NAME LIKE '/gpfs1/recorder2/%' OR NAME LIKE 'user.quota%' OR NAME LIKE 'fileset.quota%' OR NAME LIKE 'group.quota%' OR NAME LIKE '%.idx' OR NAME LIKE '%.ini' OR NAME LIKE '%.xml') ) 8

11 /* is_premigrated, is_migrated and is_resident if the file is already (pre)migrated or resident no need to remigrate again just update */ define( ) is_premigrated, (MISC_ATTRIBUTES LIKE '%M%' AND MISC_ATTRIBUTES NOT LIKE '%V%') define( ) is_migrated, (MISC_ATTRIBUTES LIKE '%V%') define( ) is_resident, (NOT MISC_ATTRIBUTES LIKE '%M%') RULE EXTERNAL POOL 'ltfsee' EXEC '/opt/ibm/ltfsee/bin/ltfsee' OPTS 'recorder1' RULE 'ee_4hour' MIGRATE TO POOL 'ltfsee' WHERE FILE_SIZE > 0 AND (((CURRENT_TIMESTAMP - MODIFICATION_TIME > INTERVAL '30' DAYS) AND is_resident) OR ((CURRENT_TIMESTAMP - ACCESS_TIME > INTERVAL '24' HOURS) AND is_premigrated)) AND NOT (exclude_list) Note: In the define exclude_list stanza, 00L4U78P7203_LL0(1) is the library work directory and must be excluded unless the work directory is not in the migration directory. Refer to the following example of GPFS policy that immediately pre-migrates data to tape after 30 days migrates the file data to tape: /* IBM Spectrum Archive (GPFS) policy premigration for directory /gpfs1/recorder1 to EE pool recorder1*/ /* exclude_list excludes all GPFS system files as well as the EE work directories other recorder directories and file names ending in.ini,.idx, and.xml */ define( 9

12 exclude_list, (PATH_NAME LIKE '%/.SpaceMan/%' OR PATH_NAME LIKE '%/.ctdb/%' OR PATH_NAME LIKE '/gpfs1/00l4u78p7203_ll0/%' OR PATH_NAME LIKE '/gpfs1/00l4u78p7203_ll1/%' OR PATH_NAME LIKE '/gpfs1/recorder2/%' OR NAME LIKE 'user.quota%' OR NAME LIKE 'fileset.quota%' OR NAME LIKE 'group.quota%' OR NAME LIKE '%.idx' OR NAME LIKE '%.ini' OR NAME LIKE '%.xml') ) /* is_premigrated, is_migrated and is_resident if the file is already (pre)migrated or resident no need to remigrate again just update */ define( ) is_premigrated, (MISC_ATTRIBUTES LIKE '%M%' AND MISC_ATTRIBUTES NOT LIKE '%V%') define( is_migrated, (MISC_ATTRIBUTES LIKE '%V%') ) define( is_resident, (NOT MISC_ATTRIBUTES LIKE '%M%') ) RULE EXTERNAL POOL 'ltfsee' EXEC '/opt/ibm/ltfsee/bin/ltfsee' OPTS 'recorder1' RULE 'premigraterec1' MIGRATE THRESHOLD(0,90,0) TO POOL 'ltfsee' WHERE FILE_SIZE > 0 AND ((CURRENT_TIMESTAMP - MODIFICATION_TIME > INTERVAL '1' HOURS) AND is_resident) AND NOT (exclude_list) Note: In the define exclude_list stanza, 00L4U78P7203_LL0(1) is the library work directory and must be excluded unless the work directory is not in the migration directory. 10

13 Milestone Corporate server configuration to allow access and control of the archive server To allow access and control of the archive server: Install GPFS / IBM Spectrum Scale for Windows on the recording server. Join the Milestone server to the mixed Windows / UNIX GPFS / IBM Spectrum Scale cluster. Download plink or similar function to allow the recording server to send commands to the archive server. On boot, GPFS automatically mounts the cluster shares as defined during the creation of the file system with the GPFS command, mmcrfs. Milestone Corporate batch recall function The IBM tape prefetch solution includes a Smart Client plug-in and a server-side service. The plug-in populates a side panel in Smart Client. It provides control for selecting either a specific camera or a view as well as the period to prefetch. When requesting a period of data to prefetch, the plug-in sends commands to a number of IBM Tape Prefetch Services, which will start fetching data from the tape system. Solution overview The Smart Client plug-in provides a control panel which is available from the Playback page of the Smart Client. The panel has settings for selecting either a specific camera or a view and a period (start and end time) to prefetch data. While prefetching data, a progress bar indicates the progress along with some status information of the process. The plug-in can be installed on any Smart Client in the system. Each Smart Client can start a prefetch process, which can run simultaneously with any existing prefetch process. IBM Tape Prefetch Service The IBM Tape Prefetch Service must be installed on every recording server that has cameras where the prefetch functionality is needed. The service responds to requests from the Smart Client plug-in and returns status to each client concerning its requests. When running a prefetch job, the service requests the files needed for the selection of the cameras within the specified time period and verifies that these are available. The service can handle multiple prefetch processes simultaneously. The amount of data being requested to prefetch must be within the limit of the allocated cache on the tape system. There is no built-in limitation, and it is up to the operator to ensure that the requested amount of video does not exceed the capacity of the cache. 11

14 Remarks The IBM tape prefetch solution is compatible with: XProtect Corporate 2014 (7.0a) XProtect Expert 2014 (7.0a) 64-bit versions of Windows OS Smart Client plug-in Install the Smart Client plug-in on the computers where this plug-in is needed. The installer for the plug-in is available in two versions, 32-bit and 64-bit. Select the installer inline with the version of Windows OS. Install the plug-in by running one of the following installation files: 32 bit system: IBMPrefetchServiceSCPluginInstaller_x86.msi 64 bit system: IBMPrefetchServiceSCPluginInstaller_x64.msi By default, the plug-in will be installed in the following folders: 32 bit system: C:\Program Files\Milestone\MIPPlugins\PrefetchServicePlugin 64 bit system: C:\Program Files\Milestone\MIPPlugins\PrefetchServicePlugin C:\Program Files (x86)\milestone\mipplugins\prefetchserviceplugin After installation, Smart Client must be restarted to load the plug-in. Note: The plug-in requires a license file in order to work. Refer to the Milestone prefetch licensing section for details. IBM Tape Prefetch Service Install the IBM Tape Prefetch Service on the computers that also have a Recording Server service running. To install the IBM Tape Prefetch Service, run the IBMPrefetchServiceInstaller_x64.msi installation file. By default, the service is installed in the following folder: C:\Program Files\Milestone\PrefetchService When installed, the IBM Tape Prefetch Service is identified by the Milestone IBM Tape Prefetch Service. Note: After installation of the IBM Tape Prefetch Service, the account that is used by the service must be changed. The service must use an account that has access to the Milestone surveillance system. Open the Windows Services management utility. Open the Properties page for the Milestone IBM Tape Prefetch Service. Navigate to the Log On page and select an account. Restart the Milestone IBM Tape Prefetch Service. To recall data form the data archive, the recording server s batch recall function requests the files from the archive by sending recall commands through Secure Shell (SSH) using plink commands. The plink application has to be installed in the IBM Tape Prefetch Service installation folder. 12

15 Note: The IBM Tape Prefetch Service typically requires additional configuration including connection information of the Milestone surveillance system. Refer to the Milestone Corporate server configuration to allow access and control of the archive server section for details. Milestone prefetch licensing The Smart Client plug-in requires a valid license file in order to work. The license is based on the software license code key of the Milestone surveillance system. Provide this information when acquiring a license file. The license must be acquired from Milestone Custom Development. The license file is called IBMTapePrefetch.lic and must be placed in the installation folder of the plug-in, typically: C:\Program Files\Milestone\MIPPlugins\PrefetchService or C:\Program Files (x86)\milestone\mipplugins\prefetchservice When the license file is in place, Smart Client must be restarted. Considerations when using the tape as an archive tier At first glance, it is often considered that the long term retention policy for video archive data is a spinning disk. What is often, however, overlooked about the retrieval process and retention process is that after seven days, the opportunity that the content will be needed for review is quite remote. The cost of keeping this data available without latency in retrieval on spinning disk, from a power and heat perspective is quite high and the opportunity of data corruption is always a factor when the archive is kept in a live and modifiable state. When creating tiers of storage for long term data access, it is recommended that the retrieval characteristics of the content be highly scrutinized and considered. If limited access to the data is expected, the value from placing this data on a tape tier is very good. Once the data is written to tape, it can be protected from erasure, copied to an offsite location, copied for transportation for evidentiary use, or put on a shelf allowing an almost limitless repository that is both green and secure. IBM LTFS technology allows the Milestone XProtect video content to be placed on a self-describing tape medium that functions much like a flash drive allowing anyone with the appropriate tape drive and LTFS Single Drive Edition to access this content for use. If security is required, the LTFS cartridges can be encrypted without any loss of performance when being written or read. This green technology allows an expanding depth of archive capability with the cost of the cartridge media being the only incremental expense. One must remember that tape is a sequential technology and must be treated a little differently than a standard disk environment. Tape does not perform well when content is written to tape in a multithreaded way. The content will be written safely, but the files will be interleaved on the tape, possibly causing latency when the files are read later. To mitigate this situation, there are few solution options that can be considered in a Milestone Corporate environment. First, if the archive pool thread count is left at one, the system will behave as expected and the tapes will not be fragmented. However, if the video content is first archived to a disk and then tiered separately to a tape external to the Milestone archive process, there are 13

16 a few advantages, and the ability to hide the single cartridge size limitation. From Milestone s perspective, the mount appears to be a large disk, while under the covers, the archive application manages data placement and the complete write commitment process. The consideration of using LTFS provides safety for the data because the metadata describing the content of the tape is stored on the tape. This mitigates any corruption of the describing database when the content is not locally stored. Additionally, as Milestone XProtect stores the video content in many tiny files, the descriptors and metadata are stored in a local and easily readable way, if LTFS is used. Recommended reference architecture Figure 3: Reference architecture for a campus network including both local and remotely connected recording servers When choosing the hardware for a traditional digital video surveillance solution, the following criteria needs to be evaluated. Number of cameras per server Frame rate and resolution of the cameras Physical location of the servers, cameras, and network infrastructure Bandwidth capabilities of the network Remote cameras or wide area networking for remote locations Whether the solution can be attached to a larger federated digital video surveillance network When choosing the management, database and recording servers, the testing team recommends enterprise Intel servers. Two socket servers provide excellent value and performance. During the testing, the Intel Xeon E processor was chosen because it would showcase the performance potential of 14

17 the Milestone and IBM alliance. The moderately priced Intel Xeon E processor should perform equally well, provided that eight core processors are chosen, or if the expected workload is light, four or six core processors could be used. The archive and LTFS storage servers have been found to be less processor and memory intensive and allow either a single socket server, or a two socket server with only one socket populated to be used. The XProtect Smart Wall and XProtect Smart Client servers also are reasonable in respect to the resources, and are much more tightly coupled to the video card chosen or if a desk side server is a better choice. Again the single socket server is a good choice. When choosing the server, the consideration for model is always of interest. Often, the deciding factor is based on the adapters in use and the number of local spindles needed for each server. Because of the significant random write requirements of a high camera digital video surveillance solution, the team recommends for all central storage solutions that an external controller based architecture be selected. The testing showed that the IBM Enterprise Storage Server (ESS) provides an excellent balance between performance and density. As a result of these storage needs, the low spindle count provided by the local server storage prevents a reasonable or scalable solution. When considering islands of cameras as outlined in Figure 3, and if you have central storage, 1U servers can be used and this storage can be shared by multiple operating components of the solution. As you move away from the central location, you have the option of a small storage array tied to a few local recording servers. In the most remote locations, you can have a server with local tier 1 storage archiving to a local tape subsystem. As you move away from the core location, network bandwidth becomes a significant factor and techniques such as, data pruning, might be needed to send the key data to the management and client sites. Figure 3 shows a recommended solution including management, recording, and client servers. One additional server type not included in the figure is the Video Analytics server. IBM offers the IBM Smart Surveillance analytics package as part of the IBM Smart Surveillance solution offering. This software package currently operates with a maximum of 32 GB of RAM and four cores of processing power per instance. To make best use of this solution, it is recommended that multiple instances of this package be installed on a single high performance server using virtualization package, such as VMware or kernelbased virtual machine (KVM). In this scenario, each Smart Surveillance solution instance should be configured with four cores and 32 GB of RAM. One of these server instances can support 16 cameras simultaneously. A server of the dual socket class can simultaneously manage and operate four server instances per physical server, providing 64 cameras worth of analytics per physical server. It is understood that not every camera in a digital video surveillance will need analytics and the physical server count can be adjusted to match the cameras requiring this analytics feature. 15

18 Reference information This section provides screen captures collected from the Milestone recording server. Figure 4: Milestone tape backup prefetch showing retrieval progress on the progress bar 16

19 Figure 5: Milestone tape backup prefetch as the files are recovered from tape 17

20 Figure 6: Milestone tape backup prefetch with progress on recall 18

21 Figure 7: Directory of the archive showing that all files are disk resident 19

22 Figure 8: Directory of the archive showing that all files are now tape resident only 20

23 Figure 9: Storage layout showing first hour of data collection stored locally on server and all files archived after one hour 21

24 Summary Milestone Systems and IBM is creating a comprehensive partnership that can provide best of breed solutions for surveillance. Innovative: Milestone Systems is internationally recognized as an innovator and thought leader in open platform Internet Protocol (IP) video management software. Milestone XProtect products operate as the core of surveillance systems for connecting, sharing, and managing all devices through a single interface that is easy to learn and operate. Easy to use: The XProtect platform is easy to use, proven in operation, and scales to support unlimited devices. The XProtect products support the wide choice of network video hardware and are designed with an application programming interface (API) that integrates seamlessly with IBM hardware platform. Best of breed: Using XProtect, you can build scalable, best of breed solutions to video enable your business, reduce costs, optimize processes, protect assets and ultimately increase value in your organization products and services. Solution partnership: By combining the innovative software and ease of use of the Milestone XProtect Corporate surveillance suite with IBM leadership servers, storage and networking, customers can be comfortable that any solution created will provide the best performance and value. The partnership also ensures that a solution investment will provide value for many years to come with the opportunity to upgrade as required while protecting the legacy investment in technology. Figure 10: Best of breed solutions for surveillance 22

25 Resources The following websites provide useful references to supplement the information contained in this paper: IBM Linear Tape File System Enterprise Edition documentation ibm.com/support/knowledgecenter/st9mbr/com.ibm.storage.hollywood.doc/ltfs_ee_icho me.html IBM General Parallel File System documentation ibm.com/support/knowledgecenter/ssfkcn/gpfs_welcome.html IBM Redbooks ibm.com/redbooks IBM Publications Center About the author Dave Taylor is a consultant and architect in IBM Systems and Technology Group ISV Enablement Organization. He has more than 20 years of experience working with the IBM System Storage and server platforms. You can reach Dave at dltaylor@us.ibm.com. 23

26 Trademarks and special notices Copyright IBM Corporation References in this document to IBM products or services do not imply that IBM intends to make them available in every country. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel, Intel Inside (logos), MMX, and Pentium are trademarks of Intel Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-ibm products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-ibm list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-ibm products. Questions on the capability of non-ibm products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is 24

27 presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Photographs shown are of engineering prototypes. Changes may be incorporated in production models. Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. 25