KNOWLEDGE BASE ARTICLE DISPLAYING SMARTVIS FACE LIVE IN A MILESTONE ENVIRONMENT

Transcription

1 KNOWLEDGE BASE ARTICLE DISPLAYING SMARTVIS FACE LIVE IN A MILESTONE ENVIRONMENT VERSION 1.0 SEPTEMBER 17 Users with SmartVis Face Live Demo and above can display both the video stream and the metadata within the Milestone XProtect system. This document explains how to install, configure and operate SmartVis Face Live Demo with Milestone XProtect Corporate. Please note that this is a version 1.0 iteration of the Bridge and will be improved in the future. Copyright 2016 Digital Barriers plc

2 Prerequisite hardware requirements The following minimum PC specification is recommended for running Milestone XProtect with SmartVis Face Live Demo: Windows 7 64-bit. Intel 4th generation Core-i5 CPU. 8GB RAM. The following minimum PC specification is recommended for the SmartVis Face Live Demo: PC, Windows 10 8 GB RAM CPU - Intel i7 GPU - NVidia GTX 1050 equivalent or higher The following minimum software versions are required: Digital Barriers Live Demo 3.3.1c Milestone XProtect 2014 (any of the software variants). XProtect Corporate 2014 (7.0d) is used in this article. A license file is required to use Milestone XProtect. Contact Milestone to obtain this or request a trial license via their website. Milestone XProtect Device Pack 8.9 Digital Barriers software can be downloaded from : MIP Plugin - % exe?dl=1 SmartVis Face Live Demo - Windows.exe?dl=0 NTP Server: Milestone XProtect software can be downloaded from ISSUED: 15 SEPTEMBER 2017 PAGE 2

3 Connection details Item IP address TeamViewer/Remote Desktop login details Windows login details SmartVis Live Demo box Windows Remote Desktop Please provide IP address of computer test / test Windows login: automatic after boot Milestone Server Teamviewer Windows Remote Desktop Windows login: Automatic after boot Milestone login (basic user): Axis M1125 Figure 1 Table of key information for setting up a system. Left blank for your information to be entered ISSUED: 15 SEPTEMBER 2017 PAGE 3

4 Global architecture The system is composed of 3 units: An AXIS camera (M1125) whose video stream contains faces An SmartVis Face Live Demo box running the SmartVis Face Live Demo 3.5 software A Milestone PC running the Milestone XProtect Corporate (2017 R1 version) and the Bridge service The System flows information in the following manner: Figure 2 Demo architecture The Axis camera sends a video stream to both the SmartVis Face Live demo application and the Milestone XProtect Server The SmartVis Face Live application analyses the video stream and detects faces within it 1. If the person is recognized as belonging to one or more watchlists (lists of persons whose face is known ), the SmartVis Face Live application generates a metadata xml stream that displays the subject name and the name of the watchlist he belongs to near the person s face in the live stream. If the person is not known, only an unknown label is displayed (cf next image). 1 The Face Live Application at the same time may run a gender detection on each face to identify the gender of the person. The gender information is not displayed by the Milestone integration ISSUED: 15 SEPTEMBER 2017 PAGE 4

5 Figure 3In this image, the Scarlet and Monica subjects belong to the High Risk watchlist, whose color is red. Bruce subject is recognized as such, but he does not belong to a watchlist, hence only the name is displayed. The same metadata (name, watchlist) are sent by the SmartVis Face Live application to the Milestone Bridge, that, in turns, sends them to XProtect Recording Server for recording and live/playback display (the display is slightly different from the SmartVis Face Live one, see the screenshots at the end of this document) The Milestone XProtect Server receives the video stream from the M1125 camera and records/displays the camera video stream in the Smart Client The Milestone XProtect Server receives the SmartVis Face Live information from the SmartVis Face Live Bridge (that, in turn, receives them from the SmartVis Face Live application) A special plugin running in the Smart Client displays the SmartVis Face Live metadata on top of the camera video stream both in live and in playback mode. The video stream and metadata need to be synched to align properly. The Milestone XProtect Server records both the video stream and the SmartVis Face Live metadata on disk for display in playback mode. How to change the IP addresses The three devices should belong to the same network and, as such, can be used in a private, closed network. Please note the key updates required The camera IP address must be changed in the Milestone Management Client and in the SmartVis Face Live Application Within Milestone, it is necessary to change the MIP Driver device ip address, using the new one that the Milestone Recording Server has. Please modify the URL in the MIP Driver device Management Client. Within Milestone, in the MIP Plug-ins section, Digital Barriers, SmartVis Face Live it is necessary to change the SmartVis Face Live server IP address with the current one assigned to the SmartVis Face Live Demo box. The new XProtect Server address must be used when opening the Milestone Management Client and the Milestone Smart Client ISSUED: 15 SEPTEMBER 2017 PAGE 5

6 The SmartVis Face Live Demo box uses the Milestone server as time server, so if this one changes its IP address it is necessary to edit (as an administrator) the file C:\Program Files (x86)\ntp\etc\ntp.conf (on the SmartVis Face Live Demo box) and change the line server iburst minpoll 6 maxpoll 7 with a new one with the good ip address. After the change, the Network Time Protocol Daemon service must be restarted. The Axis camera uses the Milestone host as NTP server. So, align the IP address for the Milestone host by changing the necessary settings under: Axis Setup, section Setup / Basic Setup / 3 Date & Time / NTP Server:. Do not forget to save the new setting. Time synchronization A windows build of the NTP server needs to be installed on both the SmartVis Face Live Demo box and the Milestone server. This is a time client that connects to internet servers to set up the host time in a precise and reliable way (much more precise and much more reliable than the standard Windows time service). It is of the utmost importance that the SmartVis Face Live server, the Axis M1125 camera and the Milestone XProtect server are synchronized to the hundreds of milliseconds. The NTP installed on the Milestone server acts both as a client and a server. If the host has internet connectivity, it will synchronize with NTP servers in the cloud. If not, it will use its own clock as reference. The Axis camera is configured to use the Milestone server as NTP server, so it will get its time from it. Finally, the Alienbox host will use NTP servers in the cloud, if the internet connectivity is available, and fall down to using the Milestone server as NTP server if there is no internet connectivity. This will insure that the three of them will be (at least) synchronized between them and, if there is internet connectivity, also synchronized with the world clock. Software details On Axis M1125 There is no software (embedded application) running on the M1125. The camera is simply a video stream source. SmartVis Face Live Application The application is a desktop application that can be launched by double-clicking on the SmartVis Face Live Demo As soon as the application is launched (and connected to a camera, the Axis M1125 is preconfigured as video source) the application opens a TCP listening port on 618. At the first connection on this port, the application answers sending an endless stream of XML documents. Only 1 stream at a time is supported, any other concurrent connections on the same port will be refused (the connection will be immediately closed). A telnet on port 618 can be used to check if the application is behaving correctly (i.e. sending the XML data) - beware of not having any other connection already established, for example from a running Face Live Alarm & Metadata Bridge. 2 This the installed version at the time of writing. This is subject to change, please check with Digital Barriers for the most up to date version. ISSUED: 15 SEPTEMBER 2017 PAGE 6

7 The application can be closed by closing the main window. If the application starts behaving oddly (for example, not sending XML out anymore) a reboot of the host is necessary (a simple restart of the application not always fixes the problem). If the application needs to be updated, it is necessary to uninstall it ( FaceRecDemo in the Programs and Features list) and install the new version by double-clicking on the installer. It is not necessary to customize the installation parameters, the default choices are the right ones. It is however necessary to reconfigure the connection to the camera as it is lost during the upgrade. SmartVis Face Live Alarm & Metadata Bridge The SmartVis Face Live Alarm & Metadata Bridge is a software component (a windows service) that: Connects to the SmartVis Face Live Application to receive the overlay, Implements a MIP Plugin hardware device that can be added to the Milestone Recording Server in order to receive the SmartVis Face Live overlay within XProtect Creates bookmarks in the video stream of the camera analyzed by the SmartVis Face Live Application whenever a new sighting event is generated by the application. A sighting event is triggered by the SmartVis Face Live Application when a known person (i.e. a person enrolled in at least one watchlist) is recognized in the camera video stream. Triggers alarm in XProtect whenever a known person is recognized in the camera video stream The SmartVis Face Live Alarm & Metadata Bridge works with two other software modules: A graphical interface called SmartVis Face Live Metadata Bridge Configuration Tool that allows the user to enter the IP address and credentials of the Milestone Management Server: ISSUED: 15 SEPTEMBER 2017 PAGE 7

8 A MIP Plugin running in the XProtect Management Client and allowing the user to associate a Milestone Video Source (i.e. the M1125 camera, or any other camera analyzed by the FaceLive Application) to the SmartVis Face Live overlay. The overlay from SmartVis Face Live is displayed on top of the chosen camera. The MIP Plugin of the Management Client allows the user to specify the IP address and listening port of the SmartVis Face Live application as well: ISSUED: 15 SEPTEMBER 2017 PAGE 8

9 Figure 4 Installing the MIP Plugin A MIP Plugin running in the Smart Client and displaying the overlay information from the SmartVis Face Live application on top of the chosen video stream. This plugin has no graphical interface and is transparent for the user. Installation The Bridge, Bridge Configuration Tool and MIP Plugin are all installed by the same installer, named DB SmartVis Face Live Bridge to Milestone XProtect <version>. At installation it is possible to choose which software module to install. Install the following items in the following locations: The SmartVis Face Live Alarm & Metadata Bridge should be installed on the Milestone host MIP Plugins for SmartVis Face Live should be installed on any host where the XProtect Management Client and/or the XProtect Smart Client are installed 3. Once the installation finished, if the SmartVis Face Live Alarm & Metadata Bridge feature has been installed, the Metadata Bridge Configuration Tool pops-up to allow the user to enter the ip address and credentials of the Milestone Management Server. Those are used by the bridge to log into the Milestone Server. Configuration Add the camera being used for the demo to the XProtect Server the Set the primary video stream from this camera to the same resolution and framerate used by the Face Live Application (usually for a M1125 the resolution is 1920x1080 and the framerate 25 or 30 - both will work). 3 This step is necessary if the XProtect Smart Client is installed on another host (than the SmartVis Face Live Demo PC and the XProtect Recording Server host, where the installation is already done. ISSUED: 15 SEPTEMBER 2017 PAGE 9

10 Figure 5 Setting up the camera in XProtect Once the MIP Plugins for SmartVis Face Live installed, it can be configured by starting the XProtect Management Client and going to the MIP Plug-ins / Digital Barriers / SmartVis Face Live. The mandatory information are: - The IP address and port (if different from 618) of the SmartVis Face Live Application - The Milestone Video Source (chosen in a combo-box among all those configured in the XProtect System) that is analyzed by the SmartVis Face Live application: ISSUED: 15 SEPTEMBER 2017 PAGE 10

11 Figure 6 Setting up in XProtect Every change in the configuration must be recorded by saving the configuration ( Ctrl-S or the Save button). After a configuration change the Alarm & Metadata Bridge service automatically reloads the configuration. The last step consists in adding to the recording server a special hardware device, called MIP Driver, that receives the metadata from the SmartVis Face Live Alarm & Metadata Bridge: 1. In the Management Client, go to Servers, Recording Servers and choose the server ( DESKTOP-OT92IH3 in our installation) 2. Right click on the server and choose Add hardware 3. Choose the option Manual, then Next 4. Add the credentials root, password, then Next 5. Scroll down the list to Other, expand it and tick MIP Driver, then Next 6. Enter the Milestone Recording server public IP address (do not use , as the bridge does not listen on the loopback address), (or the new one if it has been changed) 7. Change the Port number to Very important: manually choose MIP Driver in the Hardware model list, as (Auto-detect) will not work. Then click on Next 9. The device should be detected with Success. Click on Next twice 10. Make sure to tick the MIP Driver ( ) - Metadata 1 channel to enable it, then click on Next 11. Choose a group for the new metadata source, for example the SmartVis Face Live Metadata group. Click on Finish 12. The new MIP Driver ( ) device should be added. If you select it (or its channel 1) you should see 0 and 1 going out from the cubes in the center of the video. That means that metadata is received by the MIP Driver: ISSUED: 15 SEPTEMBER 2017 PAGE 11

12 Figure 7 MIP driver installation Usage A second view dedicated to the Face Live demo has been defined in the Smart Client (the one installed on the Milestone Recording Server), so that it is possible to commute from the TVI demo SecPol2017/Local to the Face Live demo FaceLive/M1125 with a couple of clicks: Figure 8 Selecting the camera In the XProtect Smart Client, if the camera video stream is displayed in a tile, the corresponding overlay from SmartVis Face Live is displayed as well, both in live and in playback mode (this is true only if the corresponding MIP Plugin has been installed on the host running the Smart Client). Overlay in live mode: ISSUED: 15 SEPTEMBER 2017 PAGE 12

13 Figure 9 Overlay in Live Mode Overlay in playback mode: ISSUED: 15 SEPTEMBER 2017 PAGE 13

14 Figure 10 Displaying in XProtect When a known person is recognized in the video stream, a bookmark is inserted in the recorded video timeline as well: ISSUED: 15 SEPTEMBER 2017 PAGE 14

15 Figure 11 Alarms in XProtect And an alarm is generated: ISSUED: 15 SEPTEMBER 2017 PAGE 15

16 Figure 12 Showing the overlay in XProtect The overlay displays: - Above the head, the name of the person, if recognized, or Unknown - Under the head, a second label might appear if the subject is recognized as belonging to a Watchlist. The label contains the name of the watchlist. Watchlists are defined directly in the SmartVis Face Live application, and can be associated to a different color. The foreground color of the lower label is the color of the corresponding watchlist SmartVis Face Live metadata synchronization with the live video stream in XProtect In order to have the metadata synchronized with the live video stream in XProtect it is necessary to delay the video stream by seconds. This is due to a native delay that the VLC plugin introduces when delivering images to the SmartVis Face Live Application and the SmartVis Face Live processing time. Activate the Buffering mode of the XProtect Smart Client for the specified video stream to delay the video stream Click on Setup, select the tile where the video stream is, then scroll down the left panel to the end until the Video buffering frame appears. Depending on the number of faces in the video stream the appropriate value might be 1.5 seconds or 2 seconds. ISSUED: 15 SEPTEMBER 2017 PAGE 16

17 How to demonstrate the system This is a description of a possible way of demonstrating the system, it can be adapted according to the constraints and possibilities of the event. 1) Screen, keyboard and mouse are connected to the Milestone Recording Server 2) The Smart Client runs on the Milestone Recording Server and is used to demonstrate both TVI and Face Live (two different tile layouts have been defined, and it is possible to commute between them in a few clicks) 3) In order to demonstrate effectively Face Live, it is also necessary to be able to enroll people and to assign them to a WatchList. Those steps require access to the Face Live Application. A remote desktop session from the Milestone Recording Server to the Alienbox is kept open in the Milestone Recording Server so that at any moment it is possible to access Face Live and to demonstrate the enrolling 4) Use the Smart Client to show the face detection and tracking and the sighting events when a known subject is recognized. 5) Use the playback functionality to demonstrate the overlay recording please note that this currently marks the end of the event. 6) Finally, it is possible to double-click on the sighting alarms that are displayed in the lower tile. The double click would open a special window: ISSUED: 15 SEPTEMBER 2017 PAGE 17

18 Figure 13 Replaying book marks in SmartClient ISSUED: 15 SEPTEMBER 2017 PAGE 18