CMX Analytics Documentation

Transcription

1 CMX Analytics Documentation Version CMX Analytics Definitions Detected Signals Signals sent from devices to WIFI Access Points (AP) are the basis of CMX Analytics. There are two types of signals sent by devices: A device probing signal where the device is seeking to know available WIFI networks A device data signal where the device is sending data packets to the WIFI network Device MAC Address An OUI (Organizationally Unique Identifier) is a 24-bit identifier for a device (e.g. a0:99:9b:14:ef:25). Associated Devices A device is said to be 'Associated' with a WIFI network when it has authenticated against the WIFI network. The device is 'on' the network, sending and receiving data packets. A device may continue to probe while on the network. Probing Only Devices A device is said to be 'Probing Only' when it is not associated with a WIFI network and is seeking to know available WIFI networks. Most devices associated with a WIFI network may continue to probe. Areas An area is defined as a campus, building, floor or zone. Campuses may contain a number of buildings, buildings may contain a number of floors, and floors may contain a number of zones. Zones within a single floor cannot overlap. Visit In CMX, a visit is defined as follows: The appearance of a device in a single area on a single day (where a day is defined as beginning at 00:00 and ending at 23:59). If the device appears more than once in this area, over the same day, it is counted as only one visit. A visit appearance by a device to an area is comprised of a start time and an end time. The start time is assigned by the first detected signal received from the device from the area. The end time is assigned when the device leaves the current area. To illustrate this, consider a device which has been detected in Zone A on a floor map. There are three different ways that the end time (and therefore, visit duration) can be assigned, depending on three different circumstances: 1. Location has detected the device in another area (Zone B). In this case, the end time of the device's visit to Zone A is set as the start time of the device's visit to Zone B. 2. Location has detected more than two detected signals from the device in Zone A and has now sent an exit notification for the device (either because the device disconnected or because it has not been heard from in ten minutes). The end time for the device's visit to Zone A is defined as the last known "good" detected signal from the device, meaning its last signal in Zone A. 3. Location has not heard from the device since its initial signal from Zone A ten minutes ago and issues an exit notification. In other words, Analytics only has the initial Zone A start time and an exit notification. In this case the visit is discarded by Analytics and not counted as a visit. is to ensure that Analytics does not build up a series of ten minute visits generated by Location's The motivation for including points 2 and 3 minute inactivity timeout, which would skew the dwell time statistics calculated from such visits. ten

2 Dwell Time Because a visit is defined as the appearance of a device on a single day in a single area, the dwell time is the total duration of all appearances of the device on a given day for a given area. Dwell Time Calculation Dwell time by device The dwell time for a particular device in a given area is calculated by summing all of the device's visit durations to that area (see Visit definition for an explanation of how visit duration is calculated). For example, here are two devices visiting two different zones: Device Zone visited Visit duration 1 A 10 1 B 15 1 A 5 2 A 5 2 C 30 Dwell times for each device are calculated as: Device Total dwell time in Zone A Total dwell time in Zone B Total dwell time in Zone C 1 15 (10+5) The dwell times by zone are calculated as: Zone Total dwell time A 20 (10+5+5) B 15 C 30 Widget summary view dwell time calculation The following formula is used to calculate the value for the Dwell Time Widget's summary view value: Each total dwell time value in the formula above is calculated by multiplying the number of visits by the median dwell time of those visits. For example, to calculate the total repeat associated dwell time, the median dwell time of all the repeat, associated visits for the report's given areas and days is multiplied by the number of repeat, associated visits. The following is an example of visit dwell times and the way in which the widget summary view dwell time is calculated: Type of Visits Dwell Times of Individual Visits Repeat, associated 5, 5, 5, 60, 60, 60, 60, 60 Repeat, probing 5, 5, 30, 60, 60, 60, 60 New, associated 10, 10, 60, 60, 60, 60 New, probing 5, 5, 5, 60, 60, 60, 60

3 Total Repeat Associated Dwell Time Median(5, 5, 5, 60, 60, 60, 60, 60) = * 8 = 480 Total Repeat Probing Dwell Time Median(5, 5, 30, 60, 60, 60, 60) = * 7 = 420 Total New Associated Dwell Time Median(10, 10, 60, 60, 60, 60) = * 6 = 360 Total New Probing Dwell Time Median(5, 5, 5, 60, 60, 60, 60) = * 7 = 420 Summary View Value ( ) / ( ) = 60 Widget by-hour dwell time calculation Calculating the by hour value is very similar to the summary value. The only difference is that there is no differentiation between repeat and new; they're bunched together. The total median dwell time is the median of all the visits seen over the areas and days for which the report was generated. Dwell Threshold Filter A dwell time filter may be optionally applied to most widgets. A dwell filter consists of a minimum and maximum dwell time in minutes. The filter is used to limit results to visits with dwell times that fall within the filter's range. For example, a dwell filter of will filter out visits with durations less than one hour or more than 4 hours. Dwell time filters are restricted to a defined set. Dwell Time Breakdown & Dwell Threshold Filters The Dwell Time Breakdown Widget is the only widget that is not affected by the Dwell Threshold filter. This is because the Dwell Time Breakdown Widget groups visits by pre-defined dwell time groups and applying dwell filters to these would adversely impact the breakdown.

4 Time Range Filter A time range filter may be optionally applied to most widgets. The time range filters out all visitors that were not seen during that time range. A visit is defined as the appearance of a device in a single area on a single day, so when we look at the overview dwell time (the large number on the widget), we are looking at the total time a device was seen in a single area on a single day. The time range filters out all devices that were not seen during that time range, but to calculate dwell time, it uses the total dwell for that device in that area on that day, even if some of it is outside of the time range. The time range is a filter of devices and does not limit the dwell to the time during the time range. Consider the following example: 8:00 AM 9:00 AM 10:00 AM 11:00 AM 12:00 PM 1:00 PM 2:00 PM Total Time Device A 10 min. 10 min. 20 min. Device B 10 min. 10 min. 20 min. Device C 10 min. 10 min. For this example, consider a time range from 8:00 AM 12:00 PM. Devices seen in this time range will be included in the visitor and dwell time widgets. So, Device A and Device B will be counted but Device C will not be. But to calculate the dwell time, Device A and Device B will both have dwells of 20 minutes even though some of that time for Device B was outside of the time range because dwell calculations are done with the total time by the definition of a visit. Time ranges are restricted to a defined set. This can be configured. For more details, please see the section named "Time Range Configuration." Stationary Device If a device has a dwell time in the same area of more than 22 hours, it is considered a stationary device and is filtered out of reports unless the "Include stationary devices" checkbox is ticked and the maximum value for the dwell filter is 24 hours. Dwell Time Breakdown & Stationary Devices The Dwell Time Breakdown Widget always includes stationary devices, regardless of whether or not the checkbox is ticked in the settings. Repeat Visit In CMX, a repeat visit is defined as follows: The appearance of a device in a single area on a single day that was also seen in the same area on any other day in the current month or six months prior. As of , the number of prior months is configurable. The value must be in the range [0-6]. To change the value of the number of months of repeat history using the CLI: 1. cmxctl config analytics setnummonthsrepeathistory {0-6} Analytics does not need to be restarted to enact this change. Please note, all current repeat history is deleted when you do this. Total Visits Total Visits can be found on the Summary Widget Views. These values represent the sum of all visits represented by the corresponding areas, date range, time range filter and dwell threshold filter selected. If multiple areas or days are requested, visits from each day and area are summed together. For example, if we select a week's worth of data for a single area and a device visits the area for all 7 of those days, then it will be counted as 7 visits within the Total Visits value. Likewise, if we select a day's worth of data for 10 areas and a device visits all of those areas within the day, then it will be counted as 10 visits within the total value. Correlation A correlation is a measure of how many devices seen in one area were also seen in one or more other areas. The measure does not take account of the order in which these detections occur. For example, here are 3 devices and their movements: Device Time Zone visited

5 1 12:00 A 1 12:05 B 2 12:01 A 2 12:10 C 3 13:15 B 3 14:00 A To measure the correlation with focus on Zone A between 12:00 and 14:00, the measure first counts all the devices present at A during that time (3). For each of those devices it then checks to see where else those devices were during the time period (either before or after visiting zone A): Visits to A and B 2 66% Visits to A and C 1 33% Path A path is a measure of sequential movement to and from a given area. Given a focus area, a path counts the number of devices that were seen at that area at a given date and time period. It then tracks where those devices were immediately prior to visiting the focus area and where the devices traveled to immediately after the focus area. For example, consider these device visits: Device Time Zone visited 1 12:00 A 1 12:05 B 2 12:01 A 2 12:10 C 3 13:15 B 3 13:45 A 3 14:00 C If we choose A as the focus, then the following paths exist: Path Count Devices A->B 1 1 A->C 2 2, 3 B->A 1 3 Total 4 3 Note that the path BC does not exist, even though device 3 visits both these areas, because device 3 visited Zone A between these. The total number of Paths is the total number of movements between two nodes. Standard Analytics Widgets Visitors Widget This widget shows the count of visits to selected area(s) for a particular date(s), time range filter, and dwell threshold filter. This widget also displays repeat visits, allowing a comparison between how many visits were made by devices that had been there some time in the last six months and how many visits were new (i.e. no record of a visit for that device in the last six months). A visit can be a maximum of one per day per zone, floor, building or campus per device MAC address.

6 This New vs. Repeat ratio is shown in the horizontal stacked bar of the summary view: The repeat visit feature may give unreliable results for reports that are more than 6 months in duration because of the nature of how repeat visits are stored and defined (see definition above). Generally, the number of repeat visits will increase over time as more of the same devices will have been seen. Dwell Time Widget This widget shows the dwell time of visits within selected area(s) for particular date(s), time range filter, and dwell threshold filter. The value is calculated as described above in the section "Dwell Time Calculation." In the calculation, the median dwell time of visits are used. Median is used in place of average to avoid the figure from being distorted by small numbers of very short or very long dwell times. This widget also displays New vs. Repeat dwell time. This New vs. Repeat ratio is shown in the horizontal stacked bar of the summary view: WIFI Adoption Widget This widget shows the ratio between the number of visits that were associated vs. those that were just probing (i.e. Probing Only) the network in s elected area(s) for particular date(s), time range filter, and dwell threshold filter. This ratio is shown in the horizontal stacked bar of the summary view: Note, if probing devices are filtered, this widget will show 100% of clients as associated and will not be of value. There may be times when this widget will show high 90% (i.e. 98 or 99) when probing devices are filtered. This is due to a small number of Apple devices that may appear as probing even when probing devices are filtered (please see the Random Mac Address filtering section below for more detail).

7 Dwell Time Breakdown Widget This widget shows the categorization of visitors by their dwell time in selected area(s) for particular date(s) and time range filter. With this, we aim to show the different categories of dwell visit within an area. The categories themselves are defined as the following dwell ranges: 0-5min 5-20min 20-60min min > 120min The comparison of these dwell brackets are shown on the horizontal stacked bar of the summary view: The Dwell Time Breakdown Widget is not affected by the dwell threshold filter and always includes stationary devices. Standard Widget Data Views Summary View This view is a quick summary/overview and comparison of the different categories for the selected time range filter, date range, areas and dwell threshold filter (dwell threshold filter is not applied to Dwell Time Breakdown Widget). The summary is subdivided into three regions. The top left shows a stacked horizontal bar that visually shows the widget-applicable categories as percentages. The top right shows a percentage delta with the same previous period (i.e. This Week vs Last Week). The bottom region shows a trend over various time periods. Each data point represents the number of visits detected during the time period between data points. If multiple days are selected, the total visits for the date range is the summation of visits. If a single day is chosen, the hourly breakdown of visits during that day will be displayed in the bottom region. The total figure displayed in the top portion of the widget will be the union (not the summation) of the hourly visits during that day. If multiple areas are selected, visits from those areas are added together to generate the total and by-period chart figures.

8 Chart View (Visitors, Average Dwell Time, Wi-Fi Adoption, Dwell Time Breakdown) This view presents the data in either a line, area or bar chart depending on the chosen granularity of the data. Area charts are used when a time granularity is chosen (by Month, Week, Hour, etc.), and the data shows the numbers of visitors. Line charts are used when a time granularity is chosen. With this granularity, the data shows percentages of visitors or dwell time averages. Bar charts are used when a location granularity is chosen (Building, Floor, Zone, Tag, etc.). Table View

9 This view presents the same data as all charts but in a table format (i.e. rows and columns). Heatmap The heatmap tool provides a graphical representation of the density of devices for a given floor over a single day. The tool provides either a RealTime snapshot view or an animated playback, showing device density hour by hour for the chosen floor. The heatmap data divides a floor into a grid consisting of small squares, and a device count is maintained for each square. The data is then normalized across the day. This means that the color assigned to the heatmap does not relate directly to a specific number of devices and can vary from day to day and floor to floor. Specialized Widgets Correlation Some areas within a location, such as shops or buildings, should have a certain natural interaction or correlation with each other. This can be due to their related nature, ease of accessibility or even close proximity with one another. For example in a shopping center we may have a flower shop and a card shop. We would normally expect a strong correlation between the two areas as we expect visitors visiting the flower shop will also visit the card shop and vice-versa (because flowers and cards go hand-in-hand). Using CMX Analytics Correlation, we can quantify this relationship and prove/disprove our theory. From this chart we can see that there is a 25% correlation between the Card Shop and the Flower Shop. This tells us that out of the 100 visits that occurred in the Card Shop, last week, 25% of these visitors also happened to visit the Flower Shop. This may be lower than expected especially

10 when we compare it with the higher correlation between the Card Shop and the Cake Shop. This may be due to proximity or insufficient signage. Anyway, once we have modified our shopping center environment to better direct visitors from the Card Shop to the Flower Shop, we can then check if there has been an impact. Path Analysis Certain areas within a location, such as casinos, will hold an expected or sometimes unexpected instant attraction to visitors of that location (visita tion from when people enter the location). This can be deliberate or natural depending on the circumstances and nature of the location. For example, a casino desires to maximize the time visitors spend in the gambling section of their location and so the layout of the building as well as signage, advertising etc. will aim to increase the instant attraction. From when people enter the reception of the Casino, it hopes that they will immediately visit the Gambling Area. Using CMX Analytics Path Analysis, we can easily quantify the instant allure of an area from when people enter the building. The Path analysis widget displays the percentage of paths that began in an area and moved to the focus area as well as the percentage of paths that started in the focus area and traveled to another area. In addition, the widget displays the percentage of paths which began at the focus area and did not subsequently travel to any other area (representing devices that were not seen anywhere prior to the focus area). Finally, the widget also displays the percentage of paths that came to the focus area from elsewhere, but did not subsequently travel any further. From this chart we can see that 32% of visitors who enter the Reception then go straight to the Card Tables. Path Analysis shows immediate visitation and so we can say that 32% of visitors were immediately attracted to the Card Tables upon entering the Reception area. This may be lower than what was expected by the casino and so appropriate measures, such as more advertising, can be undertaken and then measured for their success. If the casino measures this value for 4 weeks and each week the value fluctuates between 30-34% they can say that the immediate visitation is on average 32%. They can then go on to place advertising and quantify its effectiveness - by measuring while the advertising in place. If the visitation goes up to 42% on average we can show that the advertising gave a 10% improvement. Queue Time The Queue Time Widget shows the average amount of time that a device spends in a specified zone at one time. This is different from the Dwell Time Widget because the amount of time used for the dwell time calculation looks at the total time the device was seen in the area on that day. If the device were to leave and come back, the Dwell Time Widget s calculation uses the sum of all the points in time at which the device appeared. The Queue Time Widget looks at each point in time separately. For the device s dwell to count towards the average used in this widget, the device must leave the area. This feature is off by default. To enable usage of the Queue Time Widget: 1. Using the CLI, enter the following commands: a. cmxctl config featureflags analytics.queuetime true b. cmxctl analytics stop

11 2. c. cmxctl analytics start Using the GUI: a. Change the verticalization to Airport and add the Security tag (more details to do that can be found here) b. Attach a single zone to the Security tag Create a report with the Security tag and add a Dwell Time Widget (because the report is for the Security tag, it will turn into the Queue Time Widget). The Security tag can only be attached to a single zone. Analytics SSID Filtering The SSID filtering feature can be used to restrict the SSIDs from which analytics will collect data. In order to turn the SSID Filter ON/OFF, follow these steps: Go to Analytics. Click on the "Settings" tab on the sub navigation bar. Click on the "SSID Filter" tab. Note: SSIDs filtered on Location will NOT show up in analytics SSID filtering list. Clients which have no SSID can be filtered out using the "NA" option. Visits that have been recorded for the day before the SSID filter is applied will not be taken out for that day; the filter is forward moving. Real Time Analytics To monitor the number of visitors in real time, use the "Realtime Dashboard." The real time analytics component employs the Location API to show the number of users in real time. Realtime Device Counts Widget Realtime reports only have the Realtime Device Counts Widget available. This widget gives information on the number of devices currently detected on the WiFi network. It gives a breakdown of the number of Associated and Probing devices currently seen on the network. Creating a Report To create a Reatltime report: 1. Go to Analytics. 2. Click on "Realtime" tab on the sub navigation bar.

12 3. Click the "[+]" icon in the top left of the screen. 4. Select the total number of widgets and click "Done." Reports The following are export options for reports shown in the Analytics Dashboard: Schedule Print Download These actions can be taken by clicking on the report you are interested in and then clicking the triangle. The three options will show: Reports can be exported in the following formats: PDF Excel HTML Scheduling Reports When scheduling a report, it is ed to you at the specified time. To schedule a report: Go to Analytics. Click on "Dashboard" tab on the sub navigation bar. Click on the report that you are interested in. Click on "Schedule." Choose the report format.

13 6. If you choose PDF, you are able to customize the look of your report. You can add a header, comments, and a logo. When there are multiple widgets, you are also able to change the order of the widgets. When you are done, click "Next." 7. Enter the address(es) of the recipient(s). Choose what day and time to start sending the report. Choose the frequency (i.e. One time, Daily or Weekly).

14 8. Click "Schedule." You can click on the "Schedule" tab on the analytics sub navigation bar to view which reports have been scheduled. From here, you can delete the scheduling for a single report or use "Delete All" to delete all scheduled reports. Time Range Configuration Whenever an analytics report is created, a time range must be selected. The time range filters out any devices that were not seen within that time range. A detailed explanation of time ranges can be found in the section "Time Range Filter" under "CMX Analytics Definitions." Additional time ranges can be created. In order to create a new time range, do the following: Go to Analytics. Click on the "Settings" tab in the sub navigation Click "Add." Enter the name of the new time range and select the range.

15 5. Click on "Save" to save the new time range. New time ranges can be applied to reports from this point forward. Please note, however, that: New time ranges are not retrospective Deleting time ranges may result in historical data no longer being accessible A re-compute of historical data may be necessary in order to access new time ranges with old data Miscellaneous Random Mac Address Filtering A small number of devices have unrecognized OUI and send locally administered (i.e. random) MAC addresses. These are filtered out by the location engine and not represented in Analytics. The OUI list of valid MAC addresses is updated as part of the ongoing system updates. Please note that some Apple devices do not send out valid MAC addresses in situations where they do not recognize the SSID of the network. These devices are filtered out by Analytics by default and are therefore not included in probing device counts. However, Apple devices will be counted as probing in situations where they recognize the SSID of the network (because they were once associated to it) but are not currently associated to it. RSSI Strength Filtering The filtering for RSSI signal strength is based on at least one AP reporting the signal is at a value which is higher than a preset value. The default value in CMX version is The default value in CMX version is -85. Users can configure this value to what is appropriate to their specific venues. If the values were changed when the report was created, then you would have resulted in different values from vs A recommended value for this setting in many WIFI networks is -75. Short Duration Visits There can be a large number of short duration visits in any CMX Analytics dataset if APs are outdoors and can hear devices that are infrequent, such as a car driving by. The method to filter these short duration devices would be to use the dwell threshold filter and set the minimum dwell time for a report to 30 minutes. Under-Reported Visits In certain circumstances, visits from some devices may not be reported by Analytics, even though some signals were detected for those devices. This can occur in the following scenarios:

16 1. A single signal only is detected for a device during a single day In this case, just one signal is detected from the device. After ten minutes of no further activity, the Location engine will produce an exit notification for this device. The Analytics engine therefore now has the original detected signal, followed by an exit signal ten minutes later. As noted above, Analytics does not process this visit to avoid skewing dwell time statistics and the visit is ignored. 2. A number of signals are detected for a device but the device remains in the same area In this scenario, multiple signals are detected from a device, but the device remains in the same area (this can happen for stationary devices which never leave a room, for example). As mentioned above in the definition of a Visit, in order to calculate a visit's duration, a device needs to either physically leave the area it is currently in (and be detected in another area), or issue an exit notification. For example, here are some detected signals from two devices: Time of signal Device Detected zone on floor map 10:00 1 A 10:05 1 A 10:07 1 A 10:08 2 A 10:10 1 A 10:13 2 B Device 1 remains in Zone A for the entire period, and therefore, no visits are generated. By contrast, device 2 visits zone B at 10:13 so a visit will be generated for device B in Zone A from 10:08-10:13. In this example, if device 1 continues to remain in Zone A and its signals are detected less than ten minutes apart (this is the threshold of device inactivity which triggers the Location engine to send an automatic exit notification for a device), a visit will never be created. If instead, the device emits a signal which is more than ten minutes since its last one a visit will be created. For example: Time of signal Signal source Signal type 10:00 Device 1 Detected on Zone A on floor map 10:05 Device 1 Detected on Zone A on floor map 10:07 Device 1 Detected on Zone A on floor map 10:10 Device 1 Detected on Zone A on floor map 10:21 Location engine Device 1 has not been heard from in more than 10 minutes. Therefore, the Location engine has generated an EXIT notification 10:25 Device 1 Detected on Zone A on floor map In this example, there is a 15 minute gap (10:10-10:25) in the last two detections for device 1 in Zone A and the Location engine has sent an exit notification in between these two detections at 10:21. In this case, a visit will be generated for Device 1 at Zone A from 10:00-10:21. Unique Visits Deprecated This has been deprecated as of Unique visits are too computationally expensive to compute and do not fit into the revised analytics architectural model for that focuses on speed and stability. It is likely that they will be reintroduced in a future release but in a different form. For now though they are simply gone. Cisco 2017