A transfer learning framework for traffic video using neuro-fuzzy approach

Transcription

1 Sādhanā Vol. 42, No. 9, September 2017, pp DOI /s x Ó Indian Academy of Sciences A transfer learning framework for traffic video using neuro-fuzzy approach P M ASHOK KUMAR 1, * and V VAIDEHI 2 1 Department of Computer Science and Engineering, KL University, Guntur , India 2 Department of Electronics Engineering and AU-KBC Research Centre, MIT Campus, Anna University, Chennai , India profpmashok@gmail.com MS received 29 February 2016; revised 15 September 2016; accepted 30 January 2017; published 4 August 2017 Abstract. One of the main challenges in the Traffic Anomaly Detection (TAD) system is the ability to deal with unknown target scenes. As a result, the TAD system performs less in detecting anomalies. This paper introduces a novelty in the form of Adaptive Neuro-Fuzzy Inference System-Lossy-Count-based Topic Extraction (ANFIS-LCTE) for classification of anomalies in source and target traffic scenes. The process of transforming the input variables, learning the semantic rules in source scene and transferring the model to target scene achieves the transfer learning property. The proposed ANFIS-LCTE transfer learning model consists of four steps. (1) Low level visual items are extracted only for motion regions using optical flow technique. (2) Temporal transactions are created using aggregation of visual items for each set of frames. (3) An LCTE is applied for each set of temporal transaction to extract latent sequential topics. (4) ANFIS training is done with the back-propagation gradient descent method. The proposed ANFIS model framework is tested on standard dataset and performance is evaluated in terms of training performance and classification accuracies. Experimental results confirm that the proposed ANFIS-LCTE approach performs well in both source and target datasets. Keywords. ANFIS-LCTE; knowledge transfer; low-level features; topic extraction; traffic anomaly. 1. Introduction With the advent of intelligent video surveillance systems, there is a raise in demand for detecting anomalies in data streams. The main challenge in analysing video scenes is the large number of surveillance cameras, causing traffic analysts to take wrong decisions. As a result, there is a need for automatic analysis of video traffic scene. There are two main categories in recent approaches: (1) trajectory clustering and (2) topic modelling. In clusteringbased approach [1 3], similar groups are created with the help of varying length trajectories. However, in trajectorybased approaches, the main limitations are sensitive to occlusion and tracking errors, especially in dense traffic scenes. Hence, in topic-modelling-based approach [4 6], words are constructed from foreground motion pixels. LDA, DPMM and tdpmm topic modelling techniques model motion pixels with the help of statistical techniques. A major shortcoming in topic model is time complexity during both training and classification. The main drawback in these two approaches is that the authors apply their training and testing approaches in the *For correspondence same domain. This means re-deployable trained model cannot be applied readily across cross-domains [7, 8]. Hence, researchers collect new target datasets and re-train the model from scratch. This process involves cost of annotation, cost of new data, time and staff. Most of the applications like traffic video analysis, pedestrian detection and non-stationary Unmanned Aerial Vehicle (UAV) surveillance require immediate response and should work irrespective of the viewing angles. Therefore cross-domain scene interpretation has become increasingly desirable for practical deployment and non-stationary surveillance. For this, we propose a new class of transfer learning (TL) technique for transferring the semantic knowledge from training sample to target sample. This is a new method used for transferring knowledge at semantic level. In this paper, we present a novel Adaptive Neuro-Fuzzy Inference System-Lossy-Count-based Topic Extraction technique (ANFIS-LCTE) approach to deal with the aforementioned problem. First, we extract common motion patterns occurring in busy traffic junction using the LCTE algorithm. We extract parameters distance d and directional information h 1 and h 2 between the topics for motion pattern analysis. The input parameters d, h 1 and h 2 are fuzzified along with labelled observations to form 1431

2 1432 P M Ashok Kumar and V Vaidehi IF THEN rules to describe a classifier. The parameters associated with the membership functions will change through the learning process. We build a gradient vector to compute the ANFIS parameter, which in turn provides a measure of how well the fuzzy inference system is modelling the input output data for a given set of parameters. Any of several optimization routines could be applied on the gradient vector, to adjust the parameters to optimize error measure. ANFIS uses either back-propagation or a combination of least squares estimation and back-propagation for membership function parameter estimation. The result of pattern matching between rule antecedents and input signal triggers a number of fuzzy rules in parallel with various values of firing strength. We consider individually invoked actions together with combination logic. The resultant ANFIS rules are applied to traffic scenes other than using in training phase. This method detects anomalies about illegal U turns both in source and target scenes. Figure 1 shows the process for illustration. This proposed approach can overcome the shortcomings due to cross-domain problems. It detects anomalies caused due to spatio-temporal motion interaction. Our novelty lies in three aspects: 1. A novel feature representation comprising abstract spatial and temporal information. 2. A novel approach based on frequent item set, named LCTE, is proposed for discovering hidden topics. 3. Application of ANFIS model for classification of anomalies in both source and target scenes. We organize the rest of the paper as follows. Section 2 gives an overview of the recent works. Section 3 discusses about definition of key terms and problem formulation in this paper. Section 4 presents the method of learning in source scenes and detecting anomalies in target scenes. Section 5 describes the experiments and analysis conducted on the standard video dataset. Finally, section 6 has conclusion with a brief description about the ANFIS work, datasets and result analysis. 2. Related works Currently, we classify anomalous motion event detection systems into two broad categories based on the features used. The first class belongs to low-level features [9 11]. Wang et al [12] used low-level location and optical flow [12] features along with hierarchical Bayesian approach to model activities and interactions. Rabiner and Juang [13] used a combination of spatio-temporal features and hierarchical plsa for learning global behaviour correlations. The main limitation of topic models is that they fail to represent the sequential nature of activities. The works [14 16] attempt to merge temporal information in topic models by modelling the dynamics of topic distributions over time. In case of extremely crowded or partly occluded Figure 1. Illustration of topics extraction and knowledge transfer for anomaly detection.

3 A transfer learning framework for traffic video 1433 scenes, the low-level feature approaches are superior to the counterpart and produce meaningful results. Another class belongs to object level feature, in which by blob detection and tracking method motion tracks of individual objects are extracted and analysed for anomaly detection [2, 17 21]. In the latter approach, clustering algorithm clusters object trajectories based on the distance parameters like Euclidean distance, Dynamic Time Warping (DTW), longest common subsequence and Hausdorff distance. In [21], trajectory route envelope defines an average trajectory in a cluster and gives bounds on the variance within the cluster. Piciarelli and Foresti [19] used object trajectories as a tree of clusters that allow sharing of trajectory data in sub-clusters. The high-level activity is found using the graphical tree model. This type is susceptible to noise. Morris and Trivedi [10] proposed a threestaged hierarchical learning process for characterizing behaviour based on each level point of interests, spatial location and spatio-temporal dynamics using HMM. All these methods suffer from occlusion and tracking errors, especially in dense traffic scenes. Although the approaches discussed earlier achieved good performance in a variety of datasets, a severe drawback of existing approaches is that they are highly problem/domain specific in nature. Recently, Almajai et al [22] trained their anomaly system with single player tennis matches and compared to double-player tennis matches. Following this work, Xu [23] proposed geometrical matching approach that relates motion models learned from the database of source domains to the motion in the target domain. Both low-level feature and object level-based methods require training and target data to be available, which implies that their geometry of the scene and viewing angles should be the same. In order to overcome these limitations, this paper presents a novel approach for transferring the semantic knowledge using a neuro-fuzzy approach. The main idea behind this approach is to transform the low-level knowledge into high-level semantic knowledge and apply it to other related target domains. We use a novel input feature consisting of motion pixel s abstract spatial and temporal information. The proposed LCTE method extracts latent topic in each video sequence. We extract the parameters d, h 1 and h 2 from the topics, which are given as input to ANFIS structure for finding the anomalies. From the experimental results, it is clear that this approach performs well with reasonable accuracy. 3. Problem definition In general, traffic video sequences contain spatial motion patterns, which exist over a set of continuous frames. The traffic signal at the junction directs vehicles in different directions, thereby creating several spatial motion patterns. In this work, features are constructed only on motion pixels in individual frame. Each feature is considered as items in analogy with frequent item set mining in customer transactions. All the items in the set of continuous frames are aggregated to form a temporal transaction. The LCTE algorithm is then applied for all sets of transactions to discover maximal sequential frequent patterns in the video sequence. This in turn implies the common spatial motion patterns occurring in the scene. The following definitions formally describe this mining problem. 3.1 Visual item The term visual items refer to distinct motion pixels in each frame. Thus, the visual item feature (f i ) is characterized by three related attributes: abstract spatial location (x, y), motion direction h, start time t s and end time t e. On the whole, we represent the visual item feature (f i )as(x i, y i, h i, t s,i, t e,i ). 3.2 Feature aggregation Feature aggregation is defined as the process of eliminating redundant features (f i ) based on common spatial location (x, y). During this process, we update the time of finish attribute t e with current frame number, with reference to starting frame number. 3.3 Temporal transactions A temporal transaction is defined as a sequence of consecutive frames with aggregated feature items, T i = \fr 1, fr 2,, fr n [, where n represents sequence length seq_len. 3.4 Frequent sequential patterns The set of spatial items that occurs more frequently in a set of temporal transactions is termed as sequential patterns. They represent the set of most common vehicular motion patterns followed in traffic video scenes. 3.5 Latent topic Latent topics are defined as hidden groups present in feature items of video set sequences. We discover latent topics through clustering the frequent feature items based on direction h attribute. They represent unique vehicular motion pattern followed by vehicles. Topics are distinct in their direction h and spatial location (x, y) attribute. Thus any video sequence has just a combination of different topics for a valid motion pattern.

4 1434 P M Ashok Kumar and V Vaidehi 3.6 Inter-topical distance The inter-topical distance d between two topics is measured by selecting the shortest distance between two points, with each from two groups present in the video sequence. This can be further used in ANFIS classification to achieve the TL property. qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi d d min ¼ min ðx i x j Þ 2 þðy i y j Þ 2 ; 8i 2 Topic 1, 8j 2 Topic 2: 3.7 Directional topic The directional topic h is used for representing the moving direction of each topic present in the sequence. We quantized this angular information into four directions, namely EAST, WEST, NORTH and SOUTH. 3.8 ANFIS fuzzy rule An ANFIS is a kind of fuzzy inference system that uses both neural network and fuzzy logic principles. We derive the inference from a set of fuzzy IF THEN rules, which are traditionally used for approximating nonlinear functions. However, in this work, we use ANFIS for achieving the TL property across video sequence as discussed in the remaining sections of this paper. From these definitions, the problems of anomaly detection in video sequence get converted to finding latent topics and extracting the semantic knowledge by training the ANFIS classifier in offline training phase. In online testing phase, we extract latent topics and analyse parameters with ANFIS classifier for anomaly detection in source/target video sequence. extract parameters d, h 1 and h 2 from the latent topics in each target video sequence, which are given as input to ANFIS classifier for finding anomalies. The proposed method takes traffic video sequence as input. The main steps are as follows: 1. Visual item feature extraction: In this section, optical technique is used to find motion pixels. Features are constructed with abstract spatial location, temporal and motion information. Temporal information field specifies its period of existence. In this paper, both low-level feature and visual items are used interchangeably as the term item is used in connection with customer s purchased items in database transaction. 2. Temporal transaction creation: For each set of consecutive video frames, visual items in each frame are aggregated to form temporal transaction. This transaction contains unique items with abstract spatial location, directional and updated temporal information. 3. Latent sequential topic extraction: We apply the novel method of approximate maximal frequent pattern mining to each set of transactions. We cluster the items in maximal frequent sequential patterns, based on directional information, to extract latent topics. Subsequently, we assign class labels on the basis of their support parameter. 4. ANFIS: We designed ANFIS based on two linguistic variables distance d and direction h 1 and h 2 of the topics. During offline training, membership function parameters are chosen that best allow the fuzzy inference system to match with classifier output. During online testing phase, linguistic parameters d, h 1 and h 2 foreachpairoftopicsaregivenasinputto ANFIS. Composite firing strengths of the rules are calculated and sigmoidal function is applied to find the degree of belonging to a class. 4. Proposed adaptive neuro-fuzzy inference system approach for anomaly detection 4.1 Overview Recently, there has been increased interest in cross-domain knowledge transfer among computer vision community, that too particularly for traffic applications. In this section, we explore the concept of automatic learning and detection of anomalies from videos. Figure 1 illustrates the proposed process. The proposed ANFIS-LCTE approach consists of four processes: (1) low-level feature extraction, (2) temporal transaction creation, (3) latent sequential topic extraction and (4) ANFIS classifier. In offline unsupervised phase, positive and negative samples are first found out in source scene using frequentpattern analysis, followed by latent topics extraction and ANFIS classifier training. In online testing phase, we 4.2 Visual item feature extraction In this work, we extract low-level features based on abstract location, motion and temporal information features. We describe the process of extracting features in algorithm 1. In traffic surveillance videos, topics are characteristics of the frequent motion that occurs periodically in the scenes. The first video frame is conceptually divided into non-overlapping cells of size. We apply optical flow [24] for video sequence and remove static/noise pixels based on threshold of optic flow vectors. We construct visual feature based on the motion pixel s abstract location (x, y). We quantize motion direction h into any of four directions (90, 180, 270 and 360 ) for all the moving pixels and append to the feature as (x, y, h). In addition to this, temporal information is appended as (x, y, h, t s, t e ). The attribute t e specifies the time of occurrence

5 A transfer learning framework for traffic video 1435 with respect to frame number in the video sequence. This field is very helpful in detecting temporal anomalies. 4.3 Temporal transaction creation In this, we divide each video into non-overlapping sequence S i and process each frame to get visual features. The features present in one sequence are aggregated based on the similar (x, y) location and direction h to form temporal transaction. The temporal field t s represents the frame number at which it first occurred and field t e represents the frame number at which it last occurred. The reason behind using abstract location and quantizing direction is to decrease the number of distinct feature items. Table 1 shows the process described earlier. Table 1. Feature extraction algorithm. Input: Output: 4.4 Latent sequential topic extraction In this section, we propose a novel technique named LCTE for the purpose of latent topic extraction. We consider visual feature items and sequence of frames as transaction items and temporal transactions. The latent topics extraction is done with the help of maximal frequent itemset mining and aggregation of frequent items. The input to LCTE algorithm is video sequence of length seq_len, error rate e, support threshold s and window length win_len. There are two main phases in LCTE algorithm (as shown in table 2): sequential pattern generation (SPG) and topic generation. In LCTE (as shown in table 2), we maintain a data structure D for storing temporal patterns and T for storing topics. Both D and T are set to zero. Initially, LCTE performs feature extraction process and aggregation operation in step 4. Later, temporal transactions are created with the help of novel features. In the SPG algorithm (as shown in table 3), a separate pattern list D maintains the incoming aggregated features from each frame sequence. This pattern list is arranged as per direction h. The SPG algorithm maintains count parameter for each list. SPG updates the count value for existing pattern list based on the condition of checking whether the incoming feature list is a proper subset with existing pattern list in D; otherwise, SPG creates a new pattern list with the available features and the count parameter is maintained. The proposed LCTE algorithm iterates this process for each set of sequence length seq_len. For each window of transaction sequence win_len, proposed LCTE algorithm calculates support threshold s. We drop the infrequent patterns with frequency less than s - e. The LCTE algorithm restores the patterns with frequency counts that are on tolerance levels ±e. The basic assumption behind tolerance levels is that the patterns may become frequent in the next window of sequences. The resultant list D has frequent spatial items. Table 2. Lossy-count-based topic extraction (LCTE) method algorithm. Input: Output: The generated frequent items are clustered based on the directionality h and forming a unique number of topics for each pattern list in D (shown in step 18). The resultant

6 1436 P M Ashok Kumar and V Vaidehi Table 3. Sequential pattern generation algorithm. Input: Output: Table 4. Topic generation algorithm. Input: Output: Figure 2. ANFIS-based classifier. frequent pattern items are transformed into a mixture of the topics generated. Table 4 illustrates the topic generation from frequent list D. We carry the entire process of generating frequent sequential patterns and latent topics offline in an unsupervised way. In this application scenario, the sequential patterns signify spatial patterns of moving vehicles. This is proportional to the traffic density at that particular instant. Hence, the condition of subset/superset gives reasonable and correct output of maximal frequent patterns. The main idea behind the proposed LCTE approach is that frequently generated spatial patterns are always regular and mostly followed by all moving objects. LCTE allows some patterns with tolerance levels to exist in the next window. Using the LCTE approach, we extract regular patterns in the form of a mixture of topics in the video scenes. This topic helps further processing in video analysis. 4.5 ANFIS classifier The task of ANFIS classifier is to generate an appropriate fuzzy partition of the input feature space. We develop a classifier based on fuzzy IF THEN rules. In this work, a three-input zero-order sugeno models are considered and are shown as follows: Rule 1: Ifðx is A 1 Þ and ðy is B 1 Þ and ðz is C 1 Þ then f 1 ¼ r 1 Rule 2: Ifðx is A 2 Þ and ðy is B 2 Þ and ðz is C 2 Þ then f 2 ¼ r 2 where x, y and z are input parameters with A i,b i and C i as their corresponding fuzzy sets and f i are the outputs within the fuzzy region. The main goal in this training phase is to reduce the misclassified patterns as much as possible. In this paper, we specify only two classes: (1) normal and (2) abnormal. The labelling is done based on the frequent patterns and infrequent patterns. Figure 2 shows the ANFIS classifier architecture with distance d between the topics and moving direction h as the input variables. The linguistic values for variable d are LARGE, MEDIUM and SMALL. Similarly, for variable h 1 and h 2 the values are EAST, NORTH, WEST and SOUTH directions characterized by bell-shaped membership functions. The training data are categorized by two classes C 1 and C 2. Each input d, h 1 and h 2 are represented by three, four and four linguistic values, respectively; thus, we have 48 rules. The input d, h 1 and h 2 are given as input to layer 1. The output of the node i in layer 1 is the degree to which the given input satisfies the linguistic value associated with that node. Bell-shaped membership functions are used as follows:

7 A transfer learning framework for traffic video 1437 " l 1;i ðdþ ¼exp 1 # d m 2 2 1;i ; i ¼ 1; 2; 3 2 s 1;i " l 2;i ðh 1 Þ¼exp 1 # h 1 m 2 2 2;i ; i ¼ 1; 2; 3; 4 2 s 2;i " l 3;i ðh 2 Þ¼exp 1 # h 2 m 2 2 3;i ; i ¼ 1; 2; 3; 4: 2 s 3;i The parameters d, h 1 and h 2 represent the linguistic variables, where {m 1, m 2, m 3, s 1, s 2, s 3 } represents the parameter set. The bell-shaped functions vary accordingly to these parameters values. We set the initial values of these parameters such that they satisfy the properties like completeness, normality and convexity. The parameters are then tuned with a descent-type method. In layer 2, all the nodes represent some particular rule. All the nodes output the signal representing the conjunctive combination of each degree of match. This layer outputs the firing strength w i of the fuzzy rule with reference to the input parameter concerned. Minimum t-norm operator T min (d, h 1, h 2 ) is used in this paper, as it is common in most applications. The output in this layer is calculated using the equation w i ¼ min l 1;i ðdþ; l 2;i ðh 1 Þ; l 3;i ðh 2 Þ ; i ¼ 1; 2;...; 48 Each rule in layer 2 represents a certain regular motion pattern followed by vehicles. Frequent motion patterns correspond to normal vehicle behaviour and infrequent patterns are termed as abnormal events. During offline training, all the afore-mentioned parameters fire only the corresponding node in layer 3. The main functionality of layer 3 is normalization of all the firing strengths w i of the rules in layer 2 and are represented as follows: w i w i ¼ P i¼1;...;48 w ; i ¼ 1; 2;...; 48: i where w i denotes normalized firing strengths. In layer 4, the output of each node is calculated by multiplying the normalized firing strength w i and a zero-order polynomial. Thus the outputs of this layer are given by N i ¼ w i f i where f i = r i, i = 1, 2,, 48. Finally, layer 5 consists of only one node. It determines the degree of belongingness to a class. This node computes the summation of all the input signals and is given by S ¼ X48 i¼1 w i f i From this discussion, clearly the first layer and fourth layer are two adaptive layers in this ANFIS architecture. In first layer, {m 1, m 2, m 3, s 1, s 2, s 3 } are called premise parameters. The fourth layer parameters {r i } belonging to a zero-order polynomial are called as consequent parameters. We use the back-propagation gradient descent technique for the learning of both premise and consequent parameters. To speed up the learning process, we used the winner takes all concept. In this concept, only the fuzzy rule of stronger firing strength will adjust its premise and consequent parameters to match with expected output. 5. Results and discussion In this section, we illustrate the effectiveness of the proposed ANFIS-LCTE approach on TL property and online anomaly detection. First, details about the dataset and performance metrics are presented in the experimental set-up section. Next, we present a detailed discussion on the results of topic extraction, semantic rule extraction and cross-domain knowledge transfer. Finally, we evaluate the performance using ROCs of the proposed ANFIS-LCTE approach. 5.1 Experimental set-up Datasets: We validate the proposed ANFIS-LCTE approach on two publicly available datasets, namely junction dataset [25] and roundabout dataset [26], with far field view. Both of these datasets have video traffic pattern of moving vehicles of length 1.5 h, frame size and frame rate 25 fps. As a pre-processing step, frame size is readjusted to for both the datasets and one frame is processed for every 6 frames. These datasets have both traffic-light-regulated regular motion patterns and illegal U-turn-based irregular motion patterns. In one scenario, we take junction dataset [25] as source dataset and test the proposed ANFIS-LCTE model on other roundabout junction dataset [26]. Similarly, in the other scenario, we take the roundabout dataset [26] as source dataset and test the proposed ANFIS-LCTE model on junction dataset. Finally, a study of detection accuracy on the target scene other than source is presented for both the scenarios. Performance metrics: In this paper, the following parameters are used for evaluating the proposed ANFIS- LCTE approach. True positive rate (TPR) it is defined as a measure of positive frames correctly classified with respect to the total number of positive frames in the video sample: No: of True Positive Frames TPR ¼ Total No. of Positive Frames False positive rate (FPR) it is defined as a measure of positive frames incorrectly classified with respect to the total number of negative frames in the video sample: FPR ¼ No: of False Positive Frames Total No. of Negative Frames Finally, detection accuracy is compared by drawing ROC with/without TL approach.

8 1438 P M Ashok Kumar and V Vaidehi Figure Topic generation Topics extracted from traffic junction dataset. In our experiments, we divide each frame into grid size of pixels for providing spatial abstraction location. These video frames are given as input to the LCTE algorithm (as shown in table 2). First, low-level input features are extracted from each sequence transaction, which is composed of seq_len frames. Each window sequence comprises win_len transactions. The values used in our experiment are set to seq_len = 10 and win_len = 100. For each window of sequences, infrequent patterns are labelled as positive sample based on the condition f(d j ) \ (s - e). The s value is chosen as 10% of the window sequence win_len and e as 0.1% of s. Frequent patterns are labelled as negative samples. Finally, topics are extracted by grouping the items in frequent patterns based on the direction h. All the frequent patterns and topic generation are done in offline unsupervised manner. Figure 3 shows the output of LCTE algorithm on traffic junction dataset [25], with topics shown in the coloured region. Eight topics are discovered with distinguished spatial and temporal regions. These topics capture the semantic information contained in the video. Similarly, figure 4 presents the topics extracted from the roundabout traffic junction [24]. We discovered six topics represented with distinct colour. Clearly, from these experiments, we conclude that any video sequence is represented as a linear combination of the topics extracted. Figure 4. dataset. Topics extracted from roundabout traffic junction ANFIS classifier. Each ANFIS classifier was implemented using the MATLAB software. The datasets were divided into two separate datasets the training dataset and the testing dataset. The training dataset was used to train the ANFIS model, whereas the testing dataset was used to verify the accuracy and the effectiveness of the trained ANFIS model for classification of the two classes. The step size used in this paper for parameter adaptation is initially set to The step size is decreased based on the error values, with two consecutive combinations of an increase followed by a decrease. If the error measure undergoes four consecutive decreases, then we increase the step size. Training 5.3 Semantic rule extraction After topic extraction, the parameter distance d and angle h are calculated for each pair of topics in the video sequence. These parameters along with the label are given as input to Figure 5. Final generalized membership function of input d, h 1 and h 2.

9 A transfer learning framework for traffic video 1439 Figure 6. ANFIS error convergence curve. is continued until error values reach and error convergence curve is shown in figure 5. After training, the modified membership functions are drawn in figure 6 and some of the generated rule base is shown in table 5. We use the remaining test data to validate the accuracy of the ANFIS model for classification of the video scenes. In this paper, the processes of topic extraction and semantic rule generation are carried out in offline unsupervised way. domain. This is tested using two scenarios. In the first scenario, traffic junction data are taken as source dataset and training is done in offline unsupervised manner. Testing is done in online mode with roundabout traffic junction taken as target dataset. In another scenario, datasets are exchanged and the final results are shown in figure 7. Figure 7a, c and e presents the anomaly patterns in the source dataset and its corresponding detected anomalies in target dataset are presented in figure 7b, d and f. In our experiments, positive class represents anomaly and negative class represents normal behaviour. Figure 8 shows the scenario of coloured vehicles moving upwards in the frame, while normal vehicles are moving in the opposite side of the lane. Though figure 8a scenario is normal in traffic junction dataset, this is considered as illegal in roundabout and is shown under false negative in this case. Similarly, the scenario in figure 8b is illegal in traffic junction, even though this case does not arise in roundabout junction. Clearly, the performance of the system degrades only due to mismatch with varying number of lanes between source and target scenes. 5.4 Cross-domain knowledge transfer In our work, we meet the TL property by transferring the semantic knowledge acquired in source domain to target 5.5 Performance comparison We compare the performance of the proposed approach by drawing ROC curves with/without TL framework. Table 5. Rule base of the ANFIS.

10 1440 P M Ashok Kumar and V Vaidehi Figure 7. (a, c, e) Traffic junction dataset as source dataset and (b, d, f) the corresponding output in roundabout traffic junction. Figure 8. (a) False negative with traffic junction as source dataset and (b) false negative with roundabout traffic junction as source dataset. ROC curves are drawn with FPR on x-axis and TPR on y-axis for different values of parameters. Figure 9 shows the classifier performance on roundabout Figure 9. ROC curve with traffic junction source dataset and roundabout junction dataset as target dataset.

11 A transfer learning framework for traffic video 1441 Figure 10. ROC curve with roundabout source dataset and traffic junction dataset as target dataset. junction dataset and compared without TL framework. Similarly, figure 8 shows the classifier performance on traffic junction dataset with comparison to classifier without TL framework. In both figures 9 and 10, TL framework performance can be improved by taking similar datasets taken in source and target data. The results show that the proposed ANFIS-LCTE method of semantic level knowledge transfer gives encouraging results and new direction of research in the field of video analytics. 6. Conclusion This paper has proposed a novel approach in transferring semantic knowledge of normal and abnormal events using a neuro-fuzzy approach. The main contribution of this paper includes a novel low-level feature representation, LCTE algorithm and application of ANFIS. We extract topics of each video sequence and fuzzify the topic parameters to check the normal behaviour using an ANFIS classifier. We conduct experiments on two datasets, namely, traffic video and junction dataset. Results are analysed with one video as source dataset and other as target dataset. The same is repeated with swapping of the datasets. A comparative study with and without TL framework is presented using ROC curves. Results are qualitatively consistent with the activities occurring in the scene. References [1] Hu W, Xiao X, Fu Z, Xie D, Tan T and Maybank S 2006 A system for learning statistical motion patterns. IEEE Trans. Pattern Anal. Mach. Intell. 28(9): [2] Makris D and Ellis T 2005 Learning semantic scene models from observing activity in visual surveillance. IEEE Trans. Syst. Man Cybern. B 35(3): [3] Morris B T and Trivedi M M 2011 Trajectory learning for activity understanding: unsupervised, multilevel, and longterm adaptive approach. IEEE Trans. Pattern Anal. Mach. Intell. 33(11): [4] Fu W et al 2012 Learning semantic motion patterns for dynamic scenes by improved sparse topical coding. In: Proceedings of the IEEE International Conference on Multimedia and Expo, Melbourne, Australia, pp [5] Hu W et al 2013 An incremental DPMM-based method for trajectory clustering, modeling, and retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 35(5): [6] Rana S et al 2012 Large-scale statistical modeling of motion patterns: a Bayesian nonparametric approach. In: Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing, Mumbai, India, pp. 1 8 [7] Hu W, Xiao X, Fu Z, Xie D, Tan T and Maybank S J 2006 A system for learning statistical motion patterns. IEEE Trans. Pattern Anal. Mach. Intell. 28(9): [8] Morris B T and Trivedi M M 2008 A survey of vision-based trajectory learning and analysis for surveillance. IEEE Trans. Circuits Syst. Video Technol. 18(8): [9] Li J, Gong S and Xiang T 2012 Learning behavioral context. Int. J. Comput. Vis. 97(3): [10] Morris B T and Trivedi M M 2008 Learning and classification of trajectories in dynamic scenes: a general framework for live video analysis. In: AVSS Proceedings, pp [11] Hospedales T, Gong S and Xiang T 2012 Video behaviour mining using a dynamic topic model. Int. J. Comput. Vis. 98(3): [12] Wang X, Ma X and Grimson E L 2009 Unsupervised activity perception in crowded and complicated scenes using hierarchical Bayesian models. IEEE Trans. Pattern Anal. Mach. Intell. 31(3): [13] Rabiner L and Juang B 1993 Fundamentals of speech recognition. Englewood Cliffs, New Jersey: Prentice-Hall [14] Gruber A, Rosen-Zvi M and Weiss Y 2007 Hidden topic Markov model. In: Proceedings of Artificial Intelligence and Statistics (AISTATS), pp [15] Blei D and Lafferty J 2006 Dynamic topic models. In: Proceedings of the 23rd International Conference on Machine Learning, pp [16] Hospedales T, Gong S and Xiang T 2009 A Markov clustering topic model for mining behavior in video. In: Proceedings of ICCV, Kyoto, Japan, pp [17] Hu W, Xiao X, Fu Z, Xie D, Tan T and Maybank S 2006 A system for learning statistical motion patterns. IEEE Trans. Pattern Anal. Mach. Intell. 28(9): [18] Noceti N and Odone F 2012 Learning common behaviors from large sets of unlabeled temporal series. Image Vis. Comput. 30(11): [19] Piciarelli C and Foresti G L 2006 On-line trajectory clustering for anomalous events detection. Pattern Recogn. Lett. 27(15): [20] Vasquez D, Fraichard T and Laugier C 2009 Incremental learning of statistical motion patterns with growing hidden Markov models. IEEE Trans. Intell. Transp. Syst. 10(3): [21] Makris D and Ellis T 2005 Learning semantic scene models from observing activity in visual surveillance. IEEE Trans. Syst. Man Cybern. B 35(3): [22] Almajai I et al 2012 Anomaly detection and knowledge transfer in automatic sports video annotation. In: Detection

12 1442 P M Ashok Kumar and V Vaidehi and identification of rare audio visual cues, Studies in Computational Intelligence, vol. 384, pp [23] Xu X, Gong S and Hospedales T 2013 Cross-domain traffic scene understanding by motion model transfer. In: Proceedings of the 4th ACM/IEEE International Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Stream, pp [24] Lucas B D and Kanade T 1981 An iterative image registration technique with an application to stereo vision. In: Proceedings of the Image Understanding Workshop, pp [25] Li J, Gong S and Xiang T 2012 Learning behavioural context. Int. J. Comput. Vis. 97(3): [26] Loy C C, Hospedales T M, Xiang T and Gong S 2012 Stream-based joint exploration exploitation active learning. In: Proceedings of the IEEE International Conference on CVPR, pp