OBSERVING and reasoning about object motion patterns

Transcription

1 322 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO. 2, FEBRUARY 2016 Mixture of Swithing Linear Dynamis to Disover Behavior Patterns in Objet Traks Julian F. P. Kooij, Gwenn Englebienne, and Dariu M. Gavrila Abstrat We present a novel non-parametri Bayesian model to jointly disover the dynamis of low-level ations and high-level behaviors of traked objets. In our approah, ations apture both linear, low-level objet dynamis, and an additional spatial distribution on where the dynami ours. Furthermore, behavior lasses apture high-level temporal motion dependenies in Markov hains of ations, thus eah learned behavior is a swithing linear dynamial system. The number of ations and behaviors is disovered from the data itself using Dirihlet Proesses. We are espeially interested in ases where traks an exhibit large kinemati and spatial variations, e.g. person traks in open environments, as found in the visual surveillane and intelligent vehile domains. The model handles real-valued features diretly, so no information is lost by quantizing measurements into visual words, and variations in standing, walking and running an be disovered without disrete thresholds. We desribe inferene using Markov Chain Monte Carlo sampling and validate our approah on several artifiial and real-world pedestrian trak datasets from the surveillane and intelligent vehile domain. We show that our model an distinguish between relevant behavior patterns that an existing state-of-the-art hierarhial model for lustering and simpler model variants annot. The software and the artifiial and surveillane datasets are made publily available for benhmarking purposes. Index Terms Human behavior analysis, hierarhial non-parametri graphial model, swithing linear dynamial systems Ç 1 INTRODUCTION OBSERVING and reasoning about objet motion patterns are key tasks in numerous real-world appliation domains, suh as visual surveillane (e.g., [1], [2], [3], [4]), robotis (e.g., [5]), and intelligent vehiles (e.g., [6], [7], [8]). Computer vision and mahine learning tehniques an apture and analyze person trajetories, and detet anomalous movements that deviate from normative behavior found in the training data. In fixed-amera video surveillane, this ould aid human operators to monitor many video streams, and fous their attention on possible inidents. In the intelligent vehiles domain, aurate objet motion models enable sophistiated driver assistane systems to traks multiple objets simultaneously, and perform path predition for situation analysis and objet avoidane. A few issues arise when modeling omplex objet motion patterns from low-level observations. First, how is highlevel behavior strutured and omposed of low-level ations; seond, where in the observed spae does speifi behavior our; third, how an temporal dynamis of behavior be exploited? Ideally, ation deomposition, spatial ontext, and temporal dynamis should be inferred jointly from the training data. Some previous work [1], [2], J. F. P. Kooij and G. Englebienne are with the Intelligent Systems Lab, University of Amsterdam, Siene Park 904, 1098 XH Amsterdam, The Netherlands. julian.kooij@gmail.om, G.Englebienne@uva.nl. D. M. Gavrila is with the Intelligent Systems Lab, University of Amsterdam, Siene Park 904, 1098 XH Amsterdam, The Netherlands, and the Environment Pereption Department, Daimler R&D, Ulm, Germany. dariu@gavrila.net. Manusript reeived 3 Ot. 2014; revised 20 Feb. 2015; aepted 22 Apr Date of publiation 9 June 2015; date of urrent version 13 Jan Reommended for aeptane by S. Nowozin. For information on obtaining reprints of this artile, please send to: reprints@ieee.org, and referene the Digital Objet Identifier below. Digital Objet Identifier no /TPAMI [3] has modeled behavior at the image level to apture patterns that govern the whole sene (e.g., monitoring traffi flow at juntions). We, however, target individual behavior patterns where exeution of the same movement may have large spatial and kinemati variations. In this paper we propose a mixture of swithing linear dynami systems to disover normative ations and their temporal relations at the objet level. Ations desribe lowlevel motion dynamis ourring in a semanti region using traked objet loations as observations. We use ontinuous distributions in the feature spae to apture variane in ation exeution. As Fig. 1 illustrates, our unsupervised approah segments traks into sequenes of ommon ations and jointly lusters the ation sequenes into distint behavior lasses. In our Bayesian inferene sheme, the number of ations and the number of behaviors are not fixed but disovered from the data itself using Dirihlet proess (DP) mixture models. 2 PREVIOUS WORK Motion models are essential to tasks as traking [9], [10], [11], [12], path predition [6], [7], [13], [14], and anomalous trak detetion [15], [16], found in a variety of appliation domains ranging from intelligent vehiles [6], [7], [8], [11], [12], [13], [17] to surveillane [10], [15], [16], [18], [19]. For instane, methods that build traks from objet detetions [11], [20] use motion models to redue false positives and resolve data assoiation [11], [12]. When both the motion dynamis and observations an be expressed as a linear transformation of a latent state with added Gaussian noise, then the resulting model is a linear dynamial system (LDS). The Kalman filter (KF) [9] an then be used for effiient online inferene, expressing unertainty over the state by a single normal distribution. For ß 2015 IEEE. Personal use is permitted, but republiation/redistribution requires IEEE permission. See for more information.

2 KOOIJ ET AL.: MIXTURE OF SWITCHING LINEAR DYNAMICS TO DISCOVER BEHAVIOR PATTERNS IN OBJECT TRACKS 323 Fig. 1. Inferring a mixture of swithing linear dynami systems from traks (see Setion 1). Blak rosses are observations. Traks are segmented into ations, eah ation desribing a semanti region (2D Gaussian) and motion as a linear dynamial system (mean motion shown as arrow). Behaviors luster traks with similar ation hains. In this example, four ations and three behaviors are inferred. The red and green ations distinguish walking and standing in trak 1. Traks 3 and 4 are lustered in the same behavior, but trak 2 has a behavior with different ation order. bath proessing, the Kalman Smoother [21] extends the KF by a bakward pass to inorporate information from future time steps. However, a LDS annot represent alternating dynamis, suh as a pedestrian hanging from walking to standing [6], [7]. For suh maneuvering targets, Swithing Linear Dynamial Systems (SLDS) [22], [23] use a latent toplevel Markov hain of swithing states to selet one of K possible linear dynamis eah time step. The K swithing states have their own prior and transition probabilities. Unfortunately, exat inferene in a SLDS is intratable as the number of modes in the state distribution grows exponentially over time with K, the number of the swithing states [22], [24]. For online real-time inferene, assumed density filtering (ADF) [25], [26] approximates the state posterior at every time step with a simpler distribution in losed form. ADF an be applied to disrete state Dynami Bayesian Networks, known as Boyen-Koller inferene [27], and more generally to mixed disrete-ontinuous state spaes with onditional Gaussian posterior [28] suh as the SLDS [6]. Interating Multiple Model KF (IMM-KF) [9] mixes the states of several KFs running in parallel, whih has lower omputational ost than ADF for SLDS, and is ommonly found in the intelligent vehile domain [7], [13]. To learn parameters, however, omplete sequenes of swithing states must be estimated jointly, e.g. using EM, Viterbi or Variational inferene [24]. [22] presents a Markov hain monte arlo (MCMC) method to approximate the swithing state posterior with samples, while integrating over the latent positions. MCMC sampling has also proven useful when extending the SLDS, e.g., to impose distributions on persistent state durations [29]. Note that one ould learn model parameters using MCMC, and then perform online inferene using IMM-KF [30]. This paper will fous on unsupervised offline learning of motion models, but similarly does not restrit how learned models are utilized for online inferene. Another key diffiulty when learning SLDS states, and mixture models in general, is determining the number of omponents or states K. We therefore take a moment to review the use of Dirihlet proess in non-parametri mixture models that learn the number of omponents. DP mixture models are often desribed as a Stik-breaking prior over any number of omponent weights, and a base distribution over omponent parameters [23], [31]. Stik-breaking refers to the analogy of iteratively breaking off a part from a unitlength stik, so the lengths of all parts (plus remainder) always sum to one, and represent omponent weights. As this iterative proess ould in priniple be repeated indefinitely, the resulting distribution is said to be infinite dimensional. A draw from a DP yields an existing omponent with probability proportional to its stik length, or a new one proportional to the remainder (in whih ase new parameters are sampled from the base distribution, and a part is broken of the remainder). A onentration parameter expresses prior preferene for few large or many small lusters. The posterior on omponent weights is always finite, refleting the number of omponents that the data atually supports. DPs are therefore enountered in unsupervised learning of mixtures in Markov models [23], [32], and topi models for text doument analysis [31] (whih have also been applied to behavior modeling tasks in omputer vision [1], [2], [3], [15]). For instane, the hierarhial Dirihlet proess (HDP) [31] extends latent dirihlet alloation (LDA) [33], and disovers ommon topis in a orpus of unlabeled douments represented as a bag-of-words [33]. HDP regards eah doument as a DP mixture of topis, where topis (i.e. mixture omponents) are distributions over words. The doument DPs use as base distribution one global DP on topi parameters, hene douments draw topis from a shared infinite topi mixture. Multi-level mixtures are also used in the infinite hidden Markov model (HDP-HMM), whih learns the number of states in the Markov hain [31], [32]. The HDP- HMM has been further extended to the HDP-SLDS [23] to learn swithing states. Previous work shows that models for multiple dynamis an benefit from identifying where dynamis our (spatial ontext) and what the long-term behavior is (temporal ontext). For instane, [6] improves pedestrian path predition in intelligent vehiles with a SLDS where state transitions are informed by ontextual ues of sene layout and pedestrian attention (determined by head orientation). Online inferene with ADF predits various behaviors (e.g. rossing or halting at urb while (un)aware of the approahing vehile) that are omposed of two dynamis (walking and standing). Unsupervised learning of behavior usually involves lustering trak data, e.g with a pair-wise distane measures (suh as Eulidean [34] or Hausdorff [35] distane), dynami time warping (DTW) [36], or by longest ommon subsequene mathing [7]. These methods are however not probabilisti [34], [35], [36] or omputationally demanding [6], [7], and the omplexity of lustering N trajetories is OðN 2 Þ (.f. [15]). [37] lusters traks with similar start and exit regions, and learns a LDS per luster. These trak lusters provide useful insight in the underlying behavior patterns, and are exploited for rowd simulation. Instead of lustering full traks, others luster observations into semanti regions for ativity modeling and anomaly detetion [35], [38], [39]. [14] proposes a Markov deision proess to model the influene of the sene layout onto future pedestrian ations, and predit trak paths towards various possible endpoints. Here, labeled regions found by a semanti image segmentation step, inform where people tend to walk (e.g., road or sidewalk). Learned

3 324 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO. 2, FEBRUARY 2016 regions an also represent dynamis as loal flow fields of the ontinuous position/veloity pairs using Lie algebra [40], or Gaussian Proesses [5], but these methods do not onsider alternating dynamis within a single trak. Dual-HDP [15] on the other hand is a topi model that uses multi-level DPs to jointly luster unlabeled traks into behavior lasses, and deompose them into regions of motion. Eah trak is seen as a doument represented as a bag-of-words, words being quantized position / motion pairs. Topis thus luster o-ourring words, forming a region of motion, and traks with similar topi distributions are lustered into behaviors lasses. A trained HDP an detet unexpeted movements in ommon areas (e.g. vehile moving against traffi) or unommon ombinations of areas (e.g. u-turn at juntion). However, quantization an lead to sparse bins, and learned models are not suited for appliations relying on LDS [6], [9], [13]. While Dual-HDP learns long-term behavior of individuals from traked objet paths, there are also methods that ombine temporal dynamis and topi models for senewide analysis of video. With quantized optial flow features as visual words, topis desribe regions of o-ourring motion within frames. In [1] dynamis are modeled using a Markov hain on top of LDA. The state of the hain determines the topi distribution at eah frame. This approah is used to learn models for traffi juntions where the Markov hain aptures the dynamis of traffi flow. [2] extends this to an infinite mixture of infinite Markov hains with HDP, giving more flexibility. In [3] a ombination of HDP with Probabilisti Latent Sequential Motifs [41] is used to sequential patterns of sene wide flow. 3 PROPOSED APPROACH Our method proesses trak data, e.g., obtained from traked or annotated objet paths, where eah trak is an ordered list of 2D measured positions on the ground plane. In our unsupervised approah, traks are lustered into behaviors, eah behavior defining transition probabilities between ations, whih we refer to as topis in the remainder of this paper, to follow the nomenlature of topi models. Eah topi desribes for a ommon ation the spatial area, and the low-level motion dynamis with an LDS. Topis an be shared among behaviors, thus multiple behaviors may ontain the same topi but use different topi transition probabilities. Sine eah behavior is a SLDS, the full model forms a Mixture of SLDS. The temporal dynamis apture topi duration with the self-transition probability, and help distinguish between behaviors with spatially overlapping ations. In Fig. 1 for instane, traks 2 and 3 have the same ations but different behaviors. 3.1 Contributions Our main ontributions are (1) a hierarhial model to jointly disover what and how many low-level ations and high-level behavior lasses are present in unlabeled trak data, for whih (2) we present an MCMC inferene sheme. Our model (3) extends SLDS swithing states with spatial distributions (spatial ontext), and (4) an disriminate behaviors with different ation orders (temporal ontext). (5) Features are not quantized, but ations apture motion and variane diretly in the ontinuous feature spae. Dual-HDP [15] also jointly disovers ations and behavior lasses, but its bag-of-words representation does not apture temporal order, and quantizing position/motion pairs an lead to sparse data, even at low binning resolution. Like [15], we derive spatial and motion distributions from traked 2D positions, but unlike [15] use Gaussians and LDSs to represent both in the ontinuous feature spae. While SLDS has been ombined with HDP [23] before, the ombination of SLDS with spatial distributions and multi-level trak lustering is novel. Trak lusters provide insight in the types of long-term behavior in the data, and indue higher-order dependenies between ations, as opposed to a single SLDS [23] that only models first-order dependenies. This paper is based on our earlier onferene papers [16] and [42]. In [16] we learned spatial distributions of where different states our, while lustering the traks that exhibit similar state transitions. Compared to this onferene submission, we propose an improved sampling method, and extend our evaluation with more methodologial omparisons on datasets from different appliation domains. We use data from [42], where no multi-level lustering was disussed. 3.2 Multi-Level Clustering The data onsists of J traks, eah being a sequene of T j spatial 2D measurements x jt with j the trak index and t the time index. In our model the indiator variable z jt ¼ k indiates that observation x jt is generated by topi k. To simplify notation we define x j ¼fx j1 ;...;x jtj g, z j ¼fz j1 ;...;z jtj g, and we denote the supersript t in z t j to indiate all z jt of trak j exept t (throughout the paper we use boldfae notation for vetors or sets that an be indexed). Measurements x jt are used as both spatial and dynami features whih we denote as x lo jt and x dyn jt respetively. For larity of exposition, this setion first desribes multi-level lustering in our model without the low-level motion dynamis. These will be inluded in Setion 3.3. Eah topi k defines a probability distribution over the loation x lo jt on the ground plane (i.e., a semanti region) as a 2D Gaussian parametrized by m k ; S k. Further, eah trak is assigned to a behavior, indexed by j, where a behavior, or luster, defines the topi transition probabilities using a vetor ~p~p k ¼f~pkð1Þ ; ~p kð2þ ;...g for eah k, i.e pðz jt ¼ k 0 jz jt 1 ¼ k; Þ ¼~p kðk0 Þ 1. We add extra indies z j0 ¼ init for the start of the topi hain, suh that pðz j1 jinit; Þ is the initial topi distribution, and z jtj þ1 ¼ exit for trak termination [37]. z denotes all z j with j ¼,andx j, z j, j are respetively all observations, topi labels and behavior labels exept those of trak j. The multinomial topi distributions ~p~p k are sampled from a DP over a behavior-speifi topi distribution p. The various p are sampled from a DP over p 0, whih is the global topi distribution shared by all behaviors. The distribution p 0 follows a Stik-breaking onstrution and thus represents a multinomial distribution over infinite topis, although at any time during inferene only some K topis will atually 1. We shall adopt the ommon notation for this ategorial distribution as an instane of the multinomial distribution, sine indiator variables an be written as 1-of-K vetors [25].

4 KOOIJ ET AL.: MIXTURE OF SWITCHING LINEAR DYNAMICS TO DISCOVER BEHAVIOR PATTERNS IN OBJECT TRACKS 325 Fig. 2. The Mixture of SLDS, see Setions 3.2 and 3.3. Square nodes are disrete, dashed arrows are temporal dependenies. Any time during inferene, K topis and C behaviors are represented. Trak j onsists of T j observations x jt, latent positions y jt, topi label z jt and has behavior label j drawn from distribution. p is the topi distribution in behavior, and ~p~p k are transition probabilities from topi k. Behaviors sample topis from global topi distribution p 0. R is the observation noise. m k ;Q k are the LDS and m k, S k the spatial region (Gaussian) parameters of topi k. a; b; g; d are DP onentration hyperparameters, R ; Q ; R are priors on the ontinuous distributions. be used. Note how a behavior desribes a HDP-HMM [31], where the f~p~p k g form the rows of the transition matrix. Further, the distribution p onentrates probability mass on a subset of the topis present in p 0, and the transition matries ~p~p are onstrained to use those topis from p. Without the p and orresponding prior, distint behaviors that do not share topis tend to be merged. The hierarhial model, whih is represented graphially in Fig. 2, an be summarized as p 0 jd StikðdÞ (1) p jp 0 ; a DPða; p 0 Þ (2) ~p~p k jp ; b DPðb; p Þ (3) z jt jz jt 1 ; j ; f~p~p k gmultð~p~pz jt 1 j Þ (4) x lo jt jz jt; fm k ; S k gnðm zjt ; S zjt Þ (5) m k ; S k j Q NW 1 ð Q Þ; (6) where a, b and d are onentration hyper-parameters. Q ¼ðm Q 0 ; kq 0 ; nq 0 ; CQ 0 Þ are the hyper-parameters for the Normal-Inverse-Wishart (NIW) distribution [25], whih is the onjugate prior for the mean and ovariane of a multivariate Normal. Behavior labels j are sampled from the multinomial, whih also follows a Stik-breaking distribution, jg StikðgÞ; j j MultðÞ: (7) 3.3 Low-Level Dynamis The hierarhial model is extended with a SLDS on the labels z j by introduing latent 2D-position variables y jt. Topis now not only define a distribution over the 2D spae, but also the low-level dynamis of the position sequene. In fat, topi labels z j form a Markov hain of swith variables whih selet the stohasti state dynamis. y jt ¼ Ay jt 1 þ q jt q jt Nðm zjt ;Q zjt Þ (8) x dyn jt ¼ Cy jt þ r jt r jt Nð0;RÞ: (9) Matries A and C are fixed and determine the type of kinematis used. In our experiments we set A and C to identity, resulting in a fixed-veloity model where the learned veloity is aptured in the mean of the proess noise, m zjt. Unlike in [16], we let the topis share the observation noise parameters R, sine this noise is typially a systemati error that does not depend on the topi z jt. Using appropriate priors on the noise omponents, the model now also defines y jt jy jt 1 ;z jt ; fm k ;Q k gnðay jt 1 þ m zjt ;Q zjt Þ (10) x dyn jt jy jt ;RNðCy jt ;RÞ (11) m k ;Q k NW 1 ð Q Þ; R W 1 ð R Þ (12) with Q ¼ðm Q 0 ; kq 0 ; nq 0 ; CQ 0 Þ and R ¼ðn R 0 ; CR 0 Þ the hyperparameters for a NIW and Inverse-Wishart (IW) distribution respetively. 4 BAYESIAN INFERENCE Posterior inferene in our model relies on two types of Markov hain monte arlo sampling, namely Gibbs and Metropolis-Hastings (MH). Generally, MCMC onstruts a Markov hain of samples whih has as stable distribution the joint posterior of a model s latent variables s ¼fs 1 ;s 2 ;...g. Gibbs sampling draws eah s j in turn from their posterior pðs j js j ; xþ, while keeping the other s j fixed. However, in some ases this posterior is hard to obtain. The MH rule states that we an then instead sample a andidate s $ j q from a more onvenient proposal distribution qðs $ j js j; s j Þ, and aept the proposal s $ j with probability A, or rejet it and repeat the urrent s j otherwise, where A¼min pðs$ j jx; s j Þ qðs j js $ j ; s j Þ pðs j jx; s j Þ qðs $ j js j; s j Þ ; 1! : (13) The ratio in this term effetively removes the bias that is introdued by sampling from the proposal distribution q instead of the true distribution p. 4.1 Dirihlet Proess Hierarhy As in the hierarhial Dirihlet proess [31], the Stik-breaking distribution over p 0 does not need to be implemented expliitly. It an be approximated by a Dirihlet distribution, using onentration parameter d in a parameter vetor of fixed length K [31], p 0 Dirð½d=K;...; d=kšþ: (14) This yields the same distribution as the Stik-breaking prior in the limit of K!1. In pratie, we fix K to an upperlimit on the atual number of lusters, whih is disovered from the data, but redues the burden on updating all data strutures for new or deleted topis. The atual number of lusters is found by ounting the number of non-zero elements in p 0. The prior on eah p an also be expressed [31]

5 326 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO. 2, FEBRUARY 2016 as a Dirihlet distribution p Dirðap 0 Þ, and the prior on eah row in a transition matries as ~p~p k0 Dirðbp Þ. We now first formulate some posteriors that will be used in the next setions. The transition matrix an be integrated out analytially to obtain the probability of a topi sequene z j for given p (.f. [31]), pðz j jp Þ¼ Y Z p m k j j~p~pk p ~p~p k jbp d~p~p k (15) k ¼ Y " Gð P # k 0 bpðk0 Þ Þ Y Gðbpðk0Þ k Gð P þ m kðk0 Þ j Þ : Þ k 0ðbpðk0 þ m kðk0 Þ j ÞÞ k 0 Gðbp ðk0 Þ Þ Here m kðk0 Þ j are the number of state transitions from z jt 1 ¼ k to z jt ¼ k 0 in trak j. m kðk0 Þ ¼ P Þ jj j ¼ mkðk0 j are the total transition ounts in behavior. If topis z j have been assigned to behavior, the posterior pðz j jp ; z j Þ is obtained by adding the orresponding topi ourrene ounts to the prior [31], i.e., substituting bp ðk0 Þ for bp ðk0 Þ þ m kðk0 Þ in Eq. (15). For a single transition of j k at time t 1 to k 0 at t, as used in Eq. (22), this posterior redues to just pðz jt ¼ k 0 jz jt 1 ¼ k; p ; z jt Þ/bp ðk0 Þ þ m kðk0 Þ : (16) j As we see here, lustering is a result from ommon transitions obtaining a higher prior probability. Next, note that the Dirihlet prior on p is not onjugate with Eq. (15), but we an employ the auxiliary variables sheme of [31] to obtain t kðk0 Þ, whih represent the number of times topi k 0 would have been drawn from the base distribution p in the DP of ~p~p k. So the posterior on p an then be written as pðp jp 0 ; z Þ¼Dir p jap 0 þ X! t k0 : (17) k 0 Eq. (16) and (17) will be used in the next setion. The auxiliary variable sheme is also used to obtain the posterior pðp 0 jfp gþ, but then with auxiliary variable representing draws from p 0. We use this posterior to Gibbs sample toplevel topi distribution p Assigning Traks to Clusters In [16] we also Gibbs sampled topi labels z j, while keeping luster label j fixed, and in turn we sampled j while keeping z j fixed. However, the behavior and topi labels are strongly orrelated, and these Gibbs samples an mix slowly. To improve mixing of traks between lusters, we here sample from pð j jx; j ; p 0 Þ, with z j not fixed, by performing a MH step to propose a label j. The target distribution is pð j jx; j ; p 0 Þ/pð j j j Þpðx j j j ; p 0 Þ: (18) In the first term, the prior over has been integrated out analytially. Let n j be the ourrene ount of behavior in j, then due to the Stik-breaking prior pð j ¼ j j Þ/ n j for existing lass (19) g for new lass ¼ new: We use Eq. (19) also as the MH proposal distribution. The MH aept-rejet also requires the likelihood pðx j j j ; p 0 Þ of observing a trak j in luster j. Let z j be topis from other traks assigned to j, then Z pðx j j j ¼ ; p 0 Þ¼ pðx j jz j Þpðz j jp ; z j Þpðp jp 0 Þ z j p (20) ¼ E zj ;p jz j pðx ; j jz j Þ ; j¼;p 0 i.e. the expeted data likelihood under the z j distribution in luster j (for the likelihood of j ¼ new, z j is an empty set). In a SLDS, omputing the data likelihood pðx j jz j Þ of a trak j with given state sequene z j is done straightforward with the Kalman filter equations, as the linear dynamis and transition matries at eah time step are fully speified. Still, the expetation annot be evaluated in losed form, but it ould be estimated empirially by drawing joint samples for ðz j ; p Þ from the prior pðz j jp ; z j Þpðp jp 0 Þ, and averaging the obtained values for pðx j jz j Þ. However, most samples from this prior will not ontain relevant values for z j where the likelihood term has any mass. Instead, we an use importane sampling, where we again utilize a proposal distribution q to obtain relevant samples ðz j ; p Þ. We rewrite Eq. (20) as pðx j j j ; p 0 Þ¼E q pðx j jz j Þ pðz jjp ; z j Þpðp jp 0 Þ (21) qðz j ; p Þ and ompute the expetany empirially from the samples of q. We find that posteriors pðp jp 0 ; z Þ from Eq. (17) and pðz j jx j ; z j ; p Þ, (explained in the next setion) provide good proposals qðz j ; p Þ. In summary, we first obtain posterior samples for z and p, similar to the Gibbs sampling sheme in [16]. However, [16] then onstruted a full posterior to sample j, onditioned on z j and p. Here, we empirially integrate over z j in Eq. (21) to estimate the posterior Eq. (18) for a proposed luster label j within a MH aept-rejet step. 4.3 Linear Dynamis Information Filter For the proposal distribution qðz j ; p Þ in Eq. (21), we need to sample a trak s latent topi hain z j from the posterior pðz j jx j ; z j ; p Þ for a andidate luster. Instead of sampling values for the hidden positions y j [23], we shall sample z jt sequentially from the distribution pðz jt jx j ; z jt ; p Þ [16], [22], [23]. In [22] it is shown that for a SLDS the Kalman information filter an be used to ompute effiiently in OðT j KÞ, pðz jt jx j ; z jt ; p Þ/pðx jt jz jt ; x j1:t 1 Þ p z jt jz jt 1 ; p ; z jt p zjtþ1 jz jt ; p ; z jt Z pðx jtþ1:tj jy jt ; z jtþ1:tj Þpðy jt jx j1:t ; z j1:t Þdy jt : (22) The first term is the preditive distribution of the observation, found by the forward Kalman filter, the seond and third terms are topi transition probabilities from Eq. (16). The integral an be analytially solved and expressed in term of the forward and bakward filtering statistis (.f. [22]). The filter also provides Kalman Smoother estimates pðy jt jx t j ; z jþ¼nðy jt jþ, whih will be used in Setion 4.4.

6 KOOIJ ET AL.: MIXTURE OF SWITCHING LINEAR DYNAMICS TO DISCOVER BEHAVIOR PATTERNS IN OBJECT TRACKS 327 There is again strong orrelation between the subsequent topi labels, and sampled sequenes an get stuk in a single mode. To explore alternative modes within a behavior, we first reinitialize all z j randomly, and then perform five iterations over the sequene with Eq. (22) to obtain a onsistent z j sample (typially, the z j onverge after only a few iterations). 4.4 Sampling SLDS Parameters The SLDS parameters are Gibbs sampled from their estimated posteriors. For instane, the prior on dynamis ðm k ;Q k Þ is NIW, see Eq. (12). Due to onjugay with Eq. (8), its posterior is also NIW with parameters Q þk ¼ðmQ þk ; kq þk ; nq þk ; CQ þk Þ. Given Q, and the N k motion vetors q jtjzjt ¼k assoiated to k by labels z, we ould ompute posterior parameters Q þk from the sample mean, ^q k, and satter matrix ^S k ¼ P jtjz jt ¼k ððq jt ^q k Þðq jt ^q k Þ > Þ (see [25]), k Q þk ¼ kq 0 þ N k n Q þk ¼ nq 0 þ N k (23) m Q þk ¼ k Q 0 =kq þk m Q 0 þ N k =k Q þk^qk (24) C Q þk ¼ CQ 0 þ ^S k þ kq 0 N k ^qk k Q m Q 0 ^qk m Q >: 0 (25) þk However, the true motion vetors q jt ¼ y jt Ay jt 1 (Eq. (8)), and observation noise vetors r jt ¼ x dyn jt Cy jt (Eq. (9)) depend on the values of all latent y j. While we do not know the true y j, reall that the Kalman Information Filter (Setion 4.3) also provided posterior distributions over these latent positions as normals, i.e., the output of a Kalman Smoother. One ould then sample pseudo-true positions y t from those posteriors [23], but this is omputationally demanding and may lead to slower onvergene. Instead, we use the smoothed estimates of the joint distributions Nðy jt 1 ;y jt jx j ; z j Þ to ompute the distributions over the motion vetors pðq jt jx j ; z j Þ, whih are normals too and parameterized as Nðq jt jm jt ; S jt Þ. In Eq. (24)-(25) we then use the expeted values of ^q k and ^S k, E½^q k Š¼ 1 N k X jtjz jt ¼k 1 E½q jt Š ¼ X N k jtjz jt ¼k m jt (26) E½ ^S k Š¼ X ðm jt E½^q k ŠÞðm jt E½^q k ŠÞ > þ S jt : (27) jtjz jt ¼k The same strategy an be applied to estimate the IW posterior parameters þ R on the observations noise R, using expetation and variane of the noise vetors r, and þk Q for the spatial distributions ðm k ; S k Þ. 4.5 Unusual Trajetory Detetion Given normative training data x j, anomaly detetion requires a measure of normality for unseen traks x j to rank these traks, or to set a threshold to isolate unusual from normal traks. The indiative measure proposed by [15] is the normalized log-likelihood, whih is log ðpðx j jx j ÞÞ=T j. Sine the log-probability is expeted to derease linearly with T j, the normalization ompensates for trak length. For Dual-HDP a Variational Bayesian (VB) approximation [33] is used to integrate over latent topi labels z j [15]. Following [33], this approximation introdues free variational parameters whih are optimized with respet to the lower-bound of log pðx j jx j Þ. For our model the same measure an be used, but we average over N MCMC samples fz jðnþ ; fp ðnþ g; jðnþ g from the model posterior given the training data: pðx j jx j Þ 2 3 ¼ 1 X X pð j j jðnþ Þ X pðx j jz j Þp z j jz jðnþ N j ; p ðnþ 4 j 5: n j z j (28) However, averaging over the number of observations will redue the effet of a single unlikely observation. Sine we also model the temporal order of topis, we would like to detet unlikely topi transitions too, even if suh a transition ours only one. We therefore propose a different measure E j ½min t pðx jt jx j ÞŠ whih penalizes unusual temporal transitions more. It is not neessary to average this measure over the number of observations. For Dual-HDP we estimate this term from the VB approximation s lower-bound, taking the minimum over the likelihoods of the quantized words for the x j. For our model we again use the MCMC samples, E j ½min pðx jt jx j ÞŠ ¼ 1 X t N n (29) X X 4 pð j j jðnþ Þ@ min pðx jt jz j Þpðz j jz jðnþ t j ; p ðnþ j ÞA5: j 5 SPLIT AND MERGE MOVES z j The MCMC sampling sheme from Setion 4 samples luster labels j one trak at a time. However, in [16] we noted that, sometimes, assigning several similar traks to a new behavior lass would have high probability, but reating the new behavior first for a single trak has low probability. The sampler may therefore fail to explore the possibility of a new behavior. This problem in DP mixture models has also been noted by other authors [43], [44], [45]. In [16] we presented an ad-ho solution to address this issue, but here we explore split and merge moves to hange luster labels of multiple traks simultaneously. Sequentially-Alloated Merge-Split [43] (SAMS) takes a data-driven approah to propose splits and merges moves of trak lusters. Eah iteration of the SAMS sampler, two trak indies j a and j b are drawn uniformly, whih will be termed anhors. If the traks belong to the same luster, ja ¼ jb, a split of the luster is onsidered, otherwise a merge of lusters ja and jb is proposed. In ase we split a luster, two sublusters m a ¼fj a g and m b ¼fj b g are initialized with the anhors. Then, restrited Gibbs sampling

7 328 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO. 2, FEBRUARY 2016 [43] is used ( restrited sine we ignore all other lusters) to sample one-by-one a subluster label j 2fa; bg for the remaining traks in the luster, updating the likelihood statistis of a subluster after every assignment. This result in a split proposal, whih is aepted or rejeted using the Metropolis-Hastings rule. To merge two lusters, we need to apply the same proedure to estimate reversed splitting move, see [43] for details. To generate a SAMS proposal, the likelihood of a trak ourring together with some other traks needs to be evaluated multiple times. Sine estimating the data likelihood of Eq. (21) is relatively expensive, we instead estimate pðz j jz j ; p 0 Þ, the likelihood of the last sampled z j, by importane sampling with qðp Þ¼pðp jp 0 ; z Þ (Eq. (17)) as proposal distribution, pðz j jz j ; p pðz j jp ; z j 0 Þ¼E Þpðp jp 0 Þ p q : (30) qðp Þ All terms an be omputed analytially, see Setion 4.1, and we find one importane sample suffies. 6 EXPERIMENTS We target senarios with people standing and walking in open spaes. In a surveillane setting, people may enter and exit the sene at different loations, though the system has no prior knowledge about these. Pedestrian traks observed from a moving vehile also exhibit spatial struture, e.g., one typially walks parallel to the road on the sidewalk and does not stand still in front of a vehile. In ontrast to ars in driving lanes [1], [2], [3], [15] for instane, people in open spaes an walk to the same destination along different parallel routes (i.e., spatial variation) and move at speifi or varying speeds suh as standing, walking, or running (i.e., kinemati variation). Furthermore, we are interested in long-term behavior [6], [15], [37], and in loalizing regions with partiular dynamis [5], [15], [40]. Our baseline is therefore Dual-HDP [15], whih jointly disovers low-level topis and high-level behavior lusters, but relies on quantized motion features. We ompare our Mixture of SLDSs (MoSLDS) to Dual-HDP on several artifiial and real-world datasets. 2 The Dual-HDP odebook is reated by dividing the spatial extent of the data into a grid, and motion into five bins (four diretions, as in [15], and a no-motion bin for people standing still), resulting in B ¼ 500 unique words. The prior topi distribution is a symmetri Dirihlet, with all B weights set to 100=B. We also ompare the mixture to a single SLDS [23] (with the spatial distributions), by limiting our sampler to only use one behavior but using the further the same hyperparameters. For all ompared models, we start with a random assignment of topi and luster labels, generate 500 MCMC samples, use 200 for burn-in, and keep every 25th sample. We test 15 split-merge proposals per sample. As default, we set hyperparameters a ¼ b ¼ g ¼ d ¼ 1, spatial prior Q with expeted enter on the averaged 2. Our software, and the artifiial and surveillane datasets, are available for non-ommerial, researh purposes. Please visit and follow the links (Datasets, et.). position in the data, and a motion prior Q with expeted zero mean. The spatial, system and observation noise hyperparameters are set manually per experiment based on intended use (e.g., C Q 0 ontrols large/small areas, low kq 0 spreads the spatial means w.r.t. their size), and rough estimates for the data (e.g., magnitude of observation/system noise). We report parameters C 0 in terms of expeted std. dev. s 0, under the NIW and IW distributions, h s 2 i s 2 n 0. 0 C 0 ¼ 6.1 Artifiial Datasets We begin by evaluating our approah on artifiially generated data from [16]. In the first artifiial senario, we define four waypoints (labeled a to e) on the ground plane, and five behavior lasses (labeled A to E) as ordered lists of these waypoints, see Fig. 3a. The training data ontains 50 traks from A and B eah. The test data ontains 20 traks from eah of the five behaviors. Note that behavior C oinides partially with both A and B, D follows the same route as A but moves faster, and E moves in reverse diretion of B. For a behavior lass traks are generated by first sampling observations at the waypoints with added Gaussian noise h i 10 0 (S ¼ ). Then, intermediate observations are reated along this trak at a speed of 10 units per time step 0 10 (15 for behavior D), adding again Gaussian noise at eah step h i 1 0 (S ¼ ). 0 1 We set hyperparameters ðk Q 0 ; nq 0 ; sq 0 Þ¼ð:1; 50; 2Þ, ðnr 0 ; sr 0 Þ¼ ð10;:3þ, and ðk Q 0 ; nq 0 ; sq 0 Þ¼ð:1; 10; 25Þ. Both MoSLDS and Dual-HDP infer two behavior lasses from the training data, orresponding to the traks from A and B. Fig. 3b shows the topis from our model, and Fig. 3 the orresponding topi transition ounts. For instane, behavior 1 orresponds to topi hain init 1 2 exit, and mathes A. Table 1 shows the normality ratings on the test data per behavior lass. As in [16] (where the sampler neessitated other parameters), our model assigns high likelihood to the novel traks from the normative lasses A and B, but anomalous traks from C, D and E have low likelihood and ould be separated by a threshold, espeially when using the expeted minimum log-likelihood. Dual-HDP an also distinguish A and B from C, but not A and B from D and E, for the following reasons: first, as motion is only quantized into disrete diretions [15], the speed differenes between D and A are not reognized. This ould be improved by quantizing motions at different speeds too, but this introdues again the problem of determining appropriate bins and inreases the odebook size. Seond, with the bag-of-words approah the unusual temporal order of E annot be distinguished from B. With a stronger prior for the limited dynami variane, ðk Q 0 ; nq 0 ; sq 0 Þ¼ð:1; 100; 1Þ, the MoSLDS finds the same topis and behaviors as before, but if trained on all test traks, it plaes A and D into distint behaviors lasses, using topis with distint mean veloities. Without doubling the odebook, however, Dual-HDP only distinguishes three behaviors in the test traks: one for A+D, one for B+E, and a third for C.

8 KOOIJ ET AL.: MIXTURE OF SWITCHING LINEAR DYNAMICS TO DISCOVER BEHAVIOR PATTERNS IN OBJECT TRACKS 329 Fig. 3. First artifiial dataset, see Setion 6.1: (a) A few example traks generated from the five different behaviors. The lower-ase letters in the plot indiate the waypoints that define the behaviors. (b) Topis found by MoSLDS when training on only A and B. For eah topi the topi label, Gaussian semanti area, and mean system motion diretion (as arrows) are shown. () The topi transition ounts for the two found behaviors in the MoSLDS. Seond dataset: (d) Example traks. (e) Topis found by single SLDS. (f) Topis found by Mixture of SLDS. (g) Expeted min. likelihood of test traks (as dots) for SLDS and MoSLDS (mean in red, std. dev. in blue). Next, we ompare the MoSLDS to a single SLDS when subtle motion differenes our depending on behavior. A seond artifiial dataset, seen in Fig. 3d, ontains two behavior lasses for training (A and B), and two more (C and D) for testing only. Similar to the first dataset, trak data is generated using the shown waypoints, but now the added h i, exept at the exits e and f Gaussian noise has S ¼ h i 20 0 whih use S ¼. This results in a spread at the end 0 5 whih is orrelated to the initial motion of a trak. Training traks in behavior A start at waypoint a, then move via to d, and end in e. Traks in behavior B start at b, then also TABLE 1 Trak Ratings per Ground Truth Behavior, after Training on Only Traks from A and B (See Setion 6.1 and Fig. 3a), and Testing on 20 Traks from All Five Behaviors log ðpðx j jx j ÞÞ=T j log E j ½min i pðx ji jx j ÞŠ Mean Var Min Max Mean Var Min Max Mixture of SLDS normal A B anom. C D E Dual-HDP normal A B anom. C D E While both models an rate C as unusual, MoSLDS assigns low likelihood to D and E too. move via to d, but end in f. Anomalous behaviors C and D are variations of A and B where the start and end waypoints have been exhanged. The training data onsists of 100 traks of whih 70 belong to behavior A, and 30 to behavior B. The test data ontains 25 traks from all four behaviors, and we use the same parameters as for the previous dataset. Fig. 3e shows sampled topis for the single SLDS approah. The disovered topis reflet the first-order ation dynamis, sine the motion between and d holds no information on the motion from d onward. Therefore, there is a single topi at the end of the traks (i.e., topi 3) whih mostly resembles motion of behavior A (whih has more training samples). The MoSLDS an distinguish distint ations at the end of the traks, see Fig. 3f, sine these represent the motion within eah behavior better. This effet an be ontrolled with the prior by weighing topi ounts z more heavily in the behavior topi distribution p, i.e. high b. E.g. b ¼ 500 results in only posterior samples that separate these behaviors. Indeed, due to joint inferene, the high-level behavior lustering an inform the low-level ation lustering. The plots in Fig. 3g show the normality measure on the test behaviors of both approahes. Both models assign lower probability to traks from B than those in A, sine in the training data behavior B has a lower prior probability. However, on average the SLDS also assigns high probability to traks from behavior C and D, sine their motions at the start or end orrespond to behavior A. The Mixture of SLDS is better equipped to distinguish the typial and anomalous traks, due to the higher-order dependenies introdued by behaviors. On average, traks from C have lower probability than the other behaviors, sine the motion towards to exit is unlikely given the initial motion. Traks from A are overall most probable, as their ations and behavior are most ommon in the training data.

9 330 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO. 2, FEBRUARY 2016 of our model on the other hand do generalize waiting areas, as seen in the topis in Fig. 4b. This also results in less fragmented behavior lustering, see Fig. 4 versus Fig. 4e. Seond, our model finds spatially overlapping topis for people walking at different speeds and diretions. These differenes are found beause we have a prior on the variane of low-level motion, not due to speifying a speed threshold for slow or fast movement. Fig. 4. See Setion 6.2. (a) Traks (in random olors) and sreenshot from seq_hotel dataset [46]. (b) MoSLDS topis. Red topis have motion to the right, blue to the left. Bright topis without arrow have near zero mean motion. () MoSLDS behaviors (traks start blue, end red). (d)(e) Topis and behaviors found by Dual-HDP. 6.2 BIWI Walking Pedestrians Dataset Next, we make a qualitative omparison of our approah and Dual-HDP [15] on real-world trak data. The publily available BIWI Walking Pedestrians dataset [46] ontains traks top-down filmed pedestrians. seq_hotel ontains pedestrians walking along a sidewalk and some waiting for a tram. seq_eth shows people entering and exiting the building from where the video was shot. We plae the same prior on system and observation noise, ðk Q 0 ; nq 0 ; sq 0 Þ¼ ð:1; 100; 5Þ, ðn R 0 ; sr 0 Þ¼ð100; 5Þ, and look for large spatial areas, ðk Q 0 ; nq 0 ; sq 0 Þ¼ð:001; 50; 40Þ. Conlusions for seq_eth and seq_hotel are similar (as found in [16]), so we only disus the latter here. Both Dual-HDP and MoSLDS disover topis and behaviors that orrespond to people walking in straight lines or standing and waiting, see Fig. 4. While the benefit of apturing dynamis within behaviors is minimal on these traks (most traks use only a single topi in either method), we do nevertheless observe some lear differenes, espeially on the found low-level topis. First, in Dual-HDP, waiting people are represented in the topis as no-motion at one or few spatial ells, as an be seen by the white dots in the topis of Fig. 4d. These topis thus apture waiting at exatly those spatial positions, but do not generalize over people waiting in the near viinity. The spatial distributions 6.3 Surveillane Dataset In [16] we presented a novel dataset reorded at the entral hall of a large building, see Fig. 5a, using ators to perform various roles that ommonly our there at a normal working day. Traks are obtained using a multi-view traker [4], though due to many olusions not everybody is orretly traked all the time. The training data (118 traks, 2,737 observations, see Fig. 5b) ontains normative behavior only, inluding employees entering at the main entrane and walking to one of the exits, and visitors that register at the reeption and wait to be piked up by an employee. The test data of 64 traks, see Fig. 5, ontains mostly normative behavior, but with some exeptions: in the sene a terrorist souts the environment, walking in and out of view. Later, a seond terrorist joins him and mixes with the visitors. At some point, the seond terrorist shoots, and bystanders run for safety. We set weak SLDS priors, ðk Q 0 ; nq 0 ; sq 0 Þ¼ð1; 10; 2Þ, ðnr 0 ; s R 0 Þ¼ð10;:3Þ, and use ðkq 0 ; nq 0 ; sq 0 Þ¼ð:1; 50; 25Þ. As in [16], the MoSLDS disovers various ations in the training data, shown in Fig. 5d. For instane, we interpret topi 20 as the ation waiting at reeption desk, and topi 16 as walking to lower exit. Fig. 5e shows traks from the 12 behaviors present in this sample, orresponding to workers walking straight from the main entrane to an exit, visitors entering at the reeption and waiting in various parts in the hall, and workers piking up visitors. Here we have used b ¼ 10 beause, interestingly, we observed that with b ¼ 1 traks with just a few topis in ommon tend to be lustered together, espeially when behaviors have few traks. So again, inreasing b results in behaviors with more speifi topi hains. With b ¼ 1 we find about seven behaviors typially, e.g., behaviors 10 and 13 in Fig. 5e are merged. Note how behavior lusters an merge fragmented traks with similar topis (e.g., lusters 9, 13), but systemati fragmentation may lead to new behaviors (e.g., luster 19 with waiting people). Dual-HDP disovers behavior lusters and regions of motion resembling those of MoSLDS, see Figs. 5f and 5g. However, the phenomena disussed in Setion 6.2, where standing people yield very speifi topis (e.g., topi 3 in Fig. 5f), again our. Both models are used to rank the test traks, where we disern three relevant lasses: normal traks, the terrorist suspets, and fleeing visitors. Figs. 6a and 6b show the normality measures for Dual-HDP and MoSLDS respetively. With Dual-HDP, 33 out of 64 test traks ontain words from the odebook not seen in the training data, even at the low spatial resolution of bins. Evaluation on the test data is still possible, sine the unseen words have non-zero probability due to the prior. Still, the most anomalous test traks are those with many words that did not our in the training data, as found in both the normative and the suspet lass. The running

10 KOOIJ ET AL.: MIXTURE OF SWITCHING LINEAR DYNAMICS TO DISCOVER BEHAVIOR PATTERNS IN OBJECT TRACKS 331 Fig. 5. (a) Sreenshot from the surveillane dataset, Setion 6.3. (b) Traks (in random olors) in the normative training data. When people stand still their traks form dense spots. () Traks in the test data, ontaining normal people, suspiious individuals, and people running after a gunshot. (d) Topis found by MoSLDS, with their id, Gaussian semanti region, and mean motion. (e) Several found behaviors (traks start blue, end red). (f) Dual-HDP topis (" green, # purple, blue,! red, no-motion white) and (g) behaviors. visitors have relatively high likelihood, sine the veloity magnitude is disregarded. Test traks are also ompared to all training traks using dynami time warping [36] with Eulidean distane measure (this gives better results than the implementation in [16]), and the lowest DTW distane is used as normality measure. We lassify traks while varying a normality threshold and see from the ROC urves, Fig. 6e, that Fig. 6. (a) (b) Expeted min. likelihood for Dual-HDP and MoSLDS on surveillane test traks, see Setion 6.3. (),(d) Most unusual traks (irles mark trak start) found by () dynami time warping, and (d) MoSLDS. Green traks are false positives. (e) ROC urves by varying normality threshold. (f) Sreenshots of the unusual traks found by MoSLDS: fleeing visitors; wandering visitor (false positive); two terrorists exploring the spae.

11 332 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO. 2, FEBRUARY 2016 Fig. 7. (a) Traks from the intelligent vehile pedestrian dataset, Setion 6.4. (b)-(f) Samples of learned dynamis and behavior lusters for an SLDS, MoSLDS, and Dual-HDP. In (e) and (f), traks start blue, end red. Traks have been aligned suh that the longitudinal axis is the road, so rossing pedestrians move aross this axis. MoSLDS ranks the traks best. Fig. 6 shows the eight most anomalous traks for DTW, but the only true positives are at rank 6 and 8. Mathing traks globally does not onsider the ommon spatial variane in the training data. The eight most anomalous MoSLDS traks, Fig. 6d, have only two false positives at rank 4 and 7. The two most unusual traks are running visitors, whose dynamis have low likelihood. Other traks belong to the terrorists who walk unusually between waiting areas. False positives our when normal visitors walks to a different area to hat with others, whih did not our in the training data. Fig. 6f shows several sreenshots of these events. 6.4 Intelligent Vehile Pedestrians Dataset Next, we experiment with pedestrian traks as observed from the perspetive of a moving vehile. The dataset, presented in [42], onsists of pedestrian trajetories reorded in and around various European ities, with an embedded stereo vision system in a ar. Unoluded pedestrians have been manually annotated with bounding boxes. Using depth estimation from stereo vision, the pedestrian s distane to the vehile ould be measured, resulting in 2D topdown measurement (latitude and longitude) relative to the amera oordinate system. Obtained traks are ompensated for the vehile ego-motion as estimated from on-board sensors. Finally, traks are aligned in a oordinate system with the vehile at the origin traveling along the vertial axis [42], suh that pedestrians on the sidewalk move in longitudinal diretion, and lateral to ross the road, see Fig. 7a. The resulting data has 423 traks, and 4,103 observations in total. As more longitudinal than lateral variane is expeted, we set ðk Q 0 ; nq 0 ; sqx 0 ; sqy 0 Þ¼ð:5; 50;:25;:5Þ, ðnr 0 ; srx 0 ; sry 0 Þ¼ ð50;:25;:5þ; ðk Q 0 ; nq 0 ; sqx 0 ; sqy 0 Þ¼ð1; 100; 5; 15Þ. Fig. 7b shows topis found by the single SLDS approah, as in [42], and Fig. 7 topis found with MoSLDS. In both ases, we see lear lateral motion in front of the ar of people rossing the road (in red and blue) to and from areas loated left and right of the ar. These have near zero mean motion (olored white) but large motion variane, sine pedestrians at those loations move on the sidewalk in both longitudinal diretions at low veloity. We find that the SLDS and MoSLDS topis are mostly similar here, but the MoSLDS behaviors additionally apture whih topis o-our. E.g. in Fig. 7e luster 11 transitions from topi 4 to 3 (i.e., reahing the sidewalk after rossing) but then moving from 3 to 2 (i.e. rossing bak) is improbable. Cluster 2 does ontain topis 2 and 3, but not topi 4. In this data, people only stand still in speifi spatial regions, and we now see no Dual-HDP topis for isolated standing individuals (Fig. 7d), but do see regions with lateral (rossing) and longitudinal (sidewalk) motion similar to our MoSLDS. Dual-HDP does divide traks at the sidewalk and rossing areas into many small behavior lusters with little overlap, eah ontaining speifi motion at a speifi loation, see Fig. 7f (e.g., lusters 15 and 17 for rossing). 6.5 Runtime and Sampler Comparison Finally, we ompare onvergene of our full MoSLDS sampler, the sampler without split-merge jumps (Setion 5), and the single SLDS sampler. Our ode is written in Matlab, and C++ to sample the z j (Setion 4.3). On the surveillane data, a run of 500 samples takes 1 hour for the full sampler, 30 min. without split-merge, and 15 min. using only one

12 KOOIJ ET AL.: MIXTURE OF SWITCHING LINEAR DYNAMICS TO DISCOVER BEHAVIOR PATTERNS IN OBJECT TRACKS 333 Fig. 8. Log likelihood of samples, plotted against sampling iterations, see Setion 6.5. Thin lines are plotted for eah run of a sampler. Thik lines show the average over the all runs. First iterations are omitted for larity. SLDS. With J traks of length T, K topis, and C lusters, the MH proposals runs in OðJ T KÞ (Setion 4.2, 4.3), Gibbs sampling p 0 and all p in OðK CÞ (Setion 4.1), Gibbs sampling SLDS parameters in OðJ T þ KÞ (Setion 4.4), and eah SAMS proposal in OðJ T KÞ. So sampling sales linearly with trak ount and length, but longer traks may require more information filter runs to obtain good z j, and more lusters redues the hane of good label j proposals, or good SAMS proposals. We perform 10 MCMC runs on several disussed datasets. Fig. 8 shows how the log probability of the MCMC samples onverges, where thin lines are individual runs, thik lines show the average. The figure illustrates that without the split/merge moves there is a risk of the sampler getting stuk in a suboptimal region for a long time. However, this problem appears to be more prevalent in data with some often used topis that overlap spatially with others, suh as the artifiial and intelligent vehile data. In the latter for instane, the rossing motions have quite some spatial overlap with the neutral areas at the side of the road, and initially suh motions are not plaed in distint topis. The results also show that this is only a problem in some runs, and, on realworld data, is resolved after suffiient iterations. The single SLDS sampler onverges more quikly as there are fewer latent variables to estimate without the behavior lasses. 7 CONCLUSIONS We have presented a novel model that uses a Mixture of SLDS to jointly infer ommon ations and behaviors from trak data. Evaluation on test data from artifiial, surveillane, and intelligent vehile senarios, shows that our MCMC sampler an disover meaningful dynamis and trak lusters. We find that on some datasets, additional split-merge moves help the sampler to avoid loal optima more reliably, but on our real-world data all samplers eventually onvergene to similar solutions. Behavior lusters an apture higher order dependenies on the topis, and provide insight in the behavioral struture of the data. If, however, one is only interested in the low-level topis, it may suffie to use our proposed SLDS with spatial areas and a single behavior luster, as the experiments on real-world data show. For Dual-HDP, the spatial variane of walking and standing people results in sparse spatial bins, and different motions an only be distinguished if appropriate thresholds are set in advane. Our MoSLDS does not rely on feature quantization, but learns relevant motions types in the ontinuous feature spae, using priors on the variane within a motion lass, whih is espeially benefiial when testing on new traks. It is apable of inferring informative low-level motions even when traks have spatial and kinemati variations, and available training data is limited. REFERENCES [1] T. Hospedales, S. Gong, and T. Xiang, A Markov lustering topi model for mining behaviour in video, in Pro. IEEE Int. Conf. Comput. Vis., 2009, pp [2] D. Kuettel, M. Breitenstein, L. Van Gool, and V. Ferrari, What s going on? Disovering spatio-temporal dependenies in dynami senes, in Pro. IEEE Int. Conf. Comput. Vis., 2010, pp [3] R. Emonet, J. Varadarajan, and J. Odobez, Multi-amera open spae human ativity disovery for anomaly detetion, in Pro. 8th IEEE Int. Conf. Adv. Video Signal-Based Surveillane, Aug. 2011, p. 6. [4] M. Liem and D. M. Gavrila, Multi-person loalization and trak assignment in overlapping amera views, in Pro. 33rd Int. Conf. Pattern Reog., 2011, pp [5] J. Joseph, F. Doshi-Velez, A. S. Huang, and N. Roy, A Bayesian nonparametri approah to modeling motion patterns, Auton. Robots, vol. 31, no. 4, p , [6] J. F. P. Kooij, N. Shneider, F. Flohr, and D. M. Gavrila, Contextbased pedestrian path predition, in Pro. Eur. Conf. Comput. Vis., 2014, pp [7] C. G. Keller and D. M. Gavrila, Will the pedestrian ross? A study on pedestrian path predition, IEEE Trans. Intell. Transportation Syst., vol. 15, no. 2, pp , Apr [8] Z. Chen, D. Ngai, and N. Yung, Pedestrian behavior predition based on motion patterns for vehile-to-pedestrian ollision avoidane, in Pro. 11th Int. IEEE Conf. Intell. Transportation Syst., 2008, pp [9] S. Blakman and R. Popoli, Design and Analysis of Modern Traking Systems. Norwood, MA, USA: Arteh House, [10] M. Liem and D. M. Gavrila, Multi-person traking with overlapping ameras in omplex, dynami environments, in Pro. Brit. Mah. Vis. Conf., 2009, pp [11] D. Geronimo, A. M. Lopez, A. D. Sappa, and T. Graf, Survey of pedestrian detetion for advaned driver assistane systems, IEEE Trans. Pattern Anal. Mah. Intell., vol. 32, no. 7, pp , Jul [12] J. Tao and R. Klette, Traking of 2d or 3d irregular movement by a family of unsented Kalman filters, J. Inf. Commun. Convergene Eng., vol. 10, no. 3, pp , [13] N. Shneider and D. M. Gavrila, Pedestrian path predition with reursive Bayesian filters: A omparative study, in Pro. 35th German Conf. Pattern Reog., 2013, pp [14] K. M. Kitani, B. D. Ziebart, J. A. Bagnell, and M. Hebert, Ativity foreasting, in Pro. Eur. Conf. Comput. Vis., 2012, pp [15] X. Wang, K. T. Ma, G. W. Ng, and W. E. Grimson, Trajetory analysis and semanti region modeling using a nonparametri Bayesian model, in Pro. IEEE Conf. Comput. Vis. Pattern Reog., 2008, pp [16] J. F. P. Kooij, G. Englebienne, and D. M. Gavrila, A non-parametri hierarhial model to disover behavior dynamis from traks, in Pro. IEEE Conf. Comput. Vis. Pattern Reog., 2012, pp [17] V. Romero-Cano, J. I. Nieto, and G. Agamennoni, Unsupervised motion learning from a moving platform, in Pro. IEEE Symp. Intell. Veh., 2013, pp [18] P. Sovanner and M. F. Tappen, Learning pedestrian dynamis from the real world, in Pro. IEEE Int. Conf. Comput. Vis., 2009, pp

13 334 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO. 2, FEBRUARY 2016 [19] G. Antonini, S. V. Martinez, M. Bierlaire, and J. P. Thiran, Behavioral priors for detetion and traking of pedestrians in video sequenes, Int. J. Comput. Vis., vol. 69, no. 2, pp , [20] M. Enzweiler and D. M. Gavrila, Monoular pedestrian detetion: Survey and experiments, IEEE Trans. Pattern Anal. Mah. Intell., vol. 31, no. 12, pp , De [21] H. E. Rauh, C. Striebel, and F. Tung, Maximum likelihood estimates of linear dynami systems, AIAA J., vol. 3, no. 8, pp , [22] A.-V. Rosti and M. J. F. Gales, Rao-Blakwellised Gibbs sampling for swithing linear dynamial systems, in Pro. IEEE Int. Conf. Aoust., Speeh, Signal Proess., 2004, vol. 1, pp. I-809 I-812. [23] E. Fox, E. Sudderth, M. Jordan, and A. Willsky, Bayesian nonparametri inferene of swithing dynami linear models, IEEE Trans. Signal Proess., vol. 59, no. 4, pp , Apr [24] V. Pavlovi, J. M. Rehg, and J. MaCormik, Learning swithing linear models of human motion, in Pro. Adv. Neural Inf. Proess. Syst., 2000, pp [25] C. M. Bishop, Pattern Reognition and Mahine Learning, vol. 1. New York, NY, USA: Springer, [26] T. P. Minka, Expetation propagation for approximate Bayesian inferene, in Pro. 17th Conf. Unertainty Artif. Intell., 2001, pp [27] X. Boyen and D. Koller, Tratable inferene for omplex stohasti proesses, in Pro. 14th Conf. Unertainty Artif. Intell., 1998, pp [28] S. L. Lauritzen, Propagation of probabilities, means, and varianes in mixed graphial assoiation models, J. Amer. Statist. Asso., vol. 87, no. 420, pp , [29] S. M. Oh, J. M. Rehg, T. Balh, and F. Dellaert, Learning and inferring motion patterns using parametri segmental swithing linear dynami systems, Int. J. Comput. Vis., vol. 77, nos. 1 3, pp , May [30] E. Fox, Bayesian nonparametri learning of omplex dynamial phenomena, Ph.D. Thesis, MIT, Cambridge, MA, USA, [31] Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei, Hierarhial Dirihlet proesses, J. Amer. Statist. Asso., vol. 101, no. 476, pp , [32] M. J. Beal, Z. Ghahramani, and C. E. Rasmussen, The infinite hidden Markov model, in Pro. Adv. Neural Inf. Proess. Syst., 2002, vol. 14, pp [33] D. Blei, A. Ng, and M. Jordan, Latent Dirihlet alloation, J. Mah. Learn. Res., vol. 3, pp , [34] Z. Fu, W. Hu, and T. Tan, Similarity based vehile trajetory lustering and anomaly detetion, in Pro. IEEE Int. Conf. Image Proess., 2005, vol. 2, pp. II-602 II-605. [35] X. Wang, K. Tieu, and E. Grimson, Learning semanti sene models by trajetory analysis, in Pro. Eur. Conf. Comput. Vis., 2006, pp [36] E. Keogh and M. Pazzani, Saling up dynami time warping for datamining appliations, in Pro. ACM SIGKDD Int. Conf. Knowl. Disovery Data Mining, 2000, pp [37] B. Zhou, X. Wang, and X. Tang, Understanding olletive rowd behaviors: Learning a mixture model of dynami pedestrianagents, in Pro. IEEE Conf. Comput. Vis. Pattern Reog., 2012, pp [38] J. Fernyhough, A. Cohn, and D. Hogg, Generation of semanti regions from image sequenes, in Pro. Eur. Conf. Comput. Vis., 1996, pp [39] D. Makris and T. Ellis, Automati learning of an ativity-based semanti sene model, in Pro. IEEE Int. Conf. Adv. Video Signal- Based Surveillane, 2003, pp [40] D. Lin, E. Grimson, and J. Fisher, Learning visual flows: A lie algebrai approah, in Pro. IEEE Conf. Comput. Vis. Pattern Reog., 2009, pp [41] J. Varadarajan, R. Emonet, and J. M. Odobez, Probabilisti latent sequential motifs: Disovering temporal ativity patterns in video senes, in Pro. Brit. Mah. Vis. Conf., 2010, pp [42] J. F. P. Kooij, N. Shneider, and D. M. Gavrila, Analysis of pedestrian dynamis from a vehile perspetive, in Pro. IEEE Symp. Intell. Veh., 2014, pp [43] D. B. Dahl, Sequentially-alloated merge-split sampler for onjugate and nononjugate Dirihlet proess mixture models, Department of Statistis, Texas A&M University, Teh. Rep., [44] M. C. Hughes, E. B. Fox, and E. B. Sudderth, Effetive split-merge monte arlo methods for nonparametri models of sequential data, in Pro. Adv. Neural Inf. Proess. Syst., 2012, pp [45] J. Chang and J. W. Fisher III, Parallel sampling of DP mixture models using sub-luster splits, in Pro. Adv. Neural Inf. Proess. Syst., 2013, pp [46] S. Pellegrini, A. Ess, K. Shindler, and L. Van Gool, You ll never walk alone: Modeling soial behavior for multi-target traking, in Pro. IEEE Int. Conf. Comput. Vis., 2009, pp Julian F. P. Kooij reeived the MS degree in artifiial intelligene from the University of Amsterdam in He is urrently working toward the PhD degree at the University of Amsterdam, after having been employed in part at Daimler Researh and Development, Ulm, Germany. His researh interests inlude Mahine Learning and Bayesian methods for omputer vision, with a fous on modeling trajetory dynamis and unsupervised disovery of motion patterns. Gwenn Englebienne reeived the PhD degree in omputer siene from the University of Manhester in He has sine foused on automated analysis of human behavior at the University of Amsterdam, where he has developed omputer vision tehniques for traking humans aross large amera networks and mahine learning tehniques to model human behavior from networks of simple sensors. His main researh interests are in models of human behavior, espeially their interation with other humans, with the environment, and with intelligent systems. Dariu M. Gavrila reeived the PhD degree in omputer siene from the University of Maryland at College Park in Sine 1997, he has been with Daimler R&D in Ulm, Germany, where he is urrently a prinipal sientist. In 2003, he was named professor at the University of Amsterdam, in the area of intelligent pereption systems (part time). Over the past 15 years, he has foused on visual systems for deteting human presene and ativity, with appliation to intelligent vehiles, smart surveillane and soial robotis. He o-reeived the IEEE ITS Outstanding Appliation Award 2014 for his role in transferring pedestrian detetion tehnology into Meredes- Benz vehiles. His personal web site is " For more information on this or any other omputing topi, please visit our Digital Library at