Feature Selection and Occupancy Classification Using Seismic Sensors

Transcription

1 Feature Selection and Occupancy Classification Using Seismic Sensors Arun Subramanian 1, Kishan G. Mehrotra 1, Chilukuri K. Mohan 1, Pramod K. Varshney 1 and Thyagaraju Damarla 2 1 Department of Electrical Engineering and Computer Science, Syracuse University, Syracuse, NY 13244, USA 2 Army Research Laboratory, Adelphi, MD 2783, USA Abstract. In this paper, we consider the problem of indoor surveillance and propose a feature selection scheme for occupancy classification in an indoor environment. The classifier aims to determine whether there is exactly one occupant or more than one occupant. Data are obtained from six seismic sensors (geophones) that are deployed in a typical building hallway. Four proposed features exploit amplitude and temporal characteristics of the seismic time series. A neural network classifier achieves performance ranging between 77% to 95% on the test data, depending on the type of construction of the location in the building being monitored. 1 Introduction Automatic surveillance is essential for many scenarios that require remote monitoring. Several sensors of different modalities (e.g., video, acoustic, infra-red and seismic) may be deployed to monitor areas such as international borders or cleared buildings where a sustained presence of personnel is not feasible. In some cases, constraints on logistics do not permit the use of information rich sensors such as video. We consider one such scenario for indoor surveillance and investigate the use of seismic sensors for occupancy classification. Detecting the presence of objects using seismic sensors can be categorized into two broad areas: detection of (i) humans, and (ii) other objects (such as vehicles). Detection of vehicles is easier to accomplish than detection of humans, since the seismic signal from a vehicle has a higher signal to noise ratio (see [1 3]). The harmonic signatures of two vehicles of the same type are consistent, and those of different vehicle types are generally distinguishable. In contrast, classification of human occupancy using footstep signals is difficult in that different people walk with different gaits and at varying pace, resulting in vibration patterns that are difficult to identify uniquely. To detect occupants based on footstep signal processing using seismic sensors, researchers use approaches such as auto-regressive modeling, signal moments, This research was sponsored by Army Research Laboratory and was accomplished under Cooperative Agreement No. W911NF It was also supported in part by ARO grant W911NF

2 time-scale analysis and explicit experimental modeling [4 7]. Using a copula based approach, Iyengar et al. [8] fuse signals from acoustic and seismic sensors for footstep detection. Although the detection of the presence of humans has been successfully addressed by such researchers, finer-grain analysis is much more difficult. One such challenging task, addressed in this paper, is the determination of whether one or more people are walking together in the environment being monitored. This is because people walking together have a psychological tendency to walk in lock step [9]. Since the raw signals are not easily classifiable, an important step is to derive suitable features that can assist classification. One of the main contributions of this work is to develop features that enable us to distinguish between the presence of one or more than one person in the given region of interest. In Section 2, we describe the data collection process. The raw data is preprocessed to extract useful information, and to eliminate noise, using a methodology based on empirical mode decomposition (EMD), described in Section 3. Our approach for feature selection is presented in Section 4. Section 5 contains the classification results and Section 6 contains our conclusions. 2 Data collection Six GS 2DX geophones were used for the purpose of data collection. The geophones are designed to be floor mounted. The sensors were configured as a linear array. Data was collected in two (different) building hallways of similar construction. The sensors were placed along the long edge of the hallway. The distance between two adjacent sensors was maintained at 5ft. Data was acquired using a 16 bit A/D converter at a sampling rate of 5kHz. The raw signal was uniformly down-sampled to 124 per second. The approximate duration of the data collected per trial is 12 seconds. Multiple persons participated in the data collection. The footstep data thus collected consists of 12 single-person trials (i.e., a given trial has exactly one participant walking along the hallway) and 12 two-person trials (a given trial has exactly two participants walking along the hallway). Each dataset consists of 6 trials from Building 1 and 6 trials from Building 2. 3 Overview of the Preprocessing In order to derive reliable features from pseudo-periodic signals like footsteps, it is of interest to extract a clean envelope corresponding to footfalls. We have used empirical mode decomposition (EMD) [1] for this purpose. EMD is a data-driven decomposition technique that captures oscillations at several scales. Each of these scales is called a mode and the function corresponding to a mode is called an intrinsic mode function (IMF). In order to capture interesting characteristics of footsteps and eliminate inherent noise present in the data, we have used only a small subset of these modes based on the total variation (TV) norm [6] such that the sum of TV norm of the selected modes is 9% of the total

3 TV norm. For a time-series of length T, these modes are treated as random vectors with T observations. These vectors are linearly combined, using the first principal component. 1 The raw data is denoted by y (k) (t) and the processed data so obtained is denoted by x (k) (t) for i = 1,..., 6, j = 1,..., 12, k = 1, 2, t = 1,..., T ; where i represents the sensor, j refers to trial, k refers to number of persons, and t is the time index. This processed data is used in the rest of the paper to extract features for classification. 2 The effect of this preprocessing is seen in1 Fig. 3. The EMD based processing extracts a faithful envelope of the raw signal revealing the salient features of -1 each footfall (a) (b) 1.5 Fig. 11. A comparison of a two-second segment of the raw signal y (2) of envelope.5 extraction, x (2) (t), for the 2 persons case Feature Selection (t) and the result In this section, we present four features that were employed to achieve classification between presence of one person versus two persons. We assume that the detection has already been accomplished successfully, i.e., it is certain that at least one occupant is present. The seismic signal of a sensor decays rather rapidly as person(s) move away from the sensor. For each sensor separately, we, therefore, extract a five second segment of the useful data as proposed below; the discarded data is deemed to be non-informative. For each i, j, k, first we find the time index where x (k) is maximum and extract data for all time indices that are contained in ±t o on either side of the index where the maximum occurs. We choose t = , because, as stated earlier, we collect 124 samples per second. For instance, if t max was the time when x (k) takes it s maximum value, then the extracted data will be {x (k) (t) : t = t max t, t max t + 1,..., t max + t }. However, for notational convenience we renumber the time indices and denote the extracted data as {z (k) (t) : t =, 1,..., 2t }, 1 We have observed that about 8% of the signal energy is contained in the first principal component.

4 person 2 person.6 Amplitude Frequency (Hz) Fig. 2. Periodogram comparison of 1 person and 2 person trials. for all values of (i, j, k). Note that this data will be extracted from the early part of {x (k) (t)} for the first sensor (i = 1) and later part for the sixth sensor (i = 6). In the following subsections, features are extracted from these extracted data sets only. 4.1 Periodogram and autocorrelation Footsteps are quasi-periodic signals. Typically, the periodogram and autocorrelation function (ACF) are used in determining signal periodicity [2, 3, 11]. But, analysis by Houston and McGaffigan [9] shows that the use of spectral measures is not useful for counting the number of personnel in the region of interest. In the following discussion, we note that while the periodogram and autocorrelation are not directly useful for our occupancy classification problem, nevertheless they provide some valuable insights. Fig. 2 shows a comparison of the periodograms of 2 trials of the one person and two person cases. The figure suggests that the periodogram alone may be insufficient for the classification task at hand. We expect that heel-toe transitions will be borne out in the single occupant case and will be blurred in the case of two or more occupants. In order to find support for this argument, we plotted the mean of the ACF across sensors for a single period (Fig. 4.1). We use the peak frequency from the periodogram to obtain the period of the ACF. Clearly, the ACF for one person goes through a typical pattern of local maxima and minima as heel and toe spikes align with each other. When the ACF for the first period are averaged over all the training data for all sensors in the one-person case, we obtain the ACF template seen in Fig. 3(a). Therefore, the feature extraction procedure is, 1. Form the template (Fig. 3(a)) 2. Calculate the area which measures the difference between the template and the ACF for a given trial (see Fig. 4). This difference is the mean-square

5 (a) (b) Fig. 3. The autocorrelation function for a single period averaged over all sensors. (a) 1 person. (b) 2 persons. X-axis τ is in seconds. Y -axis is r (τ). Fig. 4. Comparison of autocorrelation functions. The shaded region shows the difference in area which, when integrated, gives the mean-square error between the template and the ACF for the given trial. X-axis: τ, Y -axis: r (τ). error between the two curves and is denoted as MSE (k), which is the first of the four features that we use for occupancy classification. 4.2 Signal energy Variance is a measure of signal energy and serves as an intuitive indicator of occupancy. That is, when the number of occupants is large, the signal energy will be large. Using the extracted data described earlier we calculate the variance as described below. [ S (k) ] 2 = 1 2t o 2t o ( t= z (k) (t) z(k) ) 2, (1)

6 where z (k) is the sample mean (of the 2t observations). These variances are averaged by taking their mean across all six sensors, and S (k) j = 1 6 N [ i=1 S (k) is used as one of the features to be used for classification. ] 2 (2) 4.3 Ratio of time spent in states We expect that footsteps produce a quasi-periodic time series when one person walks whereas footsteps for two persons will be irregular and spread out. For two or more persons this footstep state will occupy a greater proportion of a window of fixed duration. Based on this criterion, the feature R(states) is calculated for each value of (i, j, k) as described below. For notational convenience we do not use j and k below. As stated earlier, we use the extracted data only. We move a sliding window (one-tenth of a second long) over this data, calculate the average of the x-values within the window, and compare it with a threshold η to calculate { 1 if the average of the observations within the window is > η d pi = if the average of the observations within the window is η (3) where p = 1,..., P and P represents the number of times the sliding window fits over the data. In the ideal situation, each 1 represents the Footstep state and each represents the Silence state. The ratio p,i R(states) = d pi p,i (1 d (4) pi) captures the ratio of time spent in these states versus not in the state. These ratios, the third feature, are obtained for all values of (j, k) and are denoted as R(states) (k) j. Threshold η is calculated from the first.25 seconds of the x data which represents the background process. Let µ b and σ b denote the sample mean and standard deviation calculated from this initial.25 second period. The threshold is then set as, η = µ b + 3σ b (5) 4.4 Cross-correlation and wavelet based feature The cross-correlation sequence between adjacent sensors is one way to measure the similarity between the data collected by two sensors. We considered the crosscorrelation between the two nearest neighbors of a given sensor; for example, two nearest neighbors of sensor 2 are sensor 1 and sensor 3, respectively. As before the

7 extracted data is obtained for sensors i = 2, 3, 4, 5 and the following correlation functions are computed: r i+ (τ) = Corr(z (k) (t), z(k) (i+1),j(t + τ)) (6) r i (τ) = Corr(z (k) (t), z(k) (i 1),j(t + τ)) (7) r i± (τ) = Corr(z (k) (i+1),j(t), z(k) (i 1),j(t + τ)) (8) R i (τ) = r i+ (τ)r i (τ) r i± (τ) (9) where τ extends from -124 to 124 (i.e., up to a shift of 1 second in both directions). We expect that, in the ideal situation, R i = for the 1-person case and R i when two or more persons are walking. In practice, R i is expected to take a small value for the 1-person case and a large value in the 2 or more persons case. In Fig. 5, we observe wide major peaks and thinner auxiliary peaks in the two-person case. In other words, the 2-person information is contained at different scales. This suggests that wavelet decomposition would be useful in extracting useful information from R i (τ) person 2 persons Fig. 5. Typical plot of R i vs. τ We chose the Mexican hat wavelet and calculated the continuous wavelet transform (scalogram) of {R i (τ) : τ = 124 to 124}, C i (a, b) = R i (τ)ψ (τ b, a)dτ, (1) allτ 1 ψ (τ, a) = (1 τ 2 ) 2πa 3 a 2 exp ( τ 2 ) 2a 2

8 We note that, in theory, the scale a can have an infinite number of values. For numerical computation, we chose the maximum value of a to be 1/(2f ) where f is the peak frequency from the periodogram of the extracted data. Finally, a summary statistics can be measured in terms of the average S C, C(a, b) = 1 4 S C = b 5 C i (a, b) (11) i=2 or use this feature separately for each sensor as: max C(a, b) (12) a S Ci = b max C i (a, b). (13) a 5 Classification Procedure and Results Section 4 discussed the features investigated for classification problem of determining one vs. two occupants in a building hallway. There are two possible fusion schemes for the selected features: 1. Use the 4-dimensional feature vector F = [MSE (k) j, S (k) j, R(states) (k) j, S C ] where the individual features have been fused across sensors by taking the mean 2. Use a 17-dimensional vector comprising of MSE (k), [S(k) ]2, R(states) (k) j, and S Ci for each i. Note that there is only one value for the third feature and 4 values for the fourth feature giving a total of ( = 17)-dimensional vector. In this case the fusion is achieved by dimensionality reduction using principal component analysis (PCA) retaining those combined features that capture 98% of the total variation. PCA is performed prior to classification. In both cases, a neural network classifier is used with one hidden layer of 6 nodes 2. Classification performance is analyzed for the following five cases, Case 1 Train on Building 1 data, test on Building 2 data Case 2 Train on Building 1 data, test on Building 1 data 3. Case 3 Train on Building 2 data, test on Building 2 data 3. Case 4 Train on Building 2 data, test on Building 1 data. Case 5 Mixed, i.e., combine data from both buildings by random permutation and use half the dataset to train and remaining half to test Tables 1 and 2 contain the results on test data; cases 2, 3 and 5 are the average of 1 iterations. We conclude that, in general, the classification performance is 2 We experimented with 3 to 9 nodes in the hidden layer. Best training performance was observed with 6 nodes. 3 Training is done using a randomly selected set of 9 trials out of the 12.

9 Table 1. Classification results: 4-dimensional feature vector. True/Error classes Case 1 Case 2 Case 3 Case 4 Case 5 C 1 true (T P ) C 2 true (T N) Type I error (F A) Type II error (M) Performance (P ) 87.5% 88.33% 95.% 77.5% 9.41% Table 2. Classification results: 17-dimensional feature vector followed by PCA. True/Error classes Case 1 Case 2 Case 3 Case 4 Case 5 C 1 true (T P ) C 2 true (T N) Type I error (F A) 3 6 Type II error (M) Performance (P ) 87.91% 88.33% 96.67% 78.75% 92.5% T P : True positives, T N: True negatives, F A: False alarms, M: Misses, P = (T P + T N)/(T P + T N + F A + M) 1% very good. Best performance is observed for Case 3 (Building 2). This indicates that selected features are good. If, somehow, we can improve the method of sensor data collection, then the classification performance will further improve. The good classification performance of the classifiers trained on the data from one building and tested on the same building supports the above conclusion. To assess the contribution of each feature alone we experimented with Case 5. It was observed that the third feature (R(states) (k) j ) contributes significantly to improving the classification performance (about 7%), whereas features MSE (k) and S Ci make similar amount of contributions. The similarity in performance between these two features is not surprising because they are both derived from correlation sequences. Since the rationale for using S Ci depends on the linear configuration of sensors, one may consider dropping this feature for a simpler system design under more general topology of sensor deployment. We have noticed that in spite of dropping this feature we get a classification performance of 89.13% for Case 5. 6 Concluding Remarks We have addressed the task of distinguishing between the presence of one vs. two persons in a visually unobservable indoor region of interest, using data obtained from seismic sensors. After exploring multiple alternatives, we identified four features as being capable of assisting classification with a high degree of reliability. Classification was achieved using a neural network classifier.

10 Although the current suite of four features has yielded very good performance, further improvements may be possible by exploring new feature extraction approaches. Performance gains may be obtained by using non-linear dimensionality reduction schemes [12] as opposed to principal component analysis, which is linear. Improved classification as well as finer classification is also expected using other modalities; such as the acoustic sensors. Since the data are vibrational in nature, the signal processing and feature extraction algorithms developed in this paper can also be applied to data collected using accelerometers and acoustic sensors. References 1. Dibazar, A.A., Park, H.O., Berger, T.W.: The application of dynamic synapse neural networks on footstep and vehicle recognition. In: Proc. International Joint Conference on Neural Networks IJCNN 27. (12 17 Aug. 27) Li, D., Wong, K.D., Hu, Y.H., Sayeed, A.M.: Detection, classification, and tracking of targets. IEEE Signal Processing Magazine 19(2) (22) Tian, Y., Qi, H., Wang, X.: Target detection and classification using seismic signal processing in unattended ground sensor systems. In: Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2). Volume 4. (22) IV Bland, R.E.: Acoustic and seismic signal processing for footstep detection. Master s thesis, Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science (26) 5. Succi, G., Clapp, D., Gampert, R., Prado, G.: Footstep detection and tracking. In: Proc. SPIE - Int. Soc. Opt. Eng. (USA). Volume 4393., USA (21) Subramanian, A., Iyengar, S.G., Mehrotra, K.G., Mohan, C.K., Varshney, P.K., Damarla, T.: A data-driven personnel detection scheme for indoor surveillance using seismic sensors. In Carapezza, E.M., ed.: Unattended Ground, Sea, and Air Sensor Technologies and Applications XI. Volume 7333., SPIE (29) Sabatier, J.M., Ekimov, A.E.: A review of human signatures in urban environments using seismic and acoustic methods. In: Proc. IEEE Conference on Technologies for Homeland Security. (12 13 May 28) Iyengar, S.G., Varshney, P.K., Damarla, T.: On the detection of footsteps based on acoustic and seismic sensing. In: Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers ACSSC 27. (4 7 Nov. 27) Houston, K.M., McGaffigan, D.P.: Spectrum analysis techniques for personnel detection using seismic sensors. In Carapezza, E.M., ed.: Unattended Ground Sensor Technologies and Applications V. Volume 59., SPIE (23) Huang, N.E., Shen, Z., Long, S.R., Wu, M.C., Shih, H.H., Zheng, Q., Yen, N.C., Tung, C.C., Liu, H.H.: The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 454(1971) (1998) Vlachos, M., Yu, P.S., Castelli, V., Meek, C.: Structural periodic measures for time-series data. Data Mining and Knowledge Discovery 12(1) (Jan 26) Lee, J.A., Verleysen, M.: Nonlinear Dimensionality Reduction. Springer (27)