Adaptive Sampling for Embedded Software Systems using SVM: Application to Water Level Sensors

Transcription

1 Adaptive Sampling for Embedded Software Systems using SVM: Water Level Sensors M. Pellegrini 1, R. De Leone 2, P. Maponi 2, C. Rossi 2 1 LIF srl, Via di Porto 159, Scandicci (FI), Italy, 2 School of Science and Technology Università degli Studi di Camerino May

2 Outline of the talk 2

3 The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models 3

4 The Classification Problem The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Given l vectors x i IR m,i = 1,...,l, a vector y IR l with y i { 1,1} determine a function h(x) such that h(x i ) > 0 when y i = 1 h(x i ) < 0 when y i = 1 4

5 The Classification Problem The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Given l vectors x i IR m,i = 1,...,l, a vector y IR l with y i { 1,1} determine a function h(x) such that h(x i ) > 0 when y i = 1 h(x i ) < 0 when y i = 1 The vectors x i and y define the training set. 4

6 The Classification Problem The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Given l vectors x i IR m,i = 1,...,l, a vector y IR l with y i { 1,1} determine a function h(x) such that h(x i ) > 0 when y i = 1 h(x i ) < 0 when y i = 1 f(x) = w T x+θ h(x) = sign(f(x)) 4

7 Machines The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Determine w IR n and θ R such that 5

8 Machines The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Determine w IR n and θ R such that w T x i +θ > 1 for y i = 1 and w T x i +θ < 1 for y i = 1 5

9 Machines The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Determine w IR n and θ R such that and and the margin. w T x i +θ > 1 for y i = 1 w T x i +θ < 1 for y i = 1 ( ) 2 w as big as possible. 5

10 C Classification The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models min w,θ 1 2 wt w Separable case subject to y i ( w T x i +θ ) 1, i = 1,...,l 6

11 C Classification The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models min w,θ,ξ General non separable case 1 2 wt w+ce T ξ subject to y i ( w T x i +θ ) 1 ξ i, i = 1,...,l ξ i 0, i = 1,...,l 6

12 C Classification The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models min w,θ,ξ General non separable case 1 2 wt w+ce T ξ subject to y i ( w T x i +θ ) 1 ξ i, i = 1,...,l ξ i 0, i = 1,...,l Controls learning capacity Controls number of misclassified points 6

13 C Classification The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models min w,θ,ξ General non separable case 1 2 wt w+ce T ξ subject to y i ( w T x i +θ ) 1 ξ i, i = 1,...,l ξ i 0, i = 1,...,l Controls learning capacity Note Controls number of misclassified points Larger values of C higher penalty for misclassification 6

14 Learning in the feature space The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Map data into a feature space where they are linearly separable φ : IR n H 7

15 Primal Dual C SVM The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models min w,θ,ξ 1 2 wt w+ce T ξ subject to y i ( w,φ(x i ) +θ ) 1 ξ i, i = 1,...,l ξ i 0, i = 1,...,l min α 1 2 αt Qα e T α subject to y T α = 0 0 α Ce Q ij = y i y j K ij, K ij = K(x i,x j ) := φ(x i ),φ(x j ) 8

16 Primal Dual C SVM The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models min w,θ,ξ 1 2 wt w+ce T ξ subject to y i ( w,φ(x i ) +θ ) 1 ξ i, i = 1,...,l ξ i 0, i = 1,...,l min α 1 2 αt Qα e T α subject to y T α = 0 0 α Ce Q ij = y i y j K ij, K ij = K(x i,x j ) := φ(x i ),φ(x j ) The function K : IR n IR n IR is called the kernel function. 8

17 Primal Dual C SVM The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models min w,θ,ξ 1 2 wt w+ce T ξ subject to y i ( w,φ(x i ) +θ ) 1 ξ i, i = 1,...,l ξ i 0, i = 1,...,l Note min α 1 2 αt Qα e T α subject to y T α = 0 0 α Ce Q ij = y i y j K ij, K ij = K(x i,x j ) := φ(x i ),φ(x j ) For the construction of the dual problem the actual form of the function φ( ) is not necessary: only the quantities φ(x i ) T φ(x j ) are needed. 8

18 The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Training data: ( x 1,t 1 ), ( x 2,t 2 ),..., ( x l,t l ) x i IR n,t i IR 9

19 The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Training data: ( ) ( ) ( ) x 1,t 1, x 2,t 2,..., x l,t l x i IR n,t i IR f(x) = w T x+θ Primal Problem min w,b,ξ +,ξ 1 subject to l 2 w 2 2 +C (ξ i +ξi) ( ) i=1 w,φ(x i ) +θ t i ǫ+ξi, i = 1,...,l ( ) w,φ(x i ) +θ t i ǫ ξ i, i = 1,...,l ξ 0, ξ 0 9

20 The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Training data: ( x 1,t 1 ), ( x 2,t 2 ),..., ( x l,t l ) min subject to where Q ij := x it x j Dual Problem 1 2 (α α ) T Q(α α ) + ǫ + x i IR n,t i IR l (α i +αi) i=1 l t i (α i αi) i=1 l (α i αi) = 0 i=1 α i [0,C], α i [0,C], i = 1,...,l 9

21 The ν-svr problem The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models min w,θ,ξ,ξ,ǫ subject to Primal Problem ( 1 2 w 2 +C ν ε+ 1 (ξ i +ξ l i) ( ) i=1 w,φ(x i ) +θ t i ǫ+ξi, i = 1,...,l ( ) w,φ(x i ) +θ t i ǫ ξ i, i = 1,...,l ξ 0, ξ 0 ǫ 0 l ) 10

22 The ν-svr problem The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models Dual Problem 1 min 2 (α α ) T Q (α α )+t T (α α ) subject to e T (α α ) = 0, e T (α+α ) C ν, α i [0,C/l], αi [0,C/l], i = 1,...,l 10

23 ǫ SVR and ν-svr models The Classification Problem Machines C Classification Learning in the feature space Primal Dual C SVM The ν-svr problem ǫ SVR and ν-svr models The parameter ν [0,1] controls the number of support vectors ǫ is not anymore a parameter but it is now a variable in the primal problem The ǫ SVR model with parameters ( C,ǫ) is equivalent to the ν-svr model with parameters (l C,ν) 11

24 Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 12

25 Hydrometric level Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing The aim is to define a new sampling strategy for hydrometric level that can self-adapt based on the error between predicted and observed water level time-trend. Using this procedure it will be possible to dynamically improve the measurement accuracy of the peak stage during a flood event. 13

26 The level monitoring project Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing The project has been carried out in collaboration with Marche Region - Security and Civil Protection Department (Regione Marche - Dipartimento per le politiche integrate di sicurezza e per la protezione civile) The goal is to provide a cost/effective monitoring of the river level, in particular in the case of flooding 14

27 The Marche Region Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 15

28 The September 2006 flooding upstream Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 16

29 The September 2006 flooding upstream Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 16

30 The September 2006 flooding downstream Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 16

31 The September 2006 flooding downstream Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 16

32 The September 2006 flooding downstream Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 16

33 The Monitoring System The Aspio stream Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 17

34 The Monitoring System The on a bridge Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 17

35 The Monitoring System Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 17

36 The Monitoring System Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 17

37 The Monitoring System Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing 17

38 Data Set and preprocessing Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing Weather data of Marche Region (located in east-central Italy) for a period of four years ( ) have been used to build the models, and data for year 2009 have been used to test the models. Marche Region SIRMIP (Regional Meteorological- Hydrological Information System) database includes readings of several hydrologic and weather parameters recorded with a sample rate of 30 minutes (15 minutes for rain data). The database is available online at Hydrometric data have been pre-processed in order to obtain time series representing the averages over six hours at any given stream cross-section (average over 12 samples). 18

39 Data Set and preprocessing Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing Aspio Terme (SIRMIP station code: 113) averaged for year 2008 (Training set) 18

40 Data Set and preprocessing Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing Hydrometric time series have been min-max normalized to transform the input data into the [0, 1] range. The objective is to predict 6-hours average of hydrometric level at a stream cross-section based on n previous 6-hours averages. A value of n=20 has been chosen since the rainfall occurred in the last 5 days is a crucial information to define the antecedent moisture condition. 18

41 Data Set and preprocessing Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing Aspio Terme averaged during January to March 2009 (Test set) 18

42 Data Set and preprocessing Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing Mean Square Error (MSE) is assumed as a performance measure in this work MSE = 1 l l i=1 ( f(x i ) t i ) 2 18

43 Data Set and preprocessing Hydrometric level The level monitoring project The Marche Region The September 2006 flooding The Monitoring System Data Set and preprocessing Mean Square Error (MSE) is assumed as a performance measure in this work MSE = 1 l l i=1 SE i ( f(x i ) t i ) 2 18

44 for Aspio Terme The proposed sampling strategy for Crocette ALTRE SLIDE 19

45 for Aspio Terme for Aspio Terme The proposed sampling strategy for Crocette ALTRE SLIDE Squared error between predicted and measured water level at Aspio Terme and MSE (January to March 2009) 20

46 for Aspio Terme for Aspio Terme The proposed sampling strategy for Crocette ALTRE SLIDE Flood event occurred at Aspio Terme section (black solid line), SVM model prediction error (grey solid line) 20

47 The proposed sampling strategy for Aspio Terme The proposed sampling strategy for Crocette ALTRE SLIDE The basic idea of the method is the exploitation of the considerable prediction error committed by the SVM model during a flood event. 21

48 The proposed sampling strategy When SE i results greater than MSE for a 6-hours averaged : for Aspio Terme The proposed sampling strategy for Crocette ALTRE SLIDE Note that f(x i ) t i > MSE t i = 1 k k m j j=1 where m j is the effective measured water level and k is the number of measurements between two consecutive predictions (i.e., 12 samples in 6 hours). 21

49 The proposed sampling strategy When SE i results greater than MSE for a 6-hours averaged : for Aspio Terme The proposed sampling strategy for Crocette ALTRE SLIDE either or k j=1 k j=1 m j < k ( f(x i ) MSE ) =: T m j > k ( f(x i )+ MSE ) =: T + 21

50 The proposed sampling strategy for Aspio Terme The proposed sampling strategy for Crocette ALTRE SLIDE calculate and keep in memory 20 previous 6-hour averaged levels; run the regression model in order to predict next 6-hour averaged level; each time a new measurement m j is taken, compare the partial sum of levels with threshold T+ to test for an under-prediction; increase the sampling rate if T + is exceeded; compare the total sum of levels with threshold T to test for an over-prediction; otherwise, hold the sampling rate steady until at least one of the two inequalities is verified; decrease the sampling rate when f(x i ) t i MSE. 21

51 for Crocette for Aspio Terme The proposed sampling strategy for Crocette ALTRE SLIDE Measured level and predicted error at Crocette 22

52 for Crocette for Aspio Terme The proposed sampling strategy for Crocette ALTRE SLIDE Squared error between predicted and measured water level at Aspio Terme and MSE (January to March 2009) 22

53 Conclusiona and future developments 23

54 Conclusiona and future developments Conclusiona and future developments Machines can be successfully used in Time Series regression 24

55 Conclusiona and future developments Conclusiona and future developments Machines can be successfully used in Time Series regression A new efficient strategy sampling for can be devised based on the difference between measured level and predicted error 24

56 Conclusiona and future developments Conclusiona and future developments Machines can be successfully used in Time Series regression A new efficient strategy sampling for can be devised based on the difference between measured level and predicted error Combine information from different s to improve prediction quality. 24