Supervisors: Prof. dr. Guy De Tré, Prof. dr. Nico Van de Weghe Counsellors: Ir. Christophe Billiet, Jasper Beernaerts

Transcription

1 Efficient analysis of large amounts of motion trajectories Hans Standaert Supervisors: Prof. dr. Guy De Tré, Prof. dr. Nico Van de Weghe Counsellors: Ir. Christophe Billiet, Jasper Beernaerts Master's dissertation submitted in order to obtain the academic degree of Master of Science in Computer Science Engineering Department of Telecommunications and Information Processing Chair: Prof. dr. ir. Herwig Bruneel Vakgroep Geografie Chair: Prof. dr. Philippe De Maeyer Faculty of Engineering and Architecture Academic year

2

3 Efficient analysis of large amounts of motion trajectories Hans Standaert Supervisors: Prof. dr. Guy De Tré, Prof. dr. Nico Van de Weghe Counsellors: Ir. Christophe Billiet, Jasper Beernaerts Master's dissertation submitted in order to obtain the academic degree of Master of Science in Computer Science Engineering Department of Telecommunications and Information Processing Chair: Prof. dr. ir. Herwig Bruneel Vakgroep Geografie Chair: Prof. dr. Philippe De Maeyer Faculty of Engineering and Architecture Academic year

4 Preface I would like to thank everyone that was a part of the research I have performed during the past year. Especially, I would like to thank the following people for their role in the realization of this dissertation. First of all, I would like to thank prof. dr. Guy De Tré and prof. dr. Nico Van de Weghe for giving me the opportunity to perform this research. Second, I would like to thank drs. ir. Christophe Billiet for his guidance during this research and for his encouragement to push me and my research to the limit. With his large amount of constructive feedback, I was able to make this dissertation what it is today. I would also like to thank drs. Jasper Beernaerts for assisting me during my research on the Qualitative Trajectory Calculus. I want to thank my girlfriend and my friends for their support and for their boundless confidence. Finally, I would to thank my parents for giving me the opportunity to pursue this degree and for their support during these years. Their encouragement and support were an important factor in the pursuit of this degree. Hans Standaert, June 2018

5 Permission of Usage The author gives permission to make this master dissertation available for consultation and to copy parts of this master dissertation for personal use. In the case of any other use, the copyright terms have to be respected, in particular with regard to the obligation to state expressly the source when quoting results from this master dissertation Hans Standaert, June 2018

6 Efficient analysis of large amounts of motion trajectories by Hans Standaert Master s dissertation submitted in order to obtain the academic degree of Master of Science in Computer Science Engineering Academic year Ghent University Faculty of Engineering and Architecture Department of Telecommunications and Information Processing Chair: Prof. dr. ir. Herwig Bruneel Vakgroep Geografie Chair: Prof. dr. Philippe De Maeyer Supervisors: prof. dr. Guy De Tr, prof. dr. Nico Van de Weghe Counsellors: ir. Christophe Billiet, Jasper Beernaerts Summary In the scientific work presented in this dissertation, a heuristic is proposed that is able to cope with some of the main limitations of the exhaustive and brute force techniques that are currently used in the context of the Qualitative Trajectory Calculus (QTC), with the goal of querying a large set of so-called QTC motion data in order to retrieve similar patterns. Due to the size of this data, current techniques fail to accomplish this task within a reasonable timespan. Hence, novel techniques need to be created, which are able to surpass the aforementioned limitations. To that end, a novel heuristic is developed in the work presented in this dissertation. In order to achieve this, the following steps were subsequently taken. In a first step, the characteristics of the QTC theory were examined. Since QTC transforms coordinates of moving objects into QTC-data, the corresponding characteristics in both the original coordinate domain and the resulting QTC domain are examined. From this research, it was learned that noise has significant influence on both domains. Therefore, these influences were discussed in a separate chapter. In a second step, by examining the QTC-data, it was learned that this data is dominated by repetitive, atomic elements. In order to reduce the amount of repetitions, several preprocessing steps were applied. These preprocessing steps used lossless and lossy compression techniques in order to reduce both the size and computational complexity of the problem. By combining both compression techniques a tree-like datastructure was obtained. This datastructure is used as a compact and complete representation of the uncompressed dataset. Finally, the intended heuristic was derived by combining the findings of each step. Furthermore, the performance of the heuristic was tested, whereby the results of these tests showed that in most cases, the heuristic can be used as an alternative for the exhaustive techniques that are currently being used. In a final chapter, ideas for future research are outlined. Keywords: QTC, QTC B1, Lossless Compression, Lossy Compression, Tree Structure 1

7 Efficient analysis of large amounts of motion trajectories Hans Standaert Supervisors: prof. dr. Guy De Tré, prof. dr. Nico Van de Weghe, ir. Christophe Billiet, Jasper Beernaerts Abstract In the scientific work presented in this paper, a heuristic is proposed that is able to cope with some of the main limitations of the exhaustive and brute force techniques that are currently used in the context of the Qualitative Trajectory Calculus (QTC), with the goal of querying a large set of so-called QTC motion data in order to retrieve similar patterns. Due to the size of this data, current techniques fail to accomplish this task within a reasonable timespan. Hence, novel techniques need to be created, which are able to surpass the aforementioned limitations. To that end, a novel heuristic is developed in the work presented in this paper. This heuristic makes use of lossless and lossy compression techniques in order to reduce both the size and the computational complexity of the problem. Furthermore, the performance of the heuristic was tested, whereby the results of these tests showed that in most cases, the heuristic can be used as an alternative for the exhaustive techniques that are currently being used. Keywords QTC, QTC B1, Lossless Compression, Lossy Compression, Tree Structure I. INTRODUCTION One of the major challenges in the context of so-called Big Data is to find patterns in large amounts of data in order to discover non-trivial relationships between data. In general, the purpose of the work that was performed in the context of this research is the same. A large set of motion data, obtained by measuring the trajectories of moving objects, need to be queried in order to find motion relationships that are similar to the query. In football, for example, where the motion dataset consists of the movement trajectories of each of the players, it could be useful to verify whether or not the tactical movements practiced, occur during the game. In order to allow such a querying system, two thing are necessary: a transformation that converts the raw data, basically xy-coordinates, into motion relationships and an adequate comparison metric that is able to express the similarity between two such (motion) relationships. Note that the aforementioned transformation must be able to characterize the movement between two moving objects on a distanceless scale. Hence, the distance between the moving objects does not have an influence on the relationships expressed by this transformation, allowing us the describe the motion relationships in a purely relational way. However, a lot of research on those topics is already done by prof. dr. Nico Van de Weghe with his work on Qualitative Trajectory Calculus (QTC) [1] [4]. During this research a variant of QTC was used. This variant, QTC B1, is the most basic version of QTC and only takes into account the relative distances between the moving objects, in order to compare their motion relationships. The speed at which the objects move relative to each other is not included. The goal of the research presented in this paper is to propose a novel method used to find specific patterns in a large amount of QTC B1 motion data. Thereby, this heuristic should at least take into account following objectives: speed, accuracy, support for multiple moving objects and possibilities to find scaled versions of the pattern. Indeed, since the current techniques are too slow to be usable in a practical setting, a faster alternative is required. Next, the intended heuristic should be able to find most, preferably all, results that are found by the current techniques. However, the accuracy can not suffer from the fact that the heuristic needs to be fast. Furthermore, the heuristic should also be able to deal with motion relationships simultaneously involving multiple (more than two) moving objects. Finally, a favorable property of the heuristic would be that it is able to recognize trajectories that are executed at a different pace, and identify them as performances of the same trajectory. Since, to our knowledge, the application of Computer Science techniques in order to reduce the computational complexity of finding patterns in QTC B1 motion data is quite novel and unprecedented, no section discussing previous or related scientific work, nor a great lot of references to existing works or publications will be found throughout this paper. The rest of this paper is structured as follows: in Section II, the theoretical background of the QTC variant that is used, will briefly be discussed. Section III will discuss the preprocessing steps that are required to transform the xy-coordinates into QTC motion data. In Section IV, the proposed heuristic will be discussed in all its facets and in Section V this heuristic will be evaluated by using the aforementioned objectives. Finally, in Section VI, conclusions on the work that was done, are discussed and some lines for future research are presented. II. QUALITATIVE TRAJECTORY CALCULUS B1 The goal of QTC B1 is to provide a spatiotemporal reasoning framework in order to describe the motion between two (disjunct) moving objects and the relationships between these motions on a distanceless scale. In order to reach this goal, the calculus makes use of three unique symbols. These symbols denote whether an object moves closer to, away from or not at all with respect to an other object during a specified time 1

8 interval and are given by a, + and 0 sign respectively. Hence, afterwards, these symbols can be combined to form a so-called QTC tuple. For example, if object a moves closer to object b, and b moves away from a during some time period t, their accompanying QTC tuple is given by (, + ). A sequence of QTC tuples is hereafter called a QTC sequence. Please note that, in order to compare the motion relationships between n moving objects, a total of n 2 n such a QTC sequences are needed. Indeed, since every moving object can be compared to any of the other moving objects, the total amount equals n 2 n. However, since the QTC tuples between moving objects b and a can be derived from the QTC tuples between a and b, this total amount decreases to n(n 1) 2. Nonetheless, it is clear that with an increasing number of moving objects, the computational complexity of the problem increases exponentially. To make it easier to read and understand the QTC sequences, the nine unique QTC tuples are converted into a string representation. Therefore, the mappings of Table I are used. Table I: QTC B1 Mapping Table First Element Second Element Mapping 0 - U 0 0 A 0 + K + - V + 0 B + + L - - W - 0 C - + M III.PREPROCESSING In order to prepare the raw data, which consists of xycoordinates, for usage by the heuristic, several preprocessing steps are required. Therefore, in this section, the preprocessing steps in the coordinate domain and QTC domain will be discussed. A. Coordinate Domain First, in the coordinate domain, a moving average is applied on the x and y coordinate sequences of the moving objects. Hereby, the amount of outliers is reduced and the original trajectory is approximated in a more natural, smoothed way. Moreover, any noise that is present in the signal, will be averaged out by taking into account multiple samples at once. Second, in order to reduce the amount of data that is generated in the QTC domain, the original signal is subsampled at a sampling rate lower than 10Hz. Hereby, we made the assumption that the moving objects used in our test sets are not able to significantly change their direction within less than 100ms. As a result, the amount of data that is generated in the QTC domain as well as the computational complexity of the problem is reduced. Figure 1 summarizes the findings of the reasoning above. In this figure, the x and y coordinates of a moving object during a time interval of 10s are shown. The line in blue represents the trajectory of the original signal, whereas the line in orange was obtained after applying a moving average and afterwards subsample the signal. It is clear that this combination of preprocessing steps is indeed able to approximate the original signal without losing too much of its details. Note that a tradeoff has to be made between temporal resolution and quality of the approximated signal. Whenever the objects that are sampled change direction frequently, higher temporal resolutions are preferred. On the other hand, when the objects show less changes in direction, lower temporal resolutions can be used. Note, however, the conclusions that can be made concerning the motion relationships between these moving objects are only valid at their corresponding temporal resolution. y B. QTC Domain Original Preprocessed Coordinate preprocessing: MA - SS x Figure 1: Preprocessing coordinate domain In the QTC domain, the QTC sequences are transformed into their string representation by using the mappings of Table I. Afterwards, these string representations are compressed by using a variant of the lossless run-length encoding technique called Low Level Compression (LLC). By using this compression technique, all repetitive QTC tuples will be removed and each character will be given a start and end timestamp correspondingly. For example, consider the fictive QTC sequence given by: AAAABBBC. After applying LLC, this sequence is reduced to ( A,0,4); ( B,4,7); ( C,7,8). In this way, we are able to significantly reduce the size of the original QTC sequences without losing information. Indeed, by using the start and end timestamps of each of the characters the original signal can be fully reconstructed. Next, these compressed sequences are compressed once more, by using a lossy compression technique that is further referred to as High Level Compression (HLC). This compression technique is based on the matrix representation below: M K L C A B W U V In this matrix, the string representations of the QTC tuples are structured by using their first and second element signs as an indicator for the column and row indices respectively. Hence, a, 0 and + sign for the first elements correspond to an column index of 1, 2 and 3. On the other hand, a +, 0 and sign for the second element corresponds to an row index of 1, 2 and 3. For example, B given by QTC tuple ( +, 0 ) can be found in the last column of the second row. 2

9 Indeed, we can use the row index of each character as index for the QTC tuple, allowing us to further compress the QTC sequences. Applied to the same example as before, the previously found ( A,0,4); ( B,4,7); ( C,7,8) will be converted into (1,0,8) since A, B and C all have the same row index of 1. To show the effectiveness of both compressions, Table II shows the compression ratios that were obtained in each of the datasets that were used during this research. Remark, the compression ratio is expressed as new size divided by old size, where size refers to the size of the QTC sequence. Hence, the lower the compress ratio, the more compressed the result is. As a result, by applying these compression techniques we are (on average) able to reduce the amount of data in the QTC domain to about 10% and 6% of its original size when using LLC and HLC respectively. Table II: Compression ratios for each dataset Dataset LLC HLC Football % 5.99 % Treadmill 4.30 % 2.10 % Basketball % % Finally, both these compression techniques are combined in order to form some kind of tree structure, where the leaf nodes are represented by the results of the LLC, and the root nodes by the results of the HLC, as shown in Figure 2. In this figure, the same example that was used in order to illustrate both LLC and HLC is used. The black dot at the root represents the result of HLC, and the white dots represent the results of LLC. Note that each node is characterized by a symbol and a timestamp, allowing us to reconstruct the original signal at any point in time by decompressing the nodes that correspond to that time interval. In the following section, this tree structure will be used in order to devise the heuristic. A B 4-7 C 7-8 Figure 2: Resulting tree structure IV. PROPOSED SOLUTION This section introduces the novel heuristic that is used to find specific patterns in a large amount of QTC B1 motion data. This heuristic was devised by efficiently using the tree structure described in the previous section. Since the goal of the intended heuristic is to find patterns similar to the one of the query, only candidate solutions that are significantly similar to the query pattern need to be examined. In order to reduce the amount of candidate solutions, we can use the aforementioned tree structure to quickly assess whether a candidate solution is a good candidate or not. Indeed, whenever a candidate solution at a higher, more compressed level is too dissimilar to the query pattern, we are sure that their decompressed representations will at least have the same amount of dissimilarity, whereby these decompressed representations are found by going deeper in to the aforementioned tree structure. The heuristic can best be explained by using the illustrations in Figure 3. In this figure, the architecture of the heuristic is described by means of a decision tree. In this tree, the rectangles represent checks to which a candidate solution is applied. The arrows on the other hand, denote decisions that are made. First of all, on top of the figure the edit distance between the HLC of the current candidate solution and the HLC of the query QTC fragment is calculated. Next, the dissimilarity between both fragments is compared against the threshold at the HLC level, this threshold is hereafter referred to as γ 3. Whenever this dissimilarity is beyond the threshold value, the heuristic stops and proceeds by moving on to the next candidate solution as illustrated by means of the triangle. Otherwise, both the candidate solution and the query fragment are decompressed and their dissimilarity at the LLC level is calculated. Again, when the dissimilarity is beyond threshold γ 2 the heuristic stops and proceeds with the candidate solution next in line, otherwise the candidate solution and the query fragment are decompressed once more, resulting into their original QTC sequences. Again, their dissimilarity is calculated and compared to γ 1. In case the uncompressed candidate solutions are sufficiently similar, the heuristic stores the start and end timestamps of the first and last QTC characters in their sequences, denoted by the circle, and proceeds to the following candidate solution. Otherwise, the candidate solution is removed from the list of possible solutions. In order to calculate the dissimilarity between two patterns, the Levenshtein [5] edit distance was used. However, in order to accelerate the calculations, the algorithm was changed such that it stops whenever the edit distance between the two patterns exceeds a predefined threshold. Indeed, since only similar patterns need to be found, patterns that are too dissimilar can be ignored. In this way, the amount of calculations in order to find the dissimilarity between the two patterns is reduced to the absolute minimum. Note that the predefined thresholds that are used in the adapted version of the Levenshtein edit distance are given by γ 1, γ 2 and γ 3 correspondingly. As a result, the heuristic can be tweaked in various ways to suit a variety of situations. For example, when only QTC fragments that equal the query fragment are requested, all thresholds should be set to 0 since equal fragments have the same LLC and HLC. On the other hand to find fragments with the same LLC, γ 3 and γ 2 should be set to 0, whereas γ 1 would equal -1. By setting the first threshold to -1, the calculations at the lowest level are skipped. Since the results of each level are stored, fragments with similar LLCs can be obtained. 3

10 HLC YES < γ 3 NO less computationally intensive way. However, due to greedy approach of the heuristic, a small decrease in accuracy is obtained. In most cases, however, the decrease in accuracy is so small that this disadvantage does not outperform the fact that the results are obtained much faster. OR YES LLC NO < γ 2 The heuristic can further be improved by parallelizing the execution of the query. Indeed, since the datastructure can be divided into independent chunks of QTC-data, the query can be executed on each of these chunks in parallel. Another improvement, likely to have a greater impact than the first, can be obtained by sampling the trajectories of the moving objects only at instants where the objects change direction. In this way, no information of the trajectories is lost and the amount of data in both domains is reduced. As a result, the execution times will probably be improved. YES < γ 1 NO Figure 3: Architecture of the heuristic V. RESULTS AND DISCUSSION In this section, the proposed heuristic will be evaluated by using the objectives put forward in the first section. Since the tree structure is able to compactly represent the whole set of QTC B1 motion data, the heuristic is indeed able to reach all of its imposed objectives. By combing LLC and HLC in this tree structure, the effect of the lossy compression technique of HLC is minimized to only those cases that have a threshold value for γ 3 smaller than the threshold value of γ 2. Indeed, since the heuristic filters out candidate solutions at higher levels to decrease the amount of comparisons it has to made at lower levels, caution should be taken when setting this threshold. The results of the tests that were performed on the heuristic learned that on average a decrease in execution time of 99.91% was obtained. However, this decrease in execution time also led to a small decrease in accuracy. Indeed, depending on the relation between the values of γ 2 and γ 3, more or less candidate solutions are taken into account at the lowest level. As a result a decrease of about 10% in recall was obtained in cases where γ 3 < γ 2. However, the precision of the heuristic always remained 100%, since the heuristic can not find any other solutions as the ones that are found by the ground truth solutions of the brute fore/current technique. As a result, the amount of false positives is also 0. Finally, as discussed at the end of the previous section, by using the candidate solutions that are found at the LLC level, scaled versions of the pattern can be obtained. REFERENCES [1] M. Delafontaine and A. G. Cohn, Calculus to Reason about Moving Point Objects, pp , [2] N. Van de Weghe, a. G. Cohn, P. Bogaert, and P. De Maeyer, Representation of moving objects along a road network, Proc. of Geoinformatics, no. June, pp , [3] N. Van De Weghe, P. Bogaert, A. G. Cohn, M. Delafontaine, L. De Temmerman, T. Neutens, P. De Maeyer, and F. Witlox, How to handle incomplete knowledge concerning moving objects, CEUR Workshop Proceedings, vol. 296, pp , [4] N. Van De Weghe, A. G. Cohn, G. De Tré, and P. De Maeyer, A qualitative trajectory calculus as a basis for representing moving objects in geographical information systems, Control and Cybernetics, vol. 35, no. 1, pp , [5] V. I. Levenshtein, Binary codes capable of correcting deletions, insertions, and reversals, Soviet Physics Doklady, vol. 10, no. 8, pp , VI.CONCLUSIONS In this paper, a novel method used to find patterns similar to a query pattern in a large amount of motion data is proposed. By using a tree structure, where each level of the tree corresponds to a certain level of compression, the limitations of the current, exhaustive technique can be dealt with. Moreover, the heuristic is able to find these patterns in a more efficient and 4

11 Contents 1 Introduction Problem Outline and Context Goals and Objectives Document Structure Qualitative Trajectory Calculus The Basics of QTC QTC B Definition Adaptations Data and Preprocessing Data Preprocessing Preliminary Insights Coordinate Domain QTC Domain Dealing with Noise in QTC Coordinate Domain QTC Domain Adaptive QTC-string Comparison Terminology Preprocessing Complexity Edit Distance Basic Algorithm Advanced Algorithm

12 CONTENTS One Relation Multiple Relations Evaluation Speed Accuracy Involve Multiple MPOs Scaled Version of the Pattern Conclusion and Future Work Conclusion Future Work

13 List of Tables 2.1 QTC B1 Mapping table Compression ratios for each dataset Execution times - overview Precision and recall for the football and treadmill dataset Impact of γ 3 on precision and recall Accuracy for the football and treadmill dataset in case of multiple MPOs. 63

14 List of Figures 2.1 Summary of three of the four abstractions made in QTC k moves towards l (-) k not moving with respect to l (0) k moves away from l (+) QTC B1 relations QTC Example QTC B1 relaxation QTC matrix Preprocessing steps in the coordinate domain Comparison of successive operations during preprocessing QTC datastructure Preprocessing steps - overview Impact of noise in the coordinate domain QTC B1 angle calculations New preprocessing step Result of novel preprocessing step Conceptual Neighbourhood Diagram of QTC B Manhattan distances football dataset Graph Structure Overview Terminology overview Fragment search in a QTC sequence Computational complexity for different values of n and τ q Edit distance Basic algorithm

15 LIST OF FIGURES 5.7 Basic algorithm example Advanced algorithm Query example Solution tree Impact of the edit distance Influence of γ 1 and γ 2 in the basic algorithm Influence of γ 1 in the advanced algorithm Influence of the query length Example of finding scaled version patterns

16 List of Abbreviations QTC MPO MA LLC HLC CND Qualitative Trajectory Calculus Moving Point Object Moving Average Low Level Compression High Level Compression Conceptual Neighborhood Diagram

17 Chapter 1 Introduction This introductory chapter will give a brief overview of the work done in this Master s thesis research. In this chapter s first section, a more detailed description of the subject of this research will be given. Therefore, the subject will be situated in its current context along with its current limitations. These limitations are the starting point of the research done during the past months, and therefore the second section will present the goals and objectives of this dissertation. In the third and last section, the structure of the document will be discussed. 1.1 Problem Outline and Context Nowadays, a lot of attention is drawn to finding patterns in large amounts of data in order to find non-trivial relationships. Indeed, this is one of the major challenges addressed in the context of so-called Big Data. In some perspective, the purpose of the work that was performed in the context of this thesis is the same. More specifically, the data is obtained by measuring moving objects, and the patterns are found by comparing the motion relationships between those objects. Hence, the patterns describe the sequences of sequential coordinate data and their properties. In football, for example, where the moving objects, players, are supposed to behave in accordance with the strategy or tactics provided by their trainer, it could useful to be able to verify whether or not the tactical movements practiced, occur during the game. In order to accomplish this task, two things are necessary. First, a transformation should be applied on the raw data, basically xy-coordinates, allowing some kind of temporal reasoning. This transformation must be able to characterize the movement between two 1

18 1.2. GOALS AND OBJECTIVES 2 moving objects on a distanceless scale. Hence, the distance between the moving objects has no influence on the resulting pattern, allowing us to describe the motion relationships in a purely relational way. Second, in order to find similar patterns, an adequate comparison metric must be designed. A lot of research on those topics is already done by prof. dr. Nico Van de Weghe with his work on Qualitative Trajectory Calculus (QTC) [1, 4, 5, 6]. As a result, only QTC B1 -data will be used throughout this research. QTC B1 is a variant of QTC that only takes into account relative distances between moving objects, in order to compare the motion relationships between those objects. The speed at which objects move relative to each other is not included. In particular, this data consists of long sequences of so-called QTC tuples, which describe the motion relationship between two moving objects during a finite time interval, by using one out of three unique symbols for each element in the relation. The algorithm that will be used in order to find patterns in the QTC B1 -data already exists, but is too slow to be useful in real-world applications, due to the size of this data. Therefore, a heuristic search technique will be devised. This heuristic will result into better search performances, since it will make use of properties inherent to this QTC-data. During this research it was clear that noise has a not-negligible impact. Indeed, patterns that where found in case no noise was present, were not found by the heuristic in cases that involved noise. Therefore, additional research was done on reducing the impact of this noise, and on how to incorporate it into the heuristic. 1.2 Goals and Objectives The main goal of the work in this Master s thesis research is to find a heuristic which is able to solve the problem of finding specific patterns in a large amount of QTC B1 motion data. This heuristic should at least take into account following objectives: Speed Current algorithms use an exhaustive brute force technique leading to enormous execution times. Because these techniques are too slow to be usable in a practical setting, faster algorithms are required. One way a technique could be faster is when

19 CHAPTER 1. INTRODUCTION 3 it is able to quickly assess whether a candidate pattern is a good candidate or not, and only continue with good candidates. Accuracy The heuristic should be able to find most, preferably all, results found by current techniques. Involve multiple moving objects The technique must be able to deal with relationships simultaneously involving multiple (more than two) moving objects. Current techniques do this by separately comparing all involved relationships. This approach is somewhat inefficient w.r.t computation and/or memory usage. The intended heuristic must do better. Scaled version of the pattern Inherent to motion is the speed at which it is performed. One could perform the same trajectory at a different pace, resulting into different speeds of execution. Therefore, a favorable property of the intended heuristic is that it is able to recognize trajectories that are executed at a different pace, and identify them as performances of the same trajectory. Since the theory of QTC is relatively new, scientific research that can be used to start from in order to develop the aforementioned heuristic or to compare this masters thesis research with, is rather limited. Moreover, the research that is done during the past months is of the first of its kind. Indeed, to my knowledge, the application of Computer Science techniques in order to reduce the computational complexity of finding patterns in QTC B1 motion data is quite novel and unprecedented. Therefore, this mater thesis dissertation will not contain a section discussing previous or related scientific work, nor a great lot of references to existing works or publications. 1.3 Document Structure This dissertation is structured as follows: in chapter 2, a theoretical background of the QTC variant that is used, will be given. Chapter 3 will discuss the data and preprocessing steps needed in order to transform the xy-coordinates into QTC motion data. Chapter 4

20 1.3. DOCUMENT STRUCTURE 4 will go deeper into the theory of QTC and investigate the impact of noise in both the xycoordinates and QTC. In chapter 5, the resulting heuristic will be discussed in all its facets. Chapter 6 will compare the proposed algorithm against the currently used exhaustive technique by using different scenarios. Finally, in Chapter 7, conclusions on the work that was done are discussed along with some ideas to further improve the performance of the heuristic.

21 Chapter 2 Qualitative Trajectory Calculus This chapter provides an introduction to the theory of QTC, as this will form the basis for all further work. The first section will deal with the general idea of the theory. The rest of the chapter discusses QTC B1, which is the variant of QTC that is used throughout this thesis. 2.1 The Basics of QTC The goal of the Qualitative Trajectory Calculus (QTC) is to be able to compare trajectories of moving objects. Such spatiotemporal phenomena are typically described using coordinates in some reference frame. Since these coordinates in se do not have enough power to allow such comparisons, a novel approach called QTC was introduced by prof. dr. Nico Van de Weghe [4] back in The proposed method describes the motion of two (disjunct) moving objects and the relationships between these motions. The main incentive of the calculus is that it is difficult to compare two trajectories by only considering their coordinates. Moreover, these coordinates only provide information concerning the position of those trajectories in the predefined coordinate space. As a consequence, there is no straightforward way to obtain information about the relationships between these trajectories. In addition, the relationships of trajectories in different coordinate systems can also not be compared to each other. To cope with these limitations, QTC introduces a novel way to reason about the relationships of two or more trajectories by comparing them on a distanceless scale. Hence, the distance between the moving objects has no influence on the relationship expressed by the calculus. To obtain such a scale, QTC makes use of three unique symbols. These 5

22 2.1. THE BASICS OF QTC 6 symbols denote whether an object moves closer to, away from or not at all with respect to an other object during a specified time interval and are given by a, + and 0 sign respectively. The symbols, representing the relationship between two objects, can then be combined to form a so called QTC tuple. For example, if object a moves closer to object b, and b moves away from a during some time period t, their accompanying QTC tuple is given by (, + ). Thereby, the fist tuple element is used to describe the motion of a compared to b and the second one to describe the motion of b compared to a during t. A sequence of QTC-tuples is then called a QTC sequence. Please note that, when the elements of the tuple switch places, this new formed tuple represents the motion relationship between MPOs b and a. This property of QTC will be frequently used in upcoming chapters. In order to use QTC on real-world examples, where moving objects can have complex shapes and properties, four assumptions/abstractions have to be made [6]. These abstractions are illustrated in Figure 2.1 and explained below. Topological abstraction This first abstraction is the most obvious one, assuming that only disjunctly moving objects are allowed. It is a consequence of the fact that the relation between two identical moving objects is trivial. Relational abstraction As mentioned in the first chapter, QTC makes use of binary relationships. This implies that the only thing that matters is the movement of two objects relative to each other. It also implies that, in order to compare trajectories of n moving objects, all possible binary relationships with those n objects need to be examined. This results into a total of n 2 n, and not n 2, binary relationships, since trivial relationships are not taken into account. Object abstraction Since the shape of the object has no direct influence on the intended direction of the object, the objects are reduced to a single point. Therefore, all complex moving objects are reduced to so-called moving point objects, further referred to as MPOs. The location of the abstracted point can be chosen freely, however it is

23 CHAPTER 2. QUALITATIVE TRAJECTORY CALCULUS 7 straightforward to use the same one for each object, the center of mass for example. Temporal abstraction The complete QTC sequence is a sequence of QTC-tuples which all hold at a particular point in time. These QTC-tuples are the subject of the upcoming sections. (a) Real-world example (b) Relational abstraction (c) Object abstraction (d) Temporal abstraction Figure 2.1: Summary of three of the four abstractions made in QTC Although other variants exists, the work presented in this thesis will only consider QTCB1. This variant of QTC only takes into account the relative distance between two moving objects. The speed at which both objects move relative to each other is not included. 2.2 QTCB1 In some way, QTC can be considered an input-output model. The input is given by the sequences of spatial coordinates generated by all involved MPOs, the output consists of the QTC-tuples as described above. The spatial coordinates of an MPO are defined as the sequence of coordinates describing the motion of the MPO in its frame of reference.

24 2.2. QTC B1 8 Since an Euclidean space with a Cartesian coordinate system is assumed, these sequences of spatial coordinates will be further referred to as xy-coordinate paths Definition To convert two xy-coordinate paths into their corresponding QTC-tuples, the difference in Euclidean distance between start and end of each interval is considered. In order to pour this into a mathematical model, the following expressions are introduced: MPOs k and l, the blue and red objects in Figures 2.2, 2.3, 2.4 k t denotes the position of MPO k at time instant t d(u, v) denotes the Euclidean distance between u and v t 1 t 2 denotes that t 1 is temporally before t 2 These expressions allow us to describe the movement of k with respect to l during some time interval. Three different scenarios can be distinguished: when k is moving towards l, Figure 2.2: t 1 (t 1 t t (t 1 t t d(k t, l t) > d(k t, l t))) (2.1) t 2 (t t 2 t + (t t + t 2 d(k t, l t) > d(k t +, l t))) (2.2) + when k is moving away from l, Figure 2.4: t 1 (t 1 t t (t 1 t t d(k t, l t) < d(k t, l t))) (2.3) t 2 (t t 2 t + (t t + t 2 d(k t, l t) < d(k t +, l t))) (2.4) 0 when k is stable l (all other cases), Figure 2.3. The movement of l with respect to k can be found by switching l and k in the reasoning above. Figures 2.2, 2.3, 2.4 illustrate the mathematics behind the definitions mentioned above. In each figure, a, b and c denote d(k t, l t), d(k t, l t) and d(k t +, l t) respectively. For

25 CHAPTER 2. QUALITATIVE TRAJECTORY CALCULUS 9 example, in Figure 2.2, it is clear that a > b and that b > c, therefore it is said that k moves towards l in the time period defined by t1 and t2. k t1 k t- k t k t+ a b l t1 k t2 c l t2 l t Figure 2.2: k moves towards l (-) k t1 k t k t- a k t+ b l t1 Figure 2.3: k not moving with respect to l (0) k t2 c l t l t2

26 2.2. QTCB1 10 k t- k t1 k t+ k t a l t1 b k t2 c l t2 l t Figure 2.4: k moves away from l (+) The definition shows that the elements of the tuples are restricted to only three symbols:, 0 and +. Furthermore, it is clear that these symbols make up a set of nine uniquely defined QTC-tuples. These tuples are shown in Figure Figure 2.5: QTCB1 relations 0

27 CHAPTER 2. QUALITATIVE TRAJECTORY CALCULUS 11 To conclude this section on QTC B1, an example is given wherein the trajectories of two (fictive) MPOs are converted into their accompanying QTC characters. In Figure 2.6, the trajectories of both MPOs are given by plotting their xy-coordinates through time. Note that, in this case the x-axis and the time-axis are the same. The x and y coordinates are converted into QTC-tuples as follows: During the first time period (0-1), MPO 1 and MPO 2 move towards each other and therefore their QTC tuple is given by (, ). From t = 1 till t = 2 both MPOs move away from each other, resulting into QTC tuple ( +, + ). Finally, during the last interval, MPO 1 does not move with respect to MPO 2 and MPO 2 does not move with respect to MPO 1, denoted by QTC tuple ( 0, 0 ). As a result, the relationship between the motions of MPO 1 and 2 during time interval (0-3) is given by the following QTC sequence: (, ), ( +, + ), ( 0, 0 ). 1 QTC Example y 0 MPO 1 MPO x and t Figure 2.6: QTC Example Adaptations The theory of QTC is quite strict and therefore sometimes fails to describe realistic situations the way it is supposed to. For example, when even a small amount of noise is present in the coordinates of the MPOs, it is possible that completely different QTC-tuples are obtained. In Chapter 4, the impact this kind of noise will be discussed, but in order to be able to solve these kind of problems, following adaptations are proposed: Firstly, the

28 2.2. QTC B1 12 tuples are converted into a string representation based on the nine different characters as displayed in Table 2.1. This string representation will make it easier to understand and compress the QTC sequences. Secondly, in order to provide some kind of buffer against noise, a softer threshold between +, and 0 is introduced. This threshold defines α as the angle between old threshold and new threshold, thereby allowing some relaxation in the definition of QTC B1. This relaxation is shown in Figure 2.7. More details on this are found in Chapter 4. First Element Second Element Mapping 0 - U 0 0 A 0 + K + - V + 0 B + + L - - W - 0 C - + M Table 2.1: QTC B1 Mapping table α Figure 2.7: QTC B1 relaxation

29 Chapter 3 Data and Preprocessing The focus of this chapter is on the different kinds of preprocessing steps that are used in order to prepare the data for the querying heuristic devised in Chapter 5. These preprocessing steps will be further subdivided into two domains, being the coordinate domain and QTC domain, since each domain has its own specific kind of preprocessing. The chapter is structured as follows: Section 3.1 will briefly introduce the data that is available and used in order to verify the findings of the heuristic. Finally, Section 3.2 will discuss preprocessing in both domains. 3.1 Data Typically, the datasets are build by sampling the motion of multiple MPOs through time, where each sample has x and y coordinates for each of the MPOs. Afterwards, these long sequences of coordinates are stored into either an.xml or an.csv file. These files contain at least an identifier of the MPO from which the sample was taken, an x coordinate and a y coordinate for each of the samples. Depending on the use case of the dataset, not all datasets are specially made for motion analysis with QTC, additional information can be put into these files. For example, some metadata concerned with the identity of the MPO, or environmental data can be added. Note that the data is provided by coordinate pairs and not by QTC sequences, since the latter would require more storage space. In addition, by storing the coordinates the ground truth of the motion is preserved. More details on the amount of data generated in the QTC domain are discussed in the following section. 13

30 3.2. PREPROCESSING Preprocessing This section will go over all preprocessing steps that are used in order to prepare the input for the heuristic of Chapter 5. Furthermore, a distinction is made based on the domain were the preprocessing step is used. Therefore, Section will handle all the preprocessing steps that are used in the coordinate domain, whereas Section discusses preprocessing in the QTC domain. To indicate the importance of preprocessing, Section gives some preliminary insights on the data that is generated in the QTC domain Preliminary Insights The amount of data produced in the QTC domain is huge. For example, consider a football game where all 23 MPOs (2 times 11 players and the ball) are sampled at 10Hz. Since each game has a minimum duration of 90 minutes, in each game approximately = 54k samples are generated for only one QTC sequence. The number of QTC sequences in a dataset of 23 MPOs is equal to = 506, resulting into a total of 54k million samples per game. Compared to the coordinate domain only having 1.24 million samples (90min 60s 10samples 23 MP Os) that is an increase min s of about 2200 %. Hence, it is clear that preprocessing in the QTC domain will have a crucial role in dealing with the complexity of the problem. However, attentive readers may have noticed that by using the property introduced in Section 2.1, that stated that the QTC sequence of MPOs b and a can be obtained by switching the elements of the QTC tuples in the QTC sequence of MPOs a and b, only half of them actually need to be calculated. Mathematically, this number of QTC sequences can be expressed as: s(n) = ( ) n = 2 n(n 1) 2 (3.1) In se, this equation is very simple. The number of QTC sequences is equal to the amount of binary relationships that exists between the MPOs. Mathematically this is described as the combinatorial problem of selecting all pairs of two in a set of n unique numbers, MPOs in our case, wherein the order of the selection does not matter. Thus, selecting the binary relationship between MPOs a and b and afterwards the binary relationship between MPOs b and a, is considered one selection.

31 CHAPTER 3. DATA AND PREPROCESSING 15 In Figure 3.1, an overview of the number of QTC relations in case of n MPOs is given. The grey cells correspond to redundant relations, either because of symmetry or because of the fact that the QTC relation between identical MPOs is trivial. The white cells on the other hand, correspond to actual QTC relations. Observe that the (fictive) string representations clearly have the advantage of being able to represent the whole sequence in an evident and compact way. Also note that the number of white cells in the matrix is indeed given by equation 3.1. MPO MPO n 1 AABBBB... KKKKKL... LLMMUU... VWWUVV... 2 CCCAAB... KKLLMM... VVVUUW... 3 BBBBBB... LLMMWW WWVVVV... n Figure 3.1: QTC matrix Coordinate Domain Preprocessing in the coordinate domain is fairly simple. First of all, a moving average is applied on the coordinates generated by each MPO. More specially, the sequences of x and y coordinates are processed separately. In each sequence the value at position i is replaced by the average of its n successive neighboring values, where n denotes the window that is used in the moving average. This technique is used because it is able to get rid of outliers, if present. Furthermore, by using a moving average on both x and y coordinates, a smoother, more realistic approximation of the trajectory that was sampled from is obtained, as illustrated in Figure 3.2a. Depending on the type of dataset and