Research Overview. Provenance Themes. Dr. Martin Chapman March 17, King s College London

Transcription

1 Research Overview Provenance Themes Dr. Martin Chapman March 17, 2017 King s College London martin.chapman@kcl.ac.uk 1

2 Overview Learning the language of error Playing Hide-And-Seek Storing and processing MT300s 2

3 Learning the language of error

4 Problem Problem: One of the assertions in a program fails, based upon given input, and you want to know how the sequence of method calls in your program might have had an impact on this. static void gl_read () { do { gl_insert ( nondet_int ()); } while ( nondet_int ()); } static void gl_insert ( int value ) { struct node * node = malloc ( sizeof * node ); node -> value = value ; list_add (& node -> linkage, & gl_list ); INIT_LIST_HEAD (& node -> nested ); } static inline void list_add ( struct list_head *new, struct list_head *prev, struct list_head * next ) { next -> prev = new ; new -> next = next ; new -> prev = prev ; prev -> next = new ; } static void inspect ( const struct list_head * head ) { head = head -> prev ; const struct node * node = list_entry (head, struct node, linkage ); assert (node -> nested. prev == &node -> nested ); for ( head = head -> next ; &node -> linkage!= head ; head = head -> next ); } static inline void list_del ( struct list_head *prev, struct list_head * next ) { next -> prev = prev ; prev -> next = next ; } static inline void list_add ( struct list_head *new, struct list_head * head ) { list_add (new, head, head -> next ); } int main () { gl_read (); inspect (& gl_list ); } 3

5 Problem Problem: One of the assertions in a program fails, based upon given input, and you want to know how the sequence of method calls in your program might have had an impact on this. static void gl_read () { do { gl_insert ( nondet_int ()); } while ( nondet_int ()); } static void gl_insert ( int value ) { struct node * node = malloc ( sizeof * node ); node -> value = value ; list_add (& node -> linkage, & gl_list ); INIT_LIST_HEAD (& node -> nested ); } static inline void list_add ( struct list_head *new, struct list_head *prev, struct list_head * next ) { next -> prev = new ; new -> next = next ; new -> prev = prev ; prev -> next = new ; } static void inspect ( const struct list_head * head ) { head = head -> prev ; const struct node * node = list_entry (head, struct node, linkage ); assert (node -> nested. prev == &node -> nested ); for ( head = head -> next ; &node -> linkage!= head ; head = head -> next ); } static inline void list_del ( struct list_head *prev, struct list_head * next ) { next -> prev = prev ; prev -> next = next ; } static inline void list_add ( struct list_head *new, struct list_head * head ) { list_add (new, head, head -> next ); } int main () { gl_read (); inspect (& gl_list ); } 3

6 Problem Problem: One of the assertions in a program fails, based upon given input, and you want to know how the sequence of method calls in your program might have had an impact on this. static void gl_read () { do { gl_insert ( nondet_int ()); } while ( nondet_int ()); } static void gl_insert ( int value ) { struct node * node = malloc ( sizeof * node ); node -> value = value ; list_add (& node -> linkage, & gl_list ); INIT_LIST_HEAD (& node -> nested ); } static inline void list_add ( struct list_head *new, struct list_head *prev, struct list_head * next ) { next -> prev = new ; new -> next = next ; new -> prev = prev ; prev -> next = new ; } static void inspect ( const struct list_head * head ) { head = head -> prev ; const struct node * node = list_entry (head, struct node, linkage ); assert (node -> nested. prev == &node -> nested ); for ( head = head -> next ; &node -> linkage!= head ; head = head -> next ); } static inline void list_del ( struct list_head *prev, struct list_head * next ) { next -> prev = prev ; prev -> next = next ; } static inline void list_add ( struct list_head *new, struct list_head * head ) { list_add (new, head, head -> next ); } int main () { gl_read (); inspect (& gl_list ); } Analysing large amounts of code to understand this can be difficult. 3

7 Solution: Learning the language of error Proposed solution: Summarise all the paths that lead to a failing program assertion as a DFA [Chapman et al., 2015]. 4

8 Solution: Learning the language of error Proposed solution: Summarise all the paths that lead to a failing program assertion as a DFA [Chapman et al., 2015]. Learn Assert D inspect list add gl read gl insert gl insert list add 4

9 Solution: Learning the language of error Proposed solution: Summarise all the paths that lead to a failing program assertion as a DFA [Chapman et al., 2015]. Learn Assert D inspect list add gl read gl insert gl insert list add Much easier to analyse. Provides an overview of program behaviours, some of which may be unexpected. 4

10 Solution: Learning the language of error Proposed solution: Summarise all the paths that lead to a failing program assertion as a DFA [Chapman et al., 2015]. Learn Assert D gl read inspect gl insert gl insert list add list add Did we really want this method call loop in our program? Much easier to analyse. Provides an overview of program behaviours, some of which may be unexpected. 4

11 Implementation Paths are formed from software counterexamples (method calls that lead to a failing assertion in a program). 5

12 Implementation Paths are formed from software counterexamples (method calls that lead to a failing assertion in a program). Our software learns these counterexamples via the L* algorithm [Angluin, 1987] (where the oracle is a model checker). 5

13 Implementation Paths are formed from software counterexamples (method calls that lead to a failing assertion in a program). Our software learns these counterexamples via the L* algorithm [Angluin, 1987] (where the oracle is a model checker). Membership queries pertain to individual counterexamples, while conjecture queries pertain to full automata. 5

14 Implementation Paths are formed from software counterexamples (method calls that lead to a failing assertion in a program). Our software learns these counterexamples via the L* algorithm [Angluin, 1987] (where the oracle is a model checker). Membership queries pertain to individual counterexamples, while conjecture queries pertain to full automata. Supported by a Google faculty research award. 5

15 Case study: Automatic merging Unexpected behaviours are particularly prevalent when code is automatically merged: main {... functiona(); functionb();...} main {... functiona();... functionz();} functionz() { functionb();} main {... functiona(); functionb(); functionc();... } main {... functiona();... functionz(); } functionz() { functionb(); functionc();...} (a) Source (b) Branch A (c) Branch B (d) Merged 6

16 Capitalising on automata representation (1) Our software uses an automaton representation to draw the developer s attention to the changes introduced by the merge. 7

17 Capitalising on automata representation (1) Our software uses an automaton representation to draw the developer s attention to the changes introduced by the merge. First we generate three automata: Branch A B1 Merged Code P Branch B B2 Automaton A1 Automaton AMerged Automaton A2 7

18 Capitalising on automata representation (1) Our software uses an automaton representation to draw the developer s attention to the changes introduced by the merge. First we generate three automata: Branch A B1 Merged Code P Branch B B2 Automaton A1 Automaton AMerged Automaton A2 We then compute the following: AMerged \ A1 and AMerged \ A2 in order to show the new behaviours. 7

19 Capitalising on automata representation (2) Z C Learn assert D Figure 1: AMerged \ A1 or behavior not in Branch A Z B C Learn assert D Figure 2: AMerged \ A2 or behavior not in Branch B a a Subtracting the union of A1 and A2 (common behaviour) would also allow us to summarise all the new behaviour introduced by the merge. 8

20 Capitalising on automata representation (3) Why an automaton? 9

21 Capitalising on automata representation (3) Why an automaton? 1. Processing the source code directly in order to achieve a similar representation is likely to be inefficient (operations on automata are well established). 9

22 Capitalising on automata representation (3) Why an automaton? 1. Processing the source code directly in order to achieve a similar representation is likely to be inefficient (operations on automata are well established). 2. The automata representation is highly intelligible. 9

23 Playing Hide-And-Seek

24 Overview (1) Problem: Network attacks are becoming more frequent. 10

25 Overview (1) Problem: Network attacks are becoming more frequent. Potential solution: Construct formal decision making models (e.g. game theoretic frameworks) that capture network security scenarios in order to aid automated response (in respect of automatic processing and solution, etc.). 10

26 Overview (1) Problem: Network attacks are becoming more frequent. Potential solution: Construct formal decision making models (e.g. game theoretic frameworks) that capture network security scenarios in order to aid automated response (in respect of automatic processing and solution, etc.). An interesting class of network security models: network security games (NSGs). Typically consider the interactions between an attacker and a defender. 10

27 Overview (1) Problem: Network attacks are becoming more frequent. Potential solution: Construct formal decision making models (e.g. game theoretic frameworks) that capture network security scenarios in order to aid automated response (in respect of automatic processing and solution, etc.). An interesting class of network security models: network security games (NSGs). Typically consider the interactions between an attacker and a defender. A common approach to deriving an NSG model is to apply existing types of games to unexplored network security problems. 10

28 Overview (2) Unexplored network security problem: Multiple node attacks (e.g. botnets and attack pivots). 11

29 Overview (2) Unexplored network security problem: Multiple node attacks (e.g. botnets and attack pivots). How do we link multiple node attacks to an existing type of game? 11

30 Overview (2) Unexplored network security problem: Multiple node attacks (e.g. botnets and attack pivots). How do we link multiple node attacks to an existing type of game? The link: Multiple node attacks exhibit the two-sided search problem (looking for something that does not want to be found; the bots in a botnet (perspective of defender), or hidden, sensitive resources (perspective of attacker)) with multiple hidden entities. 11

31 Overview (3) Search games are designed to model and investigate the two-sided search problem, as interactions between a hider and a seeker. 12

32 Overview (3) Search games are designed to model and investigate the two-sided search problem, as interactions between a hider and a seeker. Hide-and-seek games, a subset of search games, are designed to do this for multiple hidden objects. 12

33 Overview (3) Search games are designed to model and investigate the two-sided search problem, as interactions between a hider and a seeker. Hide-and-seek games, a subset of search games, are designed to do this for multiple hidden objects. Initial proposal: It is logical to study hide-and-seek games in order to study multiple node attacks [Chapman et al., 2014]. The hider is the defender, and the seeker is the attacker, or vice-versa. 12

34 Hide-And-Seek Games Different permutations on same basic model. The permutation of interest to us: Two competing players; the hider and the seeker A search space; for our purposes, a network graph Hidden objects to be concealed on the network Some cost to seeker for undertaking a search; the hider is rewarded in an inverse amount. Different strategies are explored for both the hider and the seeker. 13

35 Hide-And-Seek Games Different permutations on same basic model. The permutation of interest to us: Two competing players; the hider and the seeker A search space; for our purposes, a network graph Hidden objects to be concealed on the network Some cost to seeker for undertaking a search; the hider is rewarded in an inverse amount. Different strategies are explored for both the hider and the seeker. This model is simple, but already promising in what it can capture from a multiple node attack. 13

36 Hide-And-Seek Games Different permutations on same basic model. The permutation of interest to us: Two competing players; the hider and the seeker A search space; for our purposes, a network graph Hidden objects to be concealed on the network Some cost to seeker for undertaking a search; the hider is rewarded in an inverse amount. Different strategies are explored for both the hider and the seeker. This model is simple, but already promising in what it can capture from a multiple node attack. Richer variants to the model are natural, why aren t they explored? Complexity. 13

37 Methodology We increase the richness of the model, and thus what it can capture of the security domain (e.g. timesteps, repeated interactions). We compensate for any increase in complexity by using an Empirical Game Theoretical Analysis (EGTA) approach to estimate the payoff values associated with different strategies by realising computational representations of them, and evaluating their performance in simulation. 14

38 Methodology We increase the richness of the model, and thus what it can capture of the security domain (e.g. timesteps, repeated interactions). We compensate for any increase in complexity by using an Empirical Game Theoretical Analysis (EGTA) approach to estimate the payoff values associated with different strategies by realising computational representations of them, and evaluating their performance in simulation. Also allowed us to contribute a computational platform, which can be used as the basis for Distributed Research Games (more at cyberhands.co.uk) 14

39 Methodology We increase the richness of the model, and thus what it can capture of the security domain (e.g. timesteps, repeated interactions). We compensate for any increase in complexity by using an Empirical Game Theoretical Analysis (EGTA) approach to estimate the payoff values associated with different strategies by realising computational representations of them, and evaluating their performance in simulation. Also allowed us to contribute a computational platform, which can be used as the basis for Distributed Research Games (more at cyberhands.co.uk) The performance of different strategies provides the basis for heuristics that can be applied to real security applications. 14

40 Results (1) Multiple interaction game: the same attacker (hider) and defender (seeker) meet each other multiple times. 15

41 Results (1) Multiple interaction game: the same attacker (hider) and defender (seeker) meet each other multiple times. Natural for the defender to keep data on the actions of an attacker, to help plan future strategies, by observing how the attacker interacts with the environment (e.g. where objects are hidden). 15

42 Results (1) Multiple interaction game: the same attacker (hider) and defender (seeker) meet each other multiple times. Natural for the defender to keep data on the actions of an attacker, to help plan future strategies, by observing how the attacker interacts with the environment (e.g. where objects are hidden). Therefore, natural for attacker to attempt to manipulate this data (i.e. switch the source of the data from the environment to themselves). 15

43 Results (1) Multiple interaction game: the same attacker (hider) and defender (seeker) meet each other multiple times. Natural for the defender to keep data on the actions of an attacker, to help plan future strategies, by observing how the attacker interacts with the environment (e.g. where objects are hidden). Therefore, natural for attacker to attempt to manipulate this data (i.e. switch the source of the data from the environment to themselves). Finding: deceptive strategies are not effective if the defender is sophisticated in respect of determining the source of data (i.e determining when manipulation is being attempted). 15

44 Results (2) Payoff *** hdeceptive *** hrandomset Strategy shighprobability 16

45 Storing and processing MT300s

46 Storing and processing MT300s (1) Brief: To design and build a distributed ledger POC system to process and store proprietary messages for inter- subsidiary forex transactions (MT300s) internal to a major Fortune 500 financial institution. Taking pairs of messages about forex transactions (e.g. $; $ ), and storing them on the blockchain. Distributed ledger: A generalised term for a blockchain, emphasising that it s not only currency exchanges that can be stored in this sequential, secure and replicated way. 17

47 Storing and processing MT300s (2) Why? Intermediate message processors add time and money. Cynically: become familiar with the technology that may one day supplant them. Output. A platform that: Integrates a wider range of different technologies to achieve its aim (e.g. BigChainDB, ErisDB (Permissioned chains based on Ethereum / EVM) + Tendermint)). Focuses on scalability and throughput. Research into inter-chain interaction for processing and storing data (e.g. using a separate chain to store filtered transactions). 18

48 Provenance Themes A recurring theme of both conceptual provenance, and data provenance, in my work: 1 19

49 Provenance Themes A recurring theme of both conceptual provenance, and data provenance, in my work: Learning the language of error: Understanding the functions that have had an impact on input data, using a graph based representation, and how this has lead to an error. 1 19

50 Provenance Themes A recurring theme of both conceptual provenance, and data provenance, in my work: Learning the language of error: Understanding the functions that have had an impact on input data, using a graph based representation, and how this has lead to an error. Playing Hide-And-Seek: Understanding the origin of data in order to make strategic decisions. 1 19

51 Provenance Themes A recurring theme of both conceptual provenance, and data provenance, in my work: Learning the language of error: Understanding the functions that have had an impact on input data, using a graph based representation, and how this has lead to an error. Playing Hide-And-Seek: Understanding the origin of data in order to make strategic decisions. MT300 Processing: Using a distributed ledger (blockchain) to provide a secure historic record of all the actions involving an entity 1. 1 Lots of research to be done at the intersection here! 19

52 Summary (1) Experience of, and achievements as a part of, projects that require not only good development skills, but also research capabilities. Strong programming ability. Wide range of experience working with different systems, some large in scale, or designed to be scalable. In particular systems that have required me to consider how to facilitate communication between heterogeneous entities (e.g. learn tool, HANDS platform, distributed ledger projects). Ph.D. with a focus on game theory, artificial intelligence, and elements of learning. 20

53 Summary (2) Themes of provenance, and graph-based representation, throughout work. Additional experience as a teaching academic staff member at King s: significant teaching responsibilities, in addition to administrative and pastoral responsibilities. Some system development as a part of this role. 21

54 References Angluin, D. (1987). Learning regular sets from queries and counterexamples. Information and computation, 75(2): Chapman, M., Chockler, H., Kesseli, P., Kroening, D., Strichman, O., and Tautschnig, M. (2015). Learning the language of error. In International Symposium on Automated Technology for Verification and Analysis, pages Springer. Chapman, M., Tyson, G., McBurney, P., Luck, M., and Parsons, S. (2014). Playing hide-and-seek: an abstract game for cyber security. In Proceedings of the 1st International Workshop on Agents and CyberSecurity, page 3. ACM. 22