Following the Packets: A Walk Through Bro s Internal Processing Pipeline

Transcription

1 Following the Packets: A Walk Through Bro s Internal Processing Pipeline Robin Sommer robin@icir.org Corelight, Inc. International Computer Science Institute Lawrence Berkeley National Laboratory

2 Outline Bro s Architecture & Data Flow Components Protocol & file analysis Log writer & input readers Bro Plugins

3 Bro Architecture Script Interpreter s Network Packets

4 Bro Architecture Script Interpreter s Packet Source Network Packets

5 Bro Architecture Script Interpreter s I/O Loop Packet Source Network Packets

6 Bro Architecture Script Interpreter s I/O Loop Session Table Packet Source Network Packets

7 Bro Architecture Script Interpreter s Connection I/O Loop Session Table Packet Source Network Packets

8 Bro Architecture Script Interpreter s Protocol Connection I/O Loop Session Table Packet Source Network Packets

9 Bro Architecture Script Interpreter s File Protocol Connection I/O Loop Session Table Packet Source Network Packets

10 Bro Architecture Script Interpreter s File Protocol Connection Signature I/O Loop Session Table Packet Source Network Packets

11 Bro Architecture Script Interpreter s File Protocol Connection Signature I/O Loop Session Table Packet Source Network Packets

12 Bro Architecture Script Interpreter s Timer File Protocol Connection Signature I/O Loop Session Table Packet Source Network Packets

13 Bro Architecture Script Interpreter Handlers s Timer File Protocol Connection Signature I/O Loop Session Table Packet Source Network Packets

14 Bro Architecture Script Interpreter Modules Functions Expressions Statements Values Types Handlers s Timer File Protocol Connection Signature I/O Loop Session Table Packet Source Network Packets

15 Bro Architecture Script Interpreter Modules Functions Expressions Statements Values Types Handlers BiF Elements Types Constants prototypes Functions s Log Manager Timer File Protocol Connection Signature I/O Loop Session Table Packet Source Network Packets

16 Bro Architecture Script Interpreter Modules Functions Expressions Statements Values Types Handlers BiF Elements Types Constants prototypes Functions s Remote- Serializer Log Manager Timer File Protocol Connection Signature I/O Loop Session Table Packet Source Network Packets

17 Bro Architecture Script Interpreter Modules Functions Expressions Statements Values Types Handlers BiF Elements Types Constants prototypes Functions s Communic. Process Remote- Serializer Log Manager Timer File Protocol Connection Signature I/O Loop Session Table Packet Source Network Packets

18 Bro Architecture Script Interpreter Modules Functions Expressions Statements Values Types Handlers BiF Elements Types Constants prototypes Functions s Communic. Process Remote- Serializer Log Manager Input Manager Timer File Protocol Connection Signature I/O Loop Session Table Packet Source Network Packets

19 Bro Architecture Script Interpreter Network Packet Source I/O Loop Session Table s Packets Connection Protocol File Signature Timer Log Manager Input Manager prototypes Functions Types Constants Functions Modules Statements Expressions Types Values Handlers BiF Elements Remote- Serializer Thread Manager Communic. Process

20 Bro Architecture Script Interpreter Network Packet Source I/O Loop Session Table s Packets Connection Protocol File Signature Timer Log Manager Input Manager prototypes Functions Types Constants Functions Modules Statements Expressions Types Values Handlers BiF Elements Remote- Serializer Thread Manager Communic. Process

21 Protocol & File Example: SSL Session IP

22 Protocol & File Example: SSL Session IP TCP connection_established()

23 Protocol & File Example: SSL Session IP TCP SSL connection_established() ssl_{client,server}_hello()

24 Protocol & File Example: SSL Session IP TCP SSL X.509 connection_established() ssl_{client,server}_hello() x509_certificate()

25 Protocol & File Example: SSL Session? IP TCP SSL X.509 connection_established() ssl_{client,server}_hello() x509_certificate()

26 Dynamic Protocol Detection IP TCP

27 Dynamic Protocol Detection IP TCP PIA Buffer

28 Dynamic Protocol Detection Analyzer::register_for_port(Analyzer::SSL, 443/tcp); IP TCP PIA Buffer

29 Dynamic Protocol Detection signature dpd_ssl_server { ip-proto == tcp payload /^(\x16\x03[\x00\x01\x02\x03[ ].*/ tcp-state responder enable "ssl" } Analyzer::register_for_port(Analyzer::SSL, 443/tcp); IP TCP PIA Buffer

30 Dynamic Protocol Detection signature dpd_ssl_server { ip-proto == tcp payload /^(\x16\x03[\x00\x01\x02\x03[ ].*/ tcp-state responder enable "ssl" } Analyzer::register_for_port(Analyzer::SSL, 443/tcp); IP TCP PIA Buffer SSL

31 Dynamic Protocol Detection signature dpd_ssl_server { ip-proto == tcp payload /^(\x16\x03[\x00\x01\x02\x03[ ].*/ tcp-state responder enable "ssl" } Analyzer::register_for_port(Analyzer::SSL, 443/tcp); IP TCP PIA Buffer SSL X.509

32 Dynamic Protocol Detection signature dpd_ssl_server { ip-proto == tcp payload /^(\x16\x03[\x00\x01\x02\x03[ ].*/ tcp-state responder enable "ssl" } Analyzer::register_for_port(Analyzer::SSL, 443/tcp); IP TCP PIA Buffer SSL HTTP X.509

33 Dynamic Protocol Detection signature dpd_ssl_server { ip-proto == tcp payload /^(\x16\x03[\x00\x01\x02\x03[ ].*/ tcp-state responder enable "ssl" } Analyzer::register_for_port(Analyzer::SSL, 443/tcp); IP TCP PIA Buffer SSL HTTP X.509

34 Protocol Analyzer API class Analyzer { virtual void Init(); virtual void Done(); virtual void DeliverPacket(int len, const u_char* data, bool orig, bool orig, uint64 seq, const IP_Hdr* ip, int caplen); virtual void DeliverStream(int len, const u_char* data, bool orig); virtual void Undelivered(uint64 seq, int len, bool orig); virtual void EndOfData(bool is_orig); virtual void FlipRoles(); } class TCP_ApplicationAnalyzer : public Analyzer { virtual void EndpointEOF(bool is_orig); virtual void ConnectionFinished(int half_finished); virtual void ConnectionReset(); };

35 File Analyzer API class Analyzer { virtual void Init(); virtual void Done(); virtual bool DeliverChunk(const u_char* data, uint64 len, uint64 offset); virtual bool DeliverStream(const u_char* data, uint64 len); virtual bool EndOfFile(); virtual bool Undelivered(uint64 offset, uint64 len); };

36 Bro Architecture Script Interpreter Network Packet Source I/O Loop Session Table s Packets Connection Protocol File Signature Timer Log Manager Input Manager prototypes Functions Types Constants Functions Modules Statements Expressions Types Values Handlers BiF Elements Remote- Serializer Thread Manager Communic. Process

37 Bro Architecture Script Interpreter Network Packet Source I/O Loop Session Table s Packets Connection Protocol File Signature Timer Log Manager Input Manager prototypes Functions Types Constants Functions Modules Statements Expressions Types Values Handlers BiF Elements Remote- Serializer Thread Manager Communic. Process

38 Writers & Readers Log Writers Input Readers ASCII ASCII SQLite Binary Raw file SQLite

39 Log Writer API class WriterBackend { virtual bool DoInit(const WriterInfo& info, int num_fields, virtual bool DoWrite(int num_fields, const Field* const* fields, threading::value** vals); virtual bool DoSetBuf(bool enabled); virtual bool DoFlush(double network_time); virtual bool DoRotate(const char* rotated_path, double open, double close, bool terminating); virtual bool DoFinish(double network_time); virtual bool DoHeartbeat(double network_time, double current_time); }; Each writer runs in its own thread.

40 Input Reader API class ReaderBackend { virtual bool DoInit(const ReaderInfo& info, int arg_num_fields, const threading::field* const* fields); virtual void DoClose(); virtual bool DoUpdate(); virtual bool DoHeartbeat(double network_time, double current_time); // Simple mode. void Send(const char* name, const int num_vals, threading::value* *vals); void Put(threading::Value** val); void Delete(threading::Value** val); void Clear(); void EndOfData(); // Tracking mode. void SendEntry(threading::Value** vals); void EndCurrentSend(); }; Each reader runs in its own thread.

41 Bro Plugins Build & install Bro components independently Distribute as a Bro package Log writers Input readers Protocol analyzers File analyzers Packet Sources BiF elements Bro Scripts

42 BYOP # ~/bro/aux/bro-aux/plugin-support/init-plugin icsi-plugin ICSI BroMagic Installing icsi-plugin/changes... Installing icsi-plugin/cmakelists.txt... Installing icsi-plugin/configure... Installing icsi-plugin/configure.plugin... Installing icsi-plugin/scripts/ load.bro... Installing icsi-plugin/scripts/icsi/bromagic/ load.bro... Installing icsi-plugin/scripts/init.bro... Installing icsi-plugin/src/bromagic.bif... Installing icsi-plugin/src/plugin.h... Installing icsi-plugin/src/plugin.cc [ ] # cd icsi-plugin/ #./configure --brodist=$home/bro/master Build Directory : build Bro Source Directory : /home/robin/bro/master [ ] # make && make install [ ] # bro -N ICSI::BroMagic - <Insert description> (dynamic, version 0.1) Bro::ARP - ARP Parsing (built-in) Bro::AsciiReader - ASCII input reader (built-in) Bro::AsciiWriter - ASCII log writer (built-in) Bro::AYIYA - AYIYA Analyzer (built-in) [ ]

43 Bro Architecture Script Interpreter Network Packet Source I/O Loop Session Table s Packets Connection Protocol File Signature Timer Log Manager Input Manager prototypes Functions Types Constants Functions Modules Statements Expressions Types Values Handlers BiF Elements Remote- Serializer Thread Manager Communic. Process

44 Script Interpreter s Questions? Network Packets