CTI-TC Weekly Working Sessions

Transcription

1 CTI-TC Weekly Working Sessions Meeting Date: February 21, 2017 Time: 15:00:00 UTC Purpose: Weekly CTI TC Joint Working Session Attendees: Jordan - Moderator Darley Thomson Eric Burger Taylor Jon Baker Davidson Ivan Kirillov Wunder Greg Back Shawn McCullough Paul Patrick Rich Piazza Sarah Kelly - Gurney Alex T Masato Jane Ginn - Recorder Agenda: TAXII Update STIX Updates i18n proposal Meeting Notes TAXII 2.0 Discussion Here is the link ZFH7m8R0zr08n1WD_ZSNXV0k2Z2Ebf8g/edit#heading=h.a0iri07jww47 Today we will talk about Filtering & Pagination Pagination With Pagination Did use offset fields After discussion, we decided range field Get response back Records returned We discovered when looking through Nice to know how many records you have Does anyone have any comments? I don t care whether we use range, as long as range can support Timestamps I read in the proposal that is would be supportable One way we ve talked about doing that is for it to be a Filter Server computes how many then matches Client can paginate through There are two separate considerations One Select the result set that you want Added After parameter Two- Need to be able to page through the result set, do you want to limit or page through? I think more, the later. [Gave example of how analyst would use it] Each individual page has to be Atomic Numbers can change as the client is paging through Yea, they should all be Stateless Based on your description I think that would work [Walked through text and described how that would work matching s example]

2 In that response where Server updated the number, does that actually become 300 in your example Explained further Each STIX Object is considered an item One of the things you have in 2 nd request Server sends back additional data Client has to perform some arithmetic on numbers in range and then Make another request Had a question about the syntax from HTTP Yes, I ll put a reference in the TAXII Spec [Repeated back the example] A more responsive TIP would check every hour, not every 24 hours It depends on the data feed This is only because you have asynchronous updates. If data is frequent, we need to be polling You want the server to be polling more frequently We need to have an ability to determine what is new since last time Server updated As long as the response to the GET request is in the same order Would retain result set With a limit offset condition I have some questions One, in TAXII 2.0, are we going to support PUSH? You post to a collection HTTP doesn t do Server PUSH The way you would handle that in the implementation is long polling or web sockets That is the biggest drawback about using HTTP is that Server pushing is not supporting There are issues with using Web Sockets We can always do that in 2.1 or 2.2 My other question For every response from the Server, it will give you the new Total? Yes Is it required that it be given in the same order? I think we need to make that an element of the solution [Gave an example of how it would not work if not in order] It would make sense to specify what the order should be [gave example] That makes sense If the Server does not implement Pagination, then the ordering should be Date_Added_Time Page 2

3 Is the most reasonable choice Once you have those two things in place, you can do a lot of other things [Gave another example] Gave example of keeping it open Don t have a different call. If result set is constant, then don t have to worry about missing other data. Is this group OK with using this? What happens if the Server has limited space and the number of items shift? And Server had to delete some of the items to delete to make space for the new set Server would not know because it is Stateless If the items had shifted there might be corner cases where data were missed You could make this request at any point in time The way we have written in the Spec it does not require but, with an implementation It could be done in a certain way [Talked about data becoming stale] Gave example of using UUID and Timestamp Then pass back to the Client One thing you could do is the Server could return an X-Header with an end_date Then, if you had to come back and check, you could use that for starting new set TAXII wrapper I think that will work. I would suggest that these be optional Let s add as an option I would object to making them Optional 50% would/50% wouldn t Either make them required, or not Do we need both ways of doing Pagination? Do we need the data range? There were people that needed the numbers. I m trying to honor their request Explained how the values explained what was actually sent Gave example of building a temporary table Could someone succinctly restate the problem Restated [Gave example of how sets and subsets are transmitted] Really, they are just helpers for the Client I still don t see the need for both but, it is just an extra header, so I won t object What started this discussion was the idea that the Server didn t have space Page 3

4 Gave example of an object that had been revoked. Asked a question About whether it is on the Server Just one in a series [Gave example of how that revoked object might mess up the numbering] So we think that the HTTP method might work for us [Summarized how the GET request would work to force pagination] [Summarized how the x-headers would work for start and end of request] We also said we are going to sort the responses by day_added Content range total number can change if new data is added to the data set During the request Requesting by ranges and item_count Are these two different requests? If you took that and made a subsequent request, the content range would be different You are requesting a different slice of data I think the end_date is something the Client could use for an implementation If you had a TAXII wrapper like in 1.1, it would be in there but, here, by putting it in Header, it will give you the information you need If you have any other thoughts, please add before Friday Are there any objections or reservations on the call about this approach? Heard none Filtering We had in the Specification document an approach to filtering Showed mark-up [Summarized the feedback that TAXII SC had received so far] This is roughly equivalent to using separate parameters We need to look at s approach I d like to hear more about that [Put up s Slides] when through slides At Scale Filtering will be a problem if not well developed Why not have Grammar be the way we implement Filtering? Referred to syntax for STIX Patterning Page 4

5 Most implementers that are going to support STIX 2.0 will need to implement Patterning Therefore, it makes sense to use STIX Patterning for Filtering We need to be able to support flexible approaches for filtering Is the Proposal for TAXII 2.0 just changing the Syntax to make it consistent with STIX Patterning? You are exactly right. I have that on Slide 6. We could restrict to these terms in 2.0 In the future, we could support more. Alan I like a lot of this but, for some, like, request_by_id might not be needed Could we just have simple requests You could, but I would prefer to have it just one way You may make it more complicated if you have 2 ways of doing things? Page 5

6 I like your approach I m just concerned about having TAXII Server out of the gate? Alan How many people will be supporting a TAXII Server, but not STIX [Asked a question about the Patterning grammar] Why would we have two different grammars for STIX? Although it had its genesis in the Cyber Observable context, but we are calling It STIX Patterning to extend to the entire ecosystem Gave example of a Patterning approach that would not work in this context I m going to have to support the Patterning grammar.. So, what I m looking at is That I want to reduce load If we do this, we should copy the syntax that does make sense over, and Then clarify in the notational text I m looking through the examples mostly Observables and Indicators I just said that it is to be extensible What we are really talking about is bringing over the Boolean logic of Patterning That is what makes sense to me I would be comfortable with making it the same syntax To me, what we are talking about is Spec-man-ship We define the grammar to support the whole ecosystem That is a step in the right direction Ivan This is all about abstracting Patterning into its base components Maybe we should delay this do it in the Patterning Spec first But we are trying to wrap-up TAXII if we are trying to fix this in TAXII 2.0 If we delay and keep Filtering, then we have a mess We are 3 minutes over time. Let s put a PIN in the Patterning discussion now. We have a Friday release candidate We should meet again Thursday. 7:00 PST 10:00 EST? All OK, that works Meeting Terminated ***************************************************************** Page 6