1.0. Vinod Kone Anand Mazumdar

Transcription

1 1.0 Vinod Kone Anand Mazumdar

2 MESOS Released on July 27th 2016 Biggest release ever! Released on Aug 24th 2016 Please use this one!

3 WHAT S IN 1.0? Unified Containerizer New APIs XFS CNI GPUs Release policy Agent rename Docker Volume Dynamic weights Fine-grained AuthZ Windows

4 WHY 1.0? Versioned and documented APIs Formal release and upgrade policy

5 MESOS APIS Framework API Scheduler API Executor API Internal API Scheduler Master Agent Executor Master API Operator API Agent API Operator Operator

6 API V1 GOALS Work inside firewalls / containers Allow pure language client libraries API versioning Backwards compatible API evolution Well documented

7 PROTOCOL HTTP RPC based Based on Calls, Responses and Events Client opens connections to the master/agent One (or more) connection(s) to send Calls A Call might get a Response, stream of Events, or no response

8 FRAMEWORK PROTOCOL Every call is a HTTP POST request Content-Type: application/json or application/x-protobuf Successful SUBSCRIBE call results in a 200 OK streaming response Record-IO formatted events Persistent connection Mesos-Stream-Id header (schedulers only for now) All successful non-subscribe calls result in 202 Accepted Must include Mesos-Stream-Id" header (schedulers only for now)

9 WORKFLOW SUBSCRIBE Scheduler Master

10 WORKFLOW SUBSCRIBE Scheduler Master POST /api/v1/scheduler HTTP/1.1 { "type" : "SUBSCRIBE", "subscribe" : { "framework_info" : { "user" : "foo", "name" : "Example HTTP Framework"

11 WORKFLOW SUBSCRIBE Scheduler Master Streaming Response HTTP/ OK Content-Type: application/json Transfer-Encoding: chunked Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af <event length>{ "type": "SUBSCRIBED", "subscribed": { "framework_id": { "value": " ", "heartbeat_interval_seconds": 15 <more events> <more events>

12 WORKFLOW Scheduler Master Offers Event <event-length> { "type" : "OFFERS", "offers" : [ { "offer_id":{"value": " O235235", "framework_id":{"value": " ", "agent_id":{"value": " S23523", "hostname":"agent.host", "resources":[...], "attributes":[...], "executor_ids":[] ] Agent

13 WORKFLOW ACCEPT Scheduler Master POST /api/v1/scheduler HTTP/1.1 Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af { "framework_id" : {"value" : " ", "type" : "ACCEPT", "accept" : { "offer_ids" : [ {"value" : " O12", {"value" : " O12" ], "operations" : [ {"type" : "LAUNCH", "launch" : {... ], "filters" : {... Agent

14 WORKFLOW ACCEPT Scheduler Master Response HTTP/ Accepted Agent

15 WORKFLOW ACCEPT Scheduler Master Launch Task Agent

16 WORKFLOW ACCEPT Scheduler Master Launch Task SUBSCRIBE Executor Agent

17 WORKFLOW Scheduler Master SUBSCRIBE Executor Agent POST /api/v1/executor HTTP/1.1 { "type": "SUBSCRIBE", "executor_id": { "value": "387aa966-8fc a794-5a868a60d3eb", "framework_id": { "value": "49154f1b-8cf bf13-8bd11dccd1f1"

18 WORKFLOW Scheduler Master SUBSCRIBE Executor Agent HTTP/ OK Streaming Response Content-Type: application/json Transfer-Encoding: chunked <event length> { "type" "subscribed" <more events> : "SUBSCRIBED", : {

19 WORKFLOW Scheduler Master Executor Agent Launch Event HTTP/ OK Content-Type: application/json Transfer-Encoding: chunked <event-length> { "type": "LAUNCH", "launch": ( ) <more events>

20 SCHEDULER DISCONNECTIONS & PARTITIONS Master tracks the persistent subscription connection Reconnect within framework failover timeout Periodic HEARTBEAT events sent by master

21 OPERATOR V1 API Similar to Framework API Some Calls get Responses Streaming support! Subscribe to events happening on the cluster e.g., SUBSCRIBED, TASK_ADDED, TASK_UPDATED SUBSCRIBED includes the snapshot of the state

22 WORKFLOW GET_VERSION Operator Master POST /api/v1/ HTTP/1.1 Host: masterhost:5050 Content-Type: application/json { type : GET_VERSION

23 WORKFLOW GET_VERSION Operator Master Response HTTP/ OK { type : GET_VERSION, get_version ¹ : { version_info : { version 1.0.1

24 WORKFLOW SUBSCRIBE Operator Master POST /api/v1/ HTTP/1.1 Host: masterhost:5050 Content-Type: application/json { type : SUBSCRIBE

25 WORKFLOW SUBSCRIBE Operator Master Streaming Response HTTP/ OK Content-Type: application/json Transfer-Encoding: chunked <event length>{ "type" : SUBSCRIBED, subscribed" : { get_state" : { get_tasks : { get_executors : { get_frameworks : { get_agents : { <more events>

26 WORKFLOW Operator Master TASK_ADDED Event <event-length> { "type": "TASK_ADDED", "task_added": { "task": { "name": "dummy-task", "task_id": {"value": "12344-my-task", "agent_id": {"value": S0, <more events>

27 WORKFLOW Operator Master TASK_UPDATED Event <event length>{ "type" : TASK_UPDATED, task_updated" : { "task_id" : { "value" : my-task", framework_id" : { "value" : ", "agent_id" : { "value" : " S0", "state" : "TASK_RUNNING", <more events>

28 API STATUS Scheduler API & Executor API : Stable Operator API : Experimental Client library implementations api-client-libraries/

29

30 RELEASE POLICY New release every 2 months Supported for 6 months Patch releases every month Release branches to ease back ports

31 UPGRADE POLICY Upgrade from any minor version to any minor version! 1.2 > 1.8 (no need to go through 1.3 to 1.7) Caveat: Doesn t apply to deprecated features Deprecation period for a feature is 6 months

32 WHAT S NEXT? HTTP Internal API Master Agent

33 WHAT S NEXT? /api/v1/scheduler /api/v1/executor /api/v1/agent Scheduler Master Agent Executor /api/v1 /api/v1 Operator Operator

34 UNIFIED API /api/v2 /api/v2 /api/v2 Scheduler Master Agent Executor /api/v2 /api/v2 Operator Operator

35 PROTOCOL grpc Auto-generated client libs! HTTP/2 Single bi-directional connection!

36 THANK YOU