USING OPENSTACK TO INTEGRATE NON-OPENSTACK SERVICE JUNHO YOON, ANDREW LIU, JACK NING

Transcription

1 USING OPENSTACK TO INTEGRATE NON-OPENSTACK SERVICE JUNHO YOON, ANDREW LIU, JACK NING

2 AGENDA INTRODUCTION MOTIVATIONS INTEGRATE AUTHN/AUTHZ INTEGRATE PLATFORM UI INTEGRATE PLATFORM COMMUNICATION CONTINOUS DEPLOYMENT WITH CUSTOMIZATION

3 Introduction JUNHO YOON Senior developer of NAVER ANDREW LIU Senior developer of NAVER China JACK NING Senior developer of NAVER China

4 Introduction Established in 1999, South Korea Handle more than half of internet search market in Korea Have more than 8000 employees Some apps have more than 100m users

5 Introduction Have a own IDC and a public cloud service However NOT OpenStack based

6 PASTA - IN-HOUSE PAAS projects / 800+ daily user 10+ integrated platforms so far

7

8 PASTA Architecture Today s Topic platforms platforms platforms PASTA-web Company SSO keystone Shipdock (In-house docker cluster) Users horizon cinder ceph Nova Experimental

9 ADENDA INTRODUCTION MOTIVATIONS INTEGRATE AUTHN/AUTHZ INTEGRATE PLATFORM UI INTEGRATE PLATFORM COMMUNICATION CONTINOUS DEPLOYMENT WITH CUSTOMIZATION

10 Motivations Too many platforms About40 platforms It s impossible even to remember URL No single entrance/catalog No resource utilization No common user experience Reinvent wheel

11 Motivations - authz / authn Each platforms had its own authz/authn Takes too much time for first access Has different permission set Requires even different user id/password sometimes Common problems in big company PlatformA PlatformB

12 AWS comes to our sight What does AWS provide Integrated UI/UX - consistency Organized services catalog Separated PaaS UI with the main UI Centralized user management - AWS IAM We decide to make our platforms as a PaaS like AWS

13 Component which enables PaaS PAAS INTEGRATED CONSOLE DYNAMIC RESOURCE PROVISIONING Consistent UX Integrated Authz/Authn Seamless integration b/w platforms Resource Provisioning on demand Docker Cluster? eview/221-dockerorchestration

14 Make new from scratch? Start from opensouce or commercial system? Or OpenStack

15 Composable Infrastructure Decide to adopt openstack

16 ADENDA INTRODUCTION MOTIVATIONS INTEGRATE AUTHN/AUTHZ INTEGRATE PLATFORM UI MAKE PLATFORMS INTEROPERATE PACKAGE/DEPLOY WITH CUSTOMIZATION

17 Keystone Authn/Authz in OpenStack Feature Configurable auth/identity backend Easy to extend by Adding plugin for Authz/Authn Abundant API interface OpenStack Services Keystone API Policy Backend Token Backend Catalog Backend Identity Backend Assignments Backend Credentials Backend

18 Keystone Problem we are facing: Need to integrate into our existing SSO Need to identify not logged-in user as well Want to avoid to save user s ID/PW in our DB 1) ID/PW PROJECT HORIZON 2) issue X-AUTH-TOKEN 3) connect to ENDPOINT with X-AUTH-TOKEN KEYSTONE OPENSTACK COMPONENT 4) ask the X-AUTH-TOKEN info (PROJECT ID + ROLE + USER)

19 extended Keystone v1 OAUTH2 PROVIDER 0) OAUTH Auth 1) ID/OAUTH-TOKEN or ID/PASSWORD PASTA-WEB (IN-HOUSE CONSOLE) 3) ISSUE X-AUTH-TOKEN 5) verify X-AUTH-TOKEN 4) ACCESS WITH X-AUTH-TOKEN 1.1) verify OAUTH-TOKEN KEYSTONE 2) GET IDENTITY PLATFORMS 1.2) verify PASSWORD AUTH PLUGIN LDAP IDENTITY PLUGIN USING COMPANY S SSO USING COMPANY S LDAP FOR IDENTITY IN-HOUSE LDAP

20 Auth Plugin PASSWORD AUTH AUTH PLUGIN default identity auth LOGIN Success DEFAULT AUTH Fail LOGIN Success SSO AUTH Fail SSO HTTP API /api/auth/tokeninfo to verify token LOGIN FAILED

21 Auth Plugin Keypoint is ütreat SSO token as password ütry default auth method first. If failed, use auth using SSO next üextends auth handler Keystone.auth.plugins.password.Password Keystone Configuration

22 Extended Keystone v1 OAUTH2 PROVIDER 0) OAUTH Auth 1) ID/OAUTH-TOKEN or ID/PASSWORD PASTA-WEB (IN-HOUSE CONSOLE) 3) ISSUE X-AUTH-TOKEN 5) verify X-AUTH-TOKEN 4) ACCESS WITH X-AUTH-TOKEN 1.1) verify OAUTH-TOKEN KEYSTONE 2) GET IDENTITY PLATFORMS 1.2) verify PASSWORD AUTH PLUGIN LDAP PLUGIN Problem Do not have right to save OpenStack system users in LDAP Deadly slow when retrieving all users. IN-HOUSE LDAP

23 Extended Keystone v2 OAUTH2 PROVIDER 0) OAUTH Auth 1) ID/OAUTH-TOKEN or ID/PASSWORD PASTA-WEB (IN-HOUSE CONSOLE) 3) ISSUE X-AUTH-TOKEN 5) verify X-AUTH-TOKEN 4) ACCESS WITH X-AUTH-TOKEN 1.1) verify OAUTH-TOKEN KEYSTONE 2) GET IDENTITY PLATFORMS PASTA AUTH HANDLER 1.2) verify PASSWORD HYBRID IDENTITY PLUGIN (AUTH + IDENTITY) IN-HOUSE LDAP SQL INTRODUCE HYBRID INDENTITY PLUGIN Save new user in SQL Read from only SQL when querying all users

24 Hybrid Backend Plugin About the auth part Based on keystone-hybrid-backend ü Implement LDAP Indentity ü extending SQL Indentity IDENTITY AUTH SQL Auth Success LOGIN Failed LDAP Auth Success LOGIN Failed LOGIN FAILED

25 Hybrid Backend Plugin Identity ü For API like get/update user just like the auth flow Why customzied for list large users ü LDAP USER LIST_USERS ü List all user take 10~60s in horizon ü No domain concept when adopting legacy platforms Filter by Name Yes SQL + LDAP USERS No SQL USERS Configuration

26 ADENDA INTRODUCTION MOTIVATIONS INTEGRATE PLATFORM AUTHN/AUTHZ INTEGRATE PLATFORM UI MAKE PLATFORMS INTEROPERATE PACKAGE/DEPLOY WITH CUSTOMIZATION

27 Previously our platforms. Have each own web based management console No consistent user experience Implemented using various tech set Backend : Spring/Node.js/Golang (No python ) Framework: Backbone.js/Angular/Vue.js/React/Jquery

28 Openstack - Horizon Horizon Nova Nova UI Neutron UI Cinder UI Neutron Cinder Manilla Plugin BlarBlar Plugin Manilla Blar Blar Keystone

29 Openstack - Horizon Not fit for NAVER Is not working very well with large user set Seems little bit UGLY for us Implemented with Python + Django Need to restart and test whenever some platform s UI upgraded How to evenly distribute the UI development job to each platform s developer guaranteeing consistency? Make UI independently developed without forcing to use specific tech set

30 Micro Service Architecture UI / Monolithic App Horizon Microservice Microservice Microservice Microservice Microservice Microservice Logic + Database

31 Micro Service Architecture - modified UI Integrator PASTA Microservice+UI </> Microservice+UI </> Microservice+UI Microservice+UI Microservice+UI Microservice+UI </> </> </> </> Logic + Database

32 Micro Service Architecture - modified UI integrator handles this part Each platform handles this part

33 Spring Cloud Netflix ZUUL HTTP Request pre filters routing filters post filters custom filters error filters BACKEND SERVER

34 Realized Runtime Flow OAUTH-PROVIDER 3. OAUTH 1. Prepare routing table KEYSTONE (OPENSTACK) 4. Service Permission Check & Issue X-AUTH-TOKEN 7. User permission check using X-AUTH-TOKEN 6. https//{{platform-host}}/platform-id/* X-AUTH-TOKEN 5. Decide where to route based on context path PlatformA 8. Render platform page 2. Access Pasta WEB service-id.pasta.navercorp.com/platform-id/a.txt ZUUL 8. Final HTML Rendering PlatformB PlatformC

35 Platform Info Extension Be able to keep each platform endpoint info in keystone s Service catalog and endpoints Service(Openstack Term) = Platform (PASTA Term) Use the platform name as a context path Pick internal interface URL for routing

36 Platform Info Extension Need extra room to store extra routing info Ex) Platform Icons / Display order Need separate DB to store these? Use description section with JSON

37 Service Info Extension Should store the project s extra info into keystone Project(Openstack Term) = Service (PASTA Term) OpenStack4J

38 ADENDA INTRODUCTION MOTIVATIONS INTEGRATE AUTHN/AUTHZ INTEGRATE PLATFORM UI MAKE PLATFORMS INTEROPERATE PACKAGE/DEPLOY WITH CUSTOMIZATION

39 UI Level Interoperability Already be able to call the other platform s REST API Because all platform UI share same authn/z in a user session PLATFORM-A PLATFORM-B PLATFORM-C $.get( /platform-a/api/functiona ) $.get( /platform-b/api/functionb ) $.get( /platform-c/api/functionc )

40 Backend Interoperability Need special way to communicate each other ex) Batch / Event Handler which run outside of the user session ex) Run user s platform interoperation code when event is triggered Introduce Serverless Framework (openwisk) OPENWISK PlatformA PlatformB Not cover detail here

41 ADENDA INTRODUCTION MOTIVATIONS INTEGRATE AUTHN/AUTHZ INTEGRATE PLATFORM UI MAKE PLATFORMS INTEROPERATE PACKAGE/DEPLOY WITH CUSTOMIZATION

42 PASTA- WEB PASTA OpenStack Pasta web package and delivered in traditional way How we package and deliver OpenStack?

43 Packaging/Deployment Requirement Should guarantee 7*24 service No downtime allowed Should be one-click deployment Adopt existing infrastructure (L4, MySQL) into OpenStack deployment Minimize our own customization to reduce rebase burden Kolla ü Easy to customize by adding ansible plays ü Easy to scale out ü Highly configurable

44 Kolla minimize customization Kolla plugin ü Inspired by Kolla plugin for neutron ü Contributed for keystone ü Documents: kolla plugin functionality ü Format: [<image>-plugin-<plugin-name>] ü Install the plugin as a python module kolla-build.conf

45 Kolla plugin Benefits ü Minimize Customization: only 6 commit ü Separated our customization from KOLLA Make your own plugin ü Refer to plugin functionality of KOLLA image building ü Add similar template in the dockerfile.j2 ü Change the block name based on your images ü Add plugin source in the kolla-build.conf file

46 Kolla integrate into legacy Integrate into company s existing Database and LoadBalancer üdisable DB and LB in Kolla globals.yml : enable_mariadb: "no" enable_haproxy: "no" üadd New configurations for integration

47 Kolla no downtime Requires no down time when upgrading keystone kolla-ansible option: --limit <host> ü Deploy keystone on limit node Upgrading one by one ü Ansible/site.yml: Serial attribute ü Loadbalancer: support health check by a specific URL and switch traffic automatically ü Customized ansible plays Start to deploy one keystone node Wait for keystone back Remove health check url Added health check url LB switch traffic to other nodes Waiting for LB switch traffic back Stop and deploy keystone

48 Q&A THANKS