Cross-Platform Data Models and API Using grpc

Transcription

1 Cross-Platform Data Models and API Using grpc Sebastian Hagedorn, Felix Lamouroux

2 Outline 1. Motivation & Goals 2. Choosing the Right Cross-Platform Technology 3. Introduction to Protocol Buffers and grpc 4. Collaborative API Design 5. Here Be Dragons

3 Create a Playful & Performant App that Feels Native

4 Starting Point Green field: New project, clean slate, no legacy components, engineering-driven decisions Team of experienced developers per platform Preferred languages: ios (Swift), Android (Kotlin), Web (Elm), Server-Backend (Go) No complex business logic on mobile/desktop clients

5 Goals Ensure consistent API usage Reduce boilerplate per platform Parsing & serialization Transport layer No critical 3rd party dependencies above transport layer Integrate seamlessly with native languages & tools

6 Options Considered: JNI/Djinni Interfaces defined in platform-agnostic Interface Definition Language (IDL) Platform-specific interfaces (Java or C++/Objective-C) are generated Service/shared code must be implemented in C++ Decided against it: Not usable for web client Our team lacks C++ experience for large/critical code bases Little client-side business logic in our UI-heavy project

7 Options Considered: React Native Allows cross-platform code (way) beyond data model and networking Enables JavaScript developers to write native apps Decided against it: Does not help to define a common model unless all components use React Native (backend must use node.js) Our team consists of experienced platform developers (with little JS experience) Does not align with our goal of playful & performant apps

8 Technology Stack: Protocol Buffers Binary serialization format developed by Google Model types are defined in simple Protobuf language Per-platform/per-language code is generated from Proto files Generates interface and implementation for (de)serialization Official support for many languages: Java, Python, Objective-C, C++, Go, Ruby, and C# More can be added via plugins, as Apple did for Swift:

9 Auto-generated types Golang Java Proto Objective-C Swift code generator available from Apple: Swift

10 Technology Stack: grpc A high performance, open-source universal RPC framework" (grpc.io) API endpoints are defined in IDL (Protobuf by default) Server and client libraries are generated Protobuf handles (de)serialization grpc handles networking Apps call generated endpoint APIs in native language grpc-protorpc generators & runtimes available in Swift and ObjC We chose ObjC for better stability (Swift support is declared experimental) Interdependency between Protobuf (Swift support implemented by Apple) and grpc (Swift support implemented by official project maintainers): Cannot (necessarily) update components individually Apple uses grpc-protorpc in private CloudKit server-to-server API

11 Code Speaks Louder than Words

12 Login Example: API Specs (Service) Support JSON fall-back via proxy Shared documentation also provided in auto-generated clients.

13 Login Example: API Specs (Model)

14 Login Example: ios Callsite (Swift)

15 Login Example: Backend Implementation (Go)

16 Collaborative API Design: Tooling & Setup Bridge gap between frontend & backend teams: Design API collaboratively Cross-platform API submodule Central source of truth for model and API definitions (Proto files) Tooling to generate code for all platforms (using Makefiles) Integration of runtimes and generated files solved per platform repository All developers invited to contribute via Pull Requests to shared API specs On-the-fly code generation allows local iterations Review & verification on all platforms Shared vocabulary about types (user, account, profile, )

17 Don't Let Clients Send Messages With Ignored Fields Messages that are used in requests should only contain fields that are mutable by the client. Information about an entity that is only mutable by the server should only be available as part of responses. Combine messages to decorate them (composition over inheritance)

18 Avoid Duplicate Fields Avoid repeating fields by wrapping decorating messages around existing ones.

19 Here Be Dragons

20 Beware: All Fields are Optional Protocol buffers 3 no longer allows marking fields as required Makes backwards/forward-compatibility easier: required fields invalidate the entire message regardless of context parts of an application could pass around partially-complete messages internally whole applications might simply be interested in passing message through (see routers) Ensure that each callsite using a message performs sensible validation (no blanket rejections) Provide invalid default values to fields if you need to ensure that clients set them explicitly (see enums) Only validate messages when you care about the content See also:

21 Optionality & Versioning We re mitigating [backward and forward compatibility issues] by using Protocol Buffers and controlling most new features with flags, which remain turned off during a partition upgrade, and are enabled only after all hosts are updated. From Apple s CloudKit paper (

22 Check Your Defaults Explicit UNINITIALIZED_* values in enums ensure that unset values are caught Check if your business logic makes sense for other type defaults, such as empty string or zero integers

23 Enforce Client-Side Non-Optional Values Linter enforces usage of convenience initializers Unit tests make sure that model changes do not go unnoticed

24 Tooling Quirks Custom Makefile to process Proto files Not trivial to write Integration is different per platform/ide Make modifies output file timestamps regardless of content diff Swift compiler re-builds the entire module Solution: Extra script using rsync that only makes actual changes available to Xcode Integration of grpc runtime is not trivial except when using CocoaPods Protobuf optionality is not reflected by ObjC header annotations All properties are force-unwrapped

25 We Are Hiring We are building a product team here in Cologne. Looking for ios, Android, Web, Backend, UI/UX design. Strong mobile focus with many senior developers.

26 Sebastian Hagedorn Felix Lamouroux ios Lead Developer Project Lead - New Ventures (and ios felix.lamouroux@sumup.com