Android Internals Lecture 1 Android and Low-level Optimizations Summer School 13 July 2015 This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. Android Internals, Lecture 1 1/60
Team and Schedule Android Architecture Application components IPC Linux Kernel Binder Android Framework Managers Tools Android Internals, Lecture 1 2/60
Outline Team and Schedule Android Architecture Application components IPC Linux Kernel Binder Android Framework Managers Tools Android Internals, Lecture 1 3/60
Team and Schedule UPB: Razvan Prejbeanu, Petre Eftime, Laura Gheorghe Intel: Adrian Loteanu, Cristi Levcovici, Stefan Popa, Alecsandru Patrascu Schedule 13 July - 12 August Monday 18:00-21:00 Wednesday 18:00-21:00 Saturday 10:00-13:00 Android Internals, Lecture 1 4/60
Outline Team and Schedule Android Architecture Application components IPC Linux Kernel Binder Android Framework Managers Tools Android Internals, Lecture 1 5/60
Android Architecture Source: http://commons.wikimedia.org Android Internals, Lecture 1 6/60
Android Architecture Linux kernel AOSP Runtime Libraries Application framework Applications Android Internals, Lecture 1 7/60
Linux kernel Patches on top of mainline Linux Android Mainlining Project Wakelocks (also added to Linux 3.5) Low-Memory Killer, Binder, Alarm, Logger, etc. Only suspend to memory Android Internals, Lecture 1 8/60
Android Runtime Dalvik Virtual Machine (Java VM) Runs Dalvik-specific byte-code generated from Java-based apps, system components Apache Harmony project (implementation of Java libraries) Designed for embedded systems (slow CPU, small RAM, no swap, battery) Works with.dex files instead of.jar files Dalvik Executable Format.dex is 50% smaller than corresponding.jar Android Internals, Lecture 1 9/60
Android Runtime Class1.java Class2.java... Java compiler Class1.class constant pool class definition data Class2.class constant pool class definition data... dx classes.dex header constant pool Class1 definition Class2 definition... ClassN definition data ClassN.java ClassN.class constant pool class definition data Android Internals, Lecture 1 10/60
Libraries bionic (libc) Much smaller than glibc sqlite Managing SQL databases OpenGL ES Standard software interface for 3D processing hardware WebKit Display web pages Android, Apple ios, BlackBerry, Tizen SSL Securing the communication over Internet Android Internals, Lecture 1 11/60
Application Framework Services Managers Telephony Location Activity Package Notification Content Providers Android Internals, Lecture 1 12/60
Application Framework Application process System processes Thread Application API RPC stub Binder Service 1 JNI Thread... Native application libraries Binder Service N Android Internals, Lecture 1 13/60
Outline Team and Schedule Android Architecture Application components IPC Linux Kernel Binder Android Framework Managers Tools Android Internals, Lecture 1 14/60
Application components User interaction Activities Background functionality Services Broadcast Receivers Content Providers Android Internals, Lecture 1 15/60
Activity User interface Similar to a window from window-based GUIs Cannot be minimized, maximized, resized Uses the whole visual area User interacts with a single activity at a time Activities stack Activities launch other activities Back button for returning to the previous activity No forward button Start activities with intents Android Internals, Lecture 1 16/60
Activity Lifecycle Source: http://developer.android.com Android Internals, Lecture 1 17/60
Service Performs operations in background, no UI Runs in the same process as the application Can be configured to run in another process Provide services to the other applications Communication through the Binder Android Internals, Lecture 1 18/60
Broadcast Receiver Receive broadcast announcements Low battery Reboot Application state changes Can receive global or local broadcasts Choose which broadcasts to receive Intents Intent filters Active only when receiving a broadcast Android Internals, Lecture 1 19/60
Content Provider Manage access to a structured set of data Required for sharing data with other apps File data or structured data URI for identifying the provider and the table Content Resolver uses the URI to send a query to the provider Active only when responding to a request Android Internals, Lecture 1 20/60
Outline Team and Schedule Android Architecture Application components IPC Linux Kernel Binder Android Framework Managers Tools Android Internals, Lecture 1 21/60
Intents Send message, determine action execution Purpose Start activities, start or bind services Delivering broadcast messages to receivers Dispatched by the Android system Includes action and data ACTION DIAL content://contacts/people/1 Types Explicit: directed towards a specific receiver Implicit: a receiver which can resolve the action Android Internals, Lecture 1 22/60
Binder Lightweight RPC Remote object invocation In process and interprocess Transmit parcels of data Synchronous calls (blocking) Android Internals, Lecture 1 23/60
Outline Team and Schedule Android Architecture Application components IPC Linux Kernel Binder Android Framework Managers Tools Android Internals, Lecture 1 24/60
Linux Kernel Androidized kernel Hundreds of patches over the standard kernel Device-specific functionality, fixes, enhancements Many features get into the mainline kernel Androidisms Wakelocks Low-Memory Killer Binder Anonymous Shared Memory Alarm Logger Android Internals, Lecture 1 25/60
Wakelocks The Android kernel goes to sleep as often as possible Sometimes you want to keep the system from going to sleep Input from the user, critical operations Wakelocks keep the system awake A wakelock must be obtained by the application when it needs to stay awake Apps communicate with the Power Manager Service when they require a wakelock Device drivers call in-kernel wakelock primitives Included in mainline Android Internals, Lecture 1 26/60
Low-Memory Killer Linux OOM killer Prevents the activation of the OOM killer (system unlikely to run out of memory) Kills processes with components unused for a long time Based on OOM adjustments mechanism Different OOM kill priorities for different processes The userspace may control OOM killing policies Policies applied at startup by init Modified and enforced by Activity Manager Android Internals, Lecture 1 27/60
Low-Memory Killer Levels assigned to processes based on their components Levels from -17 to 15 (high -> killed) Threshold (MinFree) for each type of process Foreground app - application in foreground Visible app - visible but not in foreground Secondary server - service Hidden app - hidden, needed by a running app Content provider - provide data Empty app - app not active Starts killing when the threshold is reached Android Internals, Lecture 1 28/60
Anonymous Shared Memory (ashmem) IPC mechanism SysV IPC can lead to resource leakage in the kernel (vulnerability) Similar to POSIX SHM, differences: Uses reference counting to destroy the memory regions Shrink mapped regions when the system needs memory To shrink a region it must be unpinned First process creates region, uses Binder to share descriptor with other processes System services rely on ashmem Surface Flinger, Audio Flinger Driver included in the staging tree Android Internals, Lecture 1 29/60
Alarm Uses the RTC and HRT functionalities The setitimer() does not work when the system is suspended (HRT) The application receives the signal when the device wakes up Using RTC, the alarm will be fired even if the system is suspended RTC hardware device Uses HRT by default When the system is about to suspend, it uses RTC Android Internals, Lecture 1 30/60
Alarm /dev/alarm character device, ioctl() SystemClock, AlarmManager class rely on the driver SystemClock - obtain and set time AlarmManager - provide alarms to apps The driver and AlarmManager use WakeLocks The app that receives the alarm runs before the system is suspended again Included in the staging tree Android Internals, Lecture 1 31/60
Logger Uses kernel buffers for logging data Circular buffers in RAM No task switch, no writing in files (compared to syslog) Avoiding write operations in files is critical on Android devices Each buffer - separate entry in /dev/log (Events, System, Radio, Main) logcat displays the Main buffer by default Android Internals, Lecture 1 32/60
Logger Log and EventLog classes - public API Developers use Log EventLog used by the system components Diagnostic events Slog - system use (AOSP) Android Internals, Lecture 1 33/60
Logging Messages Through liblog library Logging from java classes Used by logcat Formatting and filtering Log message Priority, tag and data for each event Priority: verbose, debug, info, warn, error Tag: identifies the component that generated the message Staging tree Android Internals, Lecture 1 34/60
Logging System Android Internals, Lecture 1 35/60
Outline Team and Schedule Android Architecture Application components IPC Linux Kernel Binder Android Framework Managers Tools Android Internals, Lecture 1 36/60
Binder History RPC mechanism Initially in BeOS (then bought by Palm) OpenBinder project OpenBinder developers working in Android team Android Binder does not derive from OpenBinder Clean re-write of the same functionality OpenBinder documentation for understanding the mechanism Binder driver in the staging tree from kernel 3.3 Android Internals, Lecture 1 37/60
Binder Remote object invocation Remote services as objects Interface definition and reference to it Cornerstone of Android architecture Apps talk to System Server Apps talk to other service components Developers don t use the Binder directly Use interfaces and stubs generated with the aidl tool Android Internals, Lecture 1 38/60
Binder Driver Part of the Binder implemented in a kernel driver Character device /dev/binder Uses ioctl() calls Transmit parcels of data (serialized) between entities Android Internals, Lecture 1 39/60
Outline Team and Schedule Android Architecture Application components IPC Linux Kernel Binder Android Framework Managers Tools Android Internals, Lecture 1 40/60
Android Framework On top of the native userspace android.* packages, System Services, Android Runtime Code in /frameworks directory in AOSP Key building blocks: Service Manager, Dalvik, Zygote Android Internals, Lecture 1 41/60
System Services Form an object-oriented OS on top of Linux System Server All components run in the system_server process Many Java-based services/managers, 2 C-based services Power Manager, Activity Manager, Location Manager, etc. Surface Flinger, Sensor Service (C/C++) Media Server mediaserver process C/C++ code Audio Flinger, Media Player Service, Camera Service Android Internals, Lecture 1 42/60
Service Manager Performs system service handle lookups The Yellow pages book of all services A service must be registered to the Service Manager to be available Started by init before any other service Opens /dev/binder and becomes the Context Manager of the Binder Binder ID 0 = magic object = Service Manager Android Internals, Lecture 1 43/60
Service Manager System Server registers every service with the Service Manager Any app that wants to talk to a system service: Asks the Service Manager for a handle getsystemservice() Invokes the methods of the service using the handle Not used by an app to access its own service Used by the dumpsys utility to obtain the status of the system services Android Internals, Lecture 1 44/60
Dalvik Dalvik virtual machine Java VM optimized for mobile architectures Lower memory footprint Works with.dex files instead of.jar files Incompatible with Java bytecode Register based, not stack based 16 bit instructions Android Internals, Lecture 1 45/60
Dalvik Includes Just-in-Time (JIT) compiler ARM, x86, MIPS Translates bytecode into binary machine instructions Code runs directly on the CPU, not one instruction at a time by the VM The conversion is stored and used next time the application runs Apps run much faster Android Internals, Lecture 1 46/60
ART Android Runtime (ART) Dalvik Executable format Ahead-of-Time compilation (AoT) Improved garbage collection Support for sampling profiler More debugging features More details in case of exceptions and crash reports Android Internals, Lecture 1 47/60
Zygote Daemon used to launch apps Listens to connections on its socket for requests to start apps /dev/socket/zygote When it gets a request, it forks itself and launches the app Preloads (in RAM) all Java classes and resources needed by an app Copy-on-write (COW) Classes and resources are not modified, so all apps use them from Zygote (only one copy in RAM) The System Server is started explicitly by Zygote The PPID of all apps is the PID of Zygote Android Internals, Lecture 1 48/60
Outline Team and Schedule Android Architecture Application components IPC Linux Kernel Binder Android Framework Managers Tools Android Internals, Lecture 1 49/60
Activity Manager One of the most important services in the System Server Handles application lifecycle Broadcasts intents Starting new components (activities, services) Fetching Content Providers Responsible with the Application Not Responding (ANR) messages Involved in Permission checks OOM adjustments for the Low-Memory Killer Task management Android Internals, Lecture 1 50/60
Activity Manager Starts the Launcher (with Intent.CATEGORY_HOME) When an app is started from Launcher Launcher s onclick() callback is called Launcher calls the startactivity() from ActivityManager (through Binder) ActivityManager calls startviazygote() method Opens socket to Zygote and asks to start the activity am command for invoking the functionality of the ActivityManager isuseramonkey() Android Internals, Lecture 1 51/60
Package Manager Manages the.apk files in the systems API for installing, uninstalling, upgrading.apk files system_server and installd processes Ensures that the JIT versions of the dex bytecode is available before the app is started Resolves intents pm command for invoking the functionality of the PackageManager Android Internals, Lecture 1 52/60
Power Manager Control the power state of the device Handles WakeLocks Includes the WakeLock class acquire(), release() Apps request WakeLocks from PowerManager All calls to the Power Management (kernel) go through PowerManager Can force device to go to sleep Set the brightness of the backlights Android Internals, Lecture 1 53/60
Outline Team and Schedule Android Architecture Application components IPC Linux Kernel Binder Android Framework Managers Tools Android Internals, Lecture 1 54/60
SDK Tools Android SDK Manager Download SDK packages AVD Manager Manages Android Virtual Devices (for emulator) Emulator Virtual mobile devices running on a PC Dalvik Debug Monitor Server (ddms) Debugging tool Port forwarding, screen capture, call and SMS spoofing, location spoofing, etc. Android Internals, Lecture 1 55/60
SDK Tools Android Debug Bridge (adb) Communication between the development tools and (virtual) device dx Generates the classes.dex file from several.class files Android Interface Definition Language (aidl) To allow clients from another application to access your service Generates interfaces and stubs that are used by the Binder Android Asset Packaging Tool (aapt) Create, update and view Zip-compatible archives (zip, apk, jar) Compile resources into binary assets (XML files, etc.) dexdump Disassembler tool Obtain the Dalvik bytecode from classes.dex Android Internals, Lecture 1 56/60
Android Debug Bridge Three components Client: runs on the development machine Server: background process on the development machine Daemon: background process on the (virtual) device Copy files Install applications Debug Shell on the (virtual) device Android Internals, Lecture 1 57/60
Emulator QEMU Screen, Keyboard, Network, Audio, GPS, Radio Can be accelerated through virtualization x86 System Image Intel Hardware Accelerated Execution Manager (HAXM) on Windows KVM on Linux GPU accelerated Android Internals, Lecture 1 58/60
Bibliography Karim Yaghmour, Embedded Android: Porting, Extending, and Customizing, Chapter 2 http://developer.android.com/guide/components/ activities.html http://developer.android.com/guide/components/ services.html http://developer.android.com/guide/topics/ providers/content-providers.html http://developer.android.com/guide/components/ intents-filters.html http: //developer.android.com/tools/help/index.html Android Internals, Lecture 1 59/60
Keywords Linux kernel AOSP Android Runtime Libraries Activities Services Broadcast Receivers Content Providers Intents WakeLocks Low-Memory killer Binder Ashmem Alarm Logger System Server Service Manager Dalvik ART Zygote Activity Manager Package Manager ADB Emulator Android Internals, Lecture 1 60/60