PaPP cross-platform development system version 15.0

Transcription

1 PaPP cross-platform development system version 15.0 Laust Brock-Nannestad, DTU October 14, 2015 Contents 1 Introduction Updates Setting up the development system Setting up the virtual machine Updating the virtual machine Using the tools Testing applications Building applications Networking NFS Accelerators GPU simulation Epiphany / Parallella Other tools Alten s pappadapt Python prototype Troubleshooting Speeding up simulation Interacting with the simulator Reporting bugs and issues Mailing-list support Toolchain details Libraries

2 9 Building the cross tools on other Linux systems Cross tools build procedure Known issues 17 2

3 1 Introduction Figure 1: Development system overview This document describes the fifteenth release of development system for the main PaPP hardware platforms. The current set of tools support ARM, Epiphany, x86-64, and Leon. Accelerators are partially supported through 3rd party simulators, but not integrated with the PaPP platform simulators. Version 15 is the final release to correspond with the end of the project. It upgrades the bundled applications and tools to the latest versions from the project repository and integrates the latest version of RTE s emul8 Leon simulator. The development system includes cross compilation tools and full system simulators. The system also includes a test harness for testing applications. The test harness is designed to use simulated or real hardware target platforms, but only simulation is supported in this release. An overview of the system is shown in figure 1. The development system is provided on a virtual machine image for use in VirtualBox. This saves developers the effort of compiling the tools from source. The development system is based on Xubuntu LTS for x This document describes how to: setup the development system, use the tools, and troubleshoot the tools. 1.1 Updates These are the changes for the current release: Updated applications and test cases to latest versions from repository. Updated to emul8 and the Leon/SPARC configuration. 3

4 These were the changes for the 14th release: Fixed compilation flags for Bionic which prevented some math library functions from working properly, causing programs to hang in infinite loops. Large refactoring of TurboBlysk which should yield better performance. Open MPI runtime support for x86-64 (required by UTIMP) Added UTIMP support. For more details refer to VTT and the utimp demonstration program in $HOME/appSrc. GCC updated to General security updates and smaller fixes. These were the changes for the 13th release: Bugfixes for TurboBlysk (issues with dependent and non-dependent tasks). Fixed a configuration bug in Emul8 which caused the interrupt timer to run too fast. As a result interactive use of the emulator was not possible. General security updates to XUbuntu. These were the changes for 12th release: GCC updated to GDB updated to QEMU updated to BusyBox updated to Improved system level adaptivity based on libpappadapt and TurboBlusk. General security updates to XUbuntu. These were the changes for the 11th release: Updated all platforms kernels to Linux to support SMP on Leon. Updated emul8. Integrated the radar test case directly into test harness, due to the review. The tenth release contained the following changes: 4

5 Integrated libpappadapt and BLYSK; the number of running cores and threads can now adapt at runtime. Adds system adaptivity which aggressively throttles the machine when dangerously hot. Security and bug fix update for Ubuntu LTS. The ninth release contained the following changes: Upgraded GDB to the latest version, Security and bug fix update for Ubuntu LTS. The eighth release contained the following changes: Adds support for the OpenMP 4.0 depends clause. Updated to GCC Updated microblysk to turboblysk, which supports task dependencies: Use turboblysk by linking with -lblysk. Adds support for Parallella: See section 5.2. The toolchain includes the Epihany compiler from e-sdk The Parallella support libraries are included in the ARM root image. Updated glibc to The seventh release contained the following changes: Fixed a problem with Linux for SPARC. The sixth release contained the following changes: Updated GCC to 4.8.2, glibc to 2.18, binutils to Fixed several bugs when using dynamic libraries. Integrated MicroBLYSK from KTH/SICS. Integrated libpappadapt from DTU. Added UTIMP from VTT. Added the environment variable PAPP_SRC which should point to the location of PaPP source packages in the Subversion tree, e.g. /checkout/papp/code/. This is only relevant for developers building from SVN. 5

6 The fifth release contained the following changes and additions: Added Alten/XDIN s pappadapt Python prototype. Added Python 3.3, LINX kernel modules, and other dependencies for said tool. The host now exports /home/papp/nfs_root over NFS and it is automatically mounted on boot of the simulator. This gives persistent storage across tests. See section 4.1 on NFS for more information. Miscellaneous updates to the Ubuntu host system. The fourth release contained the following changes and additions: Updated Ubuntu host system, primarily security fixes. Reduced the Linux timer interrupt (tick) rate to reduce kernel hangs when booting QEMU on x Updated the build scripts to enable building the tool chains from a Subversion checkout and not just a Subversion export. This change is only to the source code committed to Subversion and has no impact on the downloadable Virtual Machine releases. The third release contained the following changes and additions: Network support on all three platforms. There is now a virtual private network between the development system and the simulated hardware. See section 4 on Network Support. The test harness can now launch a test environment, e.g. a simulator and a specific application, without executing the application. It keeps the simulator running and the developer can access the simulator via Telnet. See section 7.2 on interacting with the simulator. Various software updates to the development system. Initial support for accelerators. Incorporated the Multi2Sim CPU+GPU simulator. Incorporated the Ephiphany C compiler and Instruction Set Simulator. See section 5 on Accelerators. These are the changes for the second release: Incorporated the emul8 emulator, from RTE, for emulating SPARC/LEON. New test harness for controlling both QEMU and emul8. 6

7 Added mklinuximg from Gaisler Research for building bootable Leon images. Updated QEMU to Updated GCC from to More consistent use of toolchains: GCC is used for compiling applications on all platforms. 4.7.x is still used for kernel and support software on some platforms. Added Mono (required by emul8). Improved this document: More details on how to build from source. General improvements to the text. 2 Setting up the development system Note: New users should follow the instructions in Setting up the virtual machine. Users of earlier releases who want to preserve their work files contained /home/papp should follow the procedure in Updating the virtual machine. 2.1 Setting up the virtual machine The development system is distributed as a VirtualBox appliance (image). It requires the VirtualBox software and a 64-bit processor with support for hardware virtualization. VirtualBox is free and can be downloaded from: The virtual machine can be downloaded from DTU at: Questions and bug-reports should be forwarded to: papp-tool-support@sics.se Importing a virtual machine is explained in the VirtualBox manual, more specifically the section Importing and exporting virtual machines. The virtual machine contains two hard disk images. The first disk image contains the host development environment and a precompiled version of the toolchains and test harness. The second disk image contains all developer generated files such as source code for applications. The two-disk split enables the development system to be updated whilst preserving any work-related files. This procedure is outlined in the following section. 7

8 2.2 Updating the virtual machine Contrary to previous releases, updates are now done in-place by downloading and running a shell script. Please download and read the guide entitled Updating the PaPP Development System. Note: if you use VirtualBox, you can use the snapshot feature to save the state of the virtual machine before an update. This makes it simple to revert should the update fail. See the VirtualBox manual for more information. 3 Using the tools The development system comes with a user account already set up. Log in with the username papp and password papp. The development system has a running SSH server, by default forwarded to the host on port and this can also be used for login. None of the tools currently require a graphical interface. The VirtualBox documentation describes how to change the port forwarding. The current release of the toolchain contains emul8, QEMU, GCC with OpenMP support compiled for the three platforms, glibc, Bionic (Android libc), and Binutils. The actual tool chains are installed in /opt/papp/tools. Building and testing applications is performed using the papp-make command. papp-make help explains its use. Primarily there are four actions, which all follow the format: papp-make ARCH=[x86_64 arm sparc] <action>+<appname> Action is one of build, clean, or test, or boot. This builds, cleans, or tests the application <appname> through the test harness. boot sets up the test environment and then hands control over to the developer. It is explained in the section on troubleshooting. ARCH specifies the architecture and defaults to ARM if not specified. For example: papp-make ARCH=sparc build+hello builds the hello project for the Leon/S- PARC architecture. papp-make clean+hello cleans the hello project removing any object files. Architecture does not matter for the cleaning process. papp-make ARCH=arm test+hello builds (if required) and tests hello on the ARM architecture. Testing is described in section Testing applications The build system is equipped with a test harness that allows applications to automatically launch and execute on the PaPP hardware platforms. The test harness is designed to target either a simulator or physical hardware connected to the development machine. By default the test harness launches a simulator. 8

9 Figure 2: The root file system consists of 1) A precompiled Busybox for the target architecture, 2) a shell script, test.sh, which sets up the target once it has booted, and finally 3) an application to be tested. Using real hardware To execute a test on a real hardware platform, specify the IP address or hostname of the machine on the command line using the PAPP_TARGET_IP parameter, e.g.: papp-make ARCH=arm PAPP_TARGET_IP=parallella.mydomain.com test+hello This will execute the test on the target. Unlike with the simulators, the target is not booted as part of the test and must already be running. It must run a Linux setup similar to the one used by the simulated platforms. Currently this has only been tested using the Parallella board and we provide a suitable Linux user land. Please see the section on Parallella support. Applications are executed on the target platform using a custom Linux kernel. The kernel loads a root file system as indicated in figure 2. The following sequence of events takes place when the test harness launches an application: 1. papp-make calls the application s Makefile to build the application for the target platform. 2. The application and test.sh are combined with the precompiled Busybox and relevant libraries to form the minimal Linux root file system. 3. The target platform is booted through the test harness. 4. The target platform boots the generated root file system from a RAM disk. 5. The target platform is configured through the rcs initialization script. This script sets up basic system components such as devices, networking, and login daemons The script is the same for all platforms and can be found in /opt/papp/src/configs/busybox_rcs. 9

10 6. The test harness logs into the target and launches the test.sh shell script. The script starts the actual test. 7. After the test finishes, the test.sh script prints out either PASS or FAIL to indicate success or failure of the application. 8. This status is interpreted by the test harness and propagated to the development system. If nothing is printed by the target for seconds 1 the test harness will time out. In addition to user created tests and the hello example, the system also includes the Barcelona OpenMP Task Suite (BOTS) 2. These are intended for regression tests of the PaPP tools. BOTS are executed using the syntax papp-make ARCH=<arch> test+bots+<benchmarkname> Which compiles and executes the selected benchmark on the target platform. For a complete list of all predefined test targets, look at the file /opt/papp/src/testlist. 3.2 Building applications All user applications are stored in the directory /home/papp/appsrc/. There is one directory per application. The papp-make command identifies each application by the name of its directory. The image contains an example application, entitled hello. hello is the quintessential Hello World style program. New applications should follow the same structure as the hello example and it may be used as a template. An application must provide two files Makefile and test.sh, as well as one directory apps, laid out in the following structure: appname/ apps/ Makefile test.sh Makefile must provide: a build recipe which compiles the application. a clean recipe which removes object code and other temporary files. test.sh is a shell script used to test the application. test.sh executes on the target. The apps directory contains all executables and data required for the test (on the target). The build rule of the makefile must place the final executable and any required data files in the apps directory. 1 Wall clock time, not simulated time

11 4 Networking A virtual private network is established between the simulator and the development system. The network uses the virtual tap69 network device and the following default settings: Subnet x Netmask Simulator static IP: Linux host system static IP: For information on how to log into the simulator using the network, please refer to section NFS /home/papp/nfs_root is exported as an NFS share and automatically mounted at /nfs inside the simulator. The share can be used as developer controlled persistent storage. By default all permissions are remapped to the papp user account. See the file /etc/exports for details. The development image uses the Linux nfs-kernel-server, but has also been tested with the freely available unfs3 server. It is possible to run this combination without administrator privileges. Note that if you use the update procedure described in section 2.2, the directory /home/papp/nfs_root must be created manually for sharing to work. 5 Accelerators The development system includes initial support for accelerators. These are described in this section. 5.1 GPU simulation Multi2Sim is a x86+gpu simulator. Run Multi2Sim with m2s <command>. Limitations: The provided Multi2Sim does not support OpenGL. The toolchain does not include GPU (OpenCL or CUDA) compilers. The visual GPU debugger that is part of Multi2Sim is not included. Multi2Sim only simulates a 32-bit x86 processor. The 32-bit compiler has the prefix i786-unknown-linux-gnu-. To invoke this compiler through papp-make, use papp-make ARCH=x86 <action+appname>. For further details vist to the Multi2Sim website: 11

12 5.2 Epiphany / Parallella The Parallella is supported through support libraries epiphany-libs, the official Epiphany compiler e-gcc, and the e-run instruction set simulator. The toolchain and instruction set simulator are installed on the development system. The Epiphany support libraries are installed on the ARM root file system. For the physical Parallella board we have prepared a special SD card image which contains the same software infrastructure used on the simulated platforms. This includes the tools developed in the project, such as Blysk and libpappadapt. Instructions on how to prepare the image are included with the image. The image can be downloaded from the same location as the development system. Once the image has been prepared and loaded onto a Parallella, it can be controlled through the test harness. Note: The e-run simulator does not simulate the timing or hardware peripherals of the chip and only a single core. For futher information, refer to the Epiphany SDK Reference manual. 6 Other tools 6.1 Alten s pappadapt Python prototype This tool is installed in /opt/papp/alten/ along with the Python bindings for the LINX protocol. For more information on usage and design, please refer to the PaPPAdapt Design Description distributed by Alten to project participants via the mailing-list. 7 Troubleshooting This section covers some tips related to debugging of the toolchain and the test environment inside the simulator. 7.1 Speeding up simulation Multicore simulation can be quite slow, especially for x86_64. When debugging issues relating to the simulator environment, it can be beneficial to reduce the number of simulated cores. The number of cores to simulate can be given as an argument to papp-make using the parameter QEMU_SMP=x, for example: papp-make QEMU_SMP=1 ARCH=x86_64 test+<benchmark> will execute the test on x86_64 simulating just a single core. The parameter can be used with the test+ and boot+ actions. The parameter has no effect on SPARC, which currently always simulates a single core. 12

13 7.2 Interacting with the simulator The scripted nature of the test harness makes it cumbersome for debugging. As an aid, it is now possible to use the test harness to only boot the simulator, but not execute the test. The environment will be set up exactly as when executing an automated test. To do this papp-make should be called with the boot+ action, for example: papp-make ARCH=x86_64 boot+hello which will compile, and setup the hello application, and then boot the simulator and wait. Once the simulator has booted, it will print the message Please press Enter to activate this console.. At this point, the simulator should be ready for remote logins. To log into the simulator open a new terminal and run telnet The login is root without a password (press ENTER). it is now possible to interact with simulated Linux environment. Multiple logins are possible. In boot mode, the test harness disables the test timeout, but still monitors the simulator s console for indication that the test passed or failed or that the machine was halted. If this condition occurs, the test harness will terminate the simulator as usual. To safely shut down the simulator, execute the command halt from within the simulated machine. 7.3 Reporting bugs and issues It is always possible to check the current release of the development system, by executing lsb_release -r. 7.4 Mailing-list support SICS has set up a support mailing-list for the development system. Questions and bugreports should be forwarded to: papp-tool-support@sics.se. 8 Toolchain details This section covers details about the toolchains in use. Interested parties are encouraged to look at the build system and source code in /opt/papp/ or the PaPP repository at SICS in /Code/WP4/. Makefiles and scripts are commented. There are 5 toolchains, with their respective compiler prefixes: Platform Prefix Version ARM/Android arm-unknown-linux-androideabi ARM/glibc arm-unknown-linux-gnueabi LEON/glibc sparc-unknown-linux-gnu LEON/glibc sparc-build-linux-gnu x86_64/glibc x86_64-unknown-linux-gnu Epiphany e

14 The ARM/glibc toolchain is included to enable building Busybox and the Linux kernel. The Android (Bionic) based toolchain is used for applications on the ARM target. On LEON, The sparc-unknown-linux-gnu toolchain is for compiling applications, whereas is used for support tools such as Busybox and the kernel. The Android toolchain uses gold as the default linker, whereas Glibc based toolchains use ld. Prefixes are added to the common executable filenames. To give an example arm-unknown-linux-androideabi-gcc is GCC for ARM/Android. 8.1 Libraries The target libraries and headers is is stored in: /opt/papp/tools/<toolchainprefix>/sys-root The tools can be run as usual, and GCC is capable of producing both statically and dynamically linked binaries. GCC s libgomp (OpenMP) is built against the target libpthread and libc (bionic/android or glibc). Since Android s libc has pthread support built-in, it is linked against an empty libpthread object file. 9 Building the cross tools on other Linux systems Users may wish to build the cross tools on other Linux systems. While this is not officially supported, and may require extra effort, this section outlines the process. We strongly recommend using a recent 64-bit Ubuntu based host system the tools are known to build there. The following tools and libraries must be installed on the host system: automake bison curl flex g++ gawk gcc gitk gperf ia32-libs 14

15 libexpat-dev libexpect-perl libgettextpo-dev libglib2.0-dev libmpc2 libtool ncurses-dev openvpn patch pkg-config socat subversion texinfo xz-utils zlib1g-dev They are listed here by their Debian/Ubuntu package names. The following steps are recommended for building the tools. 1. Check out the Android source tree using the instructions provided on the Android website. We use the android-4.2.2_r1 branch. Building may not work if you use other branches. 2. Install Oracle s JDK version 6 (OpenJDK will not work) 3. Point the environment variable ANDROID_DIR to the Android source code. If undefined, the Makefile assumes $HOME/android. 4. Check out the Code subtree of the PaPP Subversion repository. 5. Point the environment variable PAPP_SRC to the checked out copy, e.g. export PAPP_SRC=/home/user/svn/Code/. 6. Create an empty directory and point the environment variable PAPP_INSTALL_DIR to this directory. The tools will be installed here. 15

16 7. Go into the directory containing the fetched development source code, more specifically.../code/wp4/papp_development_tools. 8. Execute make ARCH=x86_64 -j <NUM_THREADS>. This should build everything for the Intel architecture, and launch a test to verify the tool chain (should end with PASS/FAIL on the console). The Intel platform has the widest compatibility. If building for the Intel platform fails, then the other architectures are also likely to fail. The build script automatically downloads and compiles support libraries beyond those mentioned above, so make sure the machine has an Internet connection. Please note the network configuration requirements listed in section 4. The virtual network must be configured for the test harness to function. 1. Once the Intel toolchain runs without errors, proceed to the ARM platform with make ARCH=arm. This will use the Android source code. 2. Finally, do the same for the last platform with make ARCH=sparc. To enable simulation on LEON, you will need to get hold of emul8 from RTE or from the virtual machine image where it is installed in /opt/papp/tools/emul8/. You should unpack emul8 to $PAPP_INSTALL_DIR/tools/emul Cross tools build procedure The cross tools, applications and emulators are built using the makefile /opt/papp/src/makefile as described in the previous section. In this section we describe the implemented procedure for building the cross tools, which is handled by the makefile /opt/papp/src/scripts/ Tools.mk. The main components of the cross tools are GCC and libc. They are built in a 5 step process: 1. Build the prerequisites of GCC. 2. Get the target libc header files. 3. Build a minimal version of GCC for the target. 4. Build libc for the target using the minimal GCC. 5. Build a full version of GCC for the target, including OpenMP. Step 1, building the prerequisites is slightly complicated. The GCC prereqiusites include binutils for the target, and the host libraries libgmp, libmpfr, libmpc, libisl, libcloog, and libelf. Binutils for the target has no dependencies except for tools such as bison, which we assume is installed on the host. The host libraries have the following depencies: 16

17 GMP<=MPFR<=MPC <=ISL<=CLOOG LIBELF Step 2, getting the target libc header files differs for glibc and Bionic (Android) based targets. The libc header files of Bionic are extracted directly from the Android source. So for Android step 2 is just a copy operation. The glibc header files are generated based on the Linux header files and the target compiler. So for glibc step 2 actually has 3 substeps: 2a Extract the target Linux header files. 2b Compile a bare metal version of GCC for the target. 2c Generate glibc header files for the target. 10 Known issues No outstanding issues. 17