Neural Network Compiler BNN Scripts User Guide

Size: px

Start display at page:

Download "Neural Network Compiler BNN Scripts User Guide"

Benedict Strickland
5 years ago
Views:

1 FPGA-UG Version 1.0 May 2018

2 Contents 1. Introduction Software Requirements Directory Structure Installation Guide Installing Dependencies Installing Packages Installing Caffe Updating BashRC Firmware File Generation Supported Layers Batch Norm Scale QuantRelu BinaryInnerProduct BinaryConvolution Input Pooling FaceDet_run.py... 6 Technical Support Revision History Figures Figure 3.1. Example Folder Structure... 3 Figure 5.1. Start of FaceDet_run.py... 6 Figure 5.2. Network Dimension and Address Definition... 6 Figure 5.3. BNN Layer Definition... 7 Figure 5.4. Fully Connected Layer Definition... 7 Figure 5.5. Internal Functions... 8 Figure 5.6. Input Arguments... 9 Figure 5.7. Preprocessing, Simulation, and Results... 9 Tables Table 3.1. Folder Description FPGA-UG

3 1. Introduction This document describes how to create a programmable firmware file which supports the BNN Accelerator IP Core. This firmware file contains information such as the command sequence and weights from your trained Neural Network. The script provided is in Python and requires manual changes. This Python script requires.caffemodel and.protofile files, and outputs the firmware file. The Python script supports the Caffe framework. 2. Software Requirements The following are the high level requirements for using the Python script. Other requirements such as Python libraries and dependencies are included in the installation section. Ubuntu LTS Python CUDA 8.0 (GPU Caffe only) 3. Directory Structure The BNN Compiler script.zip folder includes six subfolders described in Table 3.1. Table 3.1. Folder Description Folder Name Description Documentation Examples NNFPGA Scripts Caffe1 OpenSourceLicense Contains the documents that provide details on the mldevkit_tplus. Contains sample files for generating the firmware file. This folder includes input files.caffemodel and.protofile; image file; and the Python script used to generate the firmware file. These sample files are used to generate the Human Face Detection firmware file. The folder structure is shown in Figure 3.1. data contains images required to generate the firmware file facedets_bbn contains the fpga_tplus and train subfolders. fpga_tplus carries the facedet_run.py and run.sh files. The facedet_run.py is an example on how to generate the Face Detection firmware file. This file also serves as a template to build your own script which generates the firmware file. The run.sh is a shell script which calls the input files (caffemodel, proto, image file) and output file (bin, txt). train contains the Caffemodel and Proto files Contains compiled Python files which are used in generating the firmware file. These files are called in the facedetect.py script. Note where you store this folder because you need to call the directory of this folder in the Python script. Contains a single Python script which is used to run the trained network in Caffe with input images Contains a complete build of Caffe. This Caffe build contains some specific layers which are used in to support BNNs (binary neural networks). Details on how to install the Caffe build are provided in the Installation Guide section. License File for images provided in the data directory. Figure 3.1. Example Folder Structure FPGA-UG

4 4. Installation Guide This section provides information on how to install Caffe into the Ubuntu LTS system Installing Dependencies Before installing Caffe. Install these required dependencies: sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler sudo apt-get install --no-install-recommends libboost-all-dev sudo apt-get install y python-pip sudo apt-get install y python-dev sudo apt-get install y python-numpy python-scipy 4.2. Installing Packages Install the additional packages below that are required to run Caffe. Atlas: sudo apt-get install -y libatlas-base-dev GIT: sudo apt-get install git LMDB: sudo apt-get install libffi-dev python-dev build-essential pip install lmdb librosa: sudo pip install librosa skimage.io: sudo apt-get install python-skimage yaml: pip install pyyaml protobuf: sudo apt-get install python-protobuf OpenCV: sudo apt-get install python-opencv 4.3. Installing Caffe The default Caffe1 build is defaulted of GPU Version Caffe. CUDA 8.0 must be installed in order to use the GPU version of Caffe. If you wish to install the CPU version of Caffe, please modify Makefile.config in the caffe directory and change to CPU ONLY. You will also have to find split_concat_layer.cpp and comment out line 57 in order to use CPU Version of Caffe. After deciding the version to install, run the following commands to build Caffe: cd caffe mkdir build cd build cmake.. make all make install make runtest 4.4. Updating BashRC Update.bashrc with additional paths. If you are using the GPU version of Caffe, add the following into.bashrc: export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}} export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} Add the paths below whether you are using GPU OR CPU. export CAFFE_ROOT=$HOME/ML/caffe/ export PATH=$HOME/ML/caffe/python${PATH:+:${PATH}} export PYTHONPATH=$HOME/ML/caffe/python:$PYTHONPATH This completes the installation of Caffe. To check if Caffe is installed correctly, run Python then try importing Caffe. 4 FPGA-UG

5 5. Firmware File Generation This section describes the facedet_run.py script and provides details on the items that you need to change in order to generate your own firmware file Supported Layers When creating the proto file, take into consideration the layers supported for firmware generation listed below. BatchNorm Scale QuantReLu BinaryInnerProduct BinaryConvolution Input Pooling Batch Norm The BatchNorm Caffe layer is supported for implementing batch normalization operations. You are required to put a Scale layer in your network after each BatchNorm layer Scale The Scale layer in Caffe is supported. You are required to put a Scale layer in your network after each BatchNorm layer QuantRelu QuantRelu fulfills the same purpose as the ReLu in standard neural networks. The QuantReLu layer should be used instead of ReLU BinaryInnerProduct BinaryInnerProduct calculates the inner product for a binary network and should be used instead of the InnerProduct layer BinaryConvolution The BinaryConvolution layer is an added layer that functions similiarly to the convolution layer in Caffe, using binary weights and activations, and employing the same parameters Input The Input layer is supported Pooling Pooling layer is supported. The pooling layer supports the following Caffe parameters: Max Kernel_size Pad Stride FPGA-UG

5.2. FaceDet_run.py The specific sections of the Python script are described below. Figure 5.2 shows the beginning of the script. No change is required in this section.

6 5.2. FaceDet_run.py The specific sections of the Python script are described below. Figure 5.2 shows the beginning of the script. No change is required in this section. This section just imports Python libraries and calls certain paths just as the Caffe build directory, NNFPGA directory, and some of our generation firmware engine files. Figure 5.1. Start of FaceDet_run.py The network dimension and address definition section is heavily dependent on the neural network you have implemented. Input the dimensions correlating to your design. The SPRAM size is also dependent on the BNN Accelerator Soft IP Configuration. Plan in accordance with the SPRAM size of your BNN Accelerator Soft IP. With Single SPRAM configuration, you have 4k entries. With Dual SPRAM Configuration, on the other hand, you have 8k entries. The default is the Human Face Detection as shown in Figure 5.2. Figure 5.2. Network Dimension and Address Definition 6 FPGA-UG

7 The BNN layer definition section starts the definition of the run_sim. In Figure 5.3, under stage #1 starts the first BNN layer. Stage #1-3 are identical code-wise except that there is no pooling in the third BNN layer. Line 54: bin_wt=binarize_weights(cnet.params[ conv1 ][0].data) reads the weight values from the Caffe weight file (.caffemodel) and binarize conv1. Line 55: bin_threshold, bin_negpol=get_threshold(cnet, conv1, bn1, scale1,**thr_args) calculates the parameters used in HW for binary activation, batch normalization, etc. Line 56: layer.run_bwconv2d_stage(bin_wt, data_dim(input dim*. Runs simulation and generates the code. It also performs convolution, batch normalization, scaling, binarization (activation), pooling. Figure 5.3. BNN Layer Definition This section, a fully connected layer, is shown in Figure 5.4. Line 121: layer.read_data(output_dim, OUTPUT_ADDR) reads out the final output from OUTPUT_ADDR. Figure 5.4. Fully Connected Layer Definition FPGA-UG

8 The internal functions section shown in Figure 5.5. No change is needed in this section since it involves purely internal functions,. Figure 5.5. Internal Functions 8 FPGA-UG

9 The section, shown in Figure 5.6, requests for input arguments and loads Caffe Network and parameters. Refer to run.sh file in the same file direction as facedet_run.py. Figure 5.6. Input Arguments The last section reads test image and performs preprocessing such as scaling and normalization. The section also runs simulation and code generation, and lastly, reads and displays the result in the terminal. Figure 5.7. Preprocessing, Simulation, and Results FPGA-UG

10 Technical Support For assistance, submit a technical support case at Revision History Date Version Change Summary May Initial release. 10 FPGA-UG

11 7 th Floor, 111 SW 5 th Avenue Portland, OR 97204, USA T

Key Phrase Detection Using Compact CNN Accelerator IP Reference Design

Key Phrase Detection Using Compact CNN Accelerator IP FPGA-RD-02033 Version 1.2 October 2018 Contents 1. Introduction... 3 2. Related Documentation... 3 3. Software Requirements... 3 4. Hardware Requirements...