A success story of leveraging SDSoC TM to accelerate customer software algorithm

Size: px

Start display at page:

Download "A success story of leveraging SDSoC TM to accelerate customer software algorithm"

Chad Cummings
5 years ago
Views:

1 A success story of leveraging SDSoC TM to accelerate customer software algorithm Agenda: Corporate Profile Story Summary Story Detail MPSoC DEMO Wataru Takahashi Manager 1

description 3-1 Futaba-cho, Takasaki, Gunma-ken, Japan 50 million yen OKI IDS

(100% owned by Oki) March 10, 2014 Noriaki Anada 78 Development, design, sales

communication) Product development for Medical instruments, Industrial

2 Corporate name OKI IDS Co., Ltd. (OIDS) Office address Capital Stockholder Founded CEO Employees Business description 3-1 Futaba-cho, Takasaki, Gunma-ken, Japan 50 million yen OKI IDS Co.,Ltd. (100% owned by Oki) March 10, 2014 Noriaki Anada 78 Development, design, sales and consulting of firmware/hardware for multimedia (video, audio, communication) Product development for Medical instruments, Industrial instruments The largest natural water flow of all hot springs in Japan Registered as Unesco World Heritage in 2014 Takasaki city is the top producer of Daruma Doll in Japan Kusatsu Hot Spring Tomioka Silk Mill Takasaki Daruma 2

3 OKI IDS strongly supports customer s product design with highly-versatile solution products based the technologies of its specialty Target Application Solution Technology 3

4 Goal of the project was to achieve moving object detection system on ZC706. The system was designed to processes input video from camera in real time, and output specific signal when a moving object is detected. The customer already had entire algorithm developed on PC, and requested OKI to implement it onto ZC706. In so doing, part of the processing needed to be hardware accelerated. SDSoC worked efficiently for hardware acceleration of the system. SDSoC enabled hardware acceleration of key processing of the system and greatly contributed to customer satisfaction. 4

5 The processing time for 1 frame at the this point was: 400 ms for 1000 feature points -> 2 3 fps ZC706 Zynq PL HDMI camera FMC board logiwin DDR HDMI logicvc New DMAC Zynq PS Moving Object Detection processing Video intake Moving Object Detection Graphical overlay on the detected object Xylon's IP OKI IDS Design VIVADO 5

6 Corner Harris Open CV version Corner Harris Software void cornerharris(const Mat& src, Mat&dst, intblocksize,intaperturesize, double k, int bordertype=border_default) Vivado HLS version Corner Harris template<int blocksize,int Ksize,typename KT,int SRC_T,int DST_T,int ROWS,int COLS> void CornerHarris(hls::Mat<ROWS, COLS, SRC_T> &_src, hls::mat<rows, COLS, DST_T> &_dst, KTk); Auviz Corner Harris template<int ROWS, int COLS, int DEPTH, int NPC, int IN_WW, int OUT_WW, int FILTERSIZE, int MAXPNTS,int PIPELINEFLAG> void aucornerharris( hls::stream < AU_TNAME(IN_WW) > & _src_mat, hls::stream < AU_TNAME(OUT_WW) > & _dstlist, uint16_t img_height, uint16_t img_width,uint16_t _filter_width, uint16_t _block_width, uint16_t _nms_radius,uint16_t _threshold, uint16_t val, uint32_t *ncorners) 6

7 OpenCV / VivadoHLS / Auviz OpenCV VivadoHLS Auviz Software Hardware Hardware Input format Mat Format Mat Format Mat Format Output format Mat Format Mat Format List Format (xy coordinate list) NonMaxSuppression processing, pick up feat.point Not Supported Not Supported Supported 7

8 The basic principles of Corner Harris S( u, v ) = w( x, y )[ I( x+u, y+v ) I(x, y)] 2 8

9 The basic principles of Corner Harris S(u,v)= w( x, y )[ I( x+u, y+v ) I(x, y)] Take taylor expansion about I( x+u, y+v ). Partial differentials of I are I x,i y then S(u,v) w(x,y)[i x (x, y)u I y (x, y)v] 2 2 I x = I y = Put it as matrix expression; S( u, v ) (u v) A A= w(x, y) I 2 x I x I y I x I y I 2 y 9

10 The basic principles of Corner Harris It can be determined by reviewing the eiganvalues, 1 2. However the computational cost is high in this way. Instead, the following formula can be used to determine corner strength: M det( A ) k trace A ) 2 M x feature point y 10

11 SDSoC TM 11

12 Making bitstream and interface library 12

13 Making application software DMA and other interface lib 13

$kpara, float threshold_f, float threshold_f2, int *keypointnum ) { } mygoodfeature_sds( image_in, keypointlist, rows, cols, offset, kpara, threshold_f, threshold_f2, keypointnum ); #pragma SDS$ data mem_attribute( image_in:physical_contiguous ) #pragma SDS data mem_attribute( keypointlist:physical_contiguous ) #pragma SDS data mem_attribute( keypointnum:physical_contiguous ) #pragma

data mem_attribute( image_in:physical_contiguous ) #pragma SDS data mem_attribute( keypointlist:physical_contiguous ) #pragma SDS data mem_attribute( keypointnum:physical_contiguous ) #pragma

$mygoodfeature_sds( unsigned short image_in[ WIDTH * HEIGHT ], int *keypointlist, int rows, int cols, int offset, float kpara, float threshold_f, float threshold_f2, int *keypointnum ) { // [ HLS$

14 Source example Call point from app software HW Interface Description (DMA, FIFO, etc) void feature_detector( unsigned short *image_in, int *keypointlist, int rows, int cols, int offset, float kpara, float threshold_f, float threshold_f2, int *keypointnum ) { } mygoodfeature_sds( image_in, keypointlist, rows, cols, offset, kpara, threshold_f, threshold_f2, keypointnum ); #pragma SDS data mem_attribute( image_in:physical_contiguous ) #pragma SDS data mem_attribute( keypointlist:physical_contiguous ) #pragma SDS data mem_attribute( keypointnum:physical_contiguous ) #pragma SDS data access_pattern( image_in:sequential ) #pragma SDS data access_pattern( keypointlist:sequential ) #pragma SDS data access_pattern( keypointnum:sequential ) #pragma SDS data data_mover( image_in:axidma_sg ) #pragma SDS data data_mover( keypointlist:axififo ) #pragma SDS data data_mover( keypointnum:axififo ) #pragma SDS data copy( image_in[ 0:WIDTH*HEIGHT ] ) void mygoodfeature_sds( unsigned short image_in[ WIDTH * HEIGHT ], int *keypointlist, int rows, int cols, int offset, float kpara, float threshold_f, float threshold_f2, int *keypointnum ) { // [ HLS source code ] // // 14

SDSoC effects: It's possible to make an environment that enables to compare the various result easier. SDSoC can correspond to the difference in API easily.

15 SDSoC effects: It's possible to make an environment that enables to compare the various result easier. SDSoC can correspond to the difference in API easily. Open CV Corner Harris Software void cornerharris(const Mat& src, Mat&dst, intblocksize,intaperturesize, double k, int bordertype=border_default) Vivado HLS Corner Harris template<int blocksize,int Ksize,typename KT,int SRC_T,int DST_T,int ROWS,int COLS> void CornerHarris(hls::Mat<ROWS, COLS, SRC_T> &_src, hls::mat<rows, COLS, DST_T> &_dst, KTk); Auviz Corner Harris template<int ROWS, int COLS, int DEPTH, int NPC, int IN_WW, int OUT_WW, int FILTERSIZE, int MAXPNTS,int PIPELINEFLAG> void aucornerharris( hls::stream < AU_TNAME(IN_WW) > & _src_mat, hls::stream < AU_TNAME(OUT_WW) > & _dstlist, uint16_t img_height, uint16_t img_width,uint16_t _filter_width, uint16_t _block_width, uint16_t _nms_radius,uint16_t _threshold, uint16_t val, uint32_t *ncorners) 15

16 Installed the Corner Harris Detector to zcu102's PL Base design is Xilinx's TRD 16

17 Duplicator Corner Harris Select Feature-point PL x-y list Main Thread Feature Detect Thread x-y list APU View Thread Capture Pipeline V4L2 API libdrm, etc Qt API, OpenGL API DPDMA Display Pipeline Display Port 17

18 Thank you for your attention! 18

借助 SDSoC 快速開發複雜的嵌入式應用

借助 SDSoC 快速開發複雜的嵌入式應用 May 2017 What Is C/C++ Development System-level Profiling SoC application-like programming Tools and IP for system-level profiling Specify C/C++ Functions for Acceleration Full System