An Analytic Model for Embedded Machine Vision: Architecture and Performance Exploration

Transcription

1 419 An Analytic Model or Embedded Machine Vision: Architecture and Perormance Exploration Chan Kit Wai, Prahlad Vadakkepat, Tan Kok Kiong Department o Electrical and Computer Engineering, 4 Engineering Drive 3, National University o Singapore, Singapore {g , prahlad}@nus.edu.sg Abstract This paper represents a model to analyze the perormance o an embedded machine vision system. Such an application is computationally expensive and oten is a challenging problem in the ield o embedded systems. A model is derived to analyze the perormance. It predicts system perormance with minimal eorts and costs over a wide range o processor architectures. Speciically, image segmentation is evaluated with the microprocessor technology. In the same way, a custom logic is designed with the help o the model. Keywords: Analytic Model, Embedded Machine vision, Custom vision architecture 1 Introduction Image processing is oten associated with huge amount o data processing [?]. Such computational requirements oten pose a challenging problem in the ield o embedded systems. Most o the embedded systems have limitations o computational power, low data transer rate, very tight memory budget, size constraints and most importantly cost. Additionally, low energy requirement has become an important actor in recent years [?] [?]. For battery operated devices, it is preerable to have low power consumption. Such concern is important in order to reduce the recharging requency. The memory constraint in embedded technology also plays an major obstacle. Vision algorithms demand high memory requirements compared to other applications, on the contrary, most embedded systems have tight memory budget. Such limitations orbid the implementation o many vision applications. Lastly, the speed o executing an image processing task must be comparable to the speed o acquiring the image (i.e. rame image rate) [?]. As such, the limitation o computing power in embedded systems inhibit the task to be completed on time. Nevertheless, many o the vision systems have careully tackled to meet the computational time requirements. Perhaps reducing the image resolution is one the the easiest methods to handle the real time constraints. Examples o such systems are ound in eyebot [?], CMUCam [?] and Khepera vision turret [?]. Other methods include running the algorithms on a high speed computer, exploring computationally eicient algorithms and exploring various hardware architectures. While several articles in the literature discuss about complex vision algorithms, this paper ocuses on the importance o the vision system s perormance. However it does not discuss the compiler techniques or memory and computational optimization. Rather, this paper introduces a model to estimate the perormance requirements or a given task and speciications. In literature, there are three basic techniques or perormance evaluation; namely measurement, simulation and analytic modelling [?] [?]. An analytic model is oten used to predict perormance. It can evaluate the perormance with minimal eorts and costs over a wide range o choices in the system parameters and conigurations [?]. For various processor architectures, the key resources and workload requirements can be analytically modelled with suicient realism to provide insight into the bottlenecks and key parameters aecting the system perormance [?]. Nevertheless, the model ails i it is required to model the vision system in great details [?]. This paper represents a model to analyze the perormance o machine vision in embedded environments. It also discusses about the limitations o the general purpose processors and the possibilities o alternative processors. This paper is organized as ollows: Section 2 describes the analytic model or image processing. Section 3 discusses image segmentation using microprocessor. Cus-

2 420 tom architecture is provided in Section 4 and Section 5 concludes the work. 2 Analytic Model or Image Processing The data processing is represented using a pyramid architecture as shown in Figure??. The bottom level o the pyramid represents the data volume to be processed. The top level o the pyramid represents abstract inormation derived rom the image. The bottom level comprises o raw pixels acquired rom source image. Intermediate levels are typically pre-processing, segmentation, eature extraction and classiication. The top level produces the abstract data to the rest o the system. Level 1 Level 2 Level 4 Level Perormance Parameters The speciications generally reer to the response time, rame rate, image resolution, pixel transer rate and the computational task. The perormance requirements o a system is reerred to the number o processing units, the memory size and clock speed to meet the speciications provided. The deinitions used in the model to evaluate the system perormance parameters are as ollows: λ = Arrival rate o pixels r = response time w = wait time s = service time s p = service timeperpixel n = number o jobs in system n q = number o jobs in queue n s = number o jobs in service i j = population size(inpixels) CPI = cycles per instruction Queueing theory plays a very important role in analytical modelling [?]. From the Little s Law in queueing theory [?], equation (??) to (??) are used as the undamental ormulae to calculate the required vision system requirements. These requirements include the number o processors, processor s computing speed, instruction sets, arithmetic units (ALU) and memory size o the system. Level 0 Figure 1: Data processing at dierent levels The lowest level vision task oten consumes the most computing resources. Low-level task consists o pixelbased transormations such as iltering, edge detection and segmentation processes. These tasks are characterized by a large amount o data (pixels), small neighbourhood operators and relatively simple structure operations (e.g multiply, add or compare) [?]. On the other hand, high level tasks are more dynamic in nature. They are more decision oriented and do not have a repetitive execution o a set o algorithm. Hence, the model discussed in this paper is applicable speciically or low-level repetitive tasks. An analytic model is derived to provide a mathematical description o the vision system. Such approach is considered being ar less time-consuming and more lexible compared to simulation based methods. The model can be used to describe a dierent level o abstraction. The model described in Section 2.2 and 3.2 is used to estimate the processor clock speed requirements as well as exploring custom architecture or such application. The mathematical model helps to represent computing resources and the speed o the hardware. Such model allows a simple and ast calculation o dierent processor s response time. Population size (n * m) pixels Queue Figure 2: Queue model o vision system 2.2 Queuing Theory Service Unit Service Unit Service Unit The system can be modelled using the queueing analysis as illustrated in igure??. The population size, queue size and service centers reer respectively to the number o pixels, memory size, processing units in a vision system. n = λr, (1) n q = λw, (2) n s = λs, (3) where λ = a T, (4)

3 421 r = w+s, n = n q + n s, (5) 3 Image Segmentation using Microprocessor where a is the number o jobs arrived, in time T. The basic approach is to identiy whether the system is able to handle a given load, in another words, the stability condition is satiied (??). λ < mµ, (6) where m is the number o service units and µ = 1/s is the service rate o each service unit. I the number o jobs in a system grows continuously and becomes ininite, the system is said to be unstable. For stability, λ (mean arrival rate) should be less than mµ (mean service rate) [?]. To ind the stability condition, λ and s are required. As a result, s (service time or the image) and s p (service time per pixel) can be calculated as shown in equations (??) and (??) respectively. s = s p (i j), (7) s p = n ipp n cpi t clk, g=g max s p = g=1 [n ipp (g) n cpi (g)] t clk, (8) where n ipp is the number o instructions per pixel and n cpi is the number o cycles per instruction. Both these parameters are a unction o g, a group o instructions set with the same CPI. An example o such grouping is shown in Table 1. Table 1: Instruction grouping g instruction set CPI 1 MOVF 1 SUBWF 1 2 BTFSC 2 GOTO 2 The model decomposes the system into algorithms, instruction sets and instruction cycles. Instructions with dierent number o CPIs are grouped together. Thus, the total number o cycles in the program can be easily calculated. Finally, with all the input parameters, it is possible to estimate the required processing power, in particular, the processing requency. However, this model does not consider several computational overheads, like initialization routine, loops and error checking, etc. The ollowing sections discuss the usage o such model or image segmentation in two (2) dierent types o processing architectures; namely microprocessor and custom architecture. The segmentation process partitions an image into meaningul regions [?]. The nature o this process involves scanning o each pixels. A simple experimental setup or color segmentation is tested in Visual C/C++ environment. The program reads in the raw image data rom a Portable Pixel Map (PPM) ile, converts the Red, Green, Blue (RGB) primaries to the Hue, Saturation, Value (HSV) color space, perorms color segmentation and converts back to RGB ormat. Finally, it writes the image into a destination PPM ile or viewing. Figure?? illustrates the process. Acquire image rom ile Transorm image ormat rom RGB-> HSV Image Segmentation process Transorm image ormat rom HSV-> RGB Write image to ile Figure 3: Experimental setup 3.1 An Algorithmic Approach For instance, a real-time vision system with a resolution o 640x480 is required to be processed at a rate o 30 ps (rames per second). The analytical model is used to estimate the perormance o the color segmentation process. A reerence color is selected and the distance between the two color points p i = [H i,s i,v i ] and p re = [H re,s re,v re ] is given as, { Hi H d H = re i H i H re < 2π 2π H i H re otherwise d S = S i S re, d V = V i V re. (9) The segmentation process is as ollows,,

4 422 1 i (d H < d Hre and d S < d Sre w = n q s p. (14) I(i, j) = and d V < d Vre ), (10) 0 otherwise Substituting equations (??) and (??) into equation (??) gives, where d Hre, d Sre and d Vre are the acceptable sensitivity distances. From Equation (??), the image is seg- s p = r n qs p r, s p =. (15) i j i j+ n q mented into a binary image o 1 s and 0 s. The microprocessor technology is oten used or image segmentation process. It requires several load, store and branch operation to perorm data manipulation. Figure?? shows the implementations in C language and the equivalent assembly instruction code. The results obtained are shown in Figure??. C program or ( i =0 ; i < imax ; i++ ) or ( j = 0 ; j < jmax ; j++ ) { i ( ( H1 < h [ i ] [ j ] < H2 ) && ( S1 < s [ i ] [ j ] < S2 ) && ( V1 < v [ i ] [ j ] < V2) ) { v [ i ] [ j ] = 0 ; // black } else { v [ i ] [ j ] = 100; s [ i ] [ j ] = 0; } } Assembly language mov subw btsc goto mov mov mov subw btsc goto subw btsc goto Figure 4: Assembly code representation o C program 3.2 Estimating the Required Processing Speed In this example, it is desired to estimate the required processing speed. The desired response time is given to be 1/30 ps = ms and the image resolution is 640x480 pixels. Equations (??) to (??) are used as the model or calculations. As such, rom Figure??, n ipp (g = 1) is ound to be 17 with one CPI, and n ipp (g = 2) is 8, taking the assumption o branchalways-taken with a CPI o two. Subsequently, two extreme cases o n q1 =0 (0 pixels in queues) and n q2 =640x480 (max no. o pixels in queue) are used to determine the t clk. For both cases, t clk1 (n q1 ) is 3.28ns (304.8 Mhz) and t clk2 (n q2 ) is 1.64ns ( Mhz). From the calculations, it is noted that, a simple image segmentation operation requires a very high clock requency. The perormance o such a system greatly depends on the computing speed o the processor. The segmentation algorithm does not actually map to appropriate hardware resource to achieve eiciency. Most o computing time is spent on overheads instructions rather than or the actual processing o data. Hence, it is reported that most computationally complex applications spend 90% o their execution time on only 10% o their code [?]. 4 Customized Architecture or Image Segmentation (a) (b) Figure 5: (a) original image (b) segmented image using HSV color space From equation (??), the processing speed or perorming the segmentation process is estimated. t clk = where, g=g max g=1 s p [n ipp (g) n cpi (g)], (11) s p = s i j, (12) s = r w, (13) The mathematical model helps to increase response time by choosing the right hardware conigurations. The parameters corresponding to the hardware resources are identiied to improve system perormance. For instance, rom equation (??), m, the number o service units represents the number o processors servicing the jobs. I the system does not meet the condition speciied in (??), m is increased to bring the system rom an unstable condition to a stable condition. This relects in the actual implementation o increasing the number o processors in the system. Another example is observed rom equation (??). The parameters n ipp and n cpi can be reduced by selecting an appropriate instruction-set-architecture (ISA). A suitable processor ISA should contain an instruction to compare two registers and branch to a speciic location within a single cycle. Consequently, a lower processor clock speed can be used as a result o reducing n ipp.

5 Achieving One Pixel per Cycle To achieve a very eicient system, the sotware algorithm and hardware architecture must match well. This can be achieved or algorithms with high degree o regularity that are identiied to exploit parallelism. With the model as a reerence, these operations are mapped into custom unctional units to achieve a higher perormance compared to the ixed instruction set architecture. In ideal case, one can set g=g max[n g=1 ipp (g) n cpi (g)] = 1. Eectively, rom (??), t clk1 (n q1 )=108ns; which shows the reduction o clock requency by 96.95%. Such perormance with low clock requency is achieved by customizing a dedicated hardware resource. The customized architecture allows direct computation instead o conventional load store operations. Such structure is able to compute a pixel in a single instruction within a single clock cycle. To explore urther, parallel processing o multiple pixels is also possible, considering that the pixels are transerred parallelly. Otherwise, queueing o pixels is necessary, which leads to undesirable idle time. Hence, sequential processing o a single pixel provides a better solution. Figure 8: Timing diagram o segmentation process an output pixel being generated at the ollowing rising edge. ppm input ile testbench ppm output ile verilog code Figure 6: Simulation conigurations Rin<7:0> Rout<7:0> Gin<7:0> Gout<7:0> Bin<7:0> Bout<7:0> Tclk dataready Figure 9: RTL schematic or segmentation The verilog code is synthesized and the RTL schematic is shown in Figure??. The resulted schematic is simple, consisting o just a ew comparators and lip-lops. It is noted that the architecture does not really need complex logic circuits to achieve the desired perormance. Figure 7: Block diagram o image segmentation processor The customized hardware architecture is described using the Hardware Description Language (HDL); speciically verilog HDL. The simulation conigurations are shown in Figure??. Model Sim (a verilog code simulator program) is used together with the test bench, the verilog code, and the ppm image iles to simulate the vision system. The block diagram and the RTL schematic o the design are shown in Figures?? and?? respectively. It is noted that the RGB color space domain is used instead o the HSV or reasons o simplicity. The simulation results are obtained rom the waveorm viewer in ModelSim. Figure?? shows the timing diagram o a pixel being received at t clk s rising edge and (a) (b) Figure 10: (a) original image (b) segmented image using RGB color space The image is segmented into binary image, where logic 1 is represented by RGB = (0x, 0x, 0x) and logic 0 is represented by RGB = (0x01, 0x01, 0x01). The RGB values are written into a ppm image ile and the results are shown in Figure??. There are several ways to realize the implementation o custom logic. However, the most common technolo-

6 424 gies are to abricate it in ASIC (Application Speciic IC) or FPGAs (Field Programmable Gate Array). Particularly, FPGAs oer certain advantages over ASIC. It provides the beneits o customized hardware architecture and at the same time allowing dynamic reprogrammability. It is an important characteristic which allows to meet the changing requirements. 5 Conclusion The image processing analytic model is derived to estimate the various perormance parameters associated with the vision system. Such an approach is considered being ar less time-consuming and more lexible compared to simulation based approaches. The derived model allows system designers to estimate the system responses o given speciications. It helps to compare dierent processor architectures without actual implementation. For instance, the estimation o the clock requency, response time and delay time is easier. It can be extended to calculate the amount o energy consumed. The model provides the design space exploration to achieve the desired perormance. To reduce the clock requency, it is needed to reduce the number o instructions. Such hints assist in the design o the custom architecture. The results show that the custom architecture is able to achieve a reduction o 96.95% in clock requency with respect to the ixed instruction set architecture. In conclusion, the possibilities o such an architecture can be realized in a reconigurable abric o FPGA. Reerences [1] D. Crookes. Architectures or high perormance image processing: The uture. In Journal o Systems Architecture, volume 45, pages , [2] G.Stitt, B.Grattan, J.Villarreal, and Frank Vahid. Using on-chip conigurable logic to reduce embedded system sotware energy. In IEEE Symposium on Field-Programmable Custom Computing Machines, volume 10, pages , [5] Thomas Braunl. Improv amd eyebot real-time vision on-board mobile robots. In IEEE, Mechatronics and Machine Vision in Practice, volume 4, pages , [6] A. Rowe, C. Rosenberg, and I. Nourbakhsh. A low cost embedded color vision system. In IROS 2002 conerence, [7] Khepera vision turret. Web page. visited on June [8] K.Kant. Introduction to Computer System Perormance Evaluation. Mc Graw-Hill, Singapore, [9] David J. Lilja. Measuring Computer Perormance: A practitioner s guide. Cambridge University Press, United Kingdom, [10] Hisashi Kobayashi. Modeling and Analysis: An introduction to System Perormance Evaluation Methodology. Addison-Wesley, Philippines, [11] P.Heidelberger and S.S.Lavenberg. Computer perormance evaluation methodology. In IEEE Transactions on Computers, volume C-33, pages , [12] N. K. Ratha and A. K. Jain. Computer vision algorithms on reconigurable logic arrays. In IEEE Transactions, Parallel and Distributed Systems, volume 10, pages 29 43, [13] Raj Jain. The Art o Computer Systems Perormance Analysis: Techniques or Experimental Design, Measurement, Simulation, and Modeling. Wiley- Interscience, New York, [14] G. J. Awcock and R. Thomas. Applied Image Processing Book. McGraw-Hill, USA, [15] J. L. Hennessy and D. A. Patterson. Computer Architecture: A quantitative approach. Morgan Kaumann, Cali., [3] T.Simunic, L.Benini, and G.D.Micheli. Energyeicient design o battery-powered embedded systems. In IEEE Transactions on Very Large Scale Integration (VLSI) Systems, volume 9, [4] R. T. Chin and C. R. Dyer. Model-based recognition in robot vision. In ACM COMPUTING SURVEYS, volume 18, pages , 1986.