Video Acquisition between USB 2.0 CMOS Camera and Embedded FPGA system

Transcription

1 Video Acquisition between USB 2.0 CMOS Camera and Embedded FPGA system A. Abdaoui, K. Gurram, M. Singh, A. Errandani, E. Châtelet, A. Doumar and T. Elfouly Computer science department, College of Ingineering, Qatar University, P. O. Box 2713, Doha, Qatar Institue Charles Delaunay (ICD), STMR, UMR CNRS 6279 Université de Technologie de Troyes (UTT) 12, rue Marie-Curie, 10010, Troyes Cedex, France Indian Institute of Technology, Rajasthan (Old) Residency Road Ratanada, Jodhpur , Rajasthan, India Abstract In this paper, we introduce the hardware implementation of video acquisition in a sensor node of wireless sensor network with the help of USB 2.0 interface. The USB 2.0 video acquisition is based on the CY7C67300 controller and the DCC1545M image sensor. In this paper, we detail the hardware architecture and the application program design in a sensor node using field-programmable gate array (FPGA) board and USB 2.0 interface. The CY7C67300 controller is the suitable choice for FPGA Virtex 4 and 5 based USB peripherals. A simple interface module capable of transferring data rates above 400 Mbits/s is implemented to communicate with the CY7C67300 controller. In order to use the developed module, in Xilinx embedded design, we provide a custom peripheral which includes the CY interface as its core and additional logic for the connection to the external peripheral controller (EPC) and then the processor local bus (). Index Terms Embedded systems, FPGA, video acquisition, VHDL, wireless sensor network. PowerPC 405 Processor Fig. 1. DDR G P I O External Peripheral Cypress CY7C67300 USB Interrupt Bus Architecture of the Sensor node. U A R T I. INTRODUCTION Nowadays, wireless sensor network (WSN) is in great growth regarding its potential application such as detecting the relevant quantities, monitoring, assessing and evaluating information and performing decision-making and alarm functions. Hardware implementation of a sensor node, with respect to the small size requirements, low power consumption and flexible programmable treatment is considered as an important challenge. Hardware implementation, either on applicationspecific integrated circuits (ASICs) or on field programmable gate arrays (FPGAs) platform, has definitely its place especially with low-power wireless devices running ZigBee [1]. However, the use of a reconfigurable FPGA as a flexible platform to implement a sensor node is considered as the cheapest and easiest solution [2][3]. The FPGA platform based on the intellectuel property (IP) hard core implementation and embedded processing is of great importance. For these purposes, Xilinx has developed several FPGA embedded boards: Virtex 2 Pro, Virtex 4 FXT and Virtex-5 FXT families. Considering the new features of the hardware gates with embedded processor, researchers and engineers are interested on the use of USB 2.0 interfacing to connect a physical sensor with the main board of the sensor node. To the best of our knowledge, there is no work that gives the architecture of the video acquisition based on the FPGA virtex 5 with embedded processor and the USB 2.0 Cypress controller CY7C67300 for sensor node. In this paper, we present the hardware implementation and the software setup of an intelligent sensor node targeted for the video acquisition. In the sensor node, a local data processing is applied and only a partial information or decision is submitted to another centralized system of the sensor network. Finally, we use a USB protocol analyzer to record the packets exchanged between the sensor node and the camera. The remainder of the paper is organized as follows. In Section II, we present the architecture of the intelligent video sensor. An overview of the image sensor DCC1545M, employed to capture the image, is given in Section III. The next Section is dedicated to the main contribution of this paper: /11/$ IEEE

2 XPS USB2 Device Core USB 2.0 PHY ULPI USB 2.0 interface SIE Port A DPRAM Port B DMA U S B B u s Master 32 bit Register and Control Logic Slave Fig. 2. XPS USB2 Device with and ULPI interfaces. implementation of the video acquisition using the USB 2.0 protocol based on the Cypress CY7C67300 and the CMOS digital image sensor DCC1545M. Finally, we conclude the paper in Section V. II. ARCHITECTURE OF THE SENSOR INTERFACE A sensor node is a part of a wireless sensor network where each node is capable of transmitting the image or the decision regarding the captured image. For example, the position of a target, personal identification, identification of a violent environment, etc. In this architecture, we propose the use of an image sensor based on the MT9M001 chip managed by a local micro-controller and USB2.0 interface for the connection with the FPGA Board. In the following, we apply the USB2.0 protocol with the FPGA board without any operating system with the help of embedded design kit EDK. In virtex FXT board, there exist two ways to employ USB device. The first one is based on the use of the IP XPS universal serial bus 2.0 and the other one interfaces the external peripheral controller to the USB device and the. A. IP universal bus protocol solution The USB 2.0 protocol multiplexes several devices over a single in half-duplex and serial bus. The bus can run at 480 Mbps or at 12 Mbit/s and is designed to be plug-and-play. The host controls the bus and sends tokens to each device for a specific action. Each device has an address on the bus and has one or more endpoints which are sources or sinks of data. The XPS USB 2.0 Device has eight endpoints - one control endpoint (endpoint 0) and seven user endpoints. Endpoint 0 of the USB device has different requirements than the seven user endpoints. Endpoint 0 concerns the control transactions only and start with an 8-byte setup packet and then followed by the data packets. The setup packet is stored in a dedicated location in the dual port random access memory (DPRAM). When a setup packet is received, the SETUP bit of the interrupt status register (ISR) is set. s are a maximum of 64 bytes. These data packets are stored in a single bidirectional data buffer set up by the configuration memory of endpoint 0. When a data packet is transmitted or received successfully, the data buffer free and data buffer ready bits of the interrupt status register (ISR) are set respectively. The seven user endpoints of the USB 2.0 device are configured as bulk, interrupt or isochronous. In addition, endpoints can be configured as INPUT (to the host) or OUTPUT (from the host). Each of these endpoints has 2 ping-pong buffers of the same size for endpoint data. The user endpoints data buffers are unidirectional, and are configured by the Endpoint Configuration and Status register of the respective endpoint. The XPS USB2.0 device core with the is shown in (Fig. 2) B. External peripheral controller (EPC) solution The EPC is used to control peripherals that are connected externally to Xilinx FPGAs. The most commonly used external devices are LAN controllers, USB controllers, and IEEE 1394 (Fire Wire) controllers. The EPC is an slave only device. It does not support any DMA operations from the external devices. The EPC supports both multiplexed and nonmultiplexed address and data buses where the bus width can be 8, 16, or 32 bits. The standalone software application provided with this reference system is executed from the cacheable region of the external DDR memory. The Cypress CY7C67300 USB controller is interfaced to the EPC through the Host Peripheral (HPI). The reference system has the PowerPC 405 processor with the caches enabled to use the instruction cache (I-cache) and the data cache (D-cache) from the external DDR memory. The UART Lite core with interrupts, the Interrupt ( INTC) and GPIO cores are also used in the reference system. III. VIDEO SENSOR DCC 1545M Fig. 3 shows the architecture of the sensor DCC1545M with its USB 2.0 inner interface. In the following, we describe the signal exchanges data between the image sensor MT9M001 and the micro-controller inside the Thorlabs camera DCC 1545M.

3 Sensor Device Descriptors image Sensor EEPROM SDATA SCLK USB Micro EEPROM CLK (PLL) USB bus FPGA Board SETUP Transaction Setup packet IN Transaction Thorlabs Camera trigger programable I/O IN packet Fig. 3. DCC 1545M sensor architecture with USB2.0 interface. OUT Transaction A. Serial Bus Description the registers are written to and read from the MT9M001 through the two-wire serial interface bus. The sensor consists of two-wire slave serial interfaces and is controlled by the serial clock (SCLK), which is driven by the master serial interface. Data is transferred into and out of the MT9M001 through the serial data (SDATA) line. The SDATA line is pulledupto3.3voff-chipbya1.5kωresistor. Either the slave or master device can pull the SDATA line down-the serial interface protocol determines which device is allowed to pull the SDATA line down at any given time. B. Data acquisition inside the camera 1) A typical read or write sequence begins by the master sending a start bit. After the start bit, the master sends the slave device s eight-bit address. The last bit of the address determines if the request will be a read or a write, where a "0" indicates a write and a "1" indicates a read. The slave device acknowledges its address by sending an acknowledge bit back to the master. 2) If the request was a write, the master then transfers the 8-bit register address to which a write should take place. The slave sends an acknowledge bit to indicate that the register address has been received. The master then transfers the data 8-bits at a time, with the slave sending an acknowledge bit after each 8-bits. The MT9M001 uses 16-bit data for its internal registers, thus requiring two 8-bit transfers to write to one register. After 16 bits are transferred, the register address is automatically incremented, so that the next 16 bits are written to the next register address. The master stops writing by sending a start or stop bit. 3) A typical read sequence is executed as follows. First the master sends the write-mode slave address and 8-bit register address, just as in the write request. The master then sends a start bit and the read-mode slave address. The master then clocks out the register data 8-bits at a time. The master sends an acknowledge bit after each 8- Fig. 4. OUT packet USB2.0 Protocol implemented in the XC5VFX70T FPGA Board. bit transfer. The register address is auto-incremented after every 16 bits is transferred. The data transfer is stopped when the master sends a no-acknowledge bit. The main functions of the USB 2.0 protocol are summarized in Fig. 4. IV. USB 2.0 INTERFACING USING THE CYPRESS CY7C67300 AND THE DCC1545M This design is implemented on the FPGA board which mounts a Xilinx Virtex 5 XC5VFX70T device connected to the Cypress CY7C67300 USB controller. We have also used the existent CMOS DCC1545M camera based on the USB interfacing and on the chip MT9M001 (image sensor). The camera is capable of transferring digitized video over USB to the FPGA board. the overall interfacing between the USB 2.0 controller (CY7C67300) and the FPGA ship is detailed in Fig. 6. Analyzing the connections pin by pin, the 16 data bits (USB D0 to USB D15) of the FPGA chip are connected to data pins (GPIO D0 to GPIO D15) of the Cypress CY7C The other pins are used to control and to interface the USB controller with the physical connections of the four USB 2.0 ports. EZ-Host (CY7C67300) is Cypress Semiconductor s first full speed, low cost multi-port host/peripheral controller. EZ- Host is designed to easily interface to most high-performance CPUs to add USB host functionality. EZ-Host has its own 16-bit RISC processor to act as a co processor or operate in standalone mode. EZ-Host also has a programmable I/O interface block allowing a wide range of interface options. Fig. 5 shows the block diagram of the usb control Cypress Cy7C67300 used in the Virtex 5 FXT FPGA board. Fig. 7 gives a photo of the FPGA Virtex5 Board with the USB camera.

4 Fig. 5. Block diagram of the Cypress CY7C V. CONCLUSION Throughout this paper, a new architecture of an intelligent image sensor node is detailed. The feature of the proposed sensor is the reconfigurability. In this paper, we present a stand-alone hardware implementation of the image acquisition in a sensor node using the USB 2.0 interface. Since this circuit uses the USB 2.0 interface, we can transmit video stream to an FPGA board at high speed. Finally, if we need specific image processing algorithms, we can easily implement them through hardware description language (HDL) and C ANSI on the embedded processor located inside the FPGA. USB Periph. D N USB peri D D P CY7C67300 USB Host USB host D N D P GPIO0 D0 GPIO0 D1 GPIO0 D2 GPIO0 D3 GPIO0 D4 GPIO0 D5 GPIO0 D6 GPIO0 D7 RESET USB D0 USB D1 USB D2 USB D3 USB D4 USB D5 USB D6 USB D7 USB RESET Virtex 5 FPGA XC5VFX70T REFERENCES [1] P. Baronti, P. Pillai, V. Chook, S. Chessa, A. Gotta, and Y. Hu, Wireless sensor networks: A survey on the state of the art and the and ZigBee standards, Computer Communications, vol. 30, no. 7, pp , [2] G. Chalivendra, R. Srinivasan, and N. Murthy, FPGA based reconfigurable wireless sensor network protocol, in International Conference on Electronic Design, ICED 2008, 2008, pp [3] J. Patra, H. Lee, P. Meher, and E. Ang, Field Programmable Gate Array Implementation of a Neural Network-based Intelligent Sensor System, in 9th International Conference on Control, Automation, Robotics and Vision, ICARCV 06, 2006, pp [4] S. Dulman and P. Havinga, Introduction to Wireless Sensor Networks, 2nd ed., ser. Industrial information technology series, Boca Raton, Florida, USA, 2009, vol. 2, ch. 3, pp [5] Y. Hai-qiang, X. Hong-hai, and L. Juan, Design of a USB in the Data Acquisition System Based on CY7C68001, Mechanical & Electrical Engineering Technology, [6] C. YUNDONG, J. JIE, and Z. GUANGJUN, High speed CMOS image acquisition and transmission system based on USB, in Proceedings of SPIE, the International Society for Optical Engineering. Society of Photo-Optical Instrumentation Engineers, [7] W. Gang, Design of image acquisition system based on USB2. 0, Foretgn Electronig Measurement Technology, GPIO0 D8 GPIO0 D9 GPIO0 D10 GPIO0 D11 GPIO0 D12 GPIO0 D13 GPIO0 D14 GPIO0 D15 USB D8 USB D9 USB D10 USB D11 USB D12 USB D13 USB D14 USB D15 Fig. 6. Hardware connections of the USB controler with FPGA Virtex 5 Chip. [8] N. Bartzoudis and K. McDonald-Maier, An embedded sensor validation system for adaptive condition monitoring of a wind farms, pp. 5 8, 2007.

5 Thorlab Image Sensor Fig. 7. A photo of the FPGA Virtex5 Board with the USB camera