Gesture Controller. Enoch Lam, Xihao Li, Rahat Mahmood

Size: px

Start display at page:

Download "Gesture Controller. Enoch Lam, Xihao Li, Rahat Mahmood"

Jasmin Carson
5 years ago
Views:

1 Gesture Controller Enoch Lam, Xihao Li, Rahat Mahmood Prof. Paul Chow Digital Systems Design, Spring 2011

2 Outline Overview System Design Original Design Final Design Changes Algorithms Implementation Existing IP Our IP Design Process Experiences Digital Systems Design, 2011 Gesture Controller 1

3 System Design Original Design Four subsystems: 1. Gesture Parsing 2. Gesture Recognition 3. System Controller 4. Remote Communication Digital Systems Design, 2011 Gesture Controller 2

4 System Design Final Design Video to RAM MPMC Cursor Finder 1 Video Decoder (Physical Device) MicroBlaze System Controller 3 UARTLite Three subsystems: IIC - Video Decoder Configurator Command Generator 1. Cursor Finder 2. Gesture Processer Video Capture PC Commands 3. System Controller Gesture Processor 2 * Rest is infrastructure Raw Test Gesture Buffer Scaled Test Gesture Bufer Reference Gesture Buffer VGA Output Debug Screen Digital Systems Design, 2011 Gesture Controller 3

5 System Design Changes Subsystem Original Final Gesture Parsing Gesture Recognition System Controller o Find cursor (shape + colour) o Create path (interpolation) o Crop and scale gesture o Write gesture buffer o Load references from mem o Compare test vs all refs o Pass best match to system controller o Initialize system o Read ready/done signals from modules, coordinate modules o Find cursor (shape + colour) o Pass to system controller via shared memory o Compare test vs the ref loaded into internal buffer o Pass each comparison result to system controller o Generate debug video signal o Initialize system o Generate ref gestures o Load ref gestures to buffers o Compare recognizer results o Send signals to remote PC Remote Comm. o Send commands to PC o Merged into sys controller Digital Systems Design, 2011 Gesture Controller 4

6 System Design Algorithms o Gesture Parsing o Look for pixels with key colour (defined by a range) o Gesture Recognition o Normalize input gesture to reference dimensions o Run it through neural network (impl. Gesture Processor) o Input: Test gesture pixels o Neuron parameters: Reference gestures o Output: Match score o Gesture Creation o Run image through Gaussian Filter for encoding probability Digital Systems Design, 2011 Gesture Controller 5

7 Implementation Existing IP IP Source Function Video to RAM Jeffrey Goeders Buffer raw video to RAM MPMC Xilinx Memory controller IIC Xilinx Low speed communication module, configures video decoder UartLite Xilinx Serial I/O between PC and board MicroBlaze Xilinx General purpose soft processor, used as system controller BRAM Blocks Xilinx Gesture buffers Hardware Divider Xilinx Gesture processor functionality PLB Xilinx System Buses Paddle Detector* Past Project, Virtual Pong, 2010 Modified to create cursor finder Digital Systems Design, 2011 Gesture Controller 6

8 Implementation Our IP IP, module Type Function System Controller Gesture Processor Cursor Finder* Command Gen. Software Hardware Hardware Software o Runs on Microblaze o Communicates results from cursor finder to gesture processor o Interprets gesture processor s results o Signals remote PC o Implements neural network as a pipelined pixel processor o Contains local gesture buffers o Normalizes test gesture o Compares against the ref gesture residing in local buffer o Generate VGA output displaying local buffers o Looks for cursor o Passes coordinate to system controller through shared memory o Runs on remote PC o Monitors serial (USB) port for signals from board o Interprets signals and executes task based on signal o Displays system feedback messages from board Digital Systems Design, 2011 Gesture Controller 7

9 Implementation Design Process Define interfaces Model in software Independently developed all modules Test in small projects Incremental integration Digital Systems Design, 2011 Gesture Controller 8

10 Experiences Software easier than hardware Use proper source control Develop individually, debug as group Communicate interfaces in writing, verbal communication is lossy Don t need all details to start coding, they change anyways Digital Systems Design, 2011 Gesture Controller 9

11 Questions? Digital Systems Design, 2011 Gesture Controller 10

Embedded Real-Time Video Processing System on FPGA

Embedded Real-Time Video Processing System on FPGA Yahia Said 1, Taoufik Saidani 1, Fethi Smach 2, Mohamed Atri 1, and Hichem Snoussi 3 1 Laboratory of Electronics and Microelectronics (EμE), Faculty of