Drawing Glove. Project Proposal. Team 34 Joseph Xiong jcxiong2 Willie Zeng wzeng4

Transcription

1 Drawing Glove Project Proposal Team 34 Xiong jcxiong2 Zeng wzeng4 ECE445 TA: Henry Duwe September 14, 2016

2 Table of Contents 1.0 Introduction 1.1 Statement of Purpose 1.2 Objectives Goals and Benefits Functions and Features 2.0 Design 2.1 Block Diagram 2.2 Block Description User Interface Signaling Processing Output Power Supply 3.0 Requirements and Verification 3.1 Requirements and Verification Table 3.2 Tolerance Analysis 4.0 Cost and Scheduling 4.1 Cost Analysis Labor Parts Grand Total 4.2 Schedule

3 1.0 Introduction 1.1 Statement of Purpose This project provides a cheaper alternative for artists to draw digitally. Instead of using a Xbox Kinect or tablet, this project allows the user to draw accurately onto a monitor even though the user is standing 10 feet away. The user can change brush sizes and paint different colors all through the wireless use of this glove. Having a varied wireless toolset for artists is both useful and convenient, and this project is more accessible to the public since it is a cheaper alternative. 1.2 Objectives Goals and Benefits Allows artists to draw digitally in an affordable manner Intuitive to set up and draw Users do not have to load an application or buy additional parts Large canvas Functions and Features Allows user to draw wirelessly to a monitor Responds to the motion of the finger as the input The rest of the glove is used to control drawing options Accurately outputs the motion of the glove to a monitor Camera sees the motion of the glove as the user draws in the air Includes ability to draw with different colors and brush sizes Easy to set up portable camera Available for different monitors

4 2.0 Design 2.1 Block Diagram 2.2 Block Description User Interface The user interface module encompasses all of the user aspects of the glove. It includes the controls for the user that allows for the different drawing features. Besides the power supply module, this module also connects with the signaling module, specifically the IR LEDs and the RGB LED. We will create a circuit for the controls to regulate these LED outputs On/Off Control The first control will be a push button to control if the IR LEDs will be on or off. The lights will be defaulted to off and will turn on as long as the button is on as well. This

5 would save some power in the module since the user would then only draw if the button is pressed. The button will be placed under the pointer finger (palm side) of the glove so when the user positions their hand in the way they hold a pen, their thumb will be ready to press the on/off button RGB LED Control The next control is for the color of the RGB LED through three sliders or knobs. Each slider will be connected to a potentiometer that is wired to a pin of the LED and by changing the values of the slider, the color of the LED would change. This control will be located on the back of the hand for an easily accessible location IR LED Control This last control is for controlling the brush size while drawing. This is controlled through 3 IR LEDs. The ON/OFF pattern of the IR LEDs will determine which brush size to use. For example, If LED 1 AND LED 3 are on, but LED 2 is OFF the camera should detect this pattern from the LED placements and use brush size 4. If LED 1 is ON and LED 2 and LED 3 are OFF, use brush size 3, etc. There will be two buttons, one to increase the brush size and one to decrease the brush size, which will change the LED ON/OFF state accordingly Signaling This module is in control of the communication between the glove and the camera. There is no data communication between the two but is instead through the lights of the glove that the camera sees. The glove will control the lights on it which will be seen by the camera which is part of the processing module to decipher the data that the camera sends to it IR LEDs These infrared LEDs will be the key indicator that the camera will see to locate the position of the glove. The 3 IR LEDs will be placed in a triangular pattern at approximately 1 cm apart on the pointer finger of the glove. The ON/OFF state of these LEDs will be detected by the camera, which will determine the brush size RGB LED This RGB LED is how the user chooses the color of the drawing tool being used. The LED is controlled through the user interface module and choose from the whole spectrum of colors. This LED will be placed next to the IR light on the glove.

6 2.2.3 Processing This module controls the communication between the camera and the output. The data from the camera will be connected to the microcontroller to decide what to do with the data. The camera will be connected directly to the microcontroller through a ribbon cable and the microcontroller will connect to the monitor through an HDMI cable Microcontroller This microcontroller will most likely be a Raspberry Pi which will have an input from the camera and an output to the monitor. The microcontroller will take information about the location of the IR LED, color of the RGB LED to decide the color, brush size, and location that the user wants to draw and send the corresponding data to the monitor. The Raspberry Pi Version 3 with its quad core and 1 GB RAM will be sufficient enough to perform image processing at a low resolution (640x480). Memory may be an issue, therefore we will have a MicroSD card (32GB) as an add on Software On the Raspberry Pi Model B, in the software side, we will be utilizing the software OpenCV and using languages: Python and bash(shell) for image processing. The syncing of Camera to Raspberry Pi would be done in shell, while the actual image processing portion will be done on OpenCV in Python. We will use the Picamera package for Raspberry Pi. Image processing will allow us to detect the IR LED and more additional features; The microcontroller will detect if a certain pattern from the IR lights. The brush size will change according to the pattern. For example, if LED 1 and LED 3 are ON and LED 2 is OFF, the brush size will be size 3. The color of the RGB LED will also be detected, and according to the color, the brush will also change. The Camera Search or detection feature will be done by implementing many images of the IR LEDs pattern, which is necessary for Machine learning. Much like facial detection, this will be done by processing each frame of the input to match the LED pattern. And although FPS will take a blow, I m confident the input will remain at a steady 30 fps Camera The camera is the source of the motion detection. It will use image processing to process the location of the pointer IR LED as well as the pattern of the IR LEDs to find the location of the brush and the size of the brush. The color value of the RGB LED will determine the color of the brush. The location of the pointer IR LED will be in coordinates that will be saved as a x,y point. Look in Software to see how certain

7 features are done, such as search Monitor The monitor could actually be any monitor meaning that the project does not need to be constrained to be where a specific monitor is but can be used anywhere that can display or project the information from the processing module Power Supply The power supply module provides power to each of the other components of this project Power Source There will be two power sources in this project. One will be for the glove and the other is for the rest of the system. The glove will use a 9V battery to power the IR LED and RGB LED along with the controls. LED s will draw power from the 9v battery on the glove by using 4 resistors that are in series with 4 LED s, each resistor for one LED, and this resistor & LED pair will be parallel with other resistor & LED pairs. There will be a voltage regulator responsible in providing a fixed 5 voltage as the source may decay. The microcontroller and the camera, an add on, will be powered by a micro usb to battery/wall outlet at 5.1V. At 2.5A.

8 3.0 Requirements and Verification 3.1 Requirements and Verification Table Requirement Processing 1. Is able to receive input from the camera with delay of less than 1000ms 2. Is able to output data to a monitor through HDMI/VGA with resolution of at least 640 x 480 (pixels) 3. Camera records at least 30 fps 4. Provide data x,y points to the Software/Microcontroller in the location of the pointer IR LED 5. Detect alternating patterns from the IR LEDs 6. Must display the corresponding RGB LED Colors to the Monitor as it is configured on the glove. 7. Monitor is able to receive inputs of HDMI 8. SOFTWARE: Check if the video/image processing files, microcontroller operating system, and camera input data are not taking up more than 16GB (the size of the intended SD Card) Verification 1. By using software compatible with Raspberry Pi (IDLE 3) check if camera is getting any data from 5 inches away. Then check if data is from LED s by moving LED s at a constant distance of 5 inches away, moving 5 inches farther when verified and timing the delay of movement. 2. Test to see if the monitor displays the motion of the glove. 3. (Assuming that the software has image processing) Run test trials for 30 minutes of glove/ir sensor movement and see if there are any visible signs of lag in the movement of the user to the output of the monitor. At 30 fps, there is no visible lag. 4. Code in the software to display the xy points of where the IR sensor is. Continuously place the IR sensor at the same place for 3 trials and see if the data points are the same. Then move it so that the x value is decreasing and then y value is decreasing. The max value for these points is 640 x Continuously move the IR LED through the length of the user s wingspan and repeatedly change the pattern of LED s (LED1 on, LED2 on, LED3 off, etc.) to change the brush, every input should be. Every brush size can be drawn on the monitor 6. Test different combinations of red,

9 green, and blue. The tests will include only red, only green, only blue, each combination of 2 colors, all three colors, and no color. Then the displayed color and the RGB LED color will be compared by taking a picture of the two next to each other and using computer software to check if the color values are the same. 7. Check to see if one of these ports are available to connect. It should display the Raspberry Pi functions at bootup. 8. At the end of 5, 10 minute trials, check the size of the folder containing the images used for image processing. Check the size of each individual PNG picture which should be no more than 2MB. Power Supply (Glove) 1. Must be able to support 9v +/ 0.5V and 200mA +/ 100mA to accommodate for 3 IR LEDs, an RGB LED ( IR at 1.6v and RGB at 2 3v), and the circuitry involved with controlling the LEDs. The current value was chosen because the optimal current for the LEDs is about 100mA and the difference is to account for resistors and other circuitry. 2. Must be able to supply approximately 1.8 watts for this glove for at least one hour. Power Supply (Main) 1. Must be able to support 5.1V for the Microcontroller and Camera at 2.4A (USB) This voltage is the 1. Use multimeter parallel to power source. The voltage must be within the tolerance and the current at the voltage source with the completed circuitry must be within the tolerance. 2. To test the power, the voltage and current of the power source will be tested above and the power will be calculated afterwards. To test for the duration of the battery, we will test how long the glove will stay powered on by holding the on/off button and timing until the battery dies or the glove reaches the one hour mark. 1. Use Raspberry Pi with Camera attachment and see if they both turn on.

10 power supply for the Raspberry Pi. LEDs 1. For the RGB LED, emit light at a distance of at least 5 7 ft from the Camera so we can encompass the full wingspan of the user. The luminosity of RGB LED should be equivalent to (800, 4000, 900)mcd millicandelas. 2. For the IR LED, emit light at a distance of at least 5 7ft from the camera so we can encompass the full wingspan of the user. 1. Use a Photometer to detect the luminosity of the LED at a approximate distance from 7 8 ft (the user will be at this drawing at a distance of 5 7ft). Luminosity should be within +/ 50mcd of the mcd listed to the left. Also find the lowest range that can be detected by the camera and write down the value of mcd and range. The user must then stand at least 1 ft closer than the range. 2. Start by placing the IR LED close to the Raspberry Pi Camera Module and gradually move it back by 6 inches. Take photos with the Raspberry Pi Camera Module. If the IR LED is not visible or is faint, then we have reached the lower bound. We will write down this outer bound and request that the user stay 1 feet closer from the outer bound of the LED. Controls 1. On/Off button controls IR LED correctly 2. RGB sliders/potentiometers can vary the color of the RGB LED 3. Up/Down buttons changes which IR LEDs light up 1. Check connections of buttons/potentiometers by changing their state, see if the circuit responds by using multimeter or camera detection. 3.2 Tolerance Analysis Critical Component: Camera

11 The camera plays a pivotal role because the success of the project is based on the accuracy of the camera detecting the glove. The accuracy of detection for these LED lights are crucial for the camera in interpreting the location and the brush color/size which are key features we are adding. If the camera is not accurate enough to distinguish between IR LEDs and RGB LEDs, the quality of the outputted drawing will be greatly diminished hurting the overall project. Acceptable Tolerance: The first tolerance is that the camera must shoot at least 30 frames per second. This is so the motion of the glove seen on the camera would be smooth enough to be visually appealing to the user. Any less frames may cause choppiness in the motion seen on the monitor. The second tolerance is that the camera must accurately detect the location of the glove at about 5 7 ft from the camera. We have decided that an optimal distance for the user using this device would be about 5 7 ft. Any closer and the LED may shine too close to the camera and not provide an acceptable resolution to display and if the user is too far away, the camera will not be able to detect the LEDs. Test Procedure: The first test will start with an eye test to see if the motion detection is choppy. To test for accuracy, we will cut out 5 differently sized cardboard circles then stand at the optimized distance of 5 7 feet and using one circle at a time, outline the circle with the glove and see if the displayed drawing is a perfect circle. This will be repeated with the other circles to test if the displayed drawing is a perfect circle. Presentation of results: The results will be presented in a table format, checking for these tolerances once a week after the camera, microcontroller, and monitor connections are functional.

12 4.0 Cost and Scheduling 4.1 Cost Analysis Labor Name Hourly Rate Total Hours Invested Total Cost (Rate*2.5*Hours) Zeng $ $17, Xiong $ $17, TOTAL $35, Parts Item Quantity Price Raspberry Pi 3 Model B 1 $36 microsd 1 $8 Duracell 9V battery 2 $8 Raspberry Pi Camera Module 1 $25 IR LED 3 $18 RGB LED 1 $2 Potentiometer 3 $2 Resistor 13 <$1 Button 3 $3

13 Total $ Grand Total Section Total Labor $35, Parts $ Grand Total $35, Schedule Week Tasks Partner 9/12 9/18 Finalize Proposal Section 1 Proposal Section 2 Proposal Section 3 Proposal Section 4 9/19 9/25 Select 2 Cameras to choose from and write down Specs Select RGB LED Lights and IR LED lights and write down specs Select microcontroller Select power sources 9/26 10/2 Prepare Design Review first part Prepare Design Review second part Purchase microcontroller and LED s

14 Purchase camera and power supply Start glove circuit with RGB LED Check if both finished Laboratory Safety Training 10/3 10/9 Program microcontroller with IR Camera Implement IR LED lights into glove circuit Get / utilize shop for glove design Work on glove circuit with LED light functionality with buttons and sliders Test power source compatibility with signaling module 10/10 10/16 Finalize glove circuit with Power Source Test if Microcontroller is receiving data Test Power Source for monitor,microcontroller, and camera Ensure functionality of sending signals by pressing user controls (buttons and potentiometer) on the glove circuit 10/17 10/23 Code Microcontroller to output to Monitor Run tests to see if Microcontroller can display location of glove to monitor Debug Hardware issues Debug Software issues 10/24 10/30 Finish Individual Progress reports Both

15 Does the Microcontroller receive data from camera correctly and display data correctly? Run Mock Test Debug Software Issues Debug Hardware Issues Calibrate accuracy of output from microcontroller 10/31 11/6 Start working on Presentation 11/7 11/13 Continue running tests to ensure accuracy of sensor Test edge cases (find bugs) Fix errors and edge cases/bugs for Hardware Fix errors and edge cases/bugs for Software Tests for additional problems 11/14 11/20 Finalize Presentation Prepare demonstration 11/21 11/27 Prepare Final paper Finalize demonstration 11/28 12/4 Demonstration Both Finalize Final Paper Both