The NAO Robot, a case of study Robotics 2013-2014 Franchi Alessio Mauro alessiomauro.franchi@polimi.it
Who am I? Franchi Alessio Mauro Master Degree in Computer Science Engineer at Politecnico of Milan First year Ph.D. student, collaborating with prof. Gini Research areas: Cognitive robotics Symbolic reasoning Grasping If you need to contact me: mail: alessiomauro.franchi@polimi.it office: AirLab DEIB, building 20, ground floor telephone: 3565
This lesson Two sections Computer vision - OpenCV(~60 mins) Loading/reading images; Images manipulation Images filtering Videos streaming Introducing the NAO robot (~20 mins) Structure Sensors Capabilities
Computer Vision/OpenCV What is OpenCV (Open Source Computer Vision)? A set of libraries focused on real-time images processing. A project started by Intel Russia in 1998 Now are maintened by Itseez and Willow Garage 2500 optimized algorithms A full-featured CUDA and OpenCL interfaces are being actively developed right now http://opencv.org/ Built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products
Computer Vision/OpenCV Some application: Detect and recognize faces, Identify objects, Classify human actions in videos, Track camera movements and moving objects, Extract 3D models of objects and produce 3D point clouds, Recognize scenery and establish markers to overlay it with augmented reality, Etc
Augmented Reality/OpenCV The technology functions by enhancing one s current perception of reality Live direct or indirect view of a physical real-world environment whose elements are augmented/extended
Let s start with OpenCV How to install OpenCV on Windows + VisualStudio 2012 1. Go to http://opencv.org/ 2. Download auto-extracting or binary files to be compiled 3. Set environment variables a) Open command prompt and type setx -m OPENCV_DIR D:\OpenCV\Build\x86\vc11 b) Right click on Computer - > Properties -> Advanced- System settings -> Advanced tab-> Environment variable - > System variables c) Add to PATH %OPENCV_DIR%\bin d) Reboot the system
Let s start with OpenCV Now create a new Visual Studio project 1. Open Visual Studio 2012 2. Select File -> New -> Project 3. Once created right click on the project name -> Property 4. Configuration properties -> c/c++ -> General add to Additional Include Directories the following $(OPENCV_DIR)\..\..\include 5. Linker -> General add to «Additional libraries Directories the following $(OPENCV_DIR)\lib 6. Linker -> Input add to Additional Dependencies rhe following list opencv_core249d.lib;opencv_imgproc249d.lib;opencv_highgui249d.l ib;opencv_ml249d.lib;opencv_video249d.lib;opencv_features2d249d. lib;opencv_calib3d249d.lib;opencv_objdetect249d.lib;opencv_contrib 249d.lib;opencv_legacy249d.lib;opencv_flann249d.lib
Let s start with OpenCV The «Hello world» example
Some theory How images are composed Black and white Any digital image is a matrix of pixels. Any pixel have a value. The value of pixels are between 0 (=black) and a maximum depending on the number of bits of the image (8 bits = 255 = white). The image-depth refers to the bits allocated for each pixel. When the value of the pixel is increased, the intensity of that pixel is also increased. Example Image-depth 8 bit 1 channel, grayscale image Height is 4 pixel, width is 5 pixels Resolution = 4x5.
Some theory How images are composed Color Color image is composed by 3/4 planes; Red Green Blue Alpha (optional) Any pixel s final color is the combination of the corresponding values of pixels in the three/four planes (255, 0, 0) = red. (0, 255, 0) = green (255, 0, 255) = violate
Some theory Introducing the concept of feature An "interesting" part of an image Human beings are naturally able to find these «features», it is something innate. As consequences we are able to find two similar images, to compose images or to play with puzzle A simple game: are you able to tell me the exact location of each patche?
Some theory
Some theory
Let s go on with OpenCV Dilation Erosion The value of the output pixel is the maximum value of all the pixels in the input pixel's neighborhood. Pixels beyond the image border are assigned the minimum The value of the output pixel is the minimum value of all the pixels in the input pixel's neighborhood. Pixels beyond the image border are assigned the maximum
Let s go on with OpenCV Canny Edge detector 1. Filter out any noise. The Gaussian filter is used for this purpose 2. Four filters to detect horizontal, vertical and diagonal edges in the blurred image are used 3. Search for local maxima in the gradient map with non-maximum suppression 4. Thresholding with hysteresis: Lower than low threshold: pixel is not considered; Higher than high threshold: pixel is on an edge; Between the two thresholds: pixel is considered if nearby an already considered pixel.
Let s go on with OpenCV Harris corner detector 1. Corners represents a variation in the gradient in the image, we will look for this variation. So sweep a window in both direction and compute the variation of intensity Maximize the equation above to find windows with a large variation A window with a score R greater than a certain value is considered a corner
Let s go on with OpenCV The Hough transform A technique for detecting significant configurations of points in the image. Lines, segments, curves, The shapes can be expressed by a function in a new parameters space
Let s go on with OpenCV The Hough transform How points are transformed? Points are the intersection of two or more lines So points is a curve in the parameters space
Let s go on with OpenCV The Hough transform -How lines are transformed? Curves corresponding the three points on the lines intersect in a given point in the parameters space This point is the image of the lines.
The NAO robot NAO is a programmable, 58cm tall humanoid robot 25 degrees of freedom (DOF) 2 cameras, 4 directional microphones, sonar, 2 IR receiver/emitter, 1 inertial board, 9 tactile sensors and 8 pressure sensors voice synthesizer, LED lights, and 2 speakers
Links and axis Links length Axis definition The X axis is positive toward NAO s front, the Y from right to left and the Z is vertical.
Joints
Sensors Sensor network: 2 cameras, 4 directional microphones, sonar rangefinder, 2 IR emitters and receivers, 1 inertial board, 9 tactile sensors 8 pressure sensors voice synthesizer, LED lights, 2 speakers
Some examples Some basic movements Standing up; Sitting down; Walking and some more complex: Dancing; Tracking obects; Climbing stairs; Robocup. Videos!