Motivation: why study sound? Motivation (2) Related Work. Our Study. Related Work (2)

Transcription

1 Motivation: why study sound? Interactive Learning of the Acoustic Properties of Objects by a Robot Sound Producing Event Jivko Sinapov Mark Wiemer Alexander Stoytchev {jsinapov banff alexs}@iastate.edu Iowa State University [Gaver, 99] Motivation (2) Why should a robot use acoustic information? Human environments are cluttered with objects that generate sounds Help robot perceive events and objects outside of field of view Help robot perceive material properties of objects Related Work Krotkov et al. (99) and Klatzky et al. (2000): Perception of material using contact sounds. Learned sound models for tapping aluminum, brass, glass, wood, and plastic (one object per material) Richmond and Pai (2000) Robotic platform for measuring contact sounds between robot s end effector and object surfaces Models the contact sounds from different materials using spectrogram averaging [Richmond and Pai, 200] Related Work (2) TorresJara, Natale and Fitzpatrick (200) Robot taps objects and records spectrogram of sound Recognize objects using spectrogram matching Recognized 4 test objects used during training. Our Study Demonstrate object recognition using acoustic features from interaction 8 Different Objects Different behaviors: push, grasp, drop Evaluate different machine learning algorithms Tapping objects Spectrogram of tapping

2 Robot and Objects 7DOF Barret WAM arm with Barret Hand 8 Different objects: Grasping: Pushing: Dropping: Sound Feature Representation Step : segment sound wave during interaction: Object Recognition using Acoustic Properties of Objects Problem: given robot s behavior and detected sound features from interaction, predict the object. Step 2: Compute Discrete Fourier transform (DFT) of sound wave: Step : Compute 2D histogram of DFT matrix using block averaging: Example: Behavior: Sound Features: Object Class: Frequency frequency bins grasp Time 0 temporal bins 2

3 Problem Formulation Let be the set of exploratory behaviors Let be the set of objects, Let be a data point such that:,, and For each behavior that can estimate learn a model Learning Algorithms KNN Simple instancebased algorithm Uses Euclidean distance function Support Vector Machine (SVM) Discriminative approach, uses Kernel trick Probabilistic graphical model Sound Features are discretized into bins Learning Algorithms: knn, SVM, and knn: memorybased learning algorithm? Test point With k = : 2 neighbors neighbors Therefore, Pr(red) = 0. Pr(blue) = 0. Learning Algorithms: knn, SVM, and Support Vector Machine: discriminative learning algorithm. Finds maximum margin hyperplane that separates two classes 2. Uses Kernel trick to map data points into a feature space in which such a hyperplane exists [ Learning Algorithms: knn, SVM, and : a probabilistic graphical model C A E D B. Full power of statistical modeling and inference 2. Learning: learns both the structure of the network and the parameters (conditional probability tables). Numerical features are discretized into bins Using Multiple Behaviors Given trained models,, Given novel sounds,, from behaviors performed on the same object Assign prediction to object class that maximizes:

4 4 Evaluation trials recorded with each of the 8 objects with each of the behaviors Leaveoneout crossvalidation Compared performance of learning algorithms as well as behaviors Performance Measure: Results Chance accuracy = /8 =.7% Confusion Matrix for model M push using Predicted Perfect classification and no false positives for: Confusion Matrix for model M combined using Conclusion: The errors made by models M grasp, M push and M drop are uncorrelated. Predicted Learning rate of algorithms Compare performance of the model M grasp as a function of dataset size for: knn Support Vector Machine Learning Rate per Behavior with

5 Summary and Conclusions Accurate acousticbased object recognition with 8 objects and behaviors Using multiple behaviors improves recognition regardless of learning algorithm Bayesian network performed best with given feature representation Grasping and Pushing interaction produces sound features that are more informative of the object than Dropping Future Work Scaling up: Increase number of objects Vary object and robot pose Autonomous interaction Use unsupervised learning to form object sound categories More powerful feature representations Temporal features (i.e. periodicity) of sounds Use models to detect events in the world performed by others (humans or other robots)