Presenter: Felix Goldberg, Ph.D. Chief AI Scientist Artificial Intelligence Cart AIC

Transcription

1 Presenter: Felix Goldberg, Ph.D. Chief AI Scientist Artificial Intelligence Cart AIC

2 Company Masthead Prof. Mo I Meidar co-founder & President Gidon Moshkovitz co-founder & CEO Edi Bahous co-founder & CTO Over 45 years of Executive Leadership in Operations, Manufacturing, Distribution & High Technology. Prof. Meidar has special expertise in retail operation turnaround. Successful angel investor Proactive, all-rounded professional, entrepreneurial experience, results driven. 15 years experience in finance and advanced technologies. All American Athlete at the NCAA With more than 20 years of developing experience in the automotive and computer industry. At his last position, Edi was the Vice President & Head of Department Electronics Solutions at Mercedes-Benz technology in Germany.

3 Technology Timeline 1937 Today Car Autonomous car Dial phone iphone Cart AIC

4 AIC: the main components Personalized display & ads High-speed cameras with MIPI protocol Bird-view cameras looking downward All computing is done on the Jetson/Xavier The bottom may be a weight sensor (depending on configuration)

5 Our Principles Usability (free-fall) Speed (<1s) Trust (Over 98% Accuracy)

6 Technical Challenges for our AI Engine Large-scale classification (100K classes eventually) Fine-grained image classification Incremental learning Motion detection Model compression Semantic segmentation

7 Artificial Intelligence Engine Data Acquisition Detection Engine Machine Learning

8 Data Acquisition Data Acquisition Real images (from the web/crowd-sourced): Synthetic Similar to the target images Expensive to label Noisy Difficult to ensure class representation for fine-grained classification Relatively cheap to acquire Automatic labelling Structured Not sufficiently similar to the target images How to get the best of both worlds?

9 The Solution: clean real images Acquire clean data of each product from all angles in our lab Data Acquisition Advantages: Minute details are captured, allowing for fine-grained classification. Balanced classes. The labelling is produced automatically. Structured data (pose of each image and its ontological position) Coming soon: 3d modeling of products & novel viewpoint synthesis Add real data from actual stores (in the pilot phase)

10 Structured data Multi-dimensional scaling of distance in color space Data Acquisition

11 Deep Learning Infrastructure Machine Learning THE MONSTER Dell Precision Tower 7910 XCTO 2 x NVIDIA Quadro P6000 GPUs THE SUPER MONSTER NVIDIA DGX-2, 32GB RAM (2-3 speedup factor for training) Tensorflow framework, through the Keras API. Models are ported to TensorRT before deployment to the Jetson TX2/Xavier. (4-5 speedup factor for inference)

12 Augmentation tactics Machine Learning Usual Stuff Warps Crops* (crucial for partial img) Backgrounds Camera noises Custom Stuff Elastic transformation for bags Transparency for bottles Esoteric Stuff Hands Price tags

13 Classification models roadmap Machine Learning Generation 1 (Ancient history) Generation 2 (Prototype in booth) Generation 3 (Rolling out now) Generation 4 (Active research) 4K-6K images per product As few as 300 images per product Hierarchical models for fine-grained classification Cool stuff I will tell you about next time. One large network to Customized network classify them all Up to 1000 products

14 Sneak Peek into the validation set Machine Learning

15 What Did the model learn? t-sne plot of the penultimate layer Machine Learning

16 Machine Learning

17 Two approaches to free-fall Detection Engine Recognize the product in midair Recognize the product once it has landed Easier background subtraction Needs high speed cameras More complicated change detection required Less blur Motion blur Why not do both?

18 Part 1: Image capture Detection Engine Motion detection algorithm constantly monitors the aerial space of the cart. Each high-speed camera takes a burst of 5-10 images of the product while in flight. Once the product has landed, the bird-view cameras take images.

19 Part 2: Product identification Detection Engine Background subtraction is applied to the images and all crops that may contain the new object are passed to the neural network. Hand subtraction is applied before classification, if necessary. Logo detection & OCR are applied to improve recognition accuracy. If weight measurement available, it is also factored into the decision.

20 Detection Engine Let s see how it works in practice Movie

21 Thank You! We re Hiring:

22 Don t Miss Our Demo!