viewdle! - machine vision experts

Size: px

Start display at page:

Download "viewdle! - machine vision experts"

Clarence Summers
6 years ago
Views:

1 viewdle! - machine vision experts

2 topic using algorithmic metadata creation and heterogeneous computing to build the personal content management system of the future Page 2

3 Page 3

4 video of basic recognition demo Page 4

5 desktop/tablet app Page 5

6 object detection and tracking Page 6

7 beyond management...ar Page 7

8 viewdle: about us Founded in 2007 Machine vision experts 55 employees Headquartered in Palo Alto, California Research anchored by 6 PhDs 4 patents pending, 5 to be filed all related to facial recognition systems Page 8

9 a new way to look at video Page 9

10 viewdle tech productized machine vision, we started with faces built from the ground up to work in UGC environments extremely efficient processing single database for photos and videos/streams shared faceprints across platforms Page 10

11 unified experience need to power a unified experience across all the platforms a consumer uses and need to have a single experience for both pictures and videos...users dont want to tag again and they want to search in the same way Page 11

12 why faces? Page 12

13 trend average consumer is taking more pictures and videos on increasingly connected and powerful devices Page 13

14 for the consumer RESULT a ton of pictures and videos which are very hard to find or use SOLUTION tagging? NO! users hate to tag and will not do it on a more powerful device, we can solve this problem for the consumer at the point of capture, in real time Page 14

15 user flow User select photos Extract faces Group faces to clusters User tags clusters Suggest tag User uploads photos Page 15

16 user experience -Needs to be fast: users hate to wait, even worse for video -Needs sufficient precision/recall -GPU is already inside, and not typically utilized during this experience -With extra horsepower, can perform a deeper analysis Page 16

17 video -Video processing requires a lot of computational power -Due to parallel nature of GPU computation logic, multiple video frames can be processed on GPU very fast. -On new mobile platforms, CPU is not powerful enough to provide realtime video processing...starting to see OCL on these platforms Page 17

18 opencl OpenCL is going to transform these use cases to new level of user experience: Faster Lower power / mobile computing environments Fully utilize the hardware available Enable new usecases - AR Page 18

19 local why not use a massively powerful server? usecase is broken if go to the cloud, processing needs to happen locally who wants to wait to upload gigs of video? Page 19

20 generic computer vision we ll use face recognition as an example, but principles extend across machine vision Page 20

21 computer vision pipeline Page 21

22 Why H/W co s love computer vision no end in sight to processing power necessary because unlimited objects to find, many many parallel cascades of classifiers need to store unlimited number of reference objects to match against everyone in CV is trying to do something with 3D...and, there are real world use cases that extend beyond gaming: GPAR Page 22

23 Distribution of working time. Large photos 190 photos with 220 faces 8% 17% Input Face detection Feature detection Normalization Output 75%

24 Distribution of working time. Small photos 1440 photos with 2040 faces 3% 24% Input Face detection Feature detection Normalization Output 73%

25 Distribution of working time. Video 720P 3% 1% video, duration ~ 2.5 min 3% 8% 57% 28% Input Face detection Feature detection Tracking Normalization Output

26 acceleration lowest hanging fruit for acceleration: -video input (uvd) -face detection Page 26

27 uvd UVD is a dedicated video decode processing unit Offloads CPU from the decoding process Reduces power usage In Microsoft Windows works via DXVA (DirectX Video Acceleration) API We are using it to decode H.264 video Page 27

28 uvd w/o UVD w/i UVD FPS % % CPU load *Hardware: Sabine(Torpedo), Quad Core 1.8GHz Less FPS Less CPU load Offloads work to GPU Page 28

29 uvd smaller is better Processing time Video 720P CPU + no UVD CPU + UVD GPU + UVD Mode Sabine(Torpedo) APU Engineering sample Quad Core 1.8GHz Page 29

30 uvd smaller is better 140 Video 720P Processing time Sabine (Armorhead)APU Engineering sample Quad Core 2.4GHz 1.29x 0 CPU + no UVD CPU + UVD Mode GPU + UVD Page 30

31 speedup Platform Sabine(Torpedo) APU Engineering sample Quad Core 1.8GHz Sabine(Armorhead) APU Engineering sample Quad Core 2.4GHz Video 720P Results are obtained with Viewdle Video application Platform: Windows 7 x64 Page 31

32 A Haar-like feature considers adjacent rectangular regions at a specific location in a detection window, sums up the pixel intensities in these regions and calculates the difference between them. This difference is then used to categorize subsections of an image.

33 FACE DETECT

34 NOKIA N900

35 facedetect relative performance higher is better 6 5 GPU vs CPU >5x faster GPU speedup Mpx 1MPx 4MPx 6MPx Size of input images Page 35

36 detection modes! Homogeneous " Parallel (CPU only) " Ocl (GPU only)! Heterogeneous " Balanced " Split Page 36

37 detection modes - detail Parallel multi-threaded mode, everything is processed on CPU Ocl - single-threaded mode, everything is processed on GPU using OpenCL Balanced mix of parallel and ocl modes Heterogeneous automatic selection between CPU and GPU: uses GPU for small scales and CPU for large scale Page 37

38 apu performance smaller is better 250 Photos set (190 images) Processing time Sabine(Torpedo) APU Engineering sample Quad Core 1.8GHz 1.83x 0 Parallel Balanced Splitted Mode of detector Page 38

39 mobile platform smaller is better Processing time Photos set (190 images) Parallel Ocl Balanced Splitted Mode of detector AMD Phenom II N930 Quad Core 2.00 GHz ATI MOBILITY HD x Page 39

40 mobile platform (tablet/laptop) smaller is better 700 Photos set (1440 images) Processing time AMD Phenom II N930 Quad Core 2.00 GHz ATI MOBILITY HD x 1.8x 0 Parallel Ocl Balanced Splitted Mode of detector Page 40

41 desktop smaller is better Processing time Photos set (190 images) Parallel Balanced Ocl Split Mode of detector Intel Core i x3GHz AMD Radeon HD x Page 41

42 desktop 350 Photos set (1440 images) Processing time Intel Core i x3GHz AMD Radeon HD x 0 Parallel Balanced Ocl Split Mode of detector Page 42

43 speedup on photos Platform Small photos Large photos Sabine(Torpedo) APU Engineering sample Quad Core 1.8GHz AMD Phenom II N930 Quad Core 2.00 GHz ATI MOBILITY HD5650 Intel Core i x3GHz AMD Radeon HD6970 *** times for full face detection pipeline! Page 43

44 desktop/tablet app Page 44

45 Disclaimer & Attribution The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. There is no obligation to update or otherwise correct or revise this information. However, we reserve the right to revise this information and to make changes from time to time to the content hereof without obligation to notify any person of such revisions or changes. NO REPRESENTATIONS OR WARRANTIES ARE MADE WITH RESPECT TO THE CONTENTS HEREOF AND NO RESPONSIBILITY IS ASSUMED FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. ALL IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE ARE EXPRESSLY DISCLAIMED. IN NO EVENT WILL ANY LIABILITY TO ANY PERSON BE INCURRED FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. All other names used in this presentation are for informational purposes only and may be trademarks of their respective owners. The contents of this presentation were provided by individual(s) and/or company listed on the title page. The information and opinions presented in this presentation may not represent AMD s positions, strategies or opinions. Unless explicitly stated, AMD is not responsible for the content herein and no endorsements are implied. AMD FUSION DEVELOPER SUMMIT June 2011

Use cases. Faces tagging in photo and video, enabling: sharing media editing automatic media mashuping entertaining Augmented reality Games

Use cases. Faces tagging in photo and video, enabling: sharing media editing automatic media mashuping entertaining Augmented reality Games Viewdle Inc. 1 Use cases Faces tagging in photo and video, enabling: sharing media editing automatic media mashuping entertaining Augmented reality Games 2 Why OpenCL matter? OpenCL is going to bring such