The Profitable Collision of Real and Virtual Worlds T HE AI SUMMIT S AN F RANCISCO, 27 SEPTEMBER 2017 Steve Eglash Executive Director, Strategic Research Initiatives, Computer Science, Stanford University
The AI and data science revolution
Some trends in AI and data science Data mining and machine learning Algorithms that learn from data Clustering Unstructured and dark data Anomaly detection in real-time streams Probabilistic approaches Computer vision Natural language processing Decision making in complex environments Merging of real and virtual worlds
The merging of real and virtual worlds Previously, artistic animation primarily for entertainment and gaming Now, physics-based animation allows predictive modeling and creation of realistic virtual objects
Artistic animation versus physics-based animation Doug James
Physics-based animation, simulation, and predictive modeling Based on physical principles of materials and forces If the models are correct, then provides accurate images, dynamics, and interactions Allows creation of images of things that no one has ever seen Enables investigation of events in a simulated environment How? Create virtual objects that accurately reflect properties of real physical objects Why? Useful for entertainment, e-commerce, education, experimentation, and much more Can generate data Can narrow range of likely solutions
Component parts Rendering: light transport, 3D scene 2D image Geometry: creating 3D geometry for a scene Animation: moving 3D geometry around Simulation: automated animation
Applications Entertainment and gaming, obviously Virtual reality and augmented reality Textiles, garments, and fashion Retailing and e-commerce Education and training Robotics Healthcare and medicine Insurance and automotive collision repair Disaster scenario response Manufacturing and much more
Visual cloth details
Virtual cloth for e-commerce Apparel is an important category for e-commerce However, it is hard to decide the fit or imagine the look without actually trying on the clothing Returns due to the mismatch between expectation and reality can incur logistical loss and hurt customer satisfaction Virtual try-on can make online clothes shopping a smoother process for both vendors and shoppers Ron Fedkiw, Jenny Jin, Stanford University
Cloth simulation is hard The goal is to simulate high quality and visually interesting virtual cloth with realistic folds and wrinkle patterns under dynamic motion It is very difficult to simulate real world cloth faithfully even the best CG cloth for movies isn t good enough compared to real cloth
Improved cloth simulation The inequality cloth simulation model on the right produces better visual results and improved data augmentation Ron Fedkiw, Jenny Jin, Stanford University
Ultra-realistic cloth for movies These techniques can be used to make ultra-realistic virtual cloth for movies, similar to what Fedkiw et al. did with CG water for Pirates of the Caribbean
14 Faces and the uncanny valley We, as humans, subconsciously detect something wrong with near human-like characters, particularly in their face Characters that are obviously fake and non-human provoke a negative response This effect occurs when one pushes the boundaries of realism, which is where the visual effects industry is at right now Very talented artists spend lots of time tweaking results for each shot by hand Extremely hard to fix, because we often don t know what s wrong Ron Fedkiw, Michael Bao, Stanford University
15 Building a realistic anatomical model Create a highly accurate model of the face and simulate it using volumetric muscles Compute muscle activations from motion capture data by solving an inverse problem Preserves physical properties: Volume Preservation Contact and Collision Problems: Lacks the expressiveness of traditional facial blendshape animations Hard for artists to control Still uncanny; not good enough! circa 2005
Hybridizing data and simulation
Yarn-level cloth simulation Doug James (Stanford), Steve Marschner (Cornell)
18 Scarf at 1/6 th speed Jonathan Kaldor, Steve Marschner, Doug James, Cornell University
Implications for business Industry Entertainment and gaming Virtual reality and augmented reality Textiles, garments, and fashion Retailing and e-commerce Education and training Robotics Healthcare and medicine Insurance Disaster scenario response Manufacturing Some potential applications Obviously Additional sources of data for immersive experiences Digital prototyping and virtual dressing rooms Samples, online interactivity Convenient and safe experiential training Simulation, planning, tele-operation Training, telemedicine, research Automotive collision repair, for example Prediction and next best action Integrate design, manufacture, and end user And much much more!
20