Pictures from Piles of Data

Transcription

1 Pictures from Piles of Data How Graphics, Multimedia, Vision, Visualization, Animation and Cartography All Connect Michael Gleicher Dept of Computer Sciences University of Wisconsin Madison Acknowledgements All of this work is done in collaboration with a great group of students and collaborators This talk is work done with students: (who didn t want their pictures shown) Feng Liu multimedia, video (work supported by NSF, Adobe) Now at Portland State Univ Danielle Albers genomics (work supported by NSF, DOE) Kevin Ponto Virtual Environments (work supported by NIH) Greg Cipriano molecules, vis (work supported by DOE,NIH) Now at Solidworks Adrian Mayorga vis (work supported by NIH) Aaron Bryden molecule motion (work supported by NIH) Michael Correll humanities (work supported by NSF) Sean Andrist facial animation (work supported by NSF) Pictures from Piles of Data How Graphics, Multimedia, Vision, Visualization, Animation and Cartography All Connect Michael Gleicher Dept of Computer Sciences University of Wisconsin Madison Pictures from Piles of Data How Graphics, Multimedia, Vision, Visualization, Animation and Cartography All Connect Michael Gleicher Dept of Computer Sciences University of Wisconsin Madison Pictures from Piles of Data Stuff I do, Visualization, and Animation why you and might Cartography be All interested Connect How Graphics, Multimedia, Vision, What do these have in common? Analysis of Proteins Motion Synthesis for Characters Multimedia Database Information Extraction Michael Gleicher Dept of Computer Sciences University of Wisconsin Madison Scientific Data Display Video Quality Improvement Image and Video Retargeting 1

2 What do these have in common? It s all stuff I ve done in the past few years What do these have in common? It s all stuff I ve done in the past few years Does this all tie together? How can we use our understanding of human perception and artistic traditions to improve our tools for communicating and data understanding? How can we use our understanding of human perception and artistic traditions to improve our tools for communicating and data understanding? 2

3 What do these have in common? It s all stuff I ve done in the past few years It involves large amounts of data overwhelming Data is abundant what are you going to do about it? It involves creating effective presentations It requires some understanding of the data in order to simplify it Data abundance in the modern age Is there any historical precedent? The world, Birmingham Public Library Digital Collection. Figura Del Mondo Universale Charte Cosmographique, auec les Noms, Proprietez, Nature & Operations des Vents (1544) Birmingham Public Library Figura Del Mondo Universale,

4 How can we use our understanding of human perception and artistic traditions to improve our tools for communicating and data understanding. Talk Roadmap Molecular Surface Abstraction Molecular Motions Visualizing Comparisons Living Environments Lab Video Stabilization A Protein Surface An aside How do scientists look at proteins? Stick and Ball Model (internals) Work with Greg Cipriano and George Phillips An aside How do scientists look at proteins? An aside How do scientists look at proteins? Stick and Ball Model (internals) Stick and Ball Model (internals) Ribbon Diagram (internals) Molecular Surface (externals) 4

5 A Protein Surface Molecular Surface Abstraction Work with Greg Cipriano and George Phillips What s Happening? Simplification Stylized Display Surface Indications Art Abstraction Good Lighting Line Drawings Non Photorealism Visual Cognitive Science Cue reduction Provide Depth Cues Enhance Contours Tolerance of Shading Why does B fight cancer? Talk Roadmap Molecular Surface Abstraction Molecular Motions Visualizing Comparisons Living Environments Lab Video Stabilization 5

6 Molecular Motions Molecular Motions Coarse grained models Normal mode Analysis (NMA) Bryden, Phillips, Gleicher. TVCG 11. To Appear. Motion Illustration Motion Illustration Artistic Inspirations: Comic Books / Diagrams Abstract Model Illustrate The Problem: Vector per point Abstract Group to fit models Model Illustrate 6

7 Abstract Group to fit models Model Affine model per group Illustrate x = A x x = A x Abstract Group to fit models Model Affine model per group Illustrate Affine Exponentials Glyph Design x = A x p(t) = e At x = A x p(t) = e At x = A x p(t) = e At x = A x p(t) = e At 7

8 Talk Roadmap Lots of fun domains! Molecular Surface Abstraction Molecular Motions Visualizing Comparisons Living Environments Lab Video Stabilization Is there a niche for me? Visual Comparisons Visual Comparisons Mike s theory of visual comparison 0.2* Maybe provocative? There are general principles that apply across domains, data types, And if we can figure it out, it ll be easier to crank out the comparison tools/techniques quickly * This is a work in progress. Comments welcomed! 8

9 The 3 3s 3 Axes of Scalability / Hardness 3 Strategies for Scalability 3 Basic Designs Comparison is easy. Until it gets hard. 3 Axes of Hardness Number of things to compare Size/complexity of things to compare Complexity of the relationships People can only compare a few things at a time* 3 Strategies for scaling Select Subset Summarize Statistically Scan Serially All designs appear to fall into 3 categories 3 Basic Designs* My cognitive scientist friends say the magic number 7+/ 2 is an over simplification, but * Each has its pros and cons Genome Sequence Comparison No single view is sufficient for the large and complex data encountered in comparative genomics! More Case Studies in Visual Comparison: Advance the theory through practice Advance the practice through theory Mizbee: Multiscale Synteny Browser Mauve: Multiple Genomic Alignment Broad Institute Hepatitis C Database DNAVis 9

10 Multiple Sequence Data Gets Unwieldy Perceptual Principles and Signal Processing to Design Overviews of massive alignments Work with Danielle Albers, Colin Dewey, Dave O Connor, Albers, Dewey, and Gleicher. To be submitted soon. 100 bacteria More data than columns (pixels) Averaging Color Weaving Event Striping A Different Kind of Evolution... Google Books Word Count Data Decades > Word Rank > Color by position in reference 10

11 11/27/2011 A Different Kind of Evolution... Decades > Google Books Word Count Data Word Rank > Color by occurance count Visualizing English Print from 1470 to 1800 With Mike Witmore, Robin Valenza, Michael Correll, Domain Specific Tools Back to the details that cause patterns Correll, Witmore, and Gleicher. Eurovis 2011, to appear. 11

12 Comparing Distributions Splatterplots With Adrian Mayorga, to be submitted soon Talk Roadmap Molecular Surface Abstraction Molecular Motions Visualizing Comparisons How can we use our understanding of human perception and artistic traditions to improve our tools for communicating and data understanding. Living Environments Lab Video Stabilization Video Stabilization Problem: Shaky video in, less shaky (good?) video out From Shakespeare to Video Processing 3D or not 3D, that is the question Perception and Art What is good camera movement? How can we avoid the impossible? 12

13 Motivation: More video doesn t mean better video Good video takes effort! (But cameras are everywhere) Problem: Bad Camera Motion No planning No tripod Problem: Bad Camera Motion Prior Work: Image Stabilization One part of the problem: jitter Helped by Image Stabilization Three Projects Re Cinematography What can you do beyond removing jitter? Stabilization by 3D Warping How can you make bigger changes? Stabilization by Subspace Constraints How do you make it practical? Work with Feng Liu and Adobe Video Stabilization Existing approach: 2D stabilization Re Cinematography A 2D stabilization approach Track a bunch of points Fit full frame warps that best smooth point motion Limited! Doesn t model parallax Can t reason about camera motions in 3D A model of Computational Cinematography What is a good camera motion? Cannot deal with large viewpoint changes Gleicher and Liu. TOMCCAP

14 11/27/2011 A more interesting question: To swing or not to swing Source Footage Re Cinematography Result 3D Video Stabilization: Pipeline Artifacts Input: Output: 3D reconstruction via Structure from motion Line Parabola Camera path planning Low pass filter Novel view synthesis Where did she come from? 82 3D Reconstruction from Video Structure from Motion Camera Path Planning Move the camera to a new place Line Parabola Low pass filter Voodoo Camera Tracker: 14

15 Novel viewpoint rendering How do you move the camera to a different place? 3D experiences without 3D models How to make a video quality image? Build a really high quality 3D model? Too hard Not enough input data Image Based Rendering using other frames? Violates temporal constraints Computationally expensive Liu et al. SIGGRAPH Avoid Temporal Artifacts: Need Novel view from One Frame Input: Output: Novel view from one frame Impossible? Incomplete geometric model (sparse) Occlusions / Dis occlusions Where did she come from? Novel view from one frame Impossible? Incomplete geometric model (sparse) Occlusions / Dis occlusions Impossible! So Fake it! Just need visually plausible, not accuracy Viewpoint shifts will be small Avoid artifacts 3D Stabilization by Image Warping Structure from motion gives sparse points 3D camera planning gives motion of points Use sparse points to warp image 15

16 Content Preserving Warps Results & Comparisons Warp each input frame to create the output frame by least squares minimization Data term: Soft, sparse displacement constraint Smoothness term: Local similarity transformation constraint Weightings: Higher penalties in salient regions Is this a good reconstruction? NO! Camera position Output points One frame isn t so bad Especially if you crop 16

17 Limitations Act III Video Stabilization with Subspace Constraints Limitations What do we need 3D for? 17

18 Low rank constraint: [Tomasi and Kanade1992, Irani 2002] The trajectory matrix of a rigid scene imaged by a moving camera over a short period of time should approximately lie in a low dimensional subspace. Plausibility Constraints Incomplete Moving Matrix Factorization Filter the Eigen Trajectories 18

19 Talk Roadmap Molecular Surface Abstraction Molecular Motions Visualizing Comparisons Computer Graphics, Image Processing and Surgery? Living Environments Lab Video Stabilization Increasingly The Home is where Health Happens How do we prepare for this future? Living Environments Laboratory 19

20 How do we Understand behavior in the home? Design devices for the home? Living Environments Lab (LEL) is one of five themes in the Wisconsin Institute for Discovery (WID) Train people to give care in the home? Predict performance in the home? Homes are different! Traditional Laboratories may not be adequate 20

21 Visual Simulation? Two Challenges (to start with, at least) Modeling Messiness Virtual Exertions Model the variability of lived environments? Predict behavior without realistic simulation? Model the variability of lived environments? mathematic models of messiness generate plausible, random environments Predict behavior without realistic simulation? evoke realistic responses from subjects validate mappings between virtual and real Plausibility! Joint work with Kevin Ponto, Patti Brennan and Rob Radwin 21

22 Talk Roadmap Molecular Surface Abstraction Molecular Motions Visualizing Comparisons How can we use our understanding of human perception and artistic traditions to improve our tools for communicating and data understanding? Living Environments Lab Video Stabilization Thanks! To you for listening To my students and collaborators To the folks who pay the bills Michael Gleicher Dept of Computer Sciences University of Wisconsin Madison 22