Consumer Video Understanding
|
|
|
- Judith Madlyn Walton
- 5 years ago
- Views:
Transcription
1 Consumer Video Understanding A Benchmark Database + An Evaluation of Human & Machine Performance Yu-Gang Jiang, Guangnan Ye, Shih-Fu Chang, Daniel Ellis, Alexander C. Loui Columbia University Kodak Research ACM ICMR 2011, Trento, Italy, April 2011
2 We (Consumers) take photos/videos everyday/everywhere... Barack Obama Rally, Texas,
3 What are Consumer Videos? Original unedited videos captured by ordinary consumers Interesting and very diverse contents Very weakly indexed 3 tags per consumer video on YouTube vs. 9 tags each YouTube video has on average Original audio tracks are preserved; good for audiovisual joint analysis 3
4 Part I: A Database Columbia Consumer Video (CCV) Database
5 Columbia Consumer Video (CCV) Database Basketball Skiing Dog Baseball Swimming Bird Soccer Biking Graduation Cat Birthday Celebration Ice Skating Wedding Reception Wedding Ceremony Wedding Dance Music Performance Non-music Performance Parade Beach Playground 5
6 CCV Snapshot # videos: 9,317 (210 hrs in total) video genre unedited consumer videos video source YouTube.com average length 80 seconds # defined categories 20 annotation method Amazon Mechanical Turk wedding ceremony wedding reception biking graduation baseball birthday soccer playground bird wedding dance basketball beach ice skating cat parade skiing swimming dog non-music perf. music perf. The trick of digging out consumer videos from YouTube: Use default filename prefix of many digital cameras: MVI and parade
7 Existing Database? Human Action Recognition KTH & Weizmann (constrained environment) Hollywood Database (12 categories, movies) 2008 UCF Database (50 categories, YouTube Videos) 2010 Kodak Consumer Video (25 classes, videos) 2007 LabelMe Video (many classes, videos) 2009 TRECVID MED 2010 (3 classes, videos) 2010 CCV Database Unconstrained YouTube videos Higher-level complex events More videos & better defined categories More videos & larger content variations More videos & categories
8 Crowdsourcing: Amazon Mechanical Turk A web services API that allows developers to easily integrate human intelligence directly into their processing What can I do for you? Is this a parade video? o Yes o No Task Internet-scale workforce $?.?? financial rewards 8
9 MTurk: Annotation Interface Reliability of Labels: each video was assigned to four MTurk workers $ 0.02
10 Part II: not Just A Database An Evaluation of Human & Machine Performance
11 Human Recognition Performance How to measure human (MTurk workers) recognition accuracy? We manually and carefully labeled 896 videos Golden ground truth! Consolidation of the 4 sets of labels vote 2-votes 3-votes 4-votes Plus additional manual filtering of 6 positive sample sets: 94% final precision precison recall 11
12 Human Recognition Performance (cont.) precision recall workers (sorted by # of submitted HITs) precision recall workers (sorted by average labeling time per HIT) Time is shown in seconds on top of the bars
13 Confusion Matrices Ground-truth Labels Human Recognition
14 Machine Recognition System Feature extraction Classifier SIFT Spatial-temporal interest points χ2 kernel SVM Average Late Fusion MFCC audio feature Yu-Gang Jiang, Xiaohong Zeng, Guangnan Ye, Subh Bhattacharya, Dan Ellis, Mubarak Shah, Shih-Fu Chang, Columbia-UCF TRECVID2010 Multimedia Event Detection: Combining Multiple Modalities, Contextual Concepts, and Temporal Matching, NIST TRECVID Workshop,
15 Best Performance in TRECVID-2010 Multimedia event detection (MED) task Mean Mimimal Normalized Cost Run1: Run2 + Batter Reranking Run2: Run3 + Scene/Audio/Action Context Run3: Run6 + EMD Temporal Matching Run4: Run6 + Scene/Audio/Action Context Run5: Run6 + Scene/Audio Context Run6: Baseline Classification with 3 features 0.00 r2 r3 r4 r5 r6 r1 15
16 Three Audio-Visual Features SIFT (visual) D. Lowe, IJCV 04 STIP (visual) I. Laptev, IJCV 05 MFCC (audio) 16ms 16ms 16
17 Bag-of-X Representation X = SIFT / STIP / MFCC Soft weighting (Jiang, Ngo and Yang, ACM CIVR 2007) Bag-of-SIFT Bag of audio words / bag of frames: K. Lee and D. Ellis, Audio-Based Semantic Concept Classification for Consumer Video, IEEE Trans on Audio, Speech, and Language Processing,
18 Machine Recognition Accuracy Measured by average precision SIFT works the best for event detection The 3 features are highly complementary! Prior MFCC STIP SIFT SIFT+STIP SIFT+STIP+MFCC 0.7 average precision
19 1 0.8 Human vs. Machine Human has much better recall, and is much better for non-rigid objects Machine is close to human on top-list precision recall recall machine human
20 Human vs. Machine: Confusion Matrices Human Recognition Machine Recognition
21 Human vs. Machine: Result Examples true positives false positives found by human&machine found by human only found by machine only found by human only found by machine only wedding dance (93.3% vs. 92.9%) soccer (87.5% vs. 53.8%) n/a cat (93.5% vs. 46.8%) n/a 21
22 Download - Unique YouTube Video IDs, - Labels, - Training/Test Partition, - Three Audio/Visual Features Fill out this
23 Thank you!
High-level Event Recognition in Internet Videos
High-level Event Recognition in Internet Videos Yu-Gang Jiang School of Computer Science Fudan University, Shanghai, China ygj@fudan.edu.cn Joint work with Guangnan Ye 1, Subh Bhattacharya 2, Dan Ellis
Multimedia Event Detection for Large Scale Video. Benjamin Elizalde
Multimedia Event Detection for Large Scale Video Benjamin Elizalde Outline Motivation TrecVID task Related work Our approach (System, TF/IDF) Results & Processing time Conclusion & Future work Agenda 2
Real-Time Content-Based Adaptive Streaming of Sports Videos
Real-Time Content-Based Adaptive Streaming of Sports Videos Shih-Fu Chang, Di Zhong, and Raj Kumar Digital Video and Multimedia Group ADVENT University/Industry Consortium Columbia University December
IBM Research and Columbia University TRECVID-2011 Multimedia Event Detection (MED) System
IBM Research and Columbia University TRECVID-2011 Multimedia Event Detection (MED) System Liangliang Cao, Shih-Fu Chang, Noel Codella, Courtenay Cotton, Dan Ellis, Leiguang Gong, Matthew Hill, Gang Hua,
Team SRI-Sarnoff s AURORA TRECVID 2011
Team SRI-Sarnoff s AURORA System @ TRECVID 2011 Hui Cheng, Amir Tamrakar, Saad Ali, Qian Yu, Omar Javed, Jingen Liu, Ajay Divakaran, Harpreet S. Sawhney, Alex Hauptmann, Mubarak Shah, Subhabrata Bhattacharya,
Video Event Detection via Multi-modality Deep Learning
2014 22nd International Conference on Pattern Recognition Video Event Detection via Multi-modality Deep Learning I-Hong Jhuo 1 and D.T. Lee 1,2 1 Institute of Information Science, Academia Sinica, Taipei,
arxiv: v2 [cs.cv] 17 Aug 2015
![arxiv: v2 [cs.cv] 17 Aug 2015](/thumbs/74/71064441.jpg "arxiv: v2 [cs.cv] 17 Aug 2015")
EventNet: A Large Scale Structured Concept Library for Complex Event Detection in Video Guangnan Ye, Yitong Li, Hongliang Xu, Dong Liu, Shih-Fu Chang Department of Electrical Engineering, Columbia University,
Action recognition in videos
Action recognition in videos Cordelia Schmid INRIA Grenoble Joint work with V. Ferrari, A. Gaidon, Z. Harchaoui, A. Klaeser, A. Prest, H. Wang Action recognition - goal Short actions, i.e. drinking, sit
Columbia University High-Level Feature Detection: Parts-based Concept Detectors
TRECVID 2005 Workshop Columbia University High-Level Feature Detection: Parts-based Concept Detectors Dong-Qing Zhang, Shih-Fu Chang, Winston Hsu, Lexin Xie, Eric Zavesky Digital Video and Multimedia Lab
Research Article High-Level Codewords Based on Granger Causality for Video Event Detection
Advances in Multimedia Volume 2015, Article ID 698316, 10 pages http://dx.doi.org/10.1155/2015/698316 Research Article High-Level Codewords Based on Granger Causality for Video Event Detection Shao-nian
Experimenting VIREO-374: Bag-of-Visual-Words and Visual-Based Ontology for Semantic Video Indexing and Search
Experimenting VIREO-374: Bag-of-Visual-Words and Visual-Based Ontology for Semantic Video Indexing and Search Chong-Wah Ngo, Yu-Gang Jiang, Xiaoyong Wei Feng Wang, Wanlei Zhao, Hung-Khoon Tan and Xiao
EVENT DETECTION AND HUMAN BEHAVIOR RECOGNITION. Ing. Lorenzo Seidenari
EVENT DETECTION AND HUMAN BEHAVIOR RECOGNITION Ing. Lorenzo Seidenari e-mail: seidenari@dsi.unifi.it What is an Event? Dictionary.com definition: something that occurs in a certain place during a particular
Video annotation based on adaptive annular spatial partition scheme
Video annotation based on adaptive annular spatial partition scheme Guiguang Ding a), Lu Zhang, and Xiaoxu Li Key Laboratory for Information System Security, Ministry of Education, Tsinghua National Laboratory
EE 6882 Statistical Methods for Video Indexing and Analysis
EE 6882 Statistical Methods for Video Indexing and Analysis Fall 2004 Prof. ShihFu Chang http://www.ee.columbia.edu/~sfchang Lecture 1 part A (9/8/04) 1 EE E6882 SVIA Lecture #1 Part I Introduction Course
Easy Samples First: Self-paced Reranking for Zero-Example Multimedia Search
Easy Samples First: Self-paced Reranking for Zero-Example Multimedia Search Lu Jiang 1, Deyu Meng 2, Teruko Mitamura 1, Alexander G. Hauptmann 1 1 School of Computer Science, Carnegie Mellon University
Rushes Video Segmentation Using Semantic Features
Rushes Video Segmentation Using Semantic Features Athina Pappa, Vasileios Chasanis, and Antonis Ioannidis Department of Computer Science and Engineering, University of Ioannina, GR 45110, Ioannina, Greece
FCVID: Fudan-Columbia Video Dataset
FCVID: Fudan-Columbia Video Dataset Yu-Gang Jiang, Zuxuan Wu, Jun Wang, Xiangyang Xue, Shih-Fu Chang Available at: http://bigvid.fudan.edu.cn/fcvid/ F 1 OVERVIEW Recognizing visual contents in unconstrained
Understanding and Predicting Interestingness of Videos
Understanding and Predicting Interestingness of Videos 1/1 Yu-Gang Jiang, Yanran Wang, Rui Feng, Xiangyang Xue, Yingbin Zheng, Hanfang Yang School of Computer Science, Fudan University, Shanghai, China
The Stanford/Technicolor/Fraunhofer HHI Video Semantic Indexing System
The Stanford/Technicolor/Fraunhofer HHI Video Semantic Indexing System Our first participation on the TRECVID workshop A. F. de Araujo 1, F. Silveira 2, H. Lakshman 3, J. Zepeda 2, A. Sheth 2, P. Perez
Exploiting Web Images for Event Recognition in Consumer Videos: A Multiple Source Domain Adaptation Approach
Exploiting Web Images for Event Recognition in Consumer Videos: A Multiple Source Domain Adaptation Approach Lixin Duan Dong Xu Nanyang Technological University 50 Nanyang Avenue, Singapore {S080003,dongxu}@ntu.edu.sg
Extracting Spatio-temporal Local Features Considering Consecutiveness of Motions
Extracting Spatio-temporal Local Features Considering Consecutiveness of Motions Akitsugu Noguchi and Keiji Yanai Department of Computer Science, The University of Electro-Communications, 1-5-1 Chofugaoka,
Class 9 Action Recognition
Class 9 Action Recognition Liangliang Cao, April 4, 2013 EECS 6890 Topics in Information Processing Spring 2013, Columbia University http://rogerioferis.com/visualrecognitionandsearch Visual Recognition
IEEE TRANSACTIONS ON MULTIMEDIA 1. VRFP: On-the-fly Video Retrieval using Web Images and Fast Fisher Vector Products
IEEE TRANSACTIONS ON MULTIMEDIA 1 VRFP: On-the-fly Video Retrieval using Web Images and Fast Fisher Vector Products Xintong Han*, Bharat Singh*, Vlad I. Morariu, Larry S. Davis, Fellow, IEEE Abstract VRFP
QMUL-ACTIVA: Person Runs detection for the TRECVID Surveillance Event Detection task
QMUL-ACTIVA: Person Runs detection for the TRECVID Surveillance Event Detection task Fahad Daniyal and Andrea Cavallaro Queen Mary University of London Mile End Road, London E1 4NS (United Kingdom) {fahad.daniyal,andrea.cavallaro}@eecs.qmul.ac.uk
Motion Interchange Patterns for Action Recognition in Unconstrained Videos
Motion Interchange Patterns for Action Recognition in Unconstrained Videos Orit Kliper-Gross, Yaron Gurovich, Tal Hassner, Lior Wolf Weizmann Institute of Science The Open University of Israel Tel Aviv
The Caltech-UCSD Birds Dataset
The Caltech-UCSD Birds-200-2011 Dataset Catherine Wah 1, Steve Branson 1, Peter Welinder 2, Pietro Perona 2, Serge Belongie 1 1 University of California, San Diego 2 California Institute of Technology
Experiments of Image Retrieval Using Weak Attributes
Columbia University Computer Science Department Technical Report # CUCS 005-12 (2012) Experiments of Image Retrieval Using Weak Attributes Felix X. Yu, Rongrong Ji, Ming-Hen Tsai, Guangnan Ye, Shih-Fu
CS231N Section. Video Understanding 6/1/2018
CS231N Section Video Understanding 6/1/2018 Outline Background / Motivation / History Video Datasets Models Pre-deep learning CNN + RNN 3D convolution Two-stream What we ve seen in class so far... Image
Multiple Kernel Learning for Emotion Recognition in the Wild
Multiple Kernel Learning for Emotion Recognition in the Wild Karan Sikka, Karmen Dykstra, Suchitra Sathyanarayana, Gwen Littlewort and Marian S. Bartlett Machine Perception Laboratory UCSD EmotiW Challenge,
WIth ever expanding multimedia collections, multimedia
Knowledge Adaptation with Partially Shared Features for Event Detection Using Few Exemplars Zhigang Ma, Yi Yang, Nicu Sebe, and Alexander G. Hauptmann Abstract Multimedia event detection (MED) is an emerging
Minimally Needed Evidence for Complex Event Recognition in Unconstrained Videos
Minimally Needed Evidence for Complex Event Recognition in Unconstrained Videos Subhabrata Bhattacharya Felix X. Yu Shih-Fu Chang Digital Video and Multimedia Lab Columbia University, New York, USA {subh,yuxinnan,sfchang}@ee.columbia.edu
Research Statement Subhabrata Bhattacharya
Research Statement Subhabrata Bhattacharya My research in computer vision brings together machine learning, insights from psychology, computer graphics, algorithms, and a great deal of computation. Over
Wikipedia Retrieval Task ImageCLEF 2011
Wikipedia Retrieval Task ImageCLEF 2011 Theodora Tsikrika University of Applied Sciences Western Switzerland, Switzerland Jana Kludas University of Geneva, Switzerland Adrian Popescu CEA LIST, France Outline
Content-based image and video analysis. Event Recognition
Content-based image and video analysis Event Recognition 21.06.2010 What is an event? a thing that happens or takes place, Oxford Dictionary Examples: Human gestures Human actions (running, drinking, etc.)
Utilizing Semantic Word Similarity Measures for Video Retrieval
Utilizing Semantic Word Similarity Measures for Video Retrieval Yusuf Aytar Computer Vision Lab, University of Central Florida yaytar@cs.ucf.edu Mubarak Shah Computer Vision Lab, University of Central
Modeling Concept Dependencies for Event Detection
Modeling Concept Dependencies for Event Detection Ethem F. Can Center for Intelligent Information Retrieval (CIIR) School of Computer Science UMass Amherst, MA, 01002 efcan@cs.umass.edu R. Manmatha Center
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification Zuxuan Wu, Xi Wang, Yu-Gang Jiang, Hao Ye, Xiangyang Xue School of Computer Science, Shanghai Key Lab of Intelligent
Exploring Geotagged Images for Land-Use Classification
Exploring Geotagged Images for Land-Use Classification Daniel Leung Electrical Engineering & Computer Science University of California, Merced Merced, CA 95343 cleung3@ucmerced.edu Shawn Newsam Electrical
Automatic summarization of video data
Automatic summarization of video data Presented by Danila Potapov Joint work with: Matthijs Douze Zaid Harchaoui Cordelia Schmid LEAR team, nria Grenoble Khronos-Persyvact Spring School 1.04.2015 Definition
Fast and Accurate Content-based Semantic Search in 100M Internet Videos
Fast and Accurate Content-based Semantic Search in 100M Internet Videos Lu Jiang 1, Shoou-I Yu 1, Deyu Meng 2, Yi Yang 3, Teruko Mitamura 1, Alexander Hauptmann 1 1 Carnegie Mellon University 2 Xi an Jiaotong
There is No Data Like Less Data: Percepts for Video Concept Detection on Consumer-Produced Media
There is No Data Like Less Data: Percepts for Video Concept Detection on Consumer-Produced Media Benjamin Elizalde Gerald Friedland International Computer International Computer Science Institute Science
VideoStory: A New Multimedia Embedding for Few-Example Recognition and Translation of Events
VideoStory: A New Multimedia Embedding for Few-Example Recognition and Translation of Events Amirhossein Habibian, Thomas Mensink, Cees G. M. Snoek ISLA, Informatics Institute, University of Amsterdam
Explicit Performance Metric Optimization for Fusion-based Video Retrieval
Explicit Performance Metric Optimization for Fusion-based Video Retrieval Ilseo Kim, Sangmin Oh, Byungki Byun, A. G. Amitha Perera, Chin-Hui Lee Georgia Institute of Technology, Kitware Inc., Microsoft
Concept Based Framework for Detecting High Level Events in Video
Concept Based Framework for Detecting High Level Events in Video Shih Fu Chang Columbia University September 18 th, 2014 University of Maryland Acknowledgments: This work was supported by the Intelligence
Large-scale Video Classification with Convolutional Neural Networks
Large-scale Video Classification with Convolutional Neural Networks Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, Li Fei-Fei Note: Slide content mostly from : Bay Area
Understanding Sport Activities from Correspondences of Clustered Trajectories
Understanding Sport Activities from Correspondences of Clustered Trajectories Francesco Turchini, Lorenzo Seidenari, Alberto Del Bimbo http://www.micc.unifi.it/vim Introduction The availability of multimedia
Semantic Model Vectors for Complex Video Event Recognition
IEEE TRANSACTIONS ON MULTIMEDIA 1 Semantic Model Vectors for Complex Video Event Recognition Michele Merler, Student Member, IEEE, Bert Huang, Lexing Xie, Member, IEEE, Gang Hua, Senior Member, IEEE, and
Summarization of Egocentric Moving Videos for Generating Walking Route Guidance
Summarization of Egocentric Moving Videos for Generating Walking Route Guidance Masaya Okamoto and Keiji Yanai Department of Informatics, The University of Electro-Communications 1-5-1 Chofugaoka, Chofu-shi,
Additional Remarks on Designing Category-Level Attributes for Discriminative Visual Recognition
Columbia University Computer Science Department Technical Report # CUCS 007-13 (2013) Additional Remarks on Designing Category-Level Attributes for Discriminative Visual Recognition Felix X. Yu, Liangliang
RECOGNITION OF COMPLEX EVENTS IN OPEN-SOURCE WEB-SCALE VIDEOS: FEATURES, INTERMEDIATE REPRESENTATIONS AND THEIR TEMPORAL INTERACTIONS
RECOGNITION OF COMPLEX EVENTS IN OPEN-SOURCE WEB-SCALE VIDEOS: FEATURES, INTERMEDIATE REPRESENTATIONS AND THEIR TEMPORAL INTERACTIONS by SUBHABRATA BHATTACHARYA B.E. Burdwan University (India), 2003 A
Two-layers re-ranking approach based on contextual information for visual concepts detection in videos
Author manuscript, published in "CBMI 2012 - Internationnal Workshop on Content-Based Multimedia Indexing, Annecy : France (2012)" DOI : 10.1109/CBMI.2012.6269837 Two-layers re-ranking approach based on
Columbia University TRECVID-2005 Video Search and High-Level Feature Extraction
Columbia University TRECVID-2005 Video Search and High-Level Feature Extraction Shih-Fu Chang, Winston Hsu, Lyndon Kennedy, Lexing Xie, Akira Yanagawa, Eric Zavesky, Dong-Qing Zhang Digital Video and Multimedia
Action Recognition in Video by Sparse Representation on Covariance Manifolds of Silhouette Tunnels
Action Recognition in Video by Sparse Representation on Covariance Manifolds of Silhouette Tunnels Kai Guo, Prakash Ishwar, and Janusz Konrad Department of Electrical & Computer Engineering Motivation
Short-Term Audio-Visual Atoms for Generic Video Concept Classification
Short-Term Audio-Visual Atoms for Generic Video Concept Classification Authors Wei Jiang Courtenay Cotton Shih-Fu Chang Dan Ellis Alexander C. Loui Presenters Armin Samii Images from the interwebs, 2009
88 IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 14, NO. 1, FEBRUARY 2012
88 IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 14, NO. 1, FEBRUARY 2012 Semantic Model Vectors for Complex Video Event Recognition Michele Merler, Student Member, IEEE, Bert Huang, Lexing Xie, Senior Member,
In Search of Video Event Semantics
In Search of Video Event Semantics Masoud Mazloom This book was typeset by the author using latex. Printing: Off Page, Amsterdam Copyright c 2016 by M. Mazloom. All rights reserved. No part of this publication
Content-based Video Genre Classification Using Multiple Cues
Content-based Video Genre Classification Using Multiple Cues Hazım Kemal Ekenel Institute for Anthropomatics Karlsruhe Institute of Technology (KIT) 76131 Karlsruhe, Germany ekenel@kit.edu Tomas Semela
Generating Annotations for How-to Videos Using Crowdsourcing
Generating Annotations for How-to Videos Using Crowdsourcing Phu Nguyen MIT CSAIL 32 Vassar St. Cambridge, MA 02139 phun@mit.edu Abstract How-to videos can be valuable teaching tools for users, but searching
High-level event recognition in unconstrained videos
Int J Multimed Info Retr (2013) 2:73 101 DOI 10.1007/s13735-012-0024-2 TRENDS AND SURVEYS High-level event recognition in unconstrained videos Yu-Gang Jiang Subhabrata Bhattacharya Shih-Fu Chang Mubarak
CCRMA MIR Workshop 2014 Evaluating Information Retrieval Systems. Leigh M. Smith Humtap Inc.
CCRMA MIR Workshop 2014 Evaluating Information Retrieval Systems Leigh M. Smith Humtap Inc. leigh@humtap.com Basic system overview Segmentation (Frames, Onsets, Beats, Bars, Chord Changes, etc) Feature
Audio Features for Automatic Video Tagging
Faculty of Media Engineering and Technology German University in Cairo Multimedia Analysis and Data Mining Group Deutsches Forschungszentrum Für Künstliche Intelligenz GmbH Audio Features for Automatic
Hands On: Multimedia Methods for Large Scale Video Analysis (Project Meeting) Dr. Gerald Friedland,
Hands On: Multimedia Methods for Large Scale Video Analysis (Project Meeting) Dr. Gerald Friedland, fractor@icsi.berkeley.edu 1 Today Today Project Requirements Today Project Requirements Data available
Person Action Recognition/Detection
Person Action Recognition/Detection Fabrício Ceschin Visão Computacional Prof. David Menotti Departamento de Informática - Universidade Federal do Paraná 1 In object recognition: is there a chair in the
Recognizing people. Deva Ramanan
Recognizing people Deva Ramanan The goal Why focus on people? How many person-pixels are in a video? 35% 34% Movies TV 40% YouTube Let s start our discussion with a loaded question: why is visual recognition
Video Content Recognition with Deep Learning
Video Content Recognition with Deep Learning Yu-Gang Jiang Lab for Big Video Data Analytics (BigVid) Fudan University, Shanghai, China ygj@fudan.edu.cn Joint work with: Zuxuan Wu, Xi Wang, Hao Ye, Xiangyang
Query-focused Video Summarization
Query-focused Video Summarization Jacob Laurel 1, Aidean Sharghi 2, and Boqing Gong 2 1 University of Alabama at Birmingham 2 University of Central Florida {jslaurel, aidean, bgong}@crcv.ucf.edu Abstract
Evaluating multimedia features and fusion for example-based event detection
Machine Vision and Applications DOI 10.1007/s00138-013-0527-8 SPECIAL ISSUE PAPER Evaluating multimedia features and fusion for example-based event detection Gregory K. Myers Ramesh Nallapati Julien van
COSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor
COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality
CS229: Action Recognition in Tennis
CS229: Action Recognition in Tennis Aman Sikka Stanford University Stanford, CA 94305 Rajbir Kataria Stanford University Stanford, CA 94305 asikka@stanford.edu rkataria@stanford.edu 1. Motivation As active
AUTOMATIC CONSUMER VIDEO SUMMARIZATION BY AUDIO AND VISUAL ANALYSIS
AUTOMATIC CONSUMER VIDEO SUMMARIZATION BY AUDIO AND VISUAL ANALYSIS Wei Jiang, Courtenay Cotton 2, Alexander C. Loui Corporate Research and Engineering, Eastman Kodak Company, Rochester, NY 2 Electrical
Audio-Visual Atoms for Generic Video Concept Classification
Audio-Visual Atoms for Generic Video Concept Classification 14 WEI JIANG, COURTENAY COTTON, SHIH-FU CHANG, DAN ELLIS, Columbia University ALEXANDER C. LOUI, Eastman Kodak Company We investigate the challenging
ON-LINE GENRE CLASSIFICATION OF TV PROGRAMS USING AUDIO CONTENT
ON-LINE GENRE CLASSIFICATION OF TV PROGRAMS USING AUDIO CONTENT Samuel Kim 1, Panayiotis Georgiou 2, Shrikanth Narayanan 2 1 DSP Lab., Yonsei University, Seoul, Korea 2 SAIL Lab., University of Southern
Event Recognition in Videos by Learning from Heterogeneous Web Sources
013 IEEE Conference on Computer Vision and Pattern Recognition Event Recognition in Videos by Learning from Heterogeneous Web Sources Lin Chen 1, Lixin Duan, Dong Xu 1 1 School of Computer Engineering,
Sparse coding for image classification
Sparse coding for image classification Columbia University Electrical Engineering: Kun Rong(kr2496@columbia.edu) Yongzhou Xiang(yx2211@columbia.edu) Yin Cui(yc2776@columbia.edu) Outline Background Introduction
Video Event Detection Using Motion Relativity and Feature Selection
IEEE TRANSACTIONS ON MULTIMEDIA, VOL. XX, NO. XX, XX 1 Video Event Detection Using Motion Relativity and Feature Selection Feng Wang, Zhanhu Sun, Yu-Gang Jiang, and Chong-Wah Ngo Abstract Event detection
Latent Variable Models for Structured Prediction and Content-Based Retrieval
Latent Variable Models for Structured Prediction and Content-Based Retrieval Ariadna Quattoni Universitat Politècnica de Catalunya Joint work with Borja Balle, Xavier Carreras, Adrià Recasens, Antonio
Video2vec Embeddings Recognize Events when Examples are Scarce
Video2vec Embeddings Recognize Events when Examples are Sce Amirhossein Habibian, Thomas Mensink, and Cees G. M. Snoek 1 Abstract This paper aims for event recognition when video examples are sce or even
Feature Weighting via Optimal Thresholding for Video Analysis
23 IEEE International Conference on Computer Vision Feature Weighting via Optimal Thresholding for Video Analysis Zhongwen Xu Yi Yang Ivor Tsang Nicu Sebe Alexander G. Hauptmann ITEE, The University of
Attributes. Computer Vision. James Hays. Many slides from Derek Hoiem
Many slides from Derek Hoiem Attributes Computer Vision James Hays Recap: Human Computation Active Learning: Let the classifier tell you where more annotation is needed. Human-in-the-loop recognition:
Structured Models in. Dan Huttenlocher. June 2010
Structured Models in Computer Vision i Dan Huttenlocher June 2010 Structured Models Problems where output variables are mutually dependent or constrained E.g., spatial or temporal relations Such dependencies
Baseball Game Highlight & Event Detection
Baseball Game Highlight & Event Detection Student: Harry Chao Course Adviser: Winston Hu 1 Outline 1. Goal 2. Previous methods 3. My flowchart 4. My methods 5. Experimental result 6. Conclusion & Future
Short-Term Audio-Visual Atoms for Generic Video Concept Classification
Short-Term Audio-Visual Atoms for Generic Video Concept Classification Wei Jiang Courtenay Cotton Shih-Fu Chang Dan Ellis Alexander C. Loui 2 Electrical Engineering Department, Columbia University, New
Local Features and Kernels for Classifcation of Texture and Object Categories: A Comprehensive Study
Local Features and Kernels for Classifcation of Texture and Object Categories: A Comprehensive Study J. Zhang 1 M. Marszałek 1 S. Lazebnik 2 C. Schmid 1 1 INRIA Rhône-Alpes, LEAR - GRAVIR Montbonnot, France
VIREO/DVMM at TRECVID 2009: High-Level Feature Extraction, Automatic Video Search, and Content-Based Copy Detection. Abstract
VIREO/DVMM at TRECVID 2009: High-Level Feature Extraction, Automatic Video Search, and Content-Based Copy Detection Chong-Wah Ngo, Yu-Gang Jiang, Xiao-Yong Wei, Wanlei Zhao, Yang Liu, Jun Wang, Shiai Zhu,
Hands On: Multimedia Methods for Large Scale Video Analysis (Lecture) Dr. Gerald Friedland,
Hands On: Multimedia Methods for Large Scale Video Analysis (Lecture) Dr. Gerald Friedland, fractor@icsi.berkeley.edu 1 Today Recap: Some more Machine Learning Multimedia Systems An example Multimedia
Multimedia Event Detection using Visual Features
Multimedia Event Detection using Visual Features Dishan Gupta, Rahul Goutam, Amos Ng Language Technologies Institute Carnegie Mellon University Pittsburgh, PA 15213 { dishang, rgoutam, ajng }@cs.cmu.edu
YouTube-8M Video Classification
YouTube-8M Video Classification Alexandre Gauthier and Haiyu Lu Stanford University 450 Serra Mall Stanford, CA 94305 agau@stanford.edu hylu@stanford.edu Abstract Convolutional Neural Networks (CNNs) have
Interactive Event Search Through Transfer Learning
Interactive Event Search Through Transfer Learning Antony Lam, 1 Amit K. Roy-Chowdhury, 2 and Christian R. Shelton 1 1 Dept. of Computer Science & Engineering, University of California, Riverside {antonylam,cshelton}@cs.ucr.edu
Bag of Words or Bag of Features. Lecture-16 (Slides Credit: Cordelia Schmid LEAR INRIA Grenoble)
Bag of Words or Bag of Features Lecture-16 (Slides Credit: Cordelia Schmid LEAR INRIA Grenoble) Contents Interest Point Detector Interest Point Descriptor K-means clustering Support Vector Machine (SVM)
MACHINE LEARNING: CLUSTERING, AND CLASSIFICATION. Steve Tjoa June 25, 2014
MACHINE LEARNING: CLUSTERING, AND CLASSIFICATION Steve Tjoa kiemyang@gmail.com June 25, 2014 Review from Day 2 Supervised vs. Unsupervised Unsupervised - clustering Supervised binary classifiers (2 classes)
A System that Learns to Tag Videos by Watching Youtube
A System that Learns to Tag Videos by Watching Youtube Adrian Ulges 1,2, Christian Schulze 2, Daniel Keysers 2, Thomas M. Breuel 1,2 1 Department of Computer Science, Technical University of Kaiserslautern
Segmentation as Selective Search for Object Recognition in ILSVRC2011
Segmentation as Selective Search for Object Recognition in ILSVRC2011 Koen van de Sande Jasper Uijlings Arnold Smeulders Theo Gevers Nicu Sebe Cees Snoek University of Amsterdam, University of Trento ILSVRC2011
LabROSA Research Overview
LabROSA Research Overview Dan Ellis Laboratory for Recognition and Organization of Speech and Audio Dept. Electrical Eng., Columbia Univ., NY USA dpwe@ee.columbia.edu 1. Music 2. Environmental sound 3.
TagBook: A Semantic Video Representation without Supervision for Event Detection
IEEE TRANSACTIONS ON MULTIMEDIA, VOL., NO., APRIL 2016 1 TagBook: A Semantic Video Representation without Supervision for Event Detection Masoud Mazloom, Xirong Li*, and Cees G.M. Snoek, Senior Member,
Ranking Error-Correcting Output Codes for Class Retrieval
Ranking Error-Correcting Output Codes for Class Retrieval Mehdi Mirza-Mohammadi, Francesco Ciompi, Sergio Escalera, Oriol Pujol, and Petia Radeva Computer Vision Center, Campus UAB, Edifici O, 08193, Bellaterra,
UCF-CRCV at TRECVID 2014: Semantic Indexing
UCF-CRCV at TRECVID 214: Semantic Indexing Afshin Dehghan 1, Mahdi M. Kalayeh 1, Yang Zhang 1, Haroon Idrees 1, Yicong Tian 1, Amir Mazaheri 1, Mubarak Shah 1, Jingen Liu 2, and Hui Cheng 2 1 1 Center
Unsupervised Spectral Dual Assignment Clustering of Human Actions in Context
Unsupervised Spectral Dual Assignment Clustering of Human Actions in Context Simon Jones, Ling Shao Department of Electronic and Electrical Engineering The University of Sheffield, Sheffield, S1 3JD, UK
IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 14, NO. 2, APRIL Efficient Genre-Specific Semantic Video Indexing
IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 14, NO. 2, APRIL 2012 291 Efficient Genre-Specific Semantic Video Indexing Jun Wu and Marcel Worring, Member, IEEE Abstract Large video collections such as YouTube
Highlights Extraction from Unscripted Video
Highlights Extraction from Unscripted Video T 61.6030, Multimedia Retrieval Seminar presentation 04.04.2008 Harrison Mfula Helsinki University of Technology Department of Computer Science, Espoo, Finland
Zero-Example Event Detection and Recounting
Zero-Example Event Detection and Recounting Speaker: Yi-Jie Lu Yi-Jie Lu, Hao Zhang, Ting Yao, Chong-Wah Ngo On behalf of VIREO Group, City University of Hong Kong Feb. 12, 2015 Outline Multimedia Event