About Me. Bio: Research Interests:

Transcription

1 About Me Bio: Sr. Staff Researcher, Core Networks R&D, Huawei Tech USA, ~ to date Asst Prof, HK Polytechnic Univ, CTO, Mudi Tech, ~ Senior, Senior Staff, and then Principal Staff Researcher, Multimedia Research Lab, Motorola Labs, USA, Research Interests: Biometrics & Surveillance, large scale fingerprint identification, face identification, video action and event detection. Image Identification in very large repositories. Check out our visual search Android app for dangdang.com: Large scale audio/visual data analysis, storage and indexing, search and mining, Hadoop based map-reduce scheme for large scale visual analytics. QoE metrics, video adaptation and streaming, content delivery networks (CDN) optimization, especially the next gen wireless video networks. Z. Li

2 Research Motivations 2012 Z. Li

3 Devices Explosive growth of devices: Billions of cell phones/pdas Billions of computers Billions of TVs Billions of Media Players Different Multimedia Capabilities in: display, capture, storage, computing, communication Z. Li

4 Networks Better technology from equipment makers Better wireless spectrum efficiency, WiMAX/LTE High speed DLS/Cable, 100x100 Fiber optical solutions, GPON More capacity from service providers More bandwidth, better coverage, Convergence of data, voice and media service from service providers Vertical integration of application and services Z. Li

5 Content Explosive growth of digital media Web, , Audio, Video, Game News, Music, Movie, Talk show, Game, 2 nd Life. Rapid changes in the way contents are produced and consumed Personal vs Commercial Passive (TV) vs Interactive (Blog, Game) Centralized vs P2P Z. Li

6 Key Challenges Application Needs: Good Access, be able to get what you want, a storage and communication problem Mobility across devices and access sessions: anywhere, on any device, not tied to a single device/location, get what they want, with good media quality (coding) and availability (communication/networking). Intelligence and Personalization: be able to utilize the big data collected from both networks and applications, and drive more intelligent and personalized applications:» Audio/visual content search» Social media analysis, graph induced personal profiles.» Web scale media data and human behavior analysis.» Surveillance & Security Z. Li

7 Technology Gap Technology Gap? Multimedia communication:» Rapidly widening gap between wireless capacity and explosive growth of mobile video traffic, 14 times in 2015 according to Verizon.» Richer QoE metrics» More efficient coding and adaptation schemes» Re-engineering the Internet for video traffic Media storage:» Storage of multimedia content in cloud explore various error-resilience and rate-distortion tradeoff characteristics to enable differential storage service. Web scale media computing:» Web scale multimedia data analysis, indexing and retrieval, need both algorithm magic, and the massive power of cloud computing & storage Z. Li

8 Research Projects Highlights Z. Li

9 Mobile Visual Search Robust Image Identification in Large Repositories: Find robust keypoint features that can be re-captured in visual queries Find novel coding scheme for the point set topology of the keypoints for discrimination and re-ranking Mobile Query Compression : Query-by-Capture, when shopping in the malls, just took a picture/video of the product and search the online stores to compare prices. Find coding scalable coding and query processing schemes to minimize wireless traffic, while achieving good accuracy in retrieval Dangdang.com Demo! Z. Li

10 Video Analytics Video Duplicates Detection Scaled Eigen Appearance Modeling and Duplicate Likelihood pruning Robust and scalable video fingerprint for duplicate and near duplicate search and mining applications (Funded by Microsoft Research grant, HK RGC GRF grant) Excellent performance in accuracy (98% precision on 100% recall), and speed (0.03 sec response time for checking 100 hours of video) Applications:» Web scale video repository copyright protection» Video segment search and content association analysis» Video bookmarks for social media applications Z. Li

11 Video Analytics Video Action Recognition Video Action and Event Recognition: Video action and human behavior recognition based on spatio-temporal appearance volume manifold modeling, exploring tensorial, spline approaches, and novel learning solutions, e.g, aligned projection, HMM, and DBN. ( Funded by HK RGC and PolyU internal grant) Applications: surveillance, video biometrics, video search, social networks Z. Li

12 Subspace Learning on Grassmannian Manifold Subspace Indexing on Grassmannian Manifold: For a large subject set pattern recognition problem, single subspace model s DoF is not enough for robust recognition Instead, develop a rich set of transforms that better captures local data characteristics, and Develop a hierarchical index for subspaces on the Grassmann manifold. Applications: large subject set face recognition, speaker ID, and hierarchical transforms for image coding. Z. Li

13 Video Networking QoE Metric and Resource Pricing Video over Wireless Expected capacity gap of 19 times by Verizon. Novel QoE metrics and adaptation scheme for improving the elasticity of video traffic, and improve QoE Distributed computing with resource pricing to manage wireless resource Demo! summary frames d(f k, f k-1 ) summary distortion λ= 16.0e-4, D(S)=24.6, R(S)=80.8kb d(f k, f k ) Z. Li

14 Video Neworking - MobileTV Joint Source-Channel Coding Supporting highly elastic and robust QoS for a variety of mobile terminals with graceful visual quality degradation with channel conditions Practical frame-size and frame rate adaptation for wide codec support. PHY layer adaptation through DE- STC (diversity embedding spacetime coding) to induce a set of embedded channels that best suit the multicast group channel distribution APP layer source-channel coding optimization with layered video and digital fountain code Targeting in-band, or dual-band (WiFi + 3G) wireless infrastructure. Z. Li

15 Video Networking NextGen CDN Next Generation Content Networks Video accounts for > 70% of internet traffic now Optimization and distributed computing solutions in IPTV video networking, Primal-Dual decomposition, resource pricing schemes for multi-access and IPTV video networks, (RGC New Staff Grant, in collaboration with Princeton University EDGE lab) Video TCP: TCP re-engineering for content delivery networks. Reconsider the congestion measure and pricing as well as source adaptation schemes in TCP to better suit for content intensive traffics. Caching and Network Coding schemes for video sharing in Mesh Networks (in collaboration with Prof Cao). Also integration of routing and flow control in (small scale) mesh networks for video multicasting. Z. Li