Vannotea Real-time collaborative indexing, annotation, discussion tools for Film/video, images, 3D objects Ronald Schroeter DSTC ronalds@dstc.edu.au Dr. Jane Hunter, DSTC Michael Henderson, DSTC
Aims and Scope Collaborative objective subjective Search Indexing Discussion Digital Film/Video Browsing Retrieval Description Synchronous Annotation
GrangeNet
Access Grid Nodes Videoconferencing over broadband Large-scale, multiple groups Interactive collaborative meetings High quality video/audio Low latency Multiple cameras, mikes, projectors, echo cancellation Lack of collaborative support software
Access Grid Session
Requirements User friendliness High quality content delivery details matter Flexible metadata architecture distinction between objective & subjective support of international standards for maximum interoperability dynamic domain-specific UI Integration into Access Grid Nodes e.g. support of audiovisual annotations recordings from videoconferencing Flexible and seamless way of collaboration
Existing Annotation Tools IBM s Video Annotation Tool Ricoh MovieTool ZGDV s VIDETO COALA s LogCreator INA/INRIA s System all non-collaborative, asynchronous indexing and annotation systems Microsoft s Research Annotation System (MRAS), DTVI (Distributed Tutored Video Instruction) VidGrid UK esocial-science project None support real-time collaborative browsing and annotation of 25-30fps video or 3D objects
Applications Telemedicine Online discussions and content sharing between clinicians CyberSTEM Networked Telemicroscopy Remote Mentoring (Performance, Music, Sports) Indigenous Knowledge Sharing Museums and Indigenous communities Behavioral Studies Education - Film/Media/Communications Defence Surveillance, Field Operations
System Architecture
Challenges Part I Supporting various high quality multimedia formats MPEG-2, JPEG2000 and 3D objects Search, browse and retrieval of content based on two separate metadata stores Objective indexing (content & rights management) Using Tamino XML Database & W3C XQuery XSLT provides interoperability to Metadata Application Profile (MPEG-7 and Dublin Core) Incl. automatic shot detection (Mediaware SDK) Dynamic UI generation through.net Reflection Subjective annotation (knowledge management) W3C Annotea + extension to allow annotation of multimedia content (e.g. spatio-temporal fragment identifiers) Leveraging existing Annotea Servers
Challenges Part II Synchronous collaboration through.net Remoting Multiple users in control simultaneously (e.g. multiple mouse pointers, similar to gaming environment) Fine grained control over users collaborative participation Access control in collaborative sessions MPEG-2 Streaming using VideoLAN and multicast Video quality by far surpasses result of Application Sharing Protocols (e.g. NetMeeting or VNC)
User-Interface
User-Interface
User-Interface Video Semi-Automatic Indexing/Description (DC/MPEG-7) & Access/Rights/Traditional Care Content Viewer (e.g. MPEG-2, JPEG2000, 3D objects) Shared Annotation/ Discussion (Annotea)
User-Interface 3D object Semi-Automatic Indexing/Description (DC/MPEG-7) & Access/Rights/Traditional Care Content Viewer (e.g. MPEG-2, JPEG2000, 3D objects) Shared Annotation/ Discussion (Annotea)
Indexing Metadata Bibliographic title, creator, genre, dates, etc Formatting codec, framerate, audio format, aspect ratio Structural temporal segmentation Semantic Descriptions - based on domain-specific ontologies
Annotation Metadata [Ref: http://www.w3.org/2001/annotea/papers/kcap01]
Annotation Metadata [Ref: http://www.w3.org/2001/annotea/papers/kcap01]
Multimedia Annotations Context needs to point to spatial/temporal locations Segments, Frames, Points/Areas within 3D objects Extend XPointer/URI to point to audiovisual fragments <a:context>udp://server:port/mediafile.mp2 #start=1234&end=2345<a:context> 3D <a:context>udp://server:port/object.x# id=13234 <a:context>
Synchronous Collaboration Application Sharing Protocols Pro: General framework share any application Computer A application netmeeting Cons: Inefficient for digital video poor quality (5fps) Restrictive & inflexible for extended collaboration Computer B Mouse/Keys Screen/Window delta application-sharing netmeeting
Synchronous Collaboration.Net Remoting
Synchronous Collaboration Multi-User
Synchronous Collaboration.Net Remoting + MPEG-2 Multicast UDP Streaming
User-Interface Access Grid Node Video Conferencing + Collaborative Application
Future Directions VIRGIL - Virtual Meeting Archival Replace.NET Remoting with messaging-based Jabber record, index and audit collaborative sessions integration integration with CoAKTinG/MEMETIC (UK e-science project) Access control on content, annotations, application features using Shibboleth & XACML Capture, replay and annotation of real-time video streams (remote tutoring) Platform independent, browser-based implementation Domain-specific ontology-guided annotations recording of provenance data dynamic extension of ontologies
References http://www.metadata.net/filmed