Vannotea Real-time collaborative indexing, annotation, discussion tools for Film/video, images, 3D objects

Transcription

1 Vannotea Real-time collaborative indexing, annotation, discussion tools for Film/video, images, 3D objects Ronald Schroeter DSTC Dr. Jane Hunter, DSTC Michael Henderson, DSTC

2 Aims and Scope Collaborative objective subjective Search Indexing Discussion Digital Film/Video Browsing Retrieval Description Synchronous Annotation

3 GrangeNet

4 Access Grid Nodes Videoconferencing over broadband Large-scale, multiple groups Interactive collaborative meetings High quality video/audio Low latency Multiple cameras, mikes, projectors, echo cancellation Lack of collaborative support software

5 Access Grid Session

6 Requirements User friendliness High quality content delivery details matter Flexible metadata architecture distinction between objective & subjective support of international standards for maximum interoperability dynamic domain-specific UI Integration into Access Grid Nodes e.g. support of audiovisual annotations recordings from videoconferencing Flexible and seamless way of collaboration

7 Existing Annotation Tools IBM s Video Annotation Tool Ricoh MovieTool ZGDV s VIDETO COALA s LogCreator INA/INRIA s System all non-collaborative, asynchronous indexing and annotation systems Microsoft s Research Annotation System (MRAS), DTVI (Distributed Tutored Video Instruction) VidGrid UK esocial-science project None support real-time collaborative browsing and annotation of 25-30fps video or 3D objects

8 Applications Telemedicine Online discussions and content sharing between clinicians CyberSTEM Networked Telemicroscopy Remote Mentoring (Performance, Music, Sports) Indigenous Knowledge Sharing Museums and Indigenous communities Behavioral Studies Education - Film/Media/Communications Defence Surveillance, Field Operations

9 System Architecture

10 Challenges Part I Supporting various high quality multimedia formats MPEG-2, JPEG2000 and 3D objects Search, browse and retrieval of content based on two separate metadata stores Objective indexing (content & rights management) Using Tamino XML Database & W3C XQuery XSLT provides interoperability to Metadata Application Profile (MPEG-7 and Dublin Core) Incl. automatic shot detection (Mediaware SDK) Dynamic UI generation through.net Reflection Subjective annotation (knowledge management) W3C Annotea + extension to allow annotation of multimedia content (e.g. spatio-temporal fragment identifiers) Leveraging existing Annotea Servers

11 Challenges Part II Synchronous collaboration through.net Remoting Multiple users in control simultaneously (e.g. multiple mouse pointers, similar to gaming environment) Fine grained control over users collaborative participation Access control in collaborative sessions MPEG-2 Streaming using VideoLAN and multicast Video quality by far surpasses result of Application Sharing Protocols (e.g. NetMeeting or VNC)

12 User-Interface

13 User-Interface

14 User-Interface Video Semi-Automatic Indexing/Description (DC/MPEG-7) & Access/Rights/Traditional Care Content Viewer (e.g. MPEG-2, JPEG2000, 3D objects) Shared Annotation/ Discussion (Annotea)

15 User-Interface 3D object Semi-Automatic Indexing/Description (DC/MPEG-7) & Access/Rights/Traditional Care Content Viewer (e.g. MPEG-2, JPEG2000, 3D objects) Shared Annotation/ Discussion (Annotea)

16

17 Indexing Metadata Bibliographic title, creator, genre, dates, etc Formatting codec, framerate, audio format, aspect ratio Structural temporal segmentation Semantic Descriptions - based on domain-specific ontologies

18 Annotation Metadata [Ref:

19 Annotation Metadata [Ref:

20 Multimedia Annotations Context needs to point to spatial/temporal locations Segments, Frames, Points/Areas within 3D objects Extend XPointer/URI to point to audiovisual fragments <a:context>udp://server:port/mediafile.mp2 #start=1234&end=2345<a:context> 3D <a:context>udp://server:port/object.x# id=13234 <a:context>

21 Synchronous Collaboration Application Sharing Protocols Pro: General framework share any application Computer A application netmeeting Cons: Inefficient for digital video poor quality (5fps) Restrictive & inflexible for extended collaboration Computer B Mouse/Keys Screen/Window delta application-sharing netmeeting

22 Synchronous Collaboration.Net Remoting

23 Synchronous Collaboration Multi-User

24 Synchronous Collaboration.Net Remoting + MPEG-2 Multicast UDP Streaming

25 User-Interface Access Grid Node Video Conferencing + Collaborative Application

26 Future Directions VIRGIL - Virtual Meeting Archival Replace.NET Remoting with messaging-based Jabber record, index and audit collaborative sessions integration integration with CoAKTinG/MEMETIC (UK e-science project) Access control on content, annotations, application features using Shibboleth & XACML Capture, replay and annotation of real-time video streams (remote tutoring) Platform independent, browser-based implementation Domain-specific ontology-guided annotations recording of provenance data dynamic extension of ontologies

27 References