TF-Netcast. Final Report. Dan Mønster

Transcription

1 TF-Netcast Final Report Dan Mønster Sept. 2004

2

3 Contents 1 Introduction Background Aims of TF-Netcast Summary of Results Deliverable B: Report on Streaming Video Survey Content Production Streaming Portal Streaming Formats and Servers Media Players Metadata Future plans Prototype Announcement Portal Viewer s interface Broadcaster s interface Deliverable E: Information about Resources for Content Production 19 5 Deliverable F: Live-Streaming Infrastructure OpenCDN architecture OpenCDN operation Summary Deliverable G: Report on Video-on-Demand Metadata and Portal Dublin Core Conclusions iii

4 7 International Collaboration TF-Netcast Meetings Collaboration with TF-AACE Collaboration with ViDe Streaming Video Working Group Summary and Future Plans 35 A Glossary of Terms 39 References 42 iv

5 Chapter 1 Introduction This report summarises the work of TF-Netcast. TF-Netcast is a TERENA task force focusing on streaming media with an emphasis on services for the European academic sector, such as a portal for announcing live streaming events, Video-on-Demand portals, metadata for portals, and content delivery networking infrastructure. TF-Netcast also made a survey of the use of streaming media targeted towards higher education, and collected information about resources for content production. TF-Netcast was active during a one year mandate from the TERENA Technical Committee, running from March 2003 to March During this year the task force met once at the TERENA offices in Amsterdam for an inargural meeting, while the remaining task force meetings were held as multipoint videoconferences using the SWITCH videoconferencing service. A total of seven such meetings were held. The task force was chaired by Dan Mønster from UNI C and Harri Salminen from CSC acted as deputy chairman. Baiba Kaskina acted as TERENA contact person and secretary for the task force. The task force has had active participation from many people from all over Europe. The main results of TF-Netcast are the deliverables which include both software announcement portal and content delivery networking and written reports. All the deliverables and more information about TF-Netcast is available at TERENA s web site: Background From December 1999 to June 2001 the TERENA task force TF-STREAM that dealt broadly with real-time multimedia applications was active. TF-Netcast arose because of a need for continued collaboration, coordination and exchange of knowledge between professionals working with streaming media in academia and national research and educational networks (NRENs) in Europe. The first step towards the formation of TF-Netcast was a birds of a feather (BoF) session held at the 20th NORDUnet Conference in April 2002 in Copenhagen. The outcome of this BoF session 1

6 was a loosely organised group under the name Academic Netcasting Working Group. During 2002 this group held discussions on a TERENA mailing list and by the end of the year they had drafted the terms of reference for a new task force: TF-Netcast. TF-Netcast was started at an inargural meeting at the TERENA offices in Amsterdam in February 2003 and the revised terms of reference was formally approved by the TERENA Technical Council on March One of the visions of the Academic Netcasting Working Group is to set up an Academic Netcasting Channel providing high quality content to the academic community. While this could not be achieved with the limited resources and time frame available to a task force it was thought that TF-Netcast could provide a significant insight into the necessary technical infrastructure and organisational issues for an Academic Netcasting Channel. 1.2 Aims of TF-Netcast As taken from the terms of reference the aims of TF-Netcast are A to conduct a survey of the state of national efforts in producing and delivering streamingmedia content in academia in European countries, and of the (users ) needs for an international effort in this area; B to create an announcements portal, by developing metadata, portal software, policies etc. into a working prototype of an announcements portal; C to promote the production of high-quality content; D to study the feasibility of a project for setting up a live-streaming infrastructure; E to develop a common metadata model and to study the feasibility of setting up a video-ondemand portal; F to propose a model for an academic netcasting channel. The terms of reference also lists the deliverables to be produced in order to achieve those aims: A Project schedule and design document. B Report on survey. C Metadata model. D Prototype of announcements portal. E Information about resources for content production. F Report on live-streaming infrastructure. G Report on video-on-demand metadata and portal. 2

7 H Report on model for an academic netcasting channel. I Final report. Deliverable C was cancelled because it was realised that there was nearly complete overlap of this deliverable and deliverable G. Deliverable H was cancelled because the assigned leader of that deliverable withdrew from the task force and no new leader could be assigned. To agree on the definition of some of the terms used in the terms of reference a glossary explaining those terms was also included. This glossary is reproduced in Appendix A. 1.3 Summary of Results This section summarises the activities and results of TF-Netcast. Survey on streaming media TF-Netcast conducted a survey on the use of streaming media in higher education in Europe. The purpose of the survey was to get an overview of how streaming media is being used in Academia across Europe. The survey was conducted using a web based questionnaire targeted towards employees in higher education institutions who work with streaming media. The results of the survey have been published as TF-Netcast Deliverable B: Report on Streaming Video Survey [1]. Prototype announcement portal The announcement portal for TF-Netcast [2] is developed and run by CESNET. It provides a web based interface for announcing live netcasts. Submission of events can be made either through a web based interface or by sending an XML description of the event in a cryptographically signed . TF-Netcast has contributed localisation to several European languages to the CESNET developed live streaming announcement portal which is now available in nine different languages. A universal gateway module has been developed allowing submission of events by automatic means, such as e.g. the SAP harvester developed by UNINETT [3]. The portal is available at Information about resources for content production It was recognised early on that one of the major challenges in the formation of an Academic Netcasting Channel would be a sufficient amount of high quality content. It was decided to put together a list of resources for content producers that could serve as a starting point for finding information about content production. This list is published as TF-Netcast Deliverable E [4]. 3

8 live-streaming infrastructure In order to serve a very large audience or in order to save bandwidth to institutions with saturated WAN links it is necessary to have some kind of content delivery network in place. As part of TF- Netcast Alessandro Falaschi developed OpenCDN [5] which along with the report [6] consitutes deliverable F. So far OpenCDN supports only Apple s Darwin Streaming Server, but through an adaptation layer it is possible to add support for other types of server, such as RealServer and Windows Media Services. Report on video-on-demand metadata and portal An important issue in serving content on-demand is to describe the media files using metadata. A proper metadata description is necessary to enable searching media archives efficiently. In TF- Netcastdeliverable G: Report on video-on-demand metadata portals [7] Harri Salminen provides a review of the existing metadata models, discusses different methods for metadata exchange and provides recommendations for using metadata for Video-on-Demand portals. Provision of mirror sites for RIPE meetings Members of TF-Netcast provided mirrors for the Windows Media streaming from three RIPE meetings: RIPE 45, RIPE 46 and RIPE 47. In addition to RIPE s own server at the meeting venue and a server at RIPE NCC, mirrors were set up in the Czech Republic, Denmark, Greece, Italy, the Netherlands, Spain and Portugal. Each time the servers were set up manually, but based on work in TF-Netcast, viz., the announcement portal and Open CDN, such a set-up may be automated in the future. Joint Meeting with ViDe s Streaming Media Working Group In August 2003 there was a joint meeting between TF-Netcast, ViDe s Streaming Media Working Group [8] and RealNetworks. The Meeting was organised by Chris Hodge from the ViDe working group, and took place by H.323 videoconference. Unfortunately some of the U.S. sites were affected by the major power outage that took place on 14 August 2003 where the meeting took place, but otherwise the meeting was a good example of collaboration between Europe and the United States. International Collaboration TF-Netcast has played a major role as a forum for collaboration and exchange of ideas for professionals working with streaming media in higher education and research in Europe. By using videoconference as the medium for the task force meetings, the task force has been able to meet more frequently than would have been otherwise possible. 4

9 Chapter 2 Deliverable B: Report on Streaming Video Survey Leader: Andres Steijaert This chapter is a summary of TF-Netcast Deliverable B: Report on Streaming Video Survey. For all the details, please refer to the full text of deliverable B [1]. During the second quarter of 2003, the TF-Netcast group conducted a survey within its member organizations and their users, to get a better understanding of the way streaming video is used within its community and the direction in which this technology is developing. The questions were prepared by Leif Laaksonen from CSC, Egon Verharen and Andres Steijaert from SURFnet, and Dan Mønster from UNI C, and then turned into a web based questionnaire by the Statistics and Analysis group at UNI C. The survey was targeted towards people that regularly deal with streaming video: content creators, video producers, ICT staff and project leaders. In other words, the respondents are considered to be streaming specialists. The questionnaire contained a total of 112 questions which may seem overwhelming, but the questionnaire was organized in such a way that respondents would only be asked relevant questions, so that for instance a respondent would only be asked questions about encoding software and hardware if the respondent answered yes to the question does your organization encode material for streaming? The questions were organised into the following subjects: 1. Content for streaming 2. Streaming portal 3. Camera and production equipment 4. Streaming servers 5. Media players 6. Network 5

10 Number of respondents Croatia Czech Republic Denmark Finland Germany Greece Ireland Italy Latvia Netherlands Norway Poland Portugal Spain Figure 2.1: Distribution of respondents by country Switzerland United Kingdom United States of America 7. Metadata 8. Future plans The questionnaire was answered by 77 respondents from 17 different countries (see Fig. 2.1). Most of the respondents came from universities (70%) followed by National Research and Educational Networks (NRENs) (16%), while the remaining 14% came from other types of institutions typically some other higher education or research institution. 6

11 No 12% Don t know 4% No 15% Yes 88% Yes 81% Figure 2.2: Question 18: Does your organisation stream audiovisual content? Figure 2.3: Question 20: Does you organisation offer live broadcasting? Only those who answered yes to question 18 were asked this question. Daily 20% Once every year 7% Students 17.1% Other 7.7% Weekly 13% Once every few months 40% Support staff 37.6% Bi-weekly 5% Teachers 37.6% Monthly 15% Figure 2.4: Question 21: How often do you stream live content? Figure 2.5: Question 25: Please characterise the users that produce their own content within your organisation. 2.1 Content Production As many as 88% of the organisations represented in the survey streamed content themselves, while the remaining 12% relied on others to provide streaming servers (see Fig. 2.2). Of those who streamed content, 81% said they streamed live content (see Fig. 2.3). Among those who stream live content there is a great variation in how often live events are streamed ranging from daily to once a year (see Fig. 2.4). The main content producers are teachers, support staff, and only to a lesser extent students (see Fig. 2.5). 2.2 Streaming Portal Only 40% of the respondents said their organisation has a streaming portal or streaming announcement portal and 74% of those also have a video archive. Thus there should be an interest in using the TF-Netcast announcement portal. 7

12 The respondents provided links to their own portals and archives, and the full list of portals and archives can be found in Deliverable B. Looking at the different portals it is clear that there is a large variation in scope, design and functionality of the different portals. It also shows that there is a need for a common portal at least for high profile events that are of interest also outside of a narrow community. In the category of streaming announcement portal a total of 24 links were submitted. It would be much easier to consult just one portal than to cycle through even an abbreviated list of more specific portals. 2.3 Streaming Formats and Servers The streaming media industry has not converged on a single file format nor on a single transport protocol, so players and servers from different vendors usually do not interoperate. Hence the question of which formats and servers are used becomes important. The Internet Streaming Media Alliance (ISMA) [9] have tried to remedy this situation by developing a standardised way of streaming MPEG-4. In principle it is therefore possible to play content originating from any ISMA compliant server with any ISMA compliant player. Companies such as Apple and Kasenna develop servers that are capable of streaming ISMA compliant content. ISMA has not been universally adopted and proprietary formats such as RealMedia, Windows Media and QuickTime still account for the majority of content available. The older MPEG formats MPEG-1 and MPEG-2 can also be streamed, but are less popular MPEG-1 because it is relatively low quality compared to the alternatives and MPEG-2 because of its high bandwidth usage and player licensing issues. In addition there is no standard way of streaming MPEG-1 and MPEG-2 that thas been universally adopted, so different vendors have chosen each their own method. The TF-Netcast streaming video survey revealed that Windows Media is the most popular streaming format, followed closely by RealMedia and with MPEG-4 in third place (see Fig. 2.6). Quick- Time and MPEG-1,2 although less popular are also used by many respondents. In light of the popularity of the different formats it is not surprising the the most popular server software is Windows Media server (used by 36 respondents) followed by QuickTime Streaming Server and Darwin Streaming Server combined (used by 35 respondents) and RealServer (used by 32 respondents). A range of other server software was also reported, such as Kasenna MediaBase, Cisco IP/TV and VideoLAN server. 2.4 Media Players The questionnaire contained questions about which players and platforms (operating systems) were used, supported and preferred by the respondents and their organisations. 50 respondents said their organisations supports and/or recommends specific player software. When asked what media player software was used on computers in their organisation 39 respondents said they used Windows Media Player, 32 respondents said they used RealPlayer and 28 respondents said they used QuickTime Player. Other types of player software was also used or recommended, but only by a few of the respondents organisations. 8

13 50 40 Number of respondents MPEG-1 MPEG-2 MPEG-4 RealMedia Windows Media Quicktime Other Figure 2.6: Question 43: What types of streaming formats are used within your organisation? (More than one item can be selected). 9

14 The respondents were also asked which player they preferred. The distribution of answers to this question is shown in Fig Windows Media Player is clearly the most popular player preferred by 42% of the repsondents. RealPlayer comes in second with 28% of the votes and QuickTime Player is third with 16%. It is somewhat surprising that as many as 14% has listed other as their response to this question. This did not indicate that a single other player was popular among the respondets, but rather that a number of other players were preferred by those respondents. This included popular players for the Linux platform such as Mplayer, VLC and XMMS. 2.5 Metadata All respondents were asked whether their video assets and live streams are described by metadata. 30% said their video assets are described by metadata and 21% said that their live streams are. The respondents who answered yes to at least one of these questions were asked further questions about their use of metadata. Some used non-standard metadata, but many of the models were based on various standards such as Dublin Core, Qualified Dublin Core, IMS Learning Object Metadata and MPEG-7. Dublin Core, Qualified Dublin Core and IMS were the most popular. The different metadata models and various strategies for the exchange of metadata are discussed in detail in TF-Netcast deliverable G: Report on Video-on-Demand Metadata and Portals [7], also described in Chapter Future plans The last questions in the questionnaire related to any future plans the respondents might have. Most repsondents indicated that they had plans for significant changes in the near future and also that they had plans for new streaming video projects. The full list of project descriptions are available in report as is the list of new functionalities that the respondents would like to see developed. Many of these suggestions were for better interoperability and adherence to standards. Other 14% Windows Media Player 42% QuickTime Player 16% RealPlayer 28% Figure 2.7: Question 89: What is your preferred/favourite player? 10

15 8 6 Number of respondents XML Dublin Core Qualified Dublin Core IMS MPEG-7 RDF SCORM ICE, SDP ViDe Model Other (unspecified) Figure 2.8: Question 106: What standard is the metadata model based on? (More than one item can be selected). 11

16

17 Chapter 3 Prototype Announcement Portal Leader: Michal Krsek For many academic institutions it has become a routine matter to set up a live stream from an event such as a conference. One of the main factors that limits how many viewers actually tune in to the stream is how well the the stream is announced to the group of potential viewers. Traditionally there has been a reliance on already existing channels of communications such as conference web sites, mailing lists, etc. The idea behind the live streaming announcement portal is to have a single place (web site) where information about such events can be published to a very large audience. From the point of view of the viewer who would like to know about live streaming events within certain areas of interest the live streaming announcement portal also delivers the solution. It is a single web site where information about live streams covering many different topics in many different languages can be found. The TF-Netcast prototype announcement portal was developed by CESNET, and it is available at the following URL: There are two main groups of users of the portal: viewers, who use the portal to find and view live transmissions, and broadcasters, who use the portal to announce their live transmissions to potential viewers. The interfaces for each of these groups are described in detail below. 3.1 Viewer s interface A viewer who visits the portal is met with an overview of all streams announced in the current month, arranged in a calendar-like interface that is familiar and thus easy to understand for most viewers. (see Fig. 3.1). From the monthly overview the viewer can go to a daily overview (see Fig. 3.2) by clicking on the date in question, or go directly to a description of a particular live event by clicking on the title of the event. The description of each event contains a link to the webpage of the event and/or a direct link to the stream. The description contains a title, a short description, a longer description, and possibly a graphic image, e.g., a logo. A few technical details such as streaming format and bandwith are also given, so that the viewer may determine whether he is able to view the 13

18 Figure 3.1: The monthly overview is the home page of the portal transmission. The portal is international and can be localised to suit the viewer s choice of time zone and language. Rather than hiding transmissions in other languages, the portal will higlight transmissions that are in the language selected by the viewer. The languages currently supported are: Czech, Danish, Dutch, English, Finnish, German, Greek, Italian and Spanish. Once the viewer has found a transmission that he is interested in viewing, he can request a reminder to be sent by regarding the particular transmission. The viewer can choose to get reminders one day, one hour and fifteen minutes in advance of the scheduled transmission. It is also possible to subscribe to a set of predefined mailing lists each list covering transmissions in a praticular language that will alert the viewer of the upcoming transmission in the chosen language. These announcements are sent out once every week. Instead of visting live.academic.tv with a web browser, the viewer may also choose to retrieve information about scheduled transmissions using an RSS newsreader. RSS feeds are available in many different languages, so the viewer can choose to subscribe only to feeds that announce transmissions in the languages that the viewer understands. 14

19 3.2 Broadcaster s interface Figure 3.2: The daily view. Broadcasters who are registered users of the portal can submit information about live transmissions, which will then be available to viewers on the portal. Submission of information about transmissions can be done using either a web based interface or by sending a cryptographically signed containing the information in XML format. The web based interface for broadcasters looks almost identical to the interface that is presented to viewers, and which was described previoulsy. A broadcaster needs to log in to the portal to be able to submit information about broadcasts, and once logged in some extra functionality will appear in the web interface. When a broadcaster who is logged in is in the daily view a link with the title add appears at the bottom. Clicking on this link will load a web page where information about a transmission can be entered (see Fig. 3.3). The following information is entered about the transmission: start time, end time, title, description, long description, address of organiser, address of aquisitor, URL (including link to a graphic file) and subject category. The broadcaster entering the information can also choose whether to publish the event immediately or to keep it hidden. When all the information 15

20 Figure 3.3: Web based interface for submitting information to the portal. about the event has been entered it is submitted to the portal and displayed for review before final submission. After submission the transmission event will be visible to the broadcaster in the monthly and daily grids, and if it was published it will also be visible to viewers. The broadcaster can later edit, publish, hide, duplicate or delete the transmission event. The duplication feature is useful if the transmission is part of a series taking place over several days, since the details will have to be entered only once. The URL is entered in an XML format using the following syntax: <src ref="protocol://server/path" description="any_text" medium="(rm wm mp1 mp2 mp4 divx v v? mp3 ogg r a? ppt pdf ps txt html p?)" isstream="(0 1)" minbitrate="integer_in_kbps" maxbitrate="integer_in_kbps" language="language_name" 16

21 /> encoding="encoding-name" splashscreen="protocol://server/path" Setting the parameter isstream to 1 will cause the portal to create a.ram or.asx file on the fly based on the URL entered in the ref parameter and the value of the medium parameter (.ram for rm and.asx for wm). If set to 0, no action will be taken, and the URL is left unchanged. As already mentioned, submission of information about broadcasts can also be done using . In this case the body of the must contain the announcement written according to an XML document type definition made specifically for the announcement portal. The must be cryptographically signed by the broadcaster to be accepted by the portal. A detailed description of this way of submitting to the portal is beyond the scope of this document, but note that this is a very convenient way of automating submission to the portal by using software agents send the s. A reciept containing the portals internal ID of the announcement is sent back by the portal and this ID can be used to modify or delete the announcement. For further information about submitting via see reference [10]. 17

22

23 Chapter 4 Deliverable E: Information about Resources for Content Production Leader: Apostolos Karakoussis Most of the work done as part of TF-Netcast has concentrated on infrastructure and services in support of streaming video. This focus on technical matters reflects both the interests of the persons participating in the task force and the mission statemenets of the organisations they are employed by. However, it has not been entirely overlooked that the technical infrastructure is there to support video content. Without content the whole focus of TF-Netcast would not make much sense. In recognition of this fact it was decided to try to make a small contribution to helping people produce their own content. This contribution consists in Deliverable E [4] (see Fig. 4.1) which is a collection of links and references to resources that can help others produce video content. The Task-force members contributed with information about resources they could endorse and all the information was collected, categorised and edited by Apostolos Karakoussis. The resources were organised according to the categories: Media Architectures/Systems/Formats Software and Hardware Solutions Best Practices/Tutorials Glossary of Terms Miscellaneous Links The most interesting category, from the point of view of a potential content producer, is best practises/tutorials. Many of the entries in this category are links to material produced by vendors, such as RealNetworks, Microsoft and Apple. These materials sometimes have a more general character, but they are most often aimed towards training the reader in the products of the company who wrote the material. While there is nothing wrong with this, once a particular tool has been chosen, there is also a need for guides that are independent of a particular vendor. There 19

24 Figure 4.1: Deliverable E were a few examples of such guides, most notably Video Streaming: A Guide for Educational Development [11] from Click and Go Video [12], and SURFnet Content Production Guide [13], which is available only in Dutch. In the US the Moving Images Collection [14] has a section called Science goes to the Movies: Integrating moving images into the science learning experience [15]. A new web site is under development by Forskningsnettet (the Danish Research Network) under the name of UniVid [16] which will target both didactic and technical aspects of using video in the higher education sector. This web site will initially in Danish only and will be available in a first edition in November In summary TF-Netcast Deliverable E provides a valuable set of resources for persons who wish to produce their own content. Much of the material is in Egnlish, but there is also a good deal of material in other languages. 20

25 Chapter 5 Deliverable F: Live-Streaming Infrastructure Leader: Alessandro Falaschi Distributing live streaming content from a single streaming server to a very large audience, one is likely to encounter a bottleneck either because of large server load or because of network load (bandwidth saturation). One solution to both of these problems is IP multicast which has been known for a long time. While multicast presents an elegant solution to the problem of large scale distribution of live streaming content, it has unfortunately failed in reaching its potential because of a lack of deployment of IP multicast and because even where deployed it is often without a connection to other multicast enabled networks. The result is that many multicast enabled islands exist, but these are not interconnected. The situation today is that multicast is enabled in most NREN backbones and these are well interconnected forming a global multicast backbone infrastructure. Quite often, though, the multicast interconnectivity is not present at the campus level, and most commercial ISPs do not offer their customers access to the global multicast backbone. Since multicast is not ubiquitously deployed and interconnected a different solution to the problem of large scale distribution of live streaming content is needed. The obvious solution is to implement application level multicast where the stream is split, not in the routers at the transport layer, but in a relay at the application level. This is also referred to as a content delivery network (CDN). Many proprietary solutions implementing application level multicast for live streaming exist, but these are not interoperable and often support only streaming servers and clients from a single vendor. In order to achieve a vendor agnostic content delivery network it was decided to implement an open source system implementing application level multicast. Thus the Open Content Delivery Network, or OpenCDN, was developed under the leadership of Alessandro Falaschi at the University of Rome La Sapienza. The open source model has some added benefits: Firstly it is inexpensive for an NREN, a university or any other institution to set up a relay and participate in OpenCDN. Secondly it allows anyone to modify the source and tailor the system to their own specific needs. A mailing list and web page for OpenCDN has been set up by its main author and architect 21

26 Figure 5.1: Open CDN home page Alessandro Falaschi (see Fig. 5.1). The web page contains links to documentation, and instructions on how to obtain the latest version of the software. OpenCDN is also dscribed in detail in TF- Netcast deliverable F ([6]). 5.1 OpenCDN architecture OpenCDN implements a hierachical content delivery network whose topology is a tree structure, where the root node of the tree is the source and the remaining nodes are relays that relay content from upstream nodes towards the clients. The clients although not part of the OpenCDN infrastructure can be considered as the leaf nodes of the tree (see Fig. 5.2). Each OpenCDN node has streaming server software that constitutes part of the content delivery infrastructure of OpenCDN, i.e. the part of the content delivery network that is responsible for relaying content from source to clients using streaming transport protocols such as RTP. The current implementation uses Darwin Streaming Server [17], but using other servers is possible too. The OpenCDN nodes also contain a control layer that is responsible for communicating information 22

27 Client Client Last-hop Relay Transit Relay Source Client Last-hop Relay Transit Relay Figure 5.2: Open CDN architecture about the capabilities and status of the relay as well as for communication with other nodes (signalling) to set up the streams between nodes. In OpenCDN this communcation is handled by the control layer. The control layer is completely independent of the underlying content delivery infrastructure (streaming server), and the two are connected through an adaptation layer that handles all the details of how to communicate with a specific streaming server. The intelligence of OpenCDN does not reside in the nodes, but in a central entity called the Request Routing and Distribution Manager (RRDM). The role of the RRDM is to locate the most suitable relay for a given client request, to instruct the involved nodes to set up the required distribution of media, and finally to route the client to best suited relay. Thus the Request Routing and Distribution Manager is repsonsible for setting up the distribution tree based on client request and for routing clients to the most suitable relay, and it is also responsible for tearing down the distribution tree after the transmission ends. Clients, however, do not communicate with the RRDM directly, but via a portal. This, of course, should ultimately be the portal described in Chapter 3, but at this stage a small portal is running as part of the test set-up at the OpenCDN home page [5]. 5.2 OpenCDN operation With the OpenCDN architecture in mind, it can now be explained how the differet entities communicate during OpenCDN operation. The entities involved are: client, portal, RRDM, relay and 23

28 source. In the simplest case there is only one relay between the source and client, but this can easily be generalised to the case of a multi level OpenCDN deployment. This simple one level OpenCDN is shown in Fig The communication between the different entities is indicated by numbered arrows, and each of these will be explained briefly in the following. 1. The viewer clicks on a link in the portal web interface to start receiving a live transmission and the client issues an HTTP get request to the portal. 2. The portal does not by itself know where to direct the client in order to receive the stream, so it issues a setup request to the RRDM. The setup request contains information about the source that the client is requesting as well as the clients IP address. 3. The RRDM uses the information it has about the client (IP address) and the state of the content delivery network to determine which relay is the most suitable one for servicing the client in question. It then sends a do relay request to the last-hop relay that has been identified. This request contains information about both the source and the client. 4. The last-hop relay now sends a play request to the source (if a connection between the two has not already been established). This play request is transport dependent, and could be, e.g. an RTSP setup request followed by an RTSP play request. 5. In response to the play request the source starts sending an RTP media stream to the last-hop relay. 6. The last-hop relay can now reply to the do relay request with a do relay response. This response contains a status to indicate whether the relaying was set up successfully and in this case also a surrogate URI that the client can use to request the stream from the last-hop relay. 7. The RRDM can now send a setup response to the portal. This response also contains the surrogate URI. 8. The portal can now reply to the client, and the reply contains a link that points to the last-hop relay. 9. Once it has received this reply the client can issue a play request to the last-hop relay. 10. The last-hop relay replies to the play request and initiates a media stream towards the client. This is a very simplified description of the operation of OpenCDN. For further information see the Deliverable F [6] and the the OpenCDN home page [5]. 24

29 Portal 2 RRDM Client 10 Lasthop Relay 5 Source Figure 5.3: Open CDN operation. Black arrows indicate various kinds of requests and control messages. Green arrows indicate media streams. 5.3 Summary Open Content Delivery Network, OpenCDN, is a hierachical, scalable content delivery network system that is vendor and platform agnostic. Currently only Darwin Streaming Server is supported, but since it is open source it can readily be adapted to other streaming servers (such as RealServer and Windows Media Services). And since it is written in Perl it is practically independent of the underlying operating system. OpenCDN provides a crucial infrastructure for the distribution of very large live streaming transmissions where multicast is not feasible as the (only) option for distributing the content. OpenCDN is designed to communicate with a portal, so it would be an obvious extension of the work done under the auspices of TF-Netcast to integrate the live.academic.tv portal with OpenCDN. 25

30

31 Chapter 6 Deliverable G: Report on Video-on-Demand Metadata and Portal Leader: Harri Salminen It should be immediately obvious why it is important to describe video assets in an archive using metadata: Without metadata it is impossible to search for a particular asset, and even if the desired asset could be found it would be impossible to get any information about it, except by extracting the metadata from the the asset itself a process which usually requires human effort. For example it is possible to extract the necessary metadata about a movie from the credits, but since this is not in machine readable form it requires a human operator to extract the information. In some cases metadata is stored in the asset in machine readable form (this is possible both for RealMedia (.rm) and Windows Media (.asf) files). The case of metadata embedded in the media file is equivalent to storing the metadata externally (typically in a database) but the available metadata elements are usually fairly limited and not uniform across different file formats. As reported in section % of the respondents in the TF-Netcast streaming video survey said they used metadata to describe their video assets. The most popular metadata schemes among those who answered this question are Dublin Core and qualified Dublin Core. This chapter gives a brief introduction to Dublin Core and the interested reader is referred to the the more detailed description of metadata for video-on-demand given by Harri Salminen in Deliverable G [7]. 6.1 Dublin Core Dublin Core [18] is a baseline metadata standard, that contains just fifteen generic elements. Its simplicity means that most other metadata schemas (or a subset thereof) can be mapped to Dublin Core, making it ideal as a metadata exchange format, which is not surprising, since this was one of the goals of the Dublin Core Metadata Initiative (DCMI). Dublin Core is described in more detail by Harri Salminen in Deliverable G [7] and, of course, on the DCMI web site [18], so here I will only reproduce the fifteen elements of Dublin Core along with their descriptions: Title A name given to the resource 27

32 Creator An entity primarily responsible for making the content of the resource Subject A topic of the content of the resource Description An account of the content of the resource Publisher An entity responsible for making the resource available Contributor An entity responsible for making contributions to the content of the resource Date A date of an event in the lifecycle of the resource Type The nature or genre of the content of the resource Format The physical or digital manifestation of the resource Identifier An unambiguous reference to the resource within a given context Source A Reference to a resource from which the present resource is derived Language A language of the intellectual content of the resource Relation A reference to a related resource Coverage The extent or scope of the content of the resource Rights Information about rights held in and over the resource These descriptions are rather abstract, which emphasises the need to give good guideance to the people who enter metadata, so as to ensure a uniform usage of the different elements. Another way of making the description more accurate is to use qualified Dublin Core. The difference between qualified and unqualified Dublin Core is that in qualified Dublin Core each element is augmented with a set of qualifiers, so that, e.g. the element date can be further refined to one of the following: Date Created Date Valid Date Available Date Issued Date Modified Date Accepted Date Copyrighted Date Submitted 28

33 The other elements also have qualifiers that refine their otherwise broad meanings. As explained by Salminen [7] the Dublin Core element set and standardised qualifiers have been augmented with the additional element audience in what it called DCMI Metadata Terms [19]. This specifies a type vocabulary consisting of so-called vocabulary terms, viz., a list of recommended terms that may be used to describe the Dublin Core element Type. One of these recommended terms is Moving Image Dublin Core is used extensively in the academic community, also for video metadata. See Deliverable G [7] for more details about its usage and for descriptions of the following other metadata models: Learning Object Metadata (LOM) MPEG-7 MARC 21 The interested reader will also find information about persistent globally unique identifiers for video metadata (similar in function to ISBN numbers for books). Both the Digital Object Identifier (DOI) and the Open Archives Initiative (OAI) Identifier are discussed, and recommendations on usage is given. Finally the important issue of metadata exchange is treated. The motivation for metadata exchange is to enable the simultaneous search in several archives at once. Two possible solutions to this are presented and compared. These are respectively distributed querying and metadata replication. Distributed querying presents the challenge of agreeing on a common query form and mechanism. Even when this is solved the distriubuted querying approach may suffer from poor performance. The idea behind metadata exchange is to transfer the metadata itself from all the individual archives databases to one or a few union catalogs. This approach is taken by the Moving Images Collection project [14]. 6.2 Conclusions As more and more video assets appear on-line for either download or streaming it becomes increasingly important to be able search these using meaningful metadata. It is important to have good metadata in order to get good search results. With many local, regional and national archives each searchable by themselves the importance of being able to perform meta-searches in many archives at once also increases. This requires national and international portals that either perform distributed searches or that have access to union archives of metadata. TF-Netcast Deliverable G contains important information and recommendations about how to achieve these goals. 29

34

35 Chapter 7 International Collaboration Many individuals from many institutions across Europe participated actively in TF-Netcast, and even more followed the work by subscribing to the TF-Netcast mailing list. Below is an alphabetic list of institutions that participated actively defined as having attended at least one meeting. CARNet, Croatia CINECA, Italy CSC/Funet, Finland CESNET, The Czech Republic FCCN, Portugal GRNET, Greece ICM, Poland PSNC, Poland RedIRIS, Spain SURFnet, The Netherlands SWITCH, Switzerland TERENA, The Netherlands UKERNA, United Kingdom UNI C/Forskningsnettet, Denmark UNINETT, Norway Università degle Studi di Roma, Italy University of Trieste, Italy University of Wales Swansea, United Kingdom 31

36 Figure 7.1: Left: A quick poll during the last TF-Netcast meeting. Right: Baiba Kaskina from TERENA has the undivided attention of the whole task force. Note: more sites were present than could be shown in one screen. 7.1 TF-Netcast Meetings TF-Netcast only had one physical meeting at the TERENA offices in Amsterdam while the remaining meetings were held as H.323 videoconferences. All these meetings were hosted virtually by SWITCH using the SWITCHvconf videoconferencing service [20]. Ernst Heiri and Christian Schlatter from SWITCH scheduled the SWITCH multipoint control unit (MCU) for each TFNetcast meeting, and also scheduled a test conference one or two days in advance of each meeting to allow participants to check their setup and connection in advance. Examples of the continuous presence layout used on the SWITCH MCU for the TF-Netcast meetings are shown in Fig The use of videoconferences made it possible to have frequent task force meetings without incurring travel expenses. Most participating institutions already had hardware based H.323 epuipment, so they did not have to invest in new equipment in order to participate in the meetings. A few sites experienced problems with the connection, but overall the technology proved stable. At the last TF-Netcast meeting the form and frequency of the meetings were discussed and it was agreed that it would probably have been useful to have at least one more physical meeting, and that the interval between the meetings held via videoconference could have been even shorter. 7.2 Collaboration with TF-AACE At the 20th TERENA General Assembly [21] TERENA s task forces were presented and following the General Assembly TF-Netcast was contacted by Diego Lopez who is chairing TF-AACE [22] with a proposal to set up a collaboration between the two task forces. TF-AACE is working towards developing and deploying inter-operable authentication and authorisation infrastructures between NRENs in Europe. TF-AACE is seeking to cooperate with projects that have needs for such a Europe wide authentication and authorisation infrastructure, and the TF-Netcast portal is an example of an application that could benefit from such an infrastructure. 32

37 The TF-Netcast live streaming announcement portal developed by CESNET uses authentication and authorisation (AA) in order to ensure that only authorised users may submit information about live streaming events to the portal. In the present version of the portal the CESNET team manages AA locally using a database of information providers who are allowed to submit to the portal. The proposed cooperation between TF-Netcast and TF-AACE would consist in delegating the AA responsibility to the NREN using a testbed coordinated by TF-AACE. For TF-Netcast this would mean better scalability and less resources required, while TF-AACE would be given access to an international testbed for their AA infrastructure. Diego Lopez participated in the last two TF-Netcast meetings to further develop the idea. It was agreed that such a cooperation would be in the interest of both task forces, but since TF-Netcast was in the final stages it was agreed to postpone further detailed planning of the cooperation to after TF-Netcast had ended. This cooperation will be continued by the Academic Netcasting Working Group (which formed TF-Netcast in the first place), possibly in a new task force to succeed TF-Netcast(see also Chapter 8). 7.3 Collaboration with ViDe Streaming Video Working Group Throughout the lifetime of TF-Netcast Chris Hodge from the ViDe Streaming Video Working Group [8] has acted as liason between the two groups. Even though the two groups shared a common interest they had each their own set of goals and thus no formal cooperation between the two groups was ever initiated. At one occasion the ViDe Streaming Video Working Group organised a meeting with RealNetworks [23] to which members of TF-Netcast were invited. The meeting took place as an H.323 videoconference on 14 August Unfortunately there was a major power outage in the United States on that particular day which caused problems for some of the US participants who were either prevented from participating or had to particpate by phone. Apart from the technical difficulties due to these rather extraordinary cirsumstances the meeting was a success and it was agreed that such meetings could easily be held in the future. The time difference between the United States and Europe is always an obstacle for such cross Atlantic meetings, but with some planning it is usually possible to schedule meetings at a time where most people have a chance to participate. TF-Netcast has also been presented by Dan Mønster at the SURA/ViDe Annual Digital Video Workshop in 2003 and

38

39 Chapter 8 Summary and Future Plans It is fair to say that TF-Netcast has dealt with topics that are common to many NREN s and universities, viz., providing streaming media services to academic users. This is a complex area partly because there is a lack of standards 1 and partly because many of the components for such services are not readily available or are challenging to deploy and integrate, e.g., metadata description of assets, synchronised video and still images, digital rights management, systems for announcing live streams, content delivery networking, firewall issues and multicast. Thus the organisation that wants to deliver a streaming service (either live, scheduled or on-demand) must undertake a rather comprehensive development effort in order to deploy and integrate all the needed components. It may be easy to set up a streaming server where users may upload content via ftp, but it is not easy to set up a system that includes easy upload from a web page, administration of user access to assets, metadata description, support for multiple formats and platforms, integrated and synchronised video and still images (for presentations), on-line editing and transcoding, user support and detection of firewall/nat issues. TF-Netcast has worked on a few of these issues to the benefit of all those who are in the process of setting up such services. The goal of TF-Netcast was to work towards setting up a European netcasting channel. While it was clear from the outset that setting up an Academic Netcasting Channel would exceed the resources available to the task force, the goal was to clarify what was needed both in terms of techonological and organisational issues to achieve this. The work performed in TF-Netcast reflects what we believed to be the crucial issues: to understand the current use of the technology and the users needs, to stimulate the production of high quality content, to agree on a common metadata schema and ways to exchange metadata, to devise a way to deliver content widely across Europe, to set up a common portal for the announcement of live streaming events, to propose a way to organise a common netcasting channel. 1 Or at least a lack of support from the vendors for the standards that do exist. 35