The Digital Restyling of Audiobooks

Transcription

1 The Digital Restyling of Audiobooks Lavinia Egidi and Marco Furini Computer Science Department University of Piemonte Orientale Via Bellini 25/G Alessandria Italy Abstract Portable devices and wireless connections are creating a new scenario where digital information are massively entering our life. Such a scenario is craving for new applications and, in the meanwhile, several applications are simply re-written for the mobile scenario (we can check e mail, browse the web, play games, etc.). In this paper we propose a mechanism to adapt a classical digital audiobook application to the new mobile scenario. Our proposal introduces, into a digital audiobook, multimedia objects (images and text) synchronized with the audio. The proposed mechanism does not modify the digital file structure and can be protected against piracy. With our mechanism, an audiobook is no longer a simple audio representation of a written book, but a different, complete multimedia product. The mechanism is shown to be effective through a modified audiobook player. KEY WORDS Multimedia over Wireless, Multimedia Communications, Multimedia Applications, Music Distribution 1 Introduction The digital revolution is affecting the way we live and we work, but is also strongly affecting the way we have fun. Thanks to advances in technologies, the mobile scenario is now considered an important part of our life: we can watch a movie wherever we want, we can listen to hours of music without the need of carrying several audio-cds, we can check our by using a device connected to a Wi-Fi hot spot while sitting in a coffee shop. In such a scenario, there are two main entities involved: the communication industry and the consumer. The former plays a fundamental role, as it has to propose devices and applications that have to attract (or to modify) the habits of the second entity involved (the consumers). Cellulars, palms, DVD players, CD players, MP3 players are examples of devices that are part of our life. MP3, DVD, and, overall, the web, are examples of applications that massively entered our life. In some cases the integration of an attractive device and an interesting application creates a need in the consumer behavior. E-music combined with portable devices is an example of such integration, which is changing the music world. In this paper we consider a different application that can receive benefits and may be appreciated by consumers if combined with portable devices: the audiobook. Our goal is to reshape it to the digital world so that it cannot be considered as a simple audio representation of a written book. Although most people think of audiobook technology as a support for sight-impaired people and although in the early days of audiobook usage, sight-impaired people or slow learners were the main users of audiobooks, nowadays the interest around audiobooks is growing at an exception speed. A variety of people listens to audiobooks for several reasons: frequent book readers use audiobooks as they allow to read while pursuing other activities; Children s audiobooks are also a formidable part of the market, as many families use audiobooks for in-car entertainment (not to mention that many teachers are using them for educational tools); Students and commuters are appreciating audiobooks as they can be stored in small portable devices. The usage of audiobooks is increasing not only in the entertainment world, but audiobooks are broadly used also in education, for instance to learn languages. The growing interest around audiobooks is highlighted by the Audio Publishers Association (APA) [1], which estimates the size of the audiobook market at $800 million. Overall sales growth from 2002 to 2003 is up 5.1 percent. APA figures also give an interesting insight on the market rating of different formats: audiobooks come in cassette tapes, standard compact discs, MP3 CDs and downloadable format. Looking at the revenues, it is interesting to note that the audio tape revenue decreased from 2002 to 2003 (58% to 49%), while the revenue of CD sales increasing from 35% in 2002 to 45% in The most promising format is the downloadable digital format, as it showed the largest growth in the audiobook industry: in 2001, download sales were $5,143,000; In 2002, an increase of 112 percent led download sales to $10,940,000 and, in 2003 the sales were of $ 18,490,000 (plus 69 percent). The APA report leads us to conclude that the future of audiobooks is in the digital world. Both the audiobook

2 industry and the consumer may take advantage of this format. The former can reduce the production cost by eliminating manufacturing costs. The latter can buy and download an audiobook wherever and whenever he/she wants at a lower price that any other audiobook format (e.g., The Da Vinci s Code by Dan Brown costs around $40 on tape/cd, and around $20 if downloaded from a webstore). Although the reduction in cost is interesting, it is not the only benefit that audiobooks can receive from the digital world. Several multimedia features can be introduced so that the audiobook is no longer a simple audio representation of a written book. The contribution of this paper is to propose a mechanism that enriches an audiobook with multimedia contents. In this way the audiobook can be not only listened to but also watched. The written text is shown on a display, synchronized with the narrator s voice (a sort of karaoke effect); images appear timely on the screen, integrating the narrator s descriptions; explanations appear on the display for improved understanding; links to websites are introduced, to allow a deeper, more informed reading. These are only some features that can be introduced into an audiobook file. Besides the entertainment value of such additions, learning a language with an enriched audiobook can become easier and more fun. Furthermore, the simultaneous written and audio stimuli can help to memorize new words more quickly, as pointed out by several researches (this is mainly due to the simultaneous use of the left part of the brain, used to understand the written language, and to the right part of the brain, used to understand the audio part). It is very important to notice that we propose to add extra contents in a completely transparent way. The audiobook file maintains the same downloadable format (e.g., MP3). Therefore a user can play out the digital audiobook in the usual way, oblivious of multimedia objects, over any devices he/she likes (IPod, MP3 Player, DVD player). On the other hand, if the same audiofile is played out over a particular device (for instance the one we will describe later in this paper), the listener can experience all the features described above. In this paper we consider audiobook files in MP3 format and the transparency is achieved by following the same approach [2], where multimedia contents were introduced into e-music. In essence, the file structure is not modified by storing the multimedia contents in a particular file location and the file size is not greatly affected by storing only the description of the multimedia objects (hence a description language is used). A simple audiobook player has been designed to show the benefits of our approach. The remainder of this paper is organized as follows. In Section 2 we present an overview of the audiobook and the e-music world; in Section 3 we outline the characteristics of our mechanism. Since copyright protection is mandatory in the digital world, in Section 4 we investigate possible Digital Right Managament techniques and we show how intellectual property can be protected. A sim- US Million dollars 20,00 18,00 16,00 14,00 12,00 10,00 8,00 6,00 4,00 2,00 0, Year Figure 1. Audiobook Download Sales Revenue. ple audiobook reader is presented in 5 and conclusions are drawn in Section 6. 2 Preliminaries The audiobook industry has always followed the trail created by the music industry. This trend is confirmed by fact that several web stores are offering audiobooks also in the electronic format. For this reason, it is interesting to take a look at the most recent development of the e-music industry in order to forecast the next step of the audiobook industry. Hence, we highlight a recent e-music proposal that aims at transforming a simple e-music file into a multimedia rich product. We base our current proposal on the latter approach. 2.1 The Audiobook World An audiobook is the audio representation of a written book. Audiobooks became very popular (at least in US and UK) in the mid 1980s thanks to the availability of new audio devices such as walkmans and car stereo cassette players. Many book publishers responded to the growing consumer demand by starting their own audio divisions, and books on audio cassette began to appear in bookstores. Thanks to the audiobook publisher marketing policy, since the mid 1990s an audiobook section is present in any book retail store and many libraries have audiobooks available (a Library Journal survey recently showed that circulation of adult audiobooks in libraries jumped 13.5 percent in two years, while the circulation of childrens audiobooks rose 10.7 percent). Currently, about 21% of American households listen to audiobooks (compared to 12% in 1995). Due to the similarity with music, the audiobook industry went where the music industry had already paved the way. From vinyl disc to audio tapes, from audio-cd to digital audio files. This latter technology can be considered a revolution in the audiobook world because finally limits of cost and space have been reduced drastically. For instance, the main advantage of an MP3 distribution over a traditional Book-on-CD is that it is possible to fit up to 16

3 hours of audio on a single MP3-CD (and hence most books will fit on one disc). Once again, following the e-music footsteps, audiobook publishers began to offer audiobooks in digital formats and Figure 1 shows the income of the audiobook download sales (US market) in the recent years. The growth of download sales convinced audio publishers to put up web sites where consumers can buy digital audiobooks that are then directly stored on the audio device used for e-music (e.g., ipod, Handheld OS Palm Pilot, cellurars). The large interest around audiobooks is highlighted by the audiobook section present in web stores like amazon.com and Apple Music Store. It is near certain that audio publishing will see a significant format shift in the near future. What is not yet clear is the format of the digital audio file that will be used in the near future. Currently, there are a variety of digital audio file formats and the most popular are MP3 (MPEG I Layer III), AAC (Advanced Audio Coding) and WMA (Windows Media Audio). To meet the largest demand, many on-line digital audiobook sellers offer two or all three of these formats. In conclusion, it is important to highlight a characteristic of an audiobook. As well as for e-music, the e- audiobook can be considered a simple audio track: No images, no cover, and no text is given with audio file. This is a burden that may limit the audiobook to a niche of people. To avoid this, the audiobook has to become a multimedia rich product. Since similar concerns have spurred interesting developments in the e-music world, we take our inspiration there. 2.2 The e-music world Listening to an audiobook is very similar to listening to music and the audiobook industry usually follows the steps of the music industry. For this reason, we present here the direction of the e-music world. Such directions will be used in Section 3 as the basis for the proposal of our approach to transform the audiobook into a multimedia rich product. In the current music scenario, consumers buy music in digital music files or in audio-cd format. The latter is a complete multimedia product that can be listened to almost everywhere, and the former is mainly a simple audio track that can be listened to on any device (depending on the selected e-music format). The large interest around e-music is mainly due to the price (cheaper than audio-cds) and to the development of appealing devices like the IPod. Hence, so far, the product is the same, but provided over a different media. A recent proposal [2] aims at bringing multimedia contents inside MP3 files, without affecting the MP3 file size and the MP3 compatibility. The mechanism allows to insert, inside an MP3 file, new multimedia contents like a karaoke-like service, images, covers, updated information. To this aim, the multimedia contents are first described through a description language (either MPEG7- DDL, SMIL, or the proposed MECDL) and the resulting description is stored in a transparent way into the MP3 file. A player, able to get the description and to render it, allows to provide the consumer with both audio and multimedia contents (even more than those provided by audio-cds). Conversely, a player that is not able to get the description (all the currently available MP3 player) cannot render the multimedia contents, but can still play out the MP3 music file. Hence, the mechanism proposed in [2] transforms each e-music file into a multimedia rich product in a transparent way. The same approach can be very useful to the future of audiobooks. In the following we show how this can be done. 2.3 Copyright protection An important consideration in the distribution of any digital content is copyright protection. Up to some years ago, the law, along with practical and economic barriers, was considered sufficient to protect publishers and authors from piracy; but those non-legal barriers do not exist in the case of digital files. Hence, many vendors use Digital Right Management (DRM) systems to protect the audio file. This technology allows to wrap an audio file with a control mechanism that provides users with some limitations (for instance, the DRM may limit the number of possible copies or may impose the usage over a particular device). The issue is of general concern and doesn t present specific difficulties in relation to audiobooks. Nonetheless, for completeness we briefly discuss DRM techniques in a later section. 3 Our proposal In this section we present a mechanism that brings multimedia contents inside an e-audiobook. In particular, our goal is to introduce images and written text synchronized with the audio in such a way that the consumer can also: i) follow what the narrator is saying by reading the text that appears on the screen; ii) watch images that can better describe what the narrator is saying; iii) increase his/her knowledge by reading additional information and iv) receive clues about some possible external documents (e.g., links to websites). We already mentioned some possible benefits of the karaoke feature (useful to learn a foreign language or to memorize new words), and here we describe a possible use of the other features. Suppose we are listening to the audiobook The Da Vinci Code and suppose the narrator is describing the oil painting the Mona Lisa by Leonardo Da Vinci. With our mechanism the listener may have a graphical interface (either on the computer screen or on the portable device) that provides the listener with audio, images and text. In this way, the audiobook can be considered

4 a complete multimedia product. To avoid the proliferation of files and to make simple and immediate the use of audiobooks over portable devices, our mechanism inserts all the previous features inside a single audiobook file (in this paper we consider the MP3 file format) and the insertion process must meet the following constraints: i) transparency (the input file format must be the same as the output file format); ii) lightness (the input and the output file sizes should differ only lightly); iii) security (multimedia contents, especially the written text, must be protected against piracy). To meet constraints i) and ii) we consider the approach proposed in [2] that inserts multimedia contents into e-music files in a transparent way (by using one of the ID3v2 tags) and does not directly store images (or heavy multimedia contents) into the e-music file. Instead, only a link between the e-music and the multimedia contents is stored. Conversely, timing information that relates lyrics with the audio file are directly stored. In this way, the file size is not heavily increased. In the following we use this approach as a basis for developing our audiobook mechanism. Security goals can be attained using a standard DRM system (see Section 4). Notice that linked material need not be protected in this context since the added value is the link itself rather than the (public) contents of the linked page. <body unit=100 encrypted> <kar time=69>prologue<br> <kar time=144>louvre Museum, Paris. 10:46 pm <kar time=183>renewed curator Jacques Sauniere staggered through the vaulted archway <kar time=221>of the museum s Grand Gallery. He lunged for the <kar time=250>nearest painting he could see, a Caravaggio.... </body> Table 1. An example of audio-text synchronization described through MECDL. <add> </add> <a> </a> href="url" time=value Defines the additional information area. Defines an external reference. Gives a fully qualified HTTP URL. Gives the timing synchronization between an image and the audio Table 2. Additional MECDL Tags and Attributes. 3.1 Multimedia Object synchronization We use a description language in order to specify the timing with which multimedia objects must be shown on the display. Standard description languages like MPEG7-DDL and SMIL would be adequate, except that they yield long descriptions. Thus, the lightness requirement we stated above leads us to prefer the more compact MECDL [2], designed specifically for (lightweight) enrichment of music files. We exploit here MECDL s audio text synchronization capabilities, but enrich the language to allow for synchronization of other multimedia objects, still preserving the compactness of the descriptions produced. Table 1 shows an example of the way that the written book and the narration can be described with MECDL. The language is organized in tags, attributes and values. The kar tag, along with the time attribute, allows to establish a timing relation between the text (that follows the kar tag) and the audio. All the timing relations are represented by time units, defined through the unit value (in milliseconds) inside the body tag. Similarly, in order to synchronize images, we add to MECDL a tag img that, coupled with a time attribute, yields the desired effect. For instance, <img href="lisa.jpg" time=69> specifies that the lisa.jpg image must be shown after 69 time units. The image will stay on-screen until a next image is rendered. The use of images as a support of the spoken/written words may be very helpful. For instance, while an author is describing something (for instance a painting), the image may help the listener/reader to better understand the description. In children books, images can help the children to make a connection between a word and an image (for instance the book may talk about a dinosaur and the image will display a dinosaur). By using this multimedia approach the child will memorize easily the information just listened/read. 3.2 Additional information While reading a book, sometimes we would like to have (or we need to have) some additional information that the book does not provide us. To this end, we enrich the audiobook with additional text or links to webpages. While we are commuting, or we are stacked up at the airport waiting for a delayed flight, we can profit from the extra information stored on the audiobook. If we are so lucky as to have a Wi-Fi spot in the nearby, we can explore deeper, surfing the Net starting from the link recommended by the audiobook itself. Questions like: which museum has The Mona Lisa painting? or when was it painted? Who is Leonardo Da Vinci? What did he do for a living? When and where did he live? With our mechanism, the file itself or the Net, with the help of an Internet connection, can provide the

5 <add time=1310> The Mona Lisa, less commonly rendered as the Monna Lisa, is an oil painting on poplar wood by the famous Italian artist Leonardo da Vinci."> <A HREF=" Louvre</A> </add> Table 3. The use of the new tags (add and A) to provide additional information. consumer with an immediate answer. For this reason, we expand the MECDL with two additional tags. A first tag, named add, identifies the additional information area, which is concluded by the paired closing tag /add. Inside the add area it is possible to insert additional information (as plain text) and links to external documents. A time attribute inside the opening add tag is used as always to synchronize the additional information with the audio. To create a link to another document, we borrow the <A> tag from the HTML language. The syntax is: <a href="url">text to be displayed</a>. The <a> tag is used to create an anchor to link from, the href attribute is used to address the document to link to, and the words between the open and close of the anchor tag will be displayed as a hyperlink. Table 3 shows an example of the above. Suppose we want to provide additional information about The Mona Lisa painting, which is mentioned by the narrator at time Then the add tag has timing attribute of value Information are provided as plain text between the add tag pair, while the starting point for browsing the web (here the Louvre Museum) is provided by the A tag. 4 Security Issues The ease of copy of digital material, poses the problem of protecting the contents from unauthorized distribution, calling for Digital Rights Management (DRM) systems. In this section we summarize the basic criteria on which DRM systems are built. We do not attempt to propose a specific DRM system mainly because the protection of enriched audiobooks doesn t pose unusual challenges. The basic idea of DRM systems is to wrap the contents in a digital container that discloses the material only to authorized users and typically keeps track of the number of copies that the user makes. It is important to understand that DRM is a problem that admits no final and provably strong solution, since security rests here on code obscurity. A notable example is the history of cracks of MS Windows Media DRM and Apple FairPlay (see, e.g., [3, 4, 5, 6]). Briefly stated, the reason is that DRM systems must offer limited availability of protected data on untrusted environments (the playout device is under the user s control). Security, effectiveness and desirability of DRM systems is discussed in a broad and interesting survey paper [7]. The basic tool that DRM systems use is the encryption of the content that must be protected. Encryption is carried out with typical symmetric key encryption algorithms. For playout, contents must be in the clear, therefore the cryptographic key must be available to the software player. But contents in the clear must never be available to users, in order to avoid unauthorized usage or distribution. Therefore the protection is effective as long as the user can t retrieve the key from the software, and can t capture the unencrypted data as the player outputs them to the audio card. Thus, processes authorized to handling sensitive cryptographic data and the data stream in the clear must be cryptographically checked for integrity, to avoid substitution with user-controlled code. In particular, the core DRM software must run at kernel level, and its integrity should be checked at boot time, preferably by some secure hardware component. This yields a trusted basis on which all DRM operations can depend. Additional protection measures include (see, e.g., [8]) the disabling of debugging and dumps of the memory of privileged software, integrity check of any dynamic library that a privileged process attempts to load, authentication of code only after the latter has been loaded to RAM (to avoid race condition attacks). Moreover hardware handling data stream in the clear must also be authenticated, in order to avoid capture of protected contents (cf. Micorsoft s Secure Audio Path [9]). Finally, individualization of the software player and, as a consequence, of the contents release aims at hindering unauthorized sharing of contents between users, and at minimizing the effects of a key compromise (see, e.g., [9]). Typically, the player is bound either to the machine on which it runs (exploiting some unique device identifier, like, e.g., the IMEI International Mobile Entity Identification on GSM phones), or to the user (e.g., via a reading of the fingerprint). Individualization has the advantage of helping to expose the source of illegal distribution, and therefore works as a deterrent. On the down side, it also works against users privacy (as most security measures do). Figure 2 summarizes the various steps taken by the content provider and by the user. 5 The Software Player The transparency property of our mechanism allows to produce an audiobook file that can be played out over any MP3 player available. However, since currently available players would skip the multimedia contents description, a special player is needed to enjoy the multimedia contents. This player has to to play out the audiobook file and has to

6 Figure 2. A DRM system: basic steps. The content owner (a) produces an audiobook (b) and packages it in individualized, cryptographically secure packages (c), which include licenses with decryption information; the user inputs the packaged contents and the license (d) to its portable device (e) and can enjoy the audiobook (f). read the multimedia contents description stored inside the ID3v2 tag. This description is processed by the player and the multimedia contents are showed on the user s screen according to what the narrator is saying. To evaluate the benefits introduced by our mechanism, we developed an audiobook reader and Figure 3 shows its graphical interface. A box is dedicated to displaying images, another is dedicated to the karaoke effect and a third box is used for providing additional information to the listener. Note that all these information are provided in a synchronized way with respect to the audio. 6 Conclusions In this paper we proposed a mechanism to transform a simple audiobook file (the audio representation of a written book, stored in a digital format) into a multimedia rich product. We considered audiobook in MP3 format as this is one of the most used format in the audiobooks downlodable sales. We developed a mechanism that provides additional features to the audiobook consumer: i) the written text synchronized with what is being read by audiobook narrator (a sort of karaoke); ii) images rendered according to the audiobook narration; iii) additional information about a particular subject discussed in the book and iv) links to other documents. It is important to highlight that all these features come synchronized with the audio. The effectiveness of our mechanism has been showed through an audiobook media player. We also investigated how to protect the copyright material from piracy and showed possibile guidelines to the use of DRMs systems, which can provide security to our Figure 3. The visual interface of our Digital Audiobook Reader. audiobook file. The simplicity of our mechanism along with the expansion of portable devices may find new markets for the audiobook industry both in the entertainment and in the learning world. References [1] Audio Publisher Association. [online]. Available: [2] L. Egidi, M. Furini, Bringing Multimedia Contents into MP3 files, IEEE Communications Magazine, May [3] Microsoft DRM Hack revived?, April 18, 2005, [4] EXCLUSIVE: Windows Media DRM 10 Crack Updates. It Is Real!, Feb 16, 2005, msmvps.com/chrisl/archive/2005/02/16/36147.aspx [5] DVD Jon Takes Crack at itunes, Nov 26, 2003, [6] Apple s FairPlay DRM cracked, April 5, 2004, arstechnica.com/news/posts/ html [7] P. Biddle, P. England, M. Peinado, B. Willman, The darknet and the future of content distribution, ACM Workshop on DRM, [8] H. Bar-El, Y.Weiss, DRM on Open Platforms? It May Be Possible after All, Communication Networks and Services, The 2nd IEE Secure Mobile Communications Forum, [9] Microsoft Windows DRM 10,