different content presentation requirements such as operating systems, screen and audio/video capabilities, and memory (universal access)

Transcription

1

2 (1/3) NSC S different content presentation requirements such as operating systems, screen and audio/video capabilities, and memory (universal access) limitations are significantly different from (mobile learning terminal device to device. In addition, students have devices) wide variety of personal preferences of where, when, and how they would like to access to the learning content. As a result, we need a (learners) mechanism to create and deliver adaptive content from any platform in any format to any device through any network at anytime anywhere; W3C refer such mechanism to universal access mechanism. In this XMLRDFCC/PP subproject, we proposed universal UAProf mechanism consists of three subtasks: content adaptation, content delivery, and content presentation. The proposed (continuous and seamless) mechanism will provide an adapted content SyncML for each device. In first year, we finished the background and related research and established the prototype mechanism. SCORM SMIL Keywords: universal access, content SMIL adaptation, content delivery, content presentation, SMIL, Petri net SMIL CC/PP Delivering content to Web enabled mobile devices such as phones and PDA presents a challenge; different devices have different content presentation requirements such as operating systems, screen and audio/video capabilities, and memory limitations are significantly different from device to device. In addition, students have wide variety of personal preferences of where, when, and how they would like to access to Delivering content to Web enabled mobile devices such as phones and PDA presents a challenge; different devices have the learning content. As a result, we need a mechanism to create and deliver adaptive content from any platform in any format to 1

3 any device through any network at anytime anywhere; W3C refer such mechanism to universal access mechanism as shown in Figure 1. Figure 1. Concept of universal access Our proposed universal mechanism consists of three subtasks: content adaptation, content delivery, and content presentation. Content adaptation is a mechanism of dynamic adjusting content presentation to meet the constraints of network bandwidth, user preference, and devices limitation, thus it is necessary to know network, user, and devices information prior to the content rendering. Ideally, such information can be described with user/device profiles and be stored at content server, the client only keeps a profile reference for accessing its profile. To enable content adaptation, we have based on CC/PP (Composite Capabilities / Preferences Profile) recommended by the W3C for describing and resolving user/device profiles, and implement the mechanism with open source software tools such as Deli as the CC/PP parser to resolve user/device profiles, and Cocoon as our content transformation engine to provide proper content based on the resolved user/device profiles. Content delivery is a mechanism to provide real-time monitoring and tracking of user s learning state and the connection network status, so that a content server can deliver continuous and synchronized content over wired/wireless network to the user from where she/he left off. To enable such a dynamic and continuous content delivery, we propose a SyncML based (Synchronized Markup Language) Multi-Sync content delivery protocol to serve as a common synchronization protocol for data between mobile devices such as phones, PDAs, desktop PCs and servers. Content presentation is a mechanism to manage what and how to present proper content to the user. Currently, HTML-based presentation has limits when it comes to continuous multimedia presentation characterized with spatial and temporal synchronization of multiple integrated media. To enable such continuous multimedia presentation, we will create and render adaptive multimedia content based on Dublin core/mpeg-7 and SMIL for multimedia content description and composition, and implement a Web based SMIL authoring tool for both online and offline editing of SMIL document. Besides, we implement an embedded SMIL player to be used on devices for rendering SMIL based multimedia content. To content, the life cycle can be divided into content adaptation, content delivery, and content presentation phases shown on Figure 1. The original spirit is to achieve create content once, ready for universal access, i.e. a mechanism to create and deliver adaptive content from any platform in any format to any device through any network at anytime and anywhere. In this way, it is more efficiency and reasonable to encourage teachers and authors to spend more time to create thing really needed and not pay attention on detailed formatting matters. W3C refers such mechanism to universal access. There are many research published, likes (N.R. Adam, V. Athuri, I. Adiwiyaya, S. Banerjee, R. Holowczak, 2001) (T.H. Ess, 2002) (D. Hunold, A.N. Barreto, G.P. Fettweis, M. Mecking, 2000) (W3C, 2002). Content adaptation is a mechanism to dynamic adjusting content s components to meet the constraints of network bandwidth, user preference, and devices limitations. So it is necessary to know network, user, and device information prior to content rendering. To acknowledge the capability of client s device, there are two approaches to implement it. The simplest one is to parse 2

4 packets which are issued from client. The HTTP packets include the browser information like, MS IE, Netscape, or etc. Using the related information, it is easily to guess the type of end device. Then applied to suitable style sheet, an adaptive content can be generated and delivered to client. In this way, only use browser information to identify client device. So it can treat as coarse information which used only to recognize PDA or non-pda device. In this approach, the detailed capability of device can not be properly defined. The other approach is recommended by W3C, called CC/PP (Composite Capability/ Preference Profile) (CCPP, 2002) (Cocoon, 2002). All the software and hardware attributes or features can be detailed described in the CC/PP file. Using a parsing tool to resolve the profile, the system then will know what client is equipped. Choosing a suitable style sheet then an adaptive content will appear on the client s device. The next issue is the content components organization and order which included spatial and temporal property. Well organized contents can lead the student focus on the learning objectives. In traditional homepage, all the content components will be displayed only in spatial dimension. No temporal relation or property can be used among these content components. In fact, there are lot of temporal relation exist among the resources inherently especially in learning environment. Currently, there several solutions have been proposed. W3C recommended Synchronized Multimedia Integration Language (SMIL) enables simple authoring of interactive audiovisual presentations (W3C, SIML). The others like Microsoft s HTML+TIME (Timed Interactive Multimedia Extensions) which adds timing and media synchronization to HTML pages. Using a few Extensible Markup Language (XML)-based elements and attributes, you can add images, video, and sounds to an HTML page, and synchronize them with HTML text elements over a specified amount of time (Microsoft Corp., HTML+TIME). So it is obviously that using these markup languages with temporal property can increase the learning content s readability. Petri nets is a systematic analysis method, it uses graph, mathematic, and module to represent and simulate the system. Since Carl Carl Adam Petri proposed (C. A. Petri, 1962), there are many extended researches, likes Timed Petri Nets (P. Merlin and D. Farber., 1976), Coloured Petri Nets (K. Jensen., 1992 ), Object Composition Petri Nets (T. D. C. Little and A. Ghafoor, 1990), Multimedia, and Object Petri Nets (Nabil R. Adam et al., 2001). These extended Petri nets has been applied to different application and correspondence domain to provide a completely systematic analysis. The most benefit to using Petri nets is providing a mathematical model to represent system. It needs only few symbols to depict workflow about a system. It also can provide a visualized method to simulate the system dynamically. In this way, using Petri nets to model a system can provide a better representation and performance acknowledgement. The basic concept of Petri nets is shown in Figure 2. Place is represented with a white circle in which marking can be stored; transition is represented with a box which indicating status shifts. The arc indicates the direction of flow. The arc is often labeled with weight, in general case with 1. The black circle in place is the value of initial marking. It is used a number to represent the black circle more than three. For example, place may be the resource, likes video chip, text, image, or audio. The marking can be treated as a time marking. It is very useful to apply Petri nets on multimedia which involved complicated spatial and temporal relationship among medias in a content. The bottleneck or interference of playback process can be found easily. This is the reason that Petri nets is chosen to model the content adaptation and content design process in our project. 3

5 Figure 2. Petri nets Graph SMIL is an extension from extensible Markup Language (XML) which is recommended by World Wide Web Consortium (W3C). Since 1998 the initial version 1.0 was proposed, now the current version is 2.0 in which the basic syntax has been defined. (W3C, 2001) (L. Rutledge, 2001) In SMIL, every media or resource can be treated as object. Each object has its own attributes or properties and described in metadata format, likes identification, title, description, format, and etc. SMIL can be considered to provide an integrated markup platform to organize these objects. The user or teacher use SMIL to design the learning scenario, likes appearing order and position among objects, to express learning objectives. The most benefit which SMIL contributed is the capability to integrate variety multimedia sources. Through SMIL player, a unified and outstanding multimedia content will be presented. SMIL syntax is similar with Hypertext Markup Language (HTML) and easily for editing. Using the temporal syntax, it can provide the solution for ordering and synchronization among media which still an open issue in general homepages. Based on the features of Petri nets and SMIL, it is obviously found that these two subjects have their specific contributions applied to content adaptation and content design, there are still some drawbacks need to be fixed. For example, the organization of content involved ordering and synchronization in temporal and spatial domain, especially for a complicated multimedia objects presentation. In our research, it is shown that the Petri nets is emphasized on workflow and analysis. It can provide a good temporal arrangement, but without detailed spatial attributes. The SMIL is focus on integrating task among different types of multimedia, including simple layout allocation and objects sequencing. On the designing phase of learning content, it can not provide a good method to help author to model and analysis relationships among these medias, especially when a complicated sequencing and temporal scenario involved. In our project, we show the essentiality to integrate Petri nets and SMIL. Based on this, a new authoring tool is proposed to help author to generate a complicated content in a fast and intuitive way. This tool has been used in our content management system and facilitate on digital learning environment. Content Adaptation There are many types of media content can be found on the internet. Each type of media has their specific feature and corresponding requirements. However, a well organized content with suitable adaptation will do a lot of improvement on learning effects if it can be shown properly. It is known that a XML file with XSL can provide variety of presentation. So XSL is the key for content adaptation. In other words, different XSL will provide different presentation. So a suitable style sheet which is designed based on the corresponding device s capability then would be an important issue. In this year, we proposed a content adaptation mechanism is shown in Figure 3. Figure 3. Content adaptation mechanism In this system, the desired content which is expressed in XML format will be treated as the target need to be checked. Followed XML schema, it is easily found the properties of all type of resources. The CC/PP parser will parse the CC/PP files in the same time. Based on the resolved information of device, CC/PP parser will select a suitable XSL document from the library. If there is no suitable file existed, the XSLT agent will 4

6 generate a special XSL file based on the resolved device information, like color capability, size of screen, and function of sound. In general, the XSLT will handle two major works. First, determine whether these media components will be delivered to client or not. Second, if above answer is positive, proceed the adaptive process to make a friendly presentation to client based on the capability of client s device. In case present rich types of content into a small device, the following factors need to be considered: (1) Hardware capabilities: screen size, resolution, memory requirement, and audio capability. (2) Software capabilities: browser version, supported markup language, JAVA, and others API. The CC/PP parser will select suitable transformation style-sheet, XSL, based on the above device information. For example, the pocket PC s operating system, i.e. M.S. windows CE, can support text, image, and streaming media components. Then in the content, same type components can be delivered to client device through an adaptive process, likes adjust the display size in vertical and horizontal axis. The resolution also can be reduced from true color to 16-bit color dependent on the capability of display card. All these works is aimed to achieve a better presentation effect. The software environment is also included in the CC/PP file, likes version of OS, media player, and applications. CC/PP parser then will know there is a suitable media player exists or not. If not, the streaming media may not need to be delivered to client s device. In this way, the transmitted data will be kept on the necessary amount. This will also help to reduce the need of bandwidth. The browser is the last factor need to be considered. Not all the browsers can process the whole markup languages. In fact, not all the browser can not support each type of markup language, so it is necessary to develop a mechanism to transform it to an acceptable format which can be parsed by browser. For adapting content, it is necessary to render content based on the capability of student s device. The creator or author will design their content based on a completely concept or scenarios. For example, a text-based syllabus will be appeared first with the learning objective. In the same time, the teacher s pre-recorded audio file will be played. Then the others lecturing content will be played sequentially or on demand. The lecturing content also can be re-organized into basic, preliminary, or advanced categories, according to the level of student. It is easily found that there exists a strongly temporal relationship among whole content components. A good arrangement on lecturing material is the most important thing teacher need to focus on. Teacher is unnecessary to worry about how to present these contents on the PC, or NB. In this study, due to the flexibility and cost, PDA is chose to be the adaptive target. Currently, the PDA s browser has not supported the SMIL or HTML+TIME multimedia markup language. In order to implement above mentioned create content once, ready for universal access, it is necessary to create a mechanism which will help author or creator to remove the features which can not be displayed on the small devices, likes PDA. The detail process is described as followed: (1) Parsing the content and record all the detailed information about each component including types, identification, name, src, time, location, size, and etc. (2) Determining which media component can be acceptable on the student s device. (3) Based on the capability limitation, the corresponding reference on spatial domain will be sequentially arranged. (4) Adjusting the presentation format for each candidate component. It is obviously that there exist synchronization and timing relations among these components. It is necessary using an analyzing model to effectively exploit these relationships. Due to Petri Net can be used for representation the synchronization and fidelity constraints among the multimedia components, an extend Petri Net model is adopted, called MOPN (Multimedia Object Petri Net). MOPN can provide the modeling 5

7 function on the duration property to analyze the temporal and sequence relation among content s components, i.e. rendering plan. We will use this model to explain it. A MOPN is a triple, MPON = (PN, D, MF) where PN is an ordinary Petri Net, D is a duration function, and MF is set of modalities. Each component is assigned a modality and duration where modality indicates the type of player used to play this component. The duration indicates how long the component is to be played. The following Figures will illustrate our approach. The original lecturing content is composed of rich types of media, likes streaming video, text description, and images, shown in Figure 4. In order to describe the detailed relation among content s components, we use Petri Net to simulate the workflows in our system. Using the Petri Net, it is easily to analyze the relationships among the resources especially the temporal properties. In Figure 5, the ordering sequence of components is shown using previous mentioned MOPN. From Figure 5, picture 1 with text 1 will appear first, then the picture 2 with text 2, and picture 3 with text3. In the meantime, streaming video will be continuously played. This content can be display on any PC or NB with browser MS IE 6.0 and above which support HTML+TIME. Figure5. The displaying relation of Figure 4 s content components using MOPN to expressed There are still others browsers which do not support HTML+TIME. So in our system, the original content will be parsed and transformed into HTML format. During the transformation process, the time and sequence information of components will be removed. Then the component will be adapted based on the device capability. The adapted result is shown in Figure 6. From Figure 7, it can be found that the time and sequence property of component has been removed. All the components will be appeared in the same time. Figure 6. Adaptive content shown in MS browser 5.0 which not supporting HTML+TIME Figure 4. Original content which composed of pictures, corresponding texts, and streaming video. Figure 7. The displaying relation of Figure 6 s content components using MOPN to expressed Same content with adaptation applied to PDA is shown in Figure 8. From Figure 9, it 6

8 is found that all the picture and text components will be appeared in the same time. Only the streaming media need user sending a click instruction to notify system to start playing. The time and sequence property of component has been removed. It also can be seen that the presentations of pictures with corresponding text-based description has been adapted for the PDA. In this way, student can get more friendly content presentation on his device. Figure 8. Adaptive content shown in PDA them has specific format to contribute their features. To sharing and reusing rich type of resource, we use W3C recommended Synchronized Multimedia Integration Language (SMIL) to integrate these variety. Petri nets technology is introduced to enhance the temporal attributes of SMIL. The proposed multimedia rendering environment is shown as a fast generating and intuitive method. The architecture of proposed SMIL editor and rendering tool is shown in Figure 10. This system is composed of rendering center and server management. In this study, we focus on Manifestation Plan and Toolkit in the rendering center. User can use Petri nets to visualize the rendering plan in both temporal and spatial domain. The system will automatically transfer the visualized content information into SMIL format. The generated SMIL file will be transmitted to Media Sever for further process. Then using the player embedded in client device to play the SMIL file. In Figure 10, Manifestation Plan is composed of three modules, as follows: a. Flow chart (Petri nets) b. Switching box c. Rendering panel (position) This tool mainly provides a fast method to help user designing desired layout, then export in SMIL files. Figure 9. The displaying relation of Figure 8 s content components using MOPN to expressed SMIL Authoring We propose a fast and efficient working environment which integrating variety of multimedia to provide an organized media content. Currently, there are many type of digital media exist on the internet. Each of 7 Figure 10. Petri Nets based Synchronized Multimedia Rendering Environment architecture Through this interface, user can import multimedia objects, likes video, text, image, and audio. Our system can support the file format which SMIL player supported. It also provides a real time pre-view function to let

9 user check preliminary content status. In this interface, we provide an intuitive method to help user operating. That is all the multimedia objects can be put on any position in the layout by drag-and-pull. The rendered file will be stored in SMIL format. The played result of SMIL is shown in Figure 11. In first year, we finished the background and related research and established the prototype mechanism to verify our idea. In the second year, we will focus on the delivery mechanism over wired/wireless network environment. The proposed SyncML is to be developed for tracing and synchronizing user s learning state. We also will integrate the adapting and presenting mechanism with the new delivering mechanism. Figure 11. The result of SMIL In this paper, we have proposed a content adaptation mechanism to deliver adaptive content from any platform in any format to any device through any network at anytime and anywhere. We have demonstrated that transform the original component s feature to satisfy the requirement on device, change the presentation sequence to match the different browser requirement, and re-organized the content to achieve optimal presentation on the student s device. According to above result, we can let author to concentrate on content design and use our content adaptation mechanism to deliver adaptive content to different type of devices. The related researches are published on [1][3][5][7][8]. We have proposed SMIL editor and rendering tool for Multimedia Synchronization and Integration can provide a fast and intuitive method to help user generate multimedia content. The mentioned problems between Petri nets and SMIL editor have been resolved in the proposed system. There are some topics still in developing, likes the analysis and simulation modules on Petri nets to enhance the SMIL rendering Environment. The related researches are published on [6][8]. 8 N.R. Adam, V. Athuri, I. Adiwiyaya, S. Banerjee, R. Holowczak, (2001), A dynamic manifestation approach for providing universal access to digital library objects, IEEE Trans. on Knowledge and Data Engineering, Vol. 13, No. 4, July-Aug. 2001, pp T.H. Ess, (2002), Accessing devices using a Web service, IEEE Proceedings SoutheastCon, 2002, pp D. Hunold, A.N. Barreto, G.P. Fettweis, M. Mecking, (2000), Concept for universal access and connectivity in mobile radio networks, The 11th IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, 2000, pp W3C, (2002), W3C in seven points, / CCPP, (2002), Composite Capabilities / Preferences Profile, Cocoon, (2002), Apache Cocoon, W3C, SIML, Microsoft Corp., HTML+TIME, C. A. Petri, 1962, Kommunikation mit Automaten, PhD thesis P. Merlin and D. Farber., 1976, "Recoverability of communication protocols implications of a theoretical study, IEEE Transactions on Communications. K. Jensen., 1992 Coloured Petri Nets: Basic Concepts, Analysis Methods and

10 Practical Use. Vol. 1, Basic Concepts. EATCS Monographs on Theoretical Computer Science, pp Berlin: Springer-Verlag. T. D. C. Little and A. Ghafoor,1990, Synchronization and Storage Models for Multimedia Objects, IEEE Journal on Selected Areas in Communications, Vol. 8, No. 3, 1990, pp Nabil R. Adam et al., 2001, A Dynamic Manifestation Approach for Providing Universal Access to Digital Library Objects IEEE Transactions On Knowledge And Data Engineering, Vol. 13, No. 4, pp July/Aug W3C Recommendation, 2001, Synchronized Multimedia Integration Language (SMIL 2.0), Aug. 2001, Tadao Murata, "Petri nets: Properties, analysis and applications", Proceedings of the IEEE, Vol. 77, NO. 4, pp , Apr L. Rutledge, 2001, "SMIL 2.0: XML for Web Multimedia" IEEE Internet Computing, pp.78-84, Sep.-Oct As addressed in the proposal, we plan to contribute this project in the following three areas, including 1) the progress of academic researches, 2) the development of e-learning environment, and the 3) improvement of student learning status. And we have successfully met our goals; we will describe what we have done to meet the goal as follows. In terms of academic research results, we have published two journal papers and seven conference papers; we have three graduate students graduated with their thesis titles directly related to this project, and we will have two more graduate students will graduate this year under the same project. From the development of e-learning environment perspective, we have developed a learning content management system, a 9 content adaptation system, and a SMIL multimedia authoring and analysis tool. Finally, we have applied our environment and mechanism to a high school to exercise a field study and get feedback to improve our system. We summarize our current research results from three perspectives: the techniques we have developed, the tools we have been developing, and the referred papers we have published. We have developed a content authoring and content adaptation mechanisms for SMIL multimedia. Meanwhile, we have been implementing the mechanism we developed into a learning content management system; a SMIL authoring tool and a content adaptation system for various devices which could possible connect to the SMIL content server for multimedia content access. The topic and the undergoing research results of this project have great potential to be published as journal and conference papers, in the past year, we have published two journal papers and seven conference papers under this research grant NSC S [1]. S.J.H. Yang and C.C. Chen, 2003, Ann Integrated Approach for Workflow Process Modeling and Analysis Using UML and Petri Nets, MIS Review, pp , NSC S [2]. S.J.H Yang, J.J.P. Tsai, and C.C. Chen, March-April, 2003, Fuzzy Rule Base Systems Verification Using High Level Petri Nets, IEEE Trans. on Knowledge and Data Engineering, vol. 15, no. 2, March-April 2003, pp , NSC S (SCI, EI) [3]. Stephen J.H. Yang, Norman W.Y. Shao, Rick C.S. Chen, 2003, "Design and Implementation of a Universal Access Mechanism to Multimedia Learning Content," 2003 international Conference on Computer-Assisted Instruction, B4-1, April NSC S

11 [4].,, May, 2003, Software Engineering (MSE), Dec ,, 2003 NSC S ICCAI 2003, Taipei, Taiwan, NSC [8]. Stephen J.H. Yang, Norman W.Y. Shao, S Addison Y.S. Sue, 2003, Personalized [5]. Stephen J.H. Yang, Norman W.Y. Shao, Metadata Mechanism Applied to Rick C.S. Chen, 2003, "A Multimedia Adaptive Mobile Learning, Second Content Adaptation Mechanism to IEEE International Workshop on Implement Universal Access on Learning Wireless and Mobile Technologies in Environment", NCS2003, Dec Education (WMTE 2003), March 23-25, NSC S (Accepted), NSC S-008 [6]. Stephen J.H. Yang, Norman W.Y. Shao, Kevin C.Y. Kuo, 2003, "A SMIL Editor [9]. Irene Chen, Stephen J.H. Yang, Norman and Rendering Tool for Multimedia W.Y. Shao, 2003, Applying Synchronization and Integration", Multi-Sensory Learning Model with NCS2003, Dec NSC Mobile Handheld Devices to Pervasive S Learning, Second IEEE International [7]. Norman W.Y. Shao, Stephen J.H. Yang, Workshop on Wireless and Mobile Addison Y.S. Sue, 2003, "A Content Technologies in Education (WMTE Management System for Adaptive 2003), March 23-25, 2004 (Accepted), Learning Environment", IEEE Fifth NSC S International Symposium on Multimedia 10

12 11