Chapter 13 Add Multimedia to Your MIDlets

Size: px

Start display at page:

Download "Chapter 13 Add Multimedia to Your MIDlets"

Beverley Rose
6 years ago
Views:

1 Chapter 13 Add Multimedia to Your MIDlets The Mobile Media API (MMAPI), which extends the functions of Java 2 Platform, Micro Edition (J2ME), allows easy and simple access and control of basic audio and multimedia resources on devices with limited memory and processing capabilities. This chapter is intended to get you an overview of the MMAPI, starting with an explanation of the MMAPI architecture, packages, and objects, and then showing you how to create an audio- and video-capable MIDlet. 1. Introduction to the Mobile Media API (MMAPI) This section offers a general overview of MMAPI. You'll look at MMAPI features, multimedia processing, and the MMAPI architecture. What is MMAPI? MMAPI is an implementation of the Java Community Process's JSR 135. As a simple and lightweight optional package, it gives Java developers access to native multimedia services available on a given device. MMAPI Features The MMAPI brings the following capabilities to J2ME: Scalability: The API brings scalable sound and multimedia support capabilities to J2ME. It is aimed at the CLDC, Connected Device Configuration (CDC), and profiles based on CDC and CLDC. Small footprint: The API allows easy and simple access and control of basic audio and multimedia resources on devices with limited memory and processing capabilities. Support for tone generation, playback, and recording of time-based media: The API supports any time-based audio or video content. Protocol and content independence: The API is not biased toward any specific content type or protocol. Ability to subset: Developers can limit application support to particular types of content -- like basic audio, for example. Extensibility: New features can be added easily without breaking older functions. More importantly, additional formats can be easily supported, and the framework is in place for additional controls. MIT3447 OOPA Chapter 13 1 Prepared by K.T. NG

2 Options for implementers: The API offers features for different purposes. The API is designed to allow implementers to leave some features unimplemented if they cannot be supported in a particular application. Multimedia Processing There are two parts to multimedia processing: 1. Protocol handling: Reading data from a source such as a file or a streaming server into a media-processing system. 2. Content handling: Parsing or decoding the media data and rendering it to an output device such as an audio speaker or video display. MMAPI Architecture To facilitate protocol and content handling operations, the MMAPI provides the following highlevel object types: DataSource Player Control Manager DataSource A DataSource encapsulates protocol handling by hiding the details of how the data is read from its source. javax.microedition.media.protocol.datasource is the abstract parent class for all data sources in the Mobile Media API. Player A Player reads the data from a DataSource, processes it, and renders it to an output device. A Player knows how to interpret media data, be it MP3 audio data or a QuickTime movie. Players are represented by implementations of the javax.microedition.media.player interface. Utility methods in DataSource enable the Player object to handle the content. Control One or more Controls can be used to modify the behavior of a Player. You can get the Controls from a Player instance and use them while the Player is rendering data from media. For example, you could use a VolumeControl to modify the volume of a sampled audio Player. MIT3447 OOPA Chapter 13 2 Prepared by K.T. NG

3 Controls are represented by implementations of the javax.microedition.media.control interface; specific control sub-interfaces are in the javax.microedition.media.control package. To enable video capability in a MIDlet, you need to obtain a VideoControl. The initdisplaymode() method initializes the mode that determines how the video is displayed and must be called before video can be displayed. Potential modes include USE_GUI_PRIMITIVE, USE_DIRECT_VIDEO, and implementation-specific modes. USE_GUI_PRIMITIVE defines how the GUI is displayed. The mode USE_DIRECT_VIDEO defines how the video is displayed. Manager MMAPI specifies a third object, a factory mechanism known as the Manager, that enables an application to create Players from DataSources and InputStreams. The javax.microedition.media.manager class is the access point for obtaining systemdependent resources such as Players for multimedia processing. The relationship between the various components is illustrated in Figure 1. Figure 1. MMAPI components 2. Using the Mobile Media API In this section, we ll discuss MMAPI's key methods. playtone() The Manager.playTone () method, illustrated in Listing 1, plays a single tone or a very short sequence. MIT3447 OOPA Chapter 13 3 Prepared by K.T. NG

4 Listing 1. playtone() public static void playtone (int note, int duration, int volume) throws MediaException The duration is specified in milliseconds, and the volume ranges from 0 (silent) to 100 (loud). The note is specified as a number, as in a MIDI file, where 60 is middle C and 69 is a 440 Hz A. Listing 2 illustrates playtone() in action. Listing 2. playtone() in Action try Manager.playTone(60, 200, 90); catch (MediaException ex) System.out.println("can't play tone"); createplayer() The real magic of the Mobile Media API is exposed through Manager's createplayer() method. There are three different versions of this method; which version you'll use depends on the way you want get to the media data. The method in Listing 3 lets you create a Player by accessing media data from an InputStream. The type argument specifies the content type of the input media. For example, you'd use audio/midi for an audio MIDI. If null is used, Manager attempts to determine the type. Listing 3. Creating a Player From an InputStream public static Player createplayer(inputstream stream, String type) throws IOException, MediaException The version of createplayer() in Listing 4 lets you create a Player from a DataObject, an object that speaks a protocol to get access to media data. Listing 4. Creating a Player From a DataSource public static Player createplayer(datasource source) throws IOException, MediaException The simplest way to obtain a Player is to use the version of createplayer() in Listing 5 and just pass in a string that represents media data. MIT3447 OOPA Chapter 13 4 Prepared by K.T. NG

5 Listing 5. Creating a Player From a URL public static Player createplayer(string locator) throws IOException, MediaException MMAPI determines which protocol to use and gets the media data to the Player. For instance, you might specify an audio file on a Web server, as in Listing 6. Listing 6. Specifying an Audio File on a Server Player p = Manager.createPlayer(" The application uses the methods of the returned Player to control the retrieval and playback of time-based media. Life cycle methods: States of a Player The Player's life cycle consists of five states: 1. UNREALIZED: When a Player is created, it is in the UNREALIZED state. 2. REALIZED: Calling realize() moves the Player to the REALIZED state and initializes the information it needs to acquire media resources. For example, if a Player is rendering an audio file from an HTTP connection to a server, that Player reaches the REALIZED state after the HTTP request is sent to the server, the HTTP response is received, and the DataSource is ready to begin retrieving audio data. 3. PREFETCHED: Calling prefetch() moves the Player to the PREFETCHED state, establishes network connections for streaming data, and performs other initialization tasks. The state is achieved when the Player has read enough data to begin rendering. 4. STARTED: Calling start() causes a transition to the STARTED state, where the Player can process data. When the data is being rendered, the Player's state is STARTED. When it finishes processing (that is, when it reaches the end of a media stream), it returns to the PREFETCHED state. 5. CLOSED: Calling close() moves the player to the CLOSED state. The application uses the methods of the returned Player to control the retrieval and playback of time-based media. The Player's five states and the state transition methods are summarized in Figure 2. MIT3447 OOPA Chapter 13 5 Prepared by K.T. NG

6 Figure 2. States of a Player getcontrol() A Player provides controls specific to the particular types of media it processes. An application uses getcontrol() to obtain a single control, or getcontrols() to get an array of them. Public Control getcontrol(java.lang.string controltype) The controltype specifies the type of Control to be returned. For example, if a Player invokes getcontrol() with type MIDIControl, it gets back a MIDIControl. Similarly, when VideoControl is specified, the Player gets a VideoControl. If a control type is not supported, getcontrol() returns null. Supported Formats MMAPI supports several audio and video formats. Formats supported by both MMAPI and the J2ME Wireless Toolkit are: Audio: PCM and WAV MIDI: Type 0 (single track), Type 1 (multiple tracks), and SP-MIDI Video: MPEG-1 The Mobile Media API doesn't require any specific content types or protocols, but you can find out at run time what formats are supported by a particular application by calling Manager's getsupportedcontenttypes() and getsupportedprotocols() methods. If you ask Manager to give you a Player for a content type or protocol that is not supported, it will throw an exception. 3. MMAPI in Action: Building a Sample Application MIT3447 OOPA Chapter 13 6 Prepared by K.T. NG

7 In this section, you learn to develop a MIDlet with multimedia capabilities. It shows the application of the various MMAPI functions I discussed in previous sections. The MIDlet plays a simple tone, an audio clip from an URL, a video clip from an URL, and audio from a resource file. The source code is in Listing 7; it should be saved in a file named MediaMIDlet.java in the <WTK Home>\apps\MediaMIDlet\src directory. Listing 7. A Multimedia MIDlet import javax.microedition.midlet.*; import javax.microedition.lcdui.*; import javax.microedition.media.*; import javax.microedition.media.control.*; import java.util.*; import java.io.*; /** * * This is a demo midlet to show the basic audio functionalities, to * play a Tone, URL wave file and Resource file. * */ public class MediaMIDlet extends MIDlet implements CommandListener, Runnable private Player player = null; private Command exitcommand = new Command("Exit", Command.EXIT, 1); private Command playcommand = new Command("Play", Command.ITEM, 1); private ChoiceGroup menulist = new ChoiceGroup("Play List", Choice.EXCLUSIVE); private Display display; private Form form = new Form("The Mobile Media"); private static String mode=""; public MediaMIDlet() super(); display = Display.getDisplay(this); initplaylist(); form.append(menulist); form.addcommand(exitcommand); form.addcommand(playcommand); form.setcommandlistener(this); display.setcurrent(form); public void run() MIT3447 OOPA Chapter 13 7 Prepared by K.T. NG

8 try if ("URL".equals(MediaMIDlet.mode)) player = Manager.createPlayer(" player.realize(); player.prefetch(); player.start(); else if ("RESOURCE".equals(MediaMIDlet.mode)) String loc = "pattern.mid"; InputStream is = getclass().getresourceasstream(loc); player = Manager.createPlayer(is, "audio/midi"); player.realize(); player.prefetch(); player.start(); else if ("VIDEO".equals(MediaMIDlet.mode)) player = Manager.createPlayer(" player.realize(); VideoControl videocontrol = (VideoControl)player.getControl("VideoControl"); if(videocontrol!= null) Item video = (Item)videoControl.initDisplayMode( videocontrol.use_gui_primitive, null); Form v = new Form("Playing Video..."); v.append(video); display.setcurrent(v); player.prefetch(); player.start(); player.realize(); player.prefetch(); player.start(); catch (IOException iex) iex.printstacktrace(); catch (MediaException ex) System.out.println("can't create player"); catch (Throwable t) player = null; public void startapp() public void pauseapp() MIT3447 OOPA Chapter 13 8 Prepared by K.T. NG

9 public void start() Thread t = new Thread(this); t.start(); /** * Destroy must cleanup everything not handled * by the garbage collector. */ public void destroyapp(boolean unconditional) display.setcurrent(null); public void commandaction(command c, Displayable s) if (c == exitcommand) destroyapp(true); notifydestroyed(); else if (c == playcommand) int i = menulist.getselectedindex(); if (i == 0) // Simple tone try Manager.playTone(60, 200, 90); catch (MediaException ex) System.out.println("can't play tone"); else if (i == 1) MediaMIDlet.mode="URL"; start(); else if (i == 2) MediaMIDlet.mode="RESOURCE"; start(); else if (i == 3) MediaMIDlet.mode="VIDEO"; start(); private void initplaylist() menulist.append("play Simple Tone",null); menulist.append("play URL",null); menulist.append("play Resource",null); menulist.append("play Video",null); 4. Analyzing the Multimedia MIDlet MIT3447 OOPA Chapter 13 9 Prepared by K.T. NG

10 Now let's see the details of how the code works. The lines in Listing 8 indicate the required packages for MIDlet media function. Listing 8. MMAPI Imports import javax.microedition.midlet.*; import javax.microedition.lcdui.*; import javax.microedition.media.*; import javax.microedition.media.control.*; import java.util.*; import java.io.*; The MIDlet implements the CommandListener and Runnable interfaces, as illustrated in Listing 9. Listing 9. MMAPI Interfaces public class MediaMIDlet extends MIDlet implements CommandListener, Runnable CommandListener ensures that the MIDlet responds to commands like Play and Exit, as you can see in Listing 10. Listing 10. MIDlet Command Implementation private Command exitcommand = new Command("Exit", Command.EXIT, 1); private Command playcommand = new Command("Play", Command.ITEM, 1);... public void commandaction(command c, Displayable s) if (c == exitcommand) destroyapp(true); notifydestroyed(); else if (c == playcommand) // Play media from URL or Resource When a MIDlet implements a CommandListener, it must overwrite the commandaction(command c, Displayable s) method. The application creates the commands Play and Exit. The Exit command quits the application. On receiving the Play command, the application determines the selected menu option and behaves accordingly. The MIDlet implements Runnable to create a thread. In order to avoid a deadlock on networking and I/O operations, you should run a Player on a different thread from the MIDlet. To MIT3447 OOPA Chapter Prepared by K.T. NG

11 implement Runnable, you need to overwrite the run() method. All this is illustrated in Listing 11. Listing 11. Multithreaded Approach public void commandaction(command c, Displayable s) // Get selected option start(); public void start() Thread t = new Thread(this); t.start();... public void run() // Create Player based on media The initplaylist() method builds the initial menu, menulist, that is displayed on launch of the MIDlet, as shown in Listing 12. Listing 12. MIDlet List of Options private ChoiceGroup menulist = new ChoiceGroup("Play List",Choice.EXCLUSIVE);... private void initplaylist() menulist.append("play Simple Tone",null); menulist.append("play URL",null); menulist.append("play Resource",null); menulist.append("play Video",null); When the first option, Play Simple Tone, is selected, Manager.playTone() is called and a simple tone is played. When the second option, Play URL, is selected, the mode is set to URL. When the third option, Play Resource, is selected, the mode is set to RESOURCE. When the fourth option, Play Video, is selected, the mode is set to VIDEO. With all the options, the start() method is called to create a new thread. The run() method of the thread determines whether to create a Player from a URL, Resource, or Video, depending on the mode value: If the mode is URL, Manager.createPlayer(url) is called. The createplayer() method takes care of determining the type of media data. Listing 13. MIDlet Options: Play URL if ("URL".equals(MediaMIDlet.mode)) player = MIT3447 OOPA Chapter Prepared by K.T. NG

12 Manager.createPlayer(" player.realize(); player.prefetch(); player.start(); If the mode is RESOURCE, the MIDlet forms an InputStream from the indicated location. In that case, Manager.createPlayer() takes the InputStream and related audio type as parameters and creates the Player. Listing 14. MIDlet Options: Play RESOURCE if ("RESOURCE".equals(MediaMIDlet.mode)) String loc = "pattern.mid"; InputStream is = getclass().getresourceasstream(loc); player = Manager.createPlayer(is, "audio/midi"); player.realize(); player.prefetch(); player.start(); If the mode is VIDEO, Manager.createPlayer(url) is called. After the player has been created, additional steps are required to initialize the phone screen to display the video, as illustrated in Listing 15. Listing 15. Initialization for a Video Display if ("VIDEO".equals(MediaMIDlet.mode)) player = Manager.createPlayer(" player.realize(); VideoControl videocontrol = (VideoControl)player.getControl("VideoControl"); if(videocontrol!= null) Item video = (Item)videoControl.initDisplayMode( videocontrol.use_gui_primitive, null); Form v = new Form("Playing Video..."); v.append(video); display.setcurrent(v); player.prefetch(); player.start(); MIT3447 OOPA Chapter Prepared by K.T. NG

13 In all cases, MediaException and IOException should be properly handled. MediaException is thrown only if the Player cannot be created. On the other hand, IOException is thrown if an invalid URL or InputStream cannot be formed. A Throwable object takes care of any thread-related exceptions. Listing 16 outlines the basic methods in the MIDlet that need to be defined: startapp() to take care of any startup activities, pauseapp() to take care of activities while the MIDlet is paused, and destroyapp() to perform cleanup activities not handled by the garbage collector. Listing 16. MIDlet-Related Methods public void startapp()... public void pauseapp()... public void destroyapp(boolean unconditional) display.setcurrent(null); Now you understand how the code works. In the next section, you'll see what the application looks like in action. 5. Running the MIDlet The J2ME Wireless Toolkit comes with MMAPI, along with a mobile device emulator on which you can run the sample code. Start by invoking the Wireless Toolkit's Ktoolbar; on Windows, you can do this by choosing Start > Sun Java Wireless ToolKit 2.2 > KToolbar. Create a project by selecting New Project. Enter a name for the project as shown in Figure 3. Figure 3. Create a project After the project is created, create a Java file with the same name as the MIDlet class name in the <WTK Home>\apps\MediaMIDlet\src directory. Build the project by choosing Build in KToolbar, and then choose Run to run it. Figure 4 illustrates the running MIDlet in the emulator. Navigate to each of the options, and then select each MIT3447 OOPA Chapter Prepared by K.T. NG

one and press Play to hear the different tones.

are selected: http://java.sun.com/products/java-media/mma/media/test-mpeg.

$be saved in the <WTK Home Directory>\apps\MediaMIDlet\res directory.$

14 one and press Play to hear the different tones. You can also select the Video option to play a video clip, as shown in Figure 5. The MIDlet developed here plays the following resources when options 2 and 4 are selected: In order to run option 3, the MIDI file available in this chapter should be saved in the <WTK Home Directory>\apps\MediaMIDlet\res directory. Figure 4. Multimedia MIDlet menu Figure 5. Multimedia MIDlet playing a video MIT3447 OOPA Chapter Prepared by K.T. NG

15 Summary You've now seen how a MIDlet can incorporate a number of multimedia capabilities: playing various tunes, playing a WAV file from an URL, and playing a local MIDI file. You also saw how a video clip can be displayed on the phone screen. While these examples are simple, they have illustrated the powerful abilities that MMAPI can add to your mobile Java applications. MIT3447 OOPA Chapter Prepared by K.T. NG

Mobile Information Device Profile (MIDP) Alessandro Cogliati. Helsinki University of Technology Telecommunications Software and Multimedia Laboratory

Multimedia T-111.5350 Mobile Information Device Profile (MIDP) Alessandro Cogliati Helsinki University of Technology Telecommunications Software and Multimedia Laboratory 1 Outline Java Overview (Editions/Configurations/Profiles)