Introduction to Mobile Application Development Using Android Week Four Video Lectures

Transcription

1 Introduction to Mobile Application Development Using Android Week Four Video Lectures Week Four: Lecture 1: Unit 1: Multimedia Multimedia Support in Android Multimedia Support in Android We are now going to examine the multimedia support that is available in Android, in particular the audio and video formats that Android supports, and also the various classes that are available in Android for both playing back and recording multimedia, both audio and video. Android provides very comprehensive support for various multimedia formats. Almost all audio formats that are in use, including MP3, MP4E, Ogg, Midi and so on, are supported in Android. Video formats include MPEG-4, H.263, H.264, and so on. Images that you use in your Android applications, typically we prefer to use PNG format. If not, then JPEG is also acceptable. GIF, while can still be used, is not necessarily recommended by the Android developers. You can play audio or video using the various classes that are built into Android, either from media files that are stored in the Resources folder of your application. If you are going to include media files in your application, then you would normally put them into a subfolder under the Resources folder, called as the raw folder and you can import and use them within your application. Now media files can also be accessed from the file system. For example, if you store music on your SD card that can the played back by the media classes available in Android. Also, we can just stream the audio or video data over a network connection from a server, and then play them back in Android. Now Android classes that specifically support multimedia that we are going to examine next, include the MediaPlayer class. The MediaPlayer class is a very comprehensive class for supporting playback of different kinds of audio and video, including decompression and so on, and also recording of audio and video in various formats. The second class that we're going to look at is the SoundPool class. The SoundPool class is used for generating short bursts of sound, typically with very low latency where required. So, for example, if you're designing games, this would be the class that you use to produce sounds that are useful within your game. Now the controlling of the audio and video playback and the sources is achieved by using the Audio Manager that is available within Android. Now returning to our shooting game example, we will see how the use of media enhances the game itself. The shooting game that we developed in the previous exercise has no audio, but although we can still play the game. Now adding in audio support, including background music and sounds, enhances the playing experience for the users significantly. Week 4: Multimedia 1

2 So, we can take a game like this, and then turn it into a lot more enjoyable game by adding audio and video so a game like this can be turned into a game with sound and music in the background and this enhances the user's pleasure for using and playing these games. In the next few exercises we are going to examine how we add this multimedia support into the game to enhance the game further. We are now going to move onto the next exercise where we will modify the shooting game to use a SurfaceView instead of a View and this entails implementing some of the computation in a background thread. Week 4: Multimedia 2

3 Week Four: Lecture 1: Unit 4: Multimedia Exercise Shooting Game with SurfaceView Exercise Shooting Game with SurfaceView In this exercise, we are going to modify the shooting game. Previously, we used the View to implement the user interface for the shooting game. So, we extended the View to implement a custom view. Now, in this version, we are going to extend a SurfaceView, rather than a View. Now, what does the SurfaceView bring to the table? SurfaceView allows you to update the canvas in a background thread at your own leisure and when you've completed updating the canvas, then you can post the canvas to the UI. So, this enables you to implement a lot more complicated graphics updates to the canvas, because the updates are going to be taking place behind the scenes and then you will be able to update the user interface. So, as you'll see, the game looks exactly the same, except that now the canvas is handled using a SurfaceView rather than a View after the end of this exercise, I'm going to explain the difference between SurfaceView and View and the way you implement the surface view in an application in Android. In particular, when you are using SurfaceView, you need to implement, first and foremost, three different callback methods called onsurfacecreated, onsurfacechanged, and onsurfacedestroyed. In addition, you need to create a background thread. So that is where the action comes in for this particular exercise. As usual, once you download the zip file that we provide for you to get you started on this exercise, unzip the file and then import it into Android Studio. Now, once you do that, you will notice that most of the code will remain more or less the same as before. The major updates would be in the DrawView.java class. So, let's focus on the DrawView.java class and understand how we have updated this class to use the SurfaceView, rather than the view. So, once you import the project into Android Studio, open DrawView.java file and then have a look at the code and I'll explain the modifications here and then we will to go in and look at what needs to be updated in this DrawView.java file. So, once you open the file, the first thing you will notice is that now the DrawView class is extending SurfaceView, rather than view, as before and in addition, when you extend SurfaceView, as I mentioned, you need to implement some callback methods. So that's why we have this implement the callback methods. There are three callback methods that need to be implemented, which I will show you in the code later. Now, once you go into the code, you will also notice that we now have a variable called the DrawView thread that I have declared here. This variable is going to hold the reference to the thread that we need to create the background thread on which the updates are going to take place. If you are familiar with Java threads, you understand how threading works. If you're not familiar with the Java threads, it's OK. You need to understand just the parts that we have in this file to Week 4: Multimedia 3

4 get you started. When you look at this example, you realize that you might have a lot of updates to the canvas to be done. Previously, we were doing the updates inside the ondraw method. Now, the problem with that is that the ondraw method is going to be executed on the main thread of your application and this might end up slowing down your application in case you have a lot of objects to be done to the canvas. Now, it is suggested that if you have complex operations to be done to the canvas, then it is better to shift that work to a background thread. So, a background thread executes off the main UI thread. So, if you look at it from the programming perspective, if you're familiar with threads, the main UI thread is a sequence of execution that is going on. When you have a background thread, it is going to do its work separate from the main sequence of execution. So, on your CPU, your background thread is going to execute separately from your main UI thread. So, there will be no interference with the main UI thread, as such. This enables you to do the work in the background and when you have completed and updated the canvas, then you can come back and post it back to the user interface and then update the UI at that point. So, this is essentially what we're going to make use of in the SurfaceView here. Now, in addition within the DrawView constructed, you would notice that I am doing a get hold and add callback this essentially to inform the Android framework that this class is going to implement the callback methods for the SurfaceView. Now, getting down into the code, you will notice that I have three methods that I have implemented here surfacechanged, surfacecreated, and surfacedestroyed. Now, in the surfacechanged method, I am not doing anything interesting for this example. So, we're going to leave it blank. In the surfacecreated method, when the surface gets created-- essentially meaning that the UI is now up and running, and then there is a portion of the UI that has been allocated to this custom view that we implemented, the draw view, and that has to be updated. Now, this should be done in a background thread. So, when the surfacecreated method is called, the Android framework is coming back to your code and informing you that the surface isn't ready, and so you need to start working on updating the canvas at this point. So, in the surfacecreated method, we are going to start the background thread. And the way you start a background thread is to create a new thread and later on I will show you the code for this DrawView thread down below. That is where the major focus of our update is going to be. So, we create the DrawView thread and then this DrawView thread takes a parameter called that surfaceholder code, which we are going to get hold of when we are executing the updates to the canvas. So, this surfaceholder is going to come in as a parameter here. The surfaceholder enables you to get access to the canvas so that it can start updating the canvas. So, once you have created the thread, you're going to start the thread in the surfacecreated method. Week 4: Multimedia 4

5 Now, in the surfacedestroyed method, this method will be called when your UI is destroyed. So, when your activity quits, at that point, the surfacedestroyed method is called. At this point, you need to shut down your background thread, because it is no longer needed. So that's the code that we have here for shutting down the background thread. Now, the updates to the canvas is exactly the same as we did earlier. We had this method called the draw game board and so that method will still be used for updating the canvas, except that now the canvas updates are going to happen in a background thread. Now, where exactly is the code for this background thread? For that, we ill walk down to this method that we have-- this inner class that we have implemented here called the DrawView thread. This DrawView thread class is extending the thread class. When you extend a thread class inside here, you will have to implement a method called the run method. So when the thread starts executing, the code inside the run method is going to be executed. So inside this run method, we are going to do the updates to the canvas. So inside this DrawView thread class, you'll see that we have it constructor in the constructor. I am getting hold of the surface holder and that allows me to get access to the canvas. So, I'm going to store a reference to the surface holder, which came in as the parameter here. And also, I will set the name of the threat. In case I need to use it elsewhere. Now, in addition, I'm implementing another method called sector running. This method will be called to enable the thread to be running. So, this method sets a Boolean variable to indicate that the thread is running. So if, for example, you pause your game, for example, if you hit the Home key, and then move back to the launch screen, at that point your activity will be paused. So, when your activity is paused, you're going to pause your background thread also and then when your activity resumes, you're going to come back and resume the background thread. These two parts are implemented in the activity class. So, let me switch quickly to the activity class, which is the ShootingGame class and then you will notice that I have taken advantage of the to lifecycle methods-- the onpause and the onresume lifecycle methods-- and then using that to either pause the background thread or to allow the background to address resume. So, in here, in the onpause method, whenever the onpause method of my shooting game activity is called, I'm going to stop the game and when the onresume method is going to be called, I am going to resume the game. So, switching back now to DrawView.java class, let's have a look at this top game and the resume game methods inside the DrawView.java class. Switching back to the draw View.java class, down below here you will notice me implementing the stop game and the resume game method. So, in the stop game method, I will set the running Boolean variable to false, because I want to stop the game and then in the resume game method, I'm going to do set the Boolean variable to true. So, these two methods I've implemented here. Getting back to the DrawView thread class that we implemented here, you notice that we have the run method. We are going to insert the appropriate code in order to update the UI. In order to save time typing in the code, I have done the updates and then I'm going to walk you through the code here, so that you will understand exactly what is being done in this exercise. Week 4: Multimedia 5

6 Now, getting back to the run method, you notice that I am declaring a variable called canvas. And inside here, I have a while loop and for this while loop, I will executive this while loop only when the thread is running Boolean variables set to true. So, this is one way I'm going to stop updates to the canvas. So, when the thread is running, is true, then I'm going to execute the code. If that is set to false, then I'm not going to execute this code. So, if this is true, then inside here I am going to use the surfaceholder to get hold of the canvas. So first and foremost, when I'm updating the canvas, I should lock the canvas, so that only this particular thread can update the canvas, and no other thread will be updating the canvas at the same time. Now, you can think of implementing multiple threads in the background, which may be updating the canvas simultaneously. Now, in this exercise, are using only one background thread. So, this is not really essential, but in any case, if you were implementing using multiple threads, this is the first step that you should do before you update the canvas. So, you lock the canvas, and then go ahead, and then update the canvas. So, I am using the draw game board method that we had earlier to update the canvas. So, the canvas is essentially the same that we had seen earlier. So, we will update the canvas and then you see that I have this statement called sleep 30. So, this statement essentially tells this background thread to sleep for 30 milliseconds and that after that, it is going to post the updates to the user interface. Now, how do we do that? I'm going to come to that in a minute. So, this is where we are going to do the updates to the canvas. Previously, we had similar code in the ondraw method of our view. Now, this code is now shifted to the run method for the DrawView thread here. So, once you have updated this, then finally, after you have done with this part, then we're going to unlock the canvas and post. So, when called this method unlocks canvas and post with the canvas, this method essentially informs the Android framework, saying that the canvas has been updated, and it is time for the Android framework to redraw the user interface. You recall that when we used the ondraw method, we were calling the, invalidate in order to force the EndWrite framework to redraw the screen. Here, we are going to do this approach. So, you unlock the canvas and then post. And this triggers the Android framework to redraw the view on the user interface. So, this completes the updates to the shooting game exercise. Now we can go ahead and then try and run this exercise code and see how it works on the screen. So, once you deploy an application to the user interface, you're going to see this shooting game running on either your emulator or your real device, and then the playing of the game proceeds exactly the same as before. So, you can see that the game implementation hasn't changed a whole lot from the perspective of the end user, but inside we are now using the SurfaceView instead of a view to update the user interface. Week 4: Multimedia 6

7 In the next step, we're going to add some music and background sound to this game to make it a lot more fun to play in real life. Week 4: Multimedia 7

8 Week Four: Lecture 2: Unit 1: Multimedia 2D Graphics with Canvas on SurfaceView 2D Graphics with Canvas on SurfaceView In the earlier exercise, we had subclassed the view in order to draw onto the canvas. We're going to look at another option for drawing onto the canvas using a SurfaceView. The SurfaceView, as we see, is a special subclass of the view class, but the SurfaceView provides us with a dedicated drawing surface. We can update the canvas in a background thread, and once we are completed with the updating, then we post the update to the view, and the view hierarchy will be updated at that point. Now what this buys us is the flexibility of carrying out complex updates to the canvas at our leisure, and once the update is complete then, of course, the UI can be updated at that point. So, a secondary thread is required for doing this. So, all the updates will be carried out on a secondary thread instead of the main UI thread of an application. So using the secondary thread, we can carry out our work at our leisure. How do we go about implementing this in practice? To do this, we will now extend the SurfaceView class rather than the View class as we did before. We also need to implement the SurfaceHolder callback methods. Now the callback methods that are required to be implemented are three in number, surfacecreated, surfacechanged, and surfacedestroyed. Each of these will be called by the Android framework under different circumstances. As you would expect, the surfacecreated is called when the surface is ready to be drawn, and you can begin drawing at this point. When the surfacechanged is called, it is informing your application saying that there's been a change to the surface configuration, and maybe you need to adjust the way you're drawing items onto the screen. The surfacedestroyed will be called if your activity exited and at the point you no longer have the surface available for you to update, so that's the point for you to shut down your thread and then cease any updates to the canvas. So, for the surface object you can get hold of that by using the surfaceholder, and the way you get hold of the surfaceholder is by calling the getholder () method in your class. So, when you call this, then you could use the holder in order to inform the Android framework about the fact that this class is implementing all the callback methods, and the class should implement all the three callback methods, as we saw with the example earlier. Now in addition, we are required to create a secondary thread, so you need to declare a thread and then write all the parts of the code of the thread, in particular you need to implement the run method for the thread, and in the run method you're going to carry out the updates to the canvas. So, inside the thread, you get hold of the canvas by calling the surfaceholder lockcanvas method. At that point, you have to access to the canvas and then once you complete the modifications to the canvas, then you unlock the canvas and then post. So at this point, the view hierarchy will be updated with the new canvas. Week 4: Multimedia 8

9 So now we move off to our next exercise where we are going to extend the Shooting Game with some background music being played. For this, we are going to take the help of the MediaPlayer class, where we are going to initialize the player and then control the playback of the music, depending on whether your app is currently occupying the screen or not. In addition, we have a button in the menu items in the toolbar which allows you to turn on or turn off the background music when required. Week 4: Multimedia 9

10 Week Four: Lecture 2: Unit 4: Multimedia Exercise Shooting Game with Background Music Exercise Shooting Game with Background Music In our next exercise, we are going to update the shooting game app again by adding some background music to the game. So, when the game is being played, you have this annoying background music playing in the background. If you do not like to hear the music, you always have the option of clicking on this button and turning off the music. Or if you are interested, you can have the background music running in the background. So that's the next addition to the music player example. In this exercise, we're going to make use of the media player that is part of the Android framework to enable us to play this background music on our behalf. So, let's go ahead and then see how we implement this in practice. To get you started, once again download the zip file that we have provided for you for this exercise. Unzip, and then import the project into Android Studio. Once you do that, you will notice that there are some changes to this application, minor changes and then we will update a few more things in order to make this application work correctly. First and foremost, we saw that we were using a button in the menu bar to turn on and off the music. So, we're going to first learn how that is implemented. Then subsequently, we will see how the actual music playing in the background is supported within this app. In particular, you will notice that there is a new folder, called raw folder, in which I have put a few sample sound files there, and one of them being brain candy and that is what we're going to use for our background music for this exercise. In addition, I have already included the volume icons into the drawables folder, so that we can make use of it on the user interface. Now, how do you add items to the menu bar or to the toolbar, as we saw earlier? To do that, in the menu folder, we will have to create a layout file, called menu layout file. Now, for this exercise, this is already included. Now, when you create a new activity in Android, by default, a menu XML file is already included for that activity here. So, you can go ahead and then modify that in order to include additional items into the menu bar there. So, let's have a quick look at the menu_shooting_game.xml file. Now, inside here you will notice that this is an XML file, which describes the items to be included in the menu bar. One of them is the settings item, which you don't see on the menu bar, but you see it as an overflow. That is, when you click on the three vertical dots, this will show up on the screen. The other item that is included in the menu bar is the volume on/off button that we have seen being used to turn on and off the music playing in the background. Now, when you have a look at this declaration of that button in the XML code, you see that we specify the ID for it. We specify the icon to be used there and we also specify that this should be the first one in the category and then in particular, we set app: showasaction as always, meaning that this particular item should always be shown in the menu bar. Week 4: Multimedia 10

11 Now, the same thing for the settings. You see that the showasaction is shown as never, which means that the settings menu item will always be in the overflow of the menu bar. So now we see how we can include icons into the menu bar a then now we'll go into the activity code to see how the menu items are actually activated in practice. So, let's go to ShootingGame.java file and then look at how this is done in practice. Now, before I explain how the menu items work in practice, let's first examine how we're going to include the background music playing into this app. Now, if you walk into the ShootingGame.java file, you will see that I have a few to-do items that I have marked out. So, you need to fill up these in order to complete this exercise. In the instructions, we have shown you exactly what to include for each one of these, but let me go ahead and then make the modification, and then come back and explain to you what has been done to update the ShootingGame.java file to complete the exercise. Getting back to the ShootingGame.java file, let's examine this in more detail to see how this activity class has been updated in order to play the background music. First and foremost, you will notice that I have a Boolean variable here, called play_music, which if set to true, we will play the background music. If it is set to false, we will not play the background music a then I have declared a reference to an object of MediaPlayer here. We're going to see how we create the MediaPlayer and then use it to play the background music next. Now, as you come down into the oncreate () method, you will notice that right there I create a media player. Now, MediaPlayer.create is used when you are playing a music file or a sound file that you have already included in the raw folder of your application. In this case, since we have already included the music file in our application, we are going to use MediaPlayer.create. The two parameters here are the context, the second parameter specifies the name of the music file that has to be played. And I set the set looping to true meaning that at the end of playing this music clip it's going to resume and play continuously in the background. So that is how you have the background music for your games. Now, as I mentioned, we are going to start playing the background music when our application comes up onto the screen a then we will stop playing the background music when the user navigates away from the game. So, in here, we're going to take advantage of the onpause () and onresume () lifecycle methods of the activity in order to resume playing the music and stop playing the music. So,in the onresume method, we saw that we had a to-do in place of the to-do comment, I have substituted this code here. Now, since I have already created the MediaPlayer in the oncreate method, I'm just going to resume playing music by calling player start. The start is a method that is provided by the MediaPlayer class. When you call that, the playing of the music will start. Similarly, if you want to pause the music, you call the pause method of the MediaPlayer, in order to pause the music. So that is what we're going to use in the onpause method to stop playing music. Now, I am using the play music Boolean variable to decide whether I should be playing the music or not in practice. Finally, in the ondestroy method, you see that when the activity is destroyed, then I will stop playing the music completely and then release the MediaPlayer object completely. Week 4: Multimedia 11

12 So that is also essential for you to do to shut down the MediaPlayer when you quit the application. In addition, the two parts that we saw of interest to us is how we are going to activate the menu button that we used in order to pause and play the music. So how is this menu button used in practice? Now, if you are including menu buttons within your activity, then this method called oncreateoptionsmenu method is going to be called to create the menu part far your application. Now, earlier you have seen this code being included in the activity class, but we didn't actually discuss how this code is used in practice. So now you're going to see this in practice. So in here, we are going to call this Android method called getmenuinflater and then supply the menu XML file as a parameter to that to make the Android framework create the menu bar on the top for your activity. So inside here, what I am doing is that, in case my music is to be played, then I will set the icon to be one kind, which is with the volume button with a cross mark on it. If my music is not being played, then I will set the icon of the button to be a standard volume button, which, when the user clicks, will result in the music being played. So, this too are decided by using this code here. Now, in the on-option item selected, this is the method that is going to be called when you click on any of those icons in your menu bar. So for example, when you click on the volume button on the menu bar, it is going to result in a call by the Android framework to this method in your activities code. So when this method is called, you're going to decide which particular menu item has been selected, and then appropriately take action. So here we are saying if the settings menu item has been selected, then you're not doing anything, but if the menu item corresponding to the music playing is selected, then we're going to take action appropriately. If it says that the music is being played, then you pause the music and then set the button icon to be the volume button and if not, then you will resume playing the music and then set that icon to be the volume button with the cross mark on it. So, this is implemented in the on options item selected method of this class here. So, once we complete these updates to the shooting game to our Java file, then we can go ahead and execute this application and then see it running on our emulator or the real device. Once you deploy your application to the emulator or to the device, you will now see the music playing in the background as your application is running. So, you can turn off the music by clicking on the volume button. There notice that when I click on the volume button and the music is paused, then the volume button changes its icon and when I click on that to resume the music button, now you notice that the icon changes correspondingly, and you'll see the music being played. So, you can go ahead and then play the game just like before, but with the background music playing for you in the background. So, this completes this exercise. In the next exercise, we're going to extend this example by including additional sounds that will be generated under different circumstances. So,for example, when you shoot the Android guy and hit it, then you will hear an explosion sound being generated at that point. Week 4: Multimedia 12

13 Week Four: Lecture 3: Unit 1: Multimedia The MediaPlayer Class The MediaPlayer Class We are now going to discuss the MediaPlayer class, one of the major classes for supporting audio and video, both playback and recording in Android. The MediaPlayer class is a standard method for manipulating both audio and video. The playback is supported by using the MediaPlayer class and the recording part is supported by the allied class called the MediaRecorder class. Now whether you use the MediaPlayer or the MediaRecorder, they will automatically create a background thread for processing of the audio or video it requires the audio to be available either as a file, either in the raw folder of your application, or on the storage of your device. Or you should be streaming the audio or video from your server. So, all of these three possible approaches for supporting audio and video are available with the MediaPlayer and the MediaRecorder class. How do you use the MediaPlayer? Now before we employ the MediaPlayer, we have to of course, create a MediaPlayer object. So, this we do by calling the new MediaPlayer method a when you create the MediaPlayer, then you would supply the source of the media for the player. If you are using a file from the resource folder of your application, then you would use MediaPlayer.create () method and this create method takes a context and the ID of the media file to be played. If you are using a file that is stored on your device storage, or if you are streaming a file from a server, then you would first set the data source by giving a path for finding the source of the file, either on the device storage or from a server. So the first method that you would call is setdatasource (), and then after that you would call prepare () method or prepareasync () method. If your preparation takes a longer period of time, then you should be using a prepareasync () method, which will then call back your class to inform that the media is now ready for playback. We did not deal with the handling of the file or stream in the example earlier. We only dealt with the case where the media file is already in the raw folder of your application. To begin playback of your media, you would call the start () method of your MediaPlayer. To pause the playback at any point in time, you will call the pause () method a similarly for controlling the stop of the playback, then you would call the stop a if you don't want to use the MediaPlayer anymore, you call the release () method. At this point, the MediaPlayer object is destroyed a next time if you need to play, you need to create a new MediaPlayer object. The MediaPlayer provides rich API for playing out long playing media streams. So for example, if you have a music file or if you have a long video recording that you need to play back, then the MediaPlayer is the right class for playing out these recordings. Now the MediaPlayer supports seeking operations on your media and also can do buffering of the source. However, the MediaPlayer as a class is a heavyweight class, which means that to set Week 4: Multimedia 13

14 up and run it, it takes a lot of work to be done and which also means that it is very slow to initialize. So,, if you need to play back sounds that require very low latency, like for example, if you need sounds for a game, then the MediaPlayer is not the most appropriate method of playing back the sounds. However, sustained playback of music or audio is quite acceptable with the MediaPlayer class. And in general, if you are using a MediaPlayer within your application, you should normally not have more than one or two MediaPlayer objects opened at the same time. So with this, we will move on to the next exercise where we will examine a different class called the SoundPool class, which is better designed for supporting playing back of sounds with very low latency. Week 4: Multimedia 14

15 Week Four: Lecture 3: Unit 4: Multimedia Exercise Shooting Game with Sound Exercise Shooting Game with Sound In the next extension to our shooting game app, we're going to add additional sounds that will be generated when the Android guy is shot down by a bullet. So in this case when the Android guy is shot down and there is an explosion, we would generate an explosion sound to indicate that the Android guy has been shot down. So to do that, as an example, so when the Android guy is shot down, you can hear the explosion sound in the background--[explosion]--like that. So how do we generate sounds like this, on the fly, when some event happens? That's the next part of the extension to the shooting game. To implement this, we're going to make use of another class that is provided by Android framework, called as the SoundPool class. As usual, go to the course site, download the zip file, unzip, and then import the project into Android Studio. Once you do that, you will notice that we have a new class included in this project called as SoundEffects. Go ahead and open the SoundEffects.java file, and also the ShootingGame.java, and DrawView.java files. So this is where we're going to do the modifications. Let's first concentrate on the SoundEffects class. Now when you open this class, you will notice that I declared it as public enum SoundEffects. This is a way to create a Java Singleton class. In addition, you will include the statement called INSTANCE at the top of the class. So this way, we create a Singleton class. What this means is that there will be only one instance of this class in this application and this class can be accessible from many other classes. In addition, in this class we are going to make use of a sound class provided by Android framework called the SoundPool. The SoundPool class enables us to generate short bursts of sounds with very little latency. This is different from the MediaPlayer class that we have seen earlier. After this exercise, we will discuss the distinction between the SoundPool and the Media Player class and where each one is applicable in more detail. Now examining the code here, you'll see that I have implemented two methods--here called the setcontext and the play sound method, which contains appropriate support for using and generating sounds within our application. Now in the setcontext method, I will construct the SoundPool object here, and then initialize the SoundPool object with the sounds that I want to generate. So the way the SoundPool object is initialized depends on which version of Android you're using. If you're using Android version Lollipop or higher, then the first group of statements is the way you initialize the SoundPool object. If you're using the earlier version of Android, then the SoundPool object is initialized as shown in the statement below in the ELSE part of the IF statement. So here I am specifying that if the Android version build is greater than Lollipop, then the way the SoundPool object is initialized is, at first, we create something called Audio Attributes object a this Audio Attributes Object, we set up a few parameters here. Now you don't need to worry about what these are. This is a way to set up a few parameters for the attributes. If you would like to, you could read the documentation in more detail. Let me more concentrate on how the SoundPool object itself is built. Week 4: Multimedia 15

16 The SoundPool object is built by calling the SoundPool builder a then I set up the maximum number of streams to be five, which means that I can generate five different sounds simultaneously, at a single point of time a then the second parameter I supply the audio attributes, which are derived from the previous object that I created a then I build the SoundPool object. If I'm using the earlier Android platforms, the way SoundPool is initialized is to specify the max streams here. That's why I have the number five here a then I specify the sound volume by specifying that it's on the STREAM_MUSIC a then a few parameters here. In addition to the SoundPool, we will load in several sound samples. In this exercise, I am loading in one single sample, the explosion sound. Now I have already included the explosion sound sample into the raw folder. So in here, I say in SoundPool.load and then specify the explosion sound here a then also specify the other values a then I am using a SoundPool map in order to keep track of the sounds that I load into the SoundPool object. The reason why I'm using this HashMap is that if I need to play back sounds then it is very easy for me to specify which particular sound I want to play. So I'm taking the help of a HashMap for that purpose. So I am putting the HashMap and then specifying that the SoundPool, when it is loaded, it'll return a reference to the loaded value and that I am storing in my HashMap. So subsequently, if I need to play the explosion sound, I will be able to easily recall that from using the HashMap a in addition, I am setting up the AudioManager is used for supplying and specifying various properties related to playback of the audio. In particular, I want to be able to play the sound at the maximum volume that is allowed for the music streaming and that is specified by these two statements here. Now how do I play the sound when I need it? Now to do that, I am going to make use of the play sound method that I have supplied here. When the play sound method is called specifying which particular sound should be played back, then I will use the sample object and then ask it to play that particular sound by specifying, from the SoundPool map, the reference to that particular sound that I have already loaded in the sound explosion a I specify the volume at which I want the sound to be played on the studio output of the device. So these are the various parameters that I set up inside my SoundEffects class here. Now let's see how we actually make use of this in practice. Now before I go off, if you need to load additional sounds, then you will use the same procedure for loading in additional sounds into the SoundPool object. So when we do that, then the SoundPool will then decode this sound and then store it in memory ready for playback with very little latency. So that's the reason why we make use of the SoundPool class for this purpose. Now to make use of the SoundEffects class, in the shootinggame.java file, we are going to go in and then set the context for the shooting game for the SoundEffects class. So how do we do that? So when you get into the shooting game activity in the oncreate method, you would see this ToDo statement here. We are going to replace that with the appropriate statement to set the context for my sound effects class. So how do we do that? Week 4: Multimedia 16

17 I have already substituted the ToDo comment with the appropriate code, as you can see here. So if you have a Singleton class, this is how you access the Singleton class. So you would specify the name of the class and instance you see the use of the instance that I declared in the class there a then you call the method the set context method and then supply this, meaning that this activity as the context for this SoundEffects class here. Now we are similarly going to go and modify the drawview.java file and to generate the sound when it is needed. So if I walk into the drawview.java file, then you will notice that inside the draw game board method--i want to generate the explosion sound right at the time when the collision between the bullet and the Android guy occurs a then I'm replacing both of them with the explosion image. Right there, I want to also generate the explosion sound. So that's why you see that I have put in a ToDo statement right there to indicate to me that I need to generate the explosion sound. So how do we generate the explosion sound or, rather, how do we make use of the SoundEffects class to generate the explosion sound? I have now replaced the comment with the appropriate code in order to generate the explosion sound. So here you notice that I say SoundEffects.instance. So this is how I access the Singleton class and then call the play sound method and then supply the parameter, SoundEffects.soundexplosion. So this specifies to the SoundEffects class that I want to generate the explosion sound at this particular juncture. So this is how I would generally the explosion sound. So now that we have completed updating the code, let's go ahead and execute this application and see the result. Once your application starts running, you can now see that the application generates the explosion sound whenever we shoot down the Android guy. [EXPLOSION] This completes the exercise. Week 4: Multimedia 17

18 Week Four: Lecture 4: Unit 1: Multimedia Generating Sounds using SoundPool Generating Sounds using SoundPool In the previous example, we just saw the use of the SoundPool for generating short bursts of sound. For example, for generating the explosion sound when the Android guy is hit by a bullet. Now let's examine the SoundPool class in a little more detail to understand how it all works. As we saw earlier, the Media Player class is a heavyweight class and it requires a certain amount of time for the initialization before the sound can be produced. SoundPool class, on the other hand, is much better suited for situations where you need to produce sounds with a very low latency, as an example with a game. The SoundPool class basically allows you to specify a group of sounds, or audio resources, that will be managed by the SoundPool class. So when you specify these audio samples, then the SoundPool class uses the Media Player service to decode the compressed audio into standard raw, full 16-bit PCM mono or stereo stream a then it stores this audio information in memory. So that's the reason why the sample class is able to produce the sounds with very low latency. Now, what this also means is that when you actually ship your application, you can include your audio files into the raw folder a when the SoundPool object is created, it'll automatically decompress this audio samples into its full form and store it in memory for playback. Now the SoundPool class allows you to specify several different properties. First, includes the maximum number of sounds that it can play at the same time. In the example, we saw that we specified 5 as the maximum number of sounds to be played back. In addition, it allows you to prioritize the sounds. So, at any given point, if you are producing 10 different sounds, and some of the sounds don't make much sense because they are dominated by other sounds. Then the SoundPool class will allow you to specify saying that, if you need to play two different songs, then you can specify priority for them so that the SoundPool will discard those sounds that have lower priority and only concentrate on producing the sounds that have the higher priority at that given point of time a SoundPool allows you to loop sounds. Also, it allows you to change the playback rate. Essentially meaning, that you can change the picture of the sound. So for example, from a single sound sample you may be able to generate songs at different pitches by simply reading the frequency of the sound a also SoundPool allows you to specify the stereo volume for the different stereo channels the right and the left stereo channels. So this is where you can give some spatial configuration for the sound to be reproduced in your application. Now if you look at what SoundPool and MediaPlayer, and ask ourselves under what circumstances should you use one as opposed to the other? This table gives you a comparison between the two classes. Depending on what you are using your audio reproduction for, you should choose either SoundPool or the MediaPlayer. So for example, if you require low latency or to play is short set of sounds many, many times, then you would resort to SoundPool. If you need to play a video or audio that has an audio track or if you're streaming, audio from external source, or playing background music, then Media Player is your obvious choice for play music. Long playing audio is better handled by the Media Player class. Week one: Android Overview 18

19 Now, we move on to the fourth assignment where we are going to add more sounds to the shooting game. In the exercise, we saw that we added the explosion sound. Now if the Android guy falls of the screen or if the bullet moves off the screen at the top, we're going to produce additional sounds to indicate these events occurring in your game. Week 4: Multimedia 19

20 Week Four: Assignment: Unit 1: Multimedia Assignment Shooting Game Final Assignment Shooting Game Final So as the final step in the shooting game, exercises that we have been doing. In this assignment, you're going to extend the shooting game that we have provided for you to generate sounds under three different circumstances. First of course, we have already seen that when the android guy got shot down, we generated the explosion sound. Now if the android guy drops down below the screen, you will generate another sound to indicate that you have missed shooting down one guy a similarly if you shoot a bullet and then the bullet escapes from the top, you going to generate another sound. So, we're going to take the help of the SoundPool class that we have already created to generate three different kinds of sounds under three different circumstances. So, let me demonstrate these sounds in this app by increasing the volume a little bit. [MUSIC PLAYING] So, when the android guy drops down below the screen you hear the beep sound there. [BEEP] And similarly, if you shoot a bullet and then it escapes out from the top. [BEEP] You'll hear another sound like that for your feedback a of course, we have already implemented the explosion sound in the previous exercise. [EXPLOSION] So that remains as such. So, go ahead and then modify the provided code to generate the three different sounds. In addition, of course, we have added the score keeping up top to keep track of how many android guys you have shot down so far. So, when you complete this assignment, you will have a final working shooting game that you can enjoy playing at your leisure time. Week 4: Multimedia 20