Chapter 5 Sound Learning Objectives This lesson looks at audio that can be used in multimedia. At the end of the lesson, the students will be able to: Use sound in a multimedia project. Use MIDI and understand its attributes, especially relative to digitized audio. Calculate sampling sizes and considerations for digitized sound. Record, process, and edit digital audio. Determine which audio file formats are best for use in multimedia projects. Manage audio files and integrate them into multimedia projects. Estimated Time for Module: 2 hours Many people, including some multimedia creators, think of multimedia as primarily a visual medium. Although visuals certainly are important, well-chosen and implemented sound can enhance your multimedia project as well. Video editors and producers, however, must note that audiences are generally more accepting of poor video quality than poor audio quality. This lesson introduces the class to audio that can be used in multimedia projects. Preparing for Class Ideally, this class should be conducted in a computer lab with multimedia authoring and sound editing software installed. Try to have some sample sound files, both MIDI and digital audio, available for the students to play and listen to. Prerequisites for Class Before coming to class, the students should: Have read Chapter 5 in their textbooks. 000-2 5-1
Be familiar with all previous lessons. Class Preparation Notes PowerPoint slides to accompany the lesson plan can be found on the CD. You will also find the figures from Chapter 5 in Multimedia: Making It Work as JPEGs on the CD. General Teaching Tips This lesson provides an introduction to sound on computers. To achieve the primary objectives: Although every exercise provides useful hands-on experiences with sound on computers, some may be excessive for your needs. Consider shortening coverage of less important objectives, as necessary, to ensure that material useful in meeting your own curriculum goals is dealt with sufficiently. Try to have some sample audio that you can play for the students. This will be the most effective way of comparing MIDI with digital audio. Encourage the use of formats that are platform-independent (such as MP3) when describing audio formats. Emphasize the amount of space that sound files can use, and cover ways to minimize that usage. Key Terms acoustics AIFC alaw.au Audio Interchange File Format (AIFF) audio resolution 000-2 5-2
buffer CD-quality CD-ROM/XA (extended architecture) Decibels (db) digital audio Digital audio tape (DAT) digital equalization (EQ) digital signal processing (DSP) downsampling General MIDI Linear Pulse Code Modulation MACE compression MIDI keyboard.mp3 Musical Instrument Digital Interface (MIDI) normalization postproduction, post session quantization QuickTime Red Book Red Book Audio resampling sample sample size 000-2 5-3
sampling rate SDII sequencer SND sound synthesizer streaming streaming latency time stretching ulaw wave format (WAV) Lecture Outline Sound is one of the most important, but easily overlooked, aspects of any multimedia project. Even the decision to have no sound has significant implications. This lesson introduces the class to sound on computers and provides tips and information that the students will need when choosing audio for their own projects. I. The Power of Sound A. Understanding sound. 1. Sound travels as waves through the air at about 750 miles per hour or Mach 1 at sea level. 2. Acoustics is the branch of physics that studies sound. 3. Sound pressure levels are measured in decibels (db). a) Decibels are measured on a logarithmic scale. (1) A fourfold increase of sound represents a rise of 6 db. Teaching Tip 000-2 5-4
Table 5-1 on page 91 of the textbook lists db levels of common sounds encountered throughout the world. 4. Too much sound pressure can have detrimental effects. a) It can result in damage to hearing. b) On the job errors can increase. c) In residential areas, sound above 45 db is likely to disturb neighbors. B. Multimedia system sounds. 1. Most target computers already have system sounds ready to use. Exercise Instruct the students to open the sound control panel on their computers. In Windows, click Start Settings Control Panel and double-click Sounds, Sounds and Multimedia, or Sounds and Audio Devices (depending on the Windows version). On a Macintosh, click Apple Menu Control Panels Sound. Have the students determine which sounds are available and have them play several of the sounds. 2. Windows and Macintoshes also have built-in sound recording tools. II. MIDI Versus Digital Audio A. Understanding computer audio. 1. Musical Instrument Digital Interface (MIDI). a) MIDI was developed in the 1980s as a communications standard for electronic music instruments. b) MIDI is not digitized sound; it is music playback instructions in numeric form. (1) MIDI offers a low cost opportunity for developers who want to compose their own music. Teaching Tip 000-2 5-5
MIDI can be thought of as a sort of virtual sheet music for electronic instruments or perhaps a modern equivalent of the paper music rolls that were once fed into playerstyle pianos. c) MIDI files can be extremely small compared to digitally recorded audio. (1) A MIDI might be 200 1000 times smaller than CD quality digital audio files. (2) MIDIs use less RAM and system resources. (3) MIDIs are small enough to be easily embedded into Web pages. d) The sound of the playback is greatly affected by the hardware that MIDI is played on. (1) Incorrect hardware can greatly diminish playback quality. Teaching Tip Four MIDI files can be found on the Instructor Pack CD-ROM in the Chapter 5 folder. Consider playing these files for the class, preferably on several different systems with varying audio hardware. This will illustrate how hardware affects MIDI playback. Two AIFF audio files containing recordings of children singing can also be found in the Chapter 5 folder. e) The length of MIDIs can be changed without affecting the music pitch. f) MIDI cannot be effectively used to play back spoken words. g) MIDI requires knowledge of music theory and principles to create it effectively. 2. Digital audio. a) Digital audio is an actual representation of the original audio. b) Audio is digitally sampled thousands of times per second. (1) CD quality audio is sampled 44,100 times per second, thus its sampling rate is 44.1 KHz. 000-2 5-6
(2) Other sampling rates commonly used in multimedia are 22.05 KHz and 11.025 KHz. (3) Sample sizes are either 8-bit or 16-bit. c) Digital audio uses more storage space than MIDI. d) Digital audio is less affected by hardware than MIDI. e) Digital audio requires less musical talent to record. Discussion Point In what situations would MIDI be preferable over digital audio? When would digital audio be better? Exercise Find an audio clip well known to your students. If audio editing software is available, have the students open the audio file and resample that sound at various rates and depths. Play the sounds back to the entire class on good quality speakers and let them hear the difference. Have the students note file sizes, too, so they can get an idea of the trade-offs. If you have time, compress these sounds using different codecs like MP3-level2 and discuss the importance of compression for Internet delivery. B. Preparing digital audio files. 1. File size versus quality. a) The higher the sound quality, the larger the file. b) Stereo recordings sound much better, but are about twice as large. c) Simple formulas can help you calculate file size in bytes of a digital recording. (1) For a monophonic recording: sampling rate duration of recording in seconds (bit resolution / 8) 1. (2) For stereo recording: sampling rate duration of recording in seconds (bit resolution / 8) 2. 000-2 5-7
Teaching Tip Table 5-2 on page 95 of the textbook lists common sampling rates and the typical file size for audio recorded at each respective rate. 2. Set proper recording levels. a) Watch the meters on your recording device or software carefully to ensure the levels aren t too high or too low. (1) Levels that are too high create distortion. (2) Levels that are too low cause the music to be too soft and overwhelmed by residual noise. 3. Editing digital recordings. a) Editing of recorded audio is almost always necessary. b) Sound editing software tools usually allow you to: (1) Trim blank space or other material off the beginning and end of the recording. (2) Splice and assemble the audio. (3) Adjust the volume. (4) Convert to another audio format. (5) Resample or downsample for lower bit rate projects. (6) Fade-in and fade-out the beginning and end of audio. (7) Use a digital equalizer to modify the recording s characteristics. (8) Control equalization. (9) Stretch the playing time without changing pitch. (10) Conduct digital sound processing (DSP) on the audio. (11) Reverse portions of sound. C. Making MIDI audio. 000-2 5-8
1. To make MIDI audio, you need: a) Some musical talent. (1) If you do not have musical skills, MIDI artists can be hired at an affordable rate. b) MIDI sequencer software. (1) This software allows you to record, edit, and save music generated from a MIDI keyboard or instrument. (2) Input is usually provided through a MIDI keyboard or other input device. c) A sound synthesizer. (1) Typically built into the sound board on PCs, but an add-on board or peripheral for the Macintosh. 2. MIDI can synthesize the sounds of over a hundred different instruments. a) Of course, how realistic these instruments sound is greatly affected by the hardware that these are played on. b) Each instrument is identified by a numeric General MIDI Instrument Sound code. Teaching Tip Table 5-3 on pages 102 and 103 of the textbook lists General MIDI Instrument Sound codes. 3. MIDI can be finely tuned and edited as you work on your project. III. Audio File Formats A. Understanding audio formats. 1. The most common audio format in Windows is waveform audio file (WAV). 2. The most common audio formats on Macintoshes are the following: 000-2 5-9
a) Audio Interchange File Format (AIFF) Audio. b) Digitized sound (.SND). (1) SND audio usually resides in the resource fork or as a standalone file. Teaching Tip Remember, in the Macintosh OS, most files have two forks. The data fork contains most of the data of the file, and the resource fork contains information such as the application that should open the file, the icon it should have on screen, and even sounds that should be associated with the file. The Macintosh user does not see separate resource and data forks, only a single icon for the file. 3. Both Macintosh and Windows can use MIDIs. a) Actually, most audio formats are now playable on each platform. B. Sound for the World Wide Web. 1. Many audio formats are now found online. a) Any sound with an assigned Multipurpose Internet Mail Extensions (MIME) type is likely to be found online. Teaching Tip Table 5-4 on page 104 of the textbook lists common audio formats that are found on the Internet and on computers throughout the world. 2. Sound can be played over the Web in one of two ways. a) You can wait for the audio file to download and then play it. b) You can stream the file. (1) A portion of the file is loaded into a buffer, and then it plays as it downloads. (2) Streaming quality is very sensitive to connection speed and Internet traffic. 000-2 5-10
Insider Information Streaming content providers such as online news broadcasts and radio simulcasts would have the world believe that streaming media is just like listening to the radio or watching TV over the Internet. Unfortunately, it doesn t always work well because streaming content is fundamentally at odds with the infrastructure of the online world. The Internet and its transmission protocols were designed around a system of data packets. Files, Web pages, e-mail messages, and other online content are broken into small data packets before they are sent over the Internet. Packets may arrive separately and by different paths before being reassembled on the destination machine. This system sacrifices speed for reliability. Streaming media, on the other hand, demands an uninterrupted path over which to broadcast a continuous stream of data. Streaming media still works in most cases, but even the best stream can be interrupted if Internet traffic is high and packets are delayed. IV. Adding Sound to Your Multimedia Project A. Adding sound to a project. 1. Before you add sound to a project, you must decide: a) What kind of sound is needed? b) When and where to use audio? (1) Decide whether each occurrence should be MIDI or digital audio. c) Acquire source material. Discussion Point How does a person go about acquiring source material? Have the students suggest ways to obtain audio for their projects. d) Edit the audio to suit your project. (1) Determine how your multimedia authoring software handles audio. e) Test the sounds to be sure they are timed properly with the project s images. 000-2 5-11
B. Toward professional sound: The Red Book standard. 1. The recording standard for consumer CD music is ISO 10149 is called the Red Book standard. a) The sample size is 16-bit. b) The sample rate is 44.1 KHz. c) These standards are deemed to capture all the sound that is audible to a human ear. d) Software such as Toast and CD-Creator from Adaptec can translate the digital files of Red Book Audio found on consumer compact discs directly into a digital sound editing file, or decompress MP3 files into CD-Audio. 2. Space considerations. a) Red Book audio uses a lot of storage space about 1.94MB for 11 seconds of uncompressed Red Book stereo sound. b) Space can be saved by: (1) Using monaural (8-bit) audio. (2) Downsampling the audio to 22.05 KHz. (3) Choosing another format that offers higher compression. C. Production tips. Discussion Point Read or have a student read Vaughan s Law of Multimedia Minimums on page 112 of the textbook and discuss what this means. Have the students describe things in everyday life that have evolved or been developed to the point where they can adequately get the job done. Consider home stereo technology, for example. By the early 1970s, home stereo systems were widely available and offered such good audio quality that most human ears were quite satisfied. Indeed, home stereos hardly changed until the home theater boom of the 1990s, when the home stereo started to 000-2 5-12
take on a new role as part of a complete home entertainment system rather than just a device for playing music. 1. Audio recording. a) Perform initial recordings to a cheap medium, such as tape. (1) This prevents you from filling up your hard drive with unusable recordings. (2) Digital Audio Tape (DAT) may provide high recording fidelity, but it may actually record too much ambient noise. b) If the project requires CD quality sound, hire a sound studio. c) High quality audio can often be recorded from videotapes. 2. Keeping track of your sounds. a) Make sure your recorder has a good counter or time code generator. (1) Make detailed notes of edit points so that those edits can easily be reproduced later. b) Keep your material organized and backed up. (1) Name files logically and descriptively. 3. Testing and evaluation. a) Some problems are virtually impossible to predict, so testing is crucial. b) When testing for audio, pay special attention to timing. c) Test in a realistic environment, on target hardware. (1) This includes testing the sound storage system, such as a CD- ROM or Web page. Discussion Point Have the students describe in their own words why testing on target hardware is important. 4. Copyright issues. 000-2 5-13
a) Rights must be secured for any copyrighted material. b) Some vendors now offer collections of unlimited-use, royalty-free sound. Teaching Tip Remind the students that even though a particular composition, such as a Bach Suite, may be old and in the public domain, individual performances of those suites are copyrighted. When a music sharing Web service removed copyrighted hit songs from their server, performances by Yo Yo Ma playing the Bach Suites were also removed. Project You have been assigned a corporate presentation project to support some internal training for the European division of a global firm. As a result, you have been asked to capture three audio clips (10 seconds each) to provide a European flavor to the project. Each clip should contain spoken or sung non-english words. MikesRadioWorld.com is one source of live radio Web links from around the globe. Project Solution Necessary tools may include mixers, sound recorders, and media players (depending on the operating system). Each student will need a computer with speakers or earphones. The students will need to be told how to use a sound recorder, with the mixer set appropriately, to record audio that they are listening to with whatever player on that system. An example station source from Spain is RNE-Radio 1, which is a news station. Allow approximately one hour for this project. Chapter Review This lesson looked at audio that can be used in multimedia. Objectives included: Using sound in a multimedia project. Using MIDI and understand its attributes, especially relative to digitized audio. Calculating sampling sizes and considerations for digitized sound. 000-2 5-14
Recording, processing, and editing digital audio. Determining which audio file formats are best used in multimedia projects. Managing audio files and integrate them into multimedia projects. Assessment Quiz Produce a handout and have the students answer the following questions. Questions 1. What is the primary difference between digital audio and MIDI? 2. Which type of audio is best suited to being embedded in Web pages? 3. Which type of audio is best suited for spoken word audio? 4. What is the sampling rate for CD quality audio? 5. What does a sampling rate of 44.1 KHz indicate? 6. List the basic sound editing operations that most multimedia producers need. 7. What is the most common file format for digitized sounds in Windows? 8. What are the most common file formats for digitized sounds on a Mac? 9. What are the two methods for using audio on the Web? 10. What is the most common potential audio problem you should look for during testing? 11. Fill in the blank: Sound pressure is measured in. 12. True or false? Both Macintoshes and Windows PCs already have sound files installed on them that a multimedia designer can utilize. 13. True or false? MIDI files can be as much as 1,000 times smaller than digital audio files. 14. True or false? The size of digital audio files is only slightly affected by changes in quality. 000-2 5-15
15. True or false? MIDI requires some musical talent and skill to create. Answers 1. Digital audio is a recording, MIDI is a score. The first depends on the capabilities of your sound system, the other on the quality of your musical instruments and the capabilities of your sound system. 2. MIDI audio is best suited to being embedded in Web pages. 3. Digital audio is best suited for spoken word audio. 4. The sampling rate for CD quality audio is 44.1 KHz. 5. A sampling rate of 44.1 KHz indicates that the audio sampled 44,100 times per second. 6. The basic sound editing operations that most multimedia producers need include trimming, splicing, volume adjustment, format conversion, downsampling, fadeins and fade-outs, equalization, time stretching, digital signal processing, and sound reversing. 7. The most common file format for digitized sounds in Windows is waveform audio file (WAV). 8. The most common file formats for digitized sounds on a Macintosh are AIFF and SND. 9. The two methods for using audio on the Web are to download the whole file and then play it or to stream the audio to a buffer and play it as it streams in. 10. The most common potential audio problem you should look for during testing is timing problems. 11. Sound pressure is measured in decibels. 12. True. Both Macintoshes and Windows PCs already have sound files installed on them that a multimedia designer can utilize. 13. True. MIDI files can be as much as 1,000 times smaller than digital audio files. 000-2 5-16
14. False. The size of digital audio files is significantly affected by changes in quality. 15. True. MIDI requires some musical talent and skill to create. 000-2 5-17