Problem Set 6 Audio Programming Out of 40 points. All parts due at 8pm on Thursday, November 29, 2012

Transcription

1 Problem Set 6 Audio Programming Out of 40 points All parts due at 8pm on Thursday, November 29, 2012 Goals Learn the basics of audio processing in C Learn how to use audio APIs Grading Metrics The code you write will be graded according to the following metrics: 1. Correctness: How well does your code adhere to the problem s specifications and does it produce the desired results? 2. Design: How clearly, efficiently, elegantly, and/or logically is your code written? 3. Style: How well is your code commented and indented, your variables aptly named, etc.? Answers in English will be graded for completeness and accuracy. 1. Getting started Go to the following URL: Download ps6_mac.tar.gz or ps6_windows.tar.gz to your local machine and unpack the file. 2. Download and install PortAudio and libsndfile This problem set focuses solely on audio programming. You have now learned enough C to be able to understand and use libraries written by others. The two APIs you will utilize for this problem set are PortAudio and libsndfile, both of which are free, crossplatform APIs. PortAudio is a popular audio input/output library (utilized in numerous applications such as Audacity); libsndfile is a library for reading and writing audio files in many formats. The first step is to make sure you have PortAudio and libsndfile installed on your machine. Go to the following websites to download, install, and learn more about the two APIs: PortAudio API main page: of 8 -

2 Libsndfile API main page with link to download: In your home directory, create a new directory called lib/. This will be where you download and unpack the PortAudio and libsndfile packages you download. In the future, if you download other APIs, this is a good place to put them. You should read the information on these websites and configure, compile, and install the APIs as instructed. You should also take a look at any files called README, INSTALL, README.txt, or README.configure.txt files that are included in the top-level directories of each API. The process outlined in these files is typical for many APIs. Windows users: note that the libsndfile install instructions will say to install a precompiled binary because Windows doesn t usually come equipped with the tools to compile the source; do not do this you should download the source and compile it yourself as instructed for UNIX and Mac OS users (this is possible because you have installed Cygwin). Mac/Linux users: If make install doesn t work for libsndfile, type sudo make install instead (when it prompts you, enter the password for your user account). Special note for compiling on MacOS 10.7 (Lion): Download the stable release pa_stable_v19_ tgz and unpack the archive. Try compiling/installing normally. If it fails (which it might depending on the version of Lion you have), continue with the instructions below. cd into the portaudio/include/ directory and open the pa_mac_core.h in a text edit. Uncomment line 49 (#include <AudioToolbox/AudioToolbox.h>). cd back up into the portaudio/ directory. Type./configure --disable-mac-universal at the prompt to configure. Open Makefile in a text editor and delete the -Werror option on the CFLAGS line (line 21). Type make at the prompt. Then type sudo make install to install portaudio. When you have installed PortAudio and libsndfile successfully, you should be able to compile the files in the ps6/sine_player/and ps6/reverb/ directories by typing make. Now read through the API documentation: PortAudio API documentation: of 8 -

3 libsndfile API documentation: Even if you don t understand everything yet, you should at least get a general sense of the functions available to you in each API. 3. PART I: Sine wave player (20 points) Your goal in Part I is to turn your (computer) keyboard into a musical instrument. This entails using the PortAudio library for real-time audio output. What you will be required to do is map letters on the keyboard to actual musical pitches. These pitches will be produced in the form of sine waves generated at particularly frequencies. These frequencies are shown in the table below. Note that they cover two octaves of the A major scale. Note name and octave Keyboard mapping Frequency (Hz) A a B s C# d D f E q F# w G# e A r, j B k C# l D ; E u F# i G# o A p The output of your musical instrument will be in stereo, where the notes from the lower octave of the A major scale will be outputted to Channel 1, and the higher octave will be outputted to Channel 2. These two octaves will be mapped to keys that group them by hand position on the keyboard: - 3 of 8 -

4 In addition, the letter key v silences Channel 1, n silences Channel 2. And (spacebar) quits the program. To get a sense for how the program works in its final state, you can run the executable provided for you in the exe/ directory. Recall that you might have to chmod a+x sineplayer in order to make it executable. Let s look at the files that are included in the sine_player/ directory. First take a look at the Makefile. You ll notice some new flags, -lportaudio and -lncurses for Mac OS X/Linux (full pathnames to the dlls are provided in the Windows Makefile). This tells the compiler to link to the PortAudio and ncurses libraries. N ote for Windows users If you are unable to compile your program using the ncurses functions (you ll get errors indicating that functions such as initscr, cbreak, endwin, etc. are undefined), you might need to install the ncurses library using the Cygwin installer. The best way to do this is to keep your current Cygwin install as is, but just add the libraries needed for ncurses. When you arrive at the screen that lets you choose what to install, make sure you select the Keep radio button. Then enter ncurses in the search window, and the Cygwin installer will show you all the ncurses files you need to install. Check all the boxes to indicate you want to install them. In the process of installing, Cygwin might ask you to install some additional libraries you haven t selected because ncurses is dependent on them; you should go ahead and let Cywin do this. If you still have problems, please come to office hours to get help. - 4 of 8 -

5 Implementing sineplayer Using parts of sine.c, mix.c and volume.c from lecture as models, finish the program started for you in sineplayer.c. You will notice that PI is defined for you at the top of the file. You ll need it to generate waveforms of a particular frequency. In addition to completing the main function (and any other helper function you deem necessary), the major item that needs to be implemented is the PortAudio callback function for realtime audio output. You can use the solution executable in exe/ to see how the finished program is supposed to work. Warning to Windows users: I found that there was some latency when I ran the Windows executable on my machine (running Windows XP), unlike the case for Mac OS. Generating sine waves For those of you who have taken a digital signal processing class in the past might recall that a sine wave has the following (simplified) definition: - 5 of 8 -

6 y(t) = asin(ωt), where a is the amplitude (0 or 1 in our case), t is time, and ω is the angular frequency which can be described in term of frequency in Hertz: ω = 2πf, where f is the frequency in Hz (e.g. 440 Hz for A4). Each step in time is dependent on the sampling rate, i.e. each step in time is of size 1/sample_rate. One way to keep track of the last value of t that was used that is, the last time the callback requested data is to declare t as a static variable: static float t = 0; This statement will initialize t to zero, but then retain whatever value it is set to in the future. Another way is to create a table with sine wave values pre-computed and stored (as in the case for sine.c from the lecture notes). 4. Part II: Reverb (20 points) Adding reverberation is a common way to create the perception of space in digital audio. Convolution is a common signal processing technique for simulating reverberation effects. In this next section, you will be implementing a program that adds reverb to audio files by convolving the audio data with pre-recorded impulse responses. Using the lecture code as a guideline, finish the program started in reverb.c. You do not need to alter the main function. You just need to implement the function processaudio, which is called by main. The prototype for processaudio is already provided for you. Note that it takes three arguments: an input audio file name, an impulse response file name, and an output file name. Your implementation of processaudio will begin by opening the first two files and reading them into buffers that you allocate. These two input audio files can be either mono or stereo, but they must be of the same sample rate. You must quit with an error if the sampling rates for the two files differ. Convolving audio After reading in the files successfully, you must then use the convolution function provided for you in convolve.c to create a new version of the input audio with added reverberation effects. The prototype of convolve can be found in convolve.h. You should not alter convolve.h or convolve.c; you should simply include convolve.h and call the convolve function whenever you need to use it in reverb.c. This is the prototype for the function: - 6 of 8 -

7 int convolve(float *x, float *h, int lenx, int lenh, float **output); Note that it has five parameters and one return value. The array of floats x is the original audio signal (mono); h is the impulse response signal; lenx is the size of x and lenh is the size of h; output is a pointer to the output buffer. The return value is the size of the new signal that output points to. You will then write the data pointed to by output to the specified output file. Note that you will have to call convolve twice, and combine the results, if the input audio is in stereo. You should not allocate memory for output yourself just pass convolve the address of a pointer to float. The convolve function allocates whatever memory is necessary and sets your pointer to it. However, that doesn t mean you don t need to allocate memory elsewhere in your program; you will need to allocate memory when reading in the audio data from the files initially. Whatever memory you do allocate must be freed before exiting the program. Make sure to close the audio files before quitting as well, otherwise the output file might not be written correctly. You can take a look at convolve (at the bottom of the convolve.c) to see exactly what it s doing. But you do not need to understand how it works; you just need to know how to use it properly. For those of you who are curious, it implements an efficient convolution algorithm that works by taking the FFT of both the input signal and impulse response, multiplying those results, then taking the inverse FFT. This method is considerably more efficient, O(n log n), than executing all of the arithmetic operations (multiplies and adds), O(n 2 ). Nonetheless, depending on your processor speed, convolution with large signals might still take a considerable amount of time. Output file format As mentioned previously, the input audio and impulse response must be of the same sampling rate, however, they may either be mono or stereo. If both are stereo, then convolve Channel 1 of the input audio with Channel 1 of the impulse response, and Channel 2 of the input audio with Channel 2 of the impulse response. If the input audio is stereo, but the impulse response is mono, convolve each channel of the input audio with the mono signal of the impulse response. If the input audio is mono, but the impulse response is stereo, average the two channels of the impulse response and then convolve it with the input audio. Finally, the file format of the output audio must be identical to the input audio (i.e. file type and number of channels), not the impulse response. Some audio files provided for you in khz mono and stereo formats in the sounds/ directory for testing purposes. Within the mono and stereo folders, you will also see a directory called IRs/. This directory contains some sample impulse responses downloaded from an impulse response library. If you are curious to experiment and try out additional impulse responses with your new program, there is a large variety of impulse responses you can listen to and download for free here: - 7 of 8 -

8 To download impulse responses, click on the description, e.g., Stairway, University of York, then on the following page select the Impulse Responses tab to access the audio files. You can use the solution executable in exe/ to see how the finished program is supposed to work. 5. Extra Credit: Create your own impulse response (5 points) Write a program called delay.c that creates an impulse response that when convolved with another audio signal, will create an exact replica of the original audio at a 3 second delay. Your program should use libsndfile to output the impulse response in mono wav format, sampled at khz, as a file called three_sec_delay.wav. An empty file delay.c is provided for you in the reverb/ directory, and the Makefile for the reverb project will compile it for you. Your program should take no arguments it should simply create three_sec_delay.wav. 6. What to submit In order to make the submission clean, type the following in all your subdirectories: make clean ***Make sure to remove the audiofiles/ directory before submitting.*** After you have removed the audio files, append your last and first names to your ps6 directory name. e.g. from the command line this will rename your directory: mv ps6 ps6_lastname_firstname Then create an archive of the newly renamed directory. You can either create a zip file or create a tar ball from the command line. To create a tar archive, first cd into the directory above ps6_lastname_firstname/ directory. Then type: tar cvzf ps6_lastname_firstname.tgz ps6_lastname_firstname You should now see a file called ps6_lastname_firstname.tgz in your directory. Submit this file via the course Blackboard page. Make sure you submit all the files you have created or modified as part of this assignment, as well as for these files (you don t need to submit delay.c unless you implemented the extra credit problem optionally you can leave the file in the submission directory unmodified). - 8 of 8 -