EE412/CS455: Principles of Digital Audio and Video Spring 2015-2016 Course Catalog Description The course provides the student with necessary concepts and techniques regarding digital audio and video. Topics include: Introduction to multimedia systems its components and applications, digital signal processing of audio, images, and video data in multimedia systems and their formats. Introduction is also given regarding compression and communication of audio and video media. Students will also be oriented regarding the appropriate hardware platform for applications related to digital audio and video. This course is a must for Electronics Engineers as a number of electronics devices are meant for digital audio or video. Designing of such systems require knowledge of processing requirement that these media imposes on the hardware. It is also a very important applied course for CS students who want to work in industry or do research involving audio and video data. It explains how the digital audio and video signals are produced and processed based on the digital signal processing concepts, compressed and uncompressed formats in which these signals exist, how electronic devices reproduce the real life experience, design principle and high level architecture of such devices etc. Assignments and home works will be geared towards this goal. Course website will provide for assignment handouts, additional reading material, web links and other information related to the course. This course is intended to be a foundation course for the multimedia/vision/image processing concentration area both for under-graduate and graduate students of Electrical Engineering and Computer Science. Course credit can be claimed towards both programs in EE and CS as per policy. Some assignments and project for graduate students will be more advanced than for undergraduate students. COURSE OBJECTIVES The goal of this course is to arouse interest in students and to prepare necessary background to pursue further research and advanced course work in vision/ video processing and speech signal processing problems and their implementation. The goal of this course is also to facilitate students to produce better quality EE (and CS) senior projects by providing them necessary insight and experience related to techniques involving digital audio and video which is frequently used in such projects. Course Details Credit Hours 3 Core Elective Elective course for EE/CS undergraduate and graduate students Open for Student Category EE/CS Senior and Junior students; EE/CS Graduate students; Closed for Student Category Freshman, Sophomore Course Prerequisite(s)/Co-Requisite(s) Pre-requisites: EE or CS standing at following levels: Junior, Senior, Graduate; Some programming knowledge/experience in Matlab, C, C++ (like CS100 and CS200) is required which could be completed in any course. Co-requisites: None Recommended: Signal and Systems, Computer Organization and Assembly. Students with no background in Signal and Systems will have to attend a few mandatory tutorials.
Course Offering Details Lecture(s) Nbr of Lec(s) Per 2 Duration 75 min Timings and Venue Recitation (per week) Nbr of Rec (s) Per x Duration Lab (if any ) per week Nbr of Session(s) Per x Duration Tutorial (per week) Nbr of Tut(s) Per x Duration Instructor Nadeem Khan Room No. 9-309 Office Hours Email nkhan@lums.edu.pk Telephone 042-35608203 Secretary/TA TA Office Hours Course URL (if any) LMS Course Learning Outcomes EE412- CLO1: CLO2: CLO3: CLO4: The students should be able to: Use state-of-the-art multimedia signal processing tools to investigate human psychophysical principles and DSP techniques Analyze performance of different (hardware and software) resources, modules, formats, standards and techniques for representing and processing digital audio and video information in multimedia applications Analyze and evaluate (the working and performance) of a audio/video system design on module level Design and Implement an audio-visual application using state-of-the-art multimedia signal processing tools Relation to EE Program Outcomes EE-412 Related PLOs Levels of Learning Teaching Methods CLO Attainment checked in CLOs CLO1 PLO5 Psy-4 Instruction, Tutorial, Assignments Assignment1, Lab1, Lab2 CLO2 PLO2 Cog-4 Instruction, Tutorial, Assignments Midterm, Final CLO3 PLO2 Cog-4 Instruction, Tutorial, Assignments Midterm, Final CLO4 PLO3 Cog-5 Instruction, Tutorial, Assignments Course Project Grading Breakup and Policy Assignments/Labs: 18% Guided Term Project: 12% Quizzes: 15% Midterm Exam: 25% Final Exam: 30% Attendance in lectures is strongly recommended for understanding of the course material. Lab and Term Project related sessions will be graded and attendance in them is necessary.
COURSE OVERVIEW / Lecture/ Recommended Topics Module Readings Module 1: Intro: Multimedia World and Digital Audio and Video Related CLOS and Additional Remarks Session 1 Multimedia Data, Systems, Applications, Production and Authoring Applications in the area of Computer Vision, Multimedia Communication, Media Compression, Digital Libraries and Searching Luther: Ch 12 Luther: Ch 1 Orient students about modern multimedia world and present relevance of this course to Multimedia world Analog vs Digital Signals and Technology Module 2: Audio & Video Fundamentals Session 2 Session 3-6 Audio Fundamentals: Properties of natural sound and music, human hearing, Exploitation in electronic devices, Performance testing and benchmarks Video Fundamentals: Human vision fundamentals and its exploitation in reproduction in Electronic Devices, Color Systems, Performance Testing & benchmarks Luther: Ch 2, Ch 3, (Halsall:Ch. 2, Mandal: Ch.2) Understanding of the basic facts about human audio and visual perception capabilities and its utilization in electronic capturing and reproduction Assignment 1 Module 3: Signal Capturing, Analog to Digital Conversion, Hardware platforms Review of Frequency Domain Analysis of Signals Luther: Ch 5, Ch 6 Gonzalez Chapter 4 (sections), Session 7-9 Frequency Spectrum of Analog and Digital Signals Electronic Capturing of Audio and Video Assignment 2 Analog-to-Digital Conversion of Audio, Images and Video: Process and Issues Image Processing/Video Processing Hardware Module 4: Analog and Digital Signal Formats and Standards Session 10-11 Analog Standards (Short review) Luther: Ch 2, Ch 3 (sections)
Digital Formats and Standards (for raw digital data) CLO1 Module 5: Techniques of Audio and Video Signal Processing Session 12-14 Review of Basic concepts of Filtering and Convolution 1-D Filters Luther: Ch 8 Gonzalez: Ch 2, 3,4 Lab sessions on Audio Processing Assignment of Projects Session 15 Midterm (Spatial) Image Filtering - Signal Interpolation, Decimation Luther: Ch 8 Gonzalez: Ch 2, 3,4 - Processing Examples Session 16-23 - Signal Processing Hardware/ Software Requirements Lab sessions on Video Processing Project Proposal Presentation Design Presentation Module 6: Techniques of Media (Audio/Video) Compressions, Coding and Standards Project Proposal and Design Presentation Data Compression Basics: Lossless and Lossy Techniques Audio Compression Techniques and Standards Luther: Ch 9 Halsall: Ch 4 Session 24-28 Video Compression Techniques and Standards (Introduction to Transform Coding) Error Detection and Correction Project Implementation work and Intermediate review; Project Final Demonstration and Evaluation, CLO4 Implementation work and Intermediate review Final Demonstration and Evaluation
Textbook(s)/Supplementary Readings Reading Package is being prepared and will be provided. Supplementary Readings: Digital Image Processing/2E, R.C. Gonzalez, R.E. Woods, Addison-Wesley, 2008 Principles of Digital Audio and Video by Arch C. Luther, Artech House Inc., Latest Addition Multimedia Signals and Systems by Mrinal Kr. Mandal, Kluwer Academic Publishers, Latest Addition Multimedia Communications Applications, Networks, Protocols and Standards, Fred Halsall, Pearson Education (Singapore) Pte. Ltd., Latest Addition Fundamentals of Multimedia, Li and Drew, Pearson Education, Inc (USA), 2004-2006 Introduction to Multimedia Systems, Edited by G. Bhatnager, S. Mehta, S. Mitra, Elsevier, Latest Addition Examination Detail Midterm Exam Yes/No: Yes Combine Separate: Combine Duration: 90 minutes Preferred Date: TBA Exam Specifications: TBA Final Exam Yes/No: Yes Combine Separate: Combine Duration: 180 minutes Exam Specifications: TBA Prepared by: Nadeem Ahmad Khan Date: 10 November 2015