TKT-2431 SoC design Introduction to exercises
Assistants: Exercises Jussi Raasakka jussi.raasakka@tut.fi Otto Esko otto.esko@tut.fi In the project work, a simplified H.263 video encoder is implemented on Altera DE2 FPGA Development and Education board The projects work consists of a set of exercises After successfully finishing each exercise, one should have a working H.263 video encoder Exercises: Mon 14-16, Fri 10-12(TC417) Assistance not available in any other time All needed software is installed on the PCs of the class and can be used whenever the class is not reserved for other courses
Exercises cont. Attending the exercise hours is voluntary The following assignment is introduced Tools and algorithms are introduced Hints are given Questions are answered Completing each of the exercises is mandatory The returns have to be in time The returns have to be accepted Exercise work is carried out in groups of 1-2 students Groups of 2 persons are preferred
You must have: Access rights to TC417 Exercises cont. Fill the access rights application (link on the exercise page) and return to Timo Rintakoski (TH305) Lintula user account Fill in the user account application and return to Lintula mailbox
Exercises cont. Completed exercise work is valid for three successive exams Points from the exercise work You can gain points from some of the exercises See exercise pages for more detail Bonus point criteria will be explained during the first exercises http://www.tkt.cs.tut.fi/kurssit/2431
Exercise 1 / Part 1 Introduction to topic
Topic of the work A simplified H.263 video encoder on DE2 FPGA Education and Development board The exercise work consists of several phases and sub-tasks Receiving and understanding the system requirements Writing a system specification Software implementation of the encoder on desktop PC Functional verification on desktop PC Creating the SoC platform on FPGA Porting the SW implementation onto the SoC platform Verification and performance profiling for pure SW implementation HW/SW partitioning and hardware acceleration Verification and performance profiling for accelerated implementation Documentation
H.263 The basics of H.263 video encoding are explained during following exercises Students are encouraged to get familiar with video encoding algorithms in general before they start the project H.263 has a lot in common with algorithms like JPEG and MPEG-2 A very simplified version of H.263 video encoder (resembling motion JPEG) is used. Only INTRA coding (i.e. prediction of subsequent frames is not applied) The utilized key algorithms are DCT (Discrete Cosine Transform), Quantization, RLE (Run-Length Encoding), and VLC (Variable Length Code.
Software Altera Quartus II v7.2 System development and FPGA synthesis SOPC Builder for building Avalon/Nios II based systems Integrated Iogic analyzer for HW debugging Nios II IDE Software development environment for Nios II processor Nios2-terminal Terminal software for Nios II standard stream inteface via jtag uart (Mentor Graphics ModelSim) Simulating own VHDL blocks/designs ffplay video player tmndec H.263 decoder
Hardware Desktop PC w/ Linux OS Platform for the first encoder implementation Utilized to verify the encoded video bitstream Altera DE2 Development and Education Board Platform for the created Nios II based SoC
Exercise returns Exercises are returned as follows: Return for an exercise has to be made before the next week s sunday at 23:59 by E-mail Deadline dates are on the exercise pages Return your exercises to tkt2431@cs.tut.fi All the required documents have to be in either pdf or pure text-file format The subject for the email has the following form: SOCD_Ex<exercise_number>_G<group_number> where <exercise_number> is the number of the exercise in question and <group_number> is the number of your group. e.g. exercise 3, group number 13: SOCD_Ex3_G13
Bonus points Three main exercise returns are rated Excellent: 1 bonus point for the exam The returned document is very good and/or the returned source codes work correctly and are well done Accepted: no bonus The returned document or code is acceptable Rejected: no bonus, the return has to be corrected Use common sense: Do not return rubbish! All the exercises must be accepted Exercise points for the exam can be obtained: 1 point can be obtained from each of the exercises 2, 5, 12 Encoder achieves the given frame rate criteria (2p if > 75fps, 1p if > 50 fps) 2 most optimized encoder are awarded extra points Bonus exercise: Dual Nios II encoder implementation Up to 3 bonus points can be achieved
Exercise 1, Part 2 Introduction to algorithms
Requirements for Video Transmission Communication delay (latency) More important in video conferencing applications than in file-based streaming applications Should be as low as possible (< 250 ms, even 150 ms) Should be kept as constant as possible Avoiding burst of frames followed by a still image Buffering Frame rate Affects to perceived smoothness of motion Under 10 fps video stream is perceived as fast slide show Image resolution Directly proportional to data size of a raw image Depends on the application
Introduction to H.263 Standard May 1996, ITU-T recommendation v1 Block-based ( Macroblock size is 16 pixels by 16 lines ) Motion estimation for temporal redundancy reduction Same objects are likely to be present in adjacent frames Half pixel accurate motion vectors DCT for spatial redundancy reduction 8 x 8 blocks Adjacent pixel values have only a little difference Quantization (lossy) Control of compression ratio RLE and Huffman as entropy coding algorithms Lossless compression
Block Diagram of H.263 Encoder + pre-processing + DCT Q Entropy coding 1/2 pixel accurate (interpolation) - Mot. Comp v(u,v) Mot. Est. Prediction error computation In Intra mode, MBs are coded directly Q -1 IDCT motion vector v(u,v) Previous reconstructed pictures (same image as the decoder observes) 7 0 4 0 0 0 0 1 1 9 3 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 1 0 bits out (Huffman, VLC) No need to send zeros in 8x8 block to the decoder
Discrete Cosine Transform (DCT) Assumption: Adjacent pixels differ only a little from each other Thus, data in the frequency domain is easier to compress Spatial domain compression Pixels are grouped into blocks and the blocks are then transformed into frequency domain Essential information is then in more compact form Important DCT-coefficients in upper-left corner, that is, in low frequencies Compression is achieved by discarding the less important information of the transformed block Quantization of coefficients DCT itself is a lossless transform Limited accuracy with coefficients, however, leads to some loss of information
Entropy Encoding Next, the quantized coefficients are compressed in a lossless manner using entropy encoding Run-length coding o o o Lower amplitude coefficient likely to be zero Arrange successive quantized non-zero coefficients into combinations of (LAST, RUN, LEVEL) Last = Whether this is the final non-zero coefficient in the block RUN = Number of preceding zeros LEVEL = sign and magnitude of the non-zero coefficient Coefficients are processed in zig-zag order Due to the fact that running zeros are most likely located at higher frequencies Huffman coding (variable length coding) o After RLE coefficients are encoded based on the statistical characteristics Shorter codewords for symbols which occur with high probability
H.263 Project work A simplified version of H.263 video is created only INTRA coding (i.e. no motion estimation/compenstation) Key algorithms: DCT, quantization, RLE and VLC) Supported image resolution is QCIF (176 x 144) Encoder: pre-processing DCT Q Entropy coding 011001011 Decoder: 011001011 Entropy decoding Q -1 IDCT Reconstructed pictures
Documentation Verification Design flow Requirements Specification Performance analysis SW Implementation Performance analysis HW/SW partitioning Performance analysis Final Implementation
Specification On this week the specification of the encoder is started Required C source codes for the encoder are given Can be downloaded from course web-pages You have to write a simple specification for the video encoder system you are going to implement Specification does not have to be long The quality of the content matters 4-7 pages in total is enough (including the chapters introduced on next week) Remember: LONG!= GOOD Don t submit 31 pages of BS A diagram of the video encoding flow is required Control and data flow diagram describing how the pre-given H.263 functions are used
Specification (2) 1. Introduction What is being specified 2. Flow of encoding Present different phases of the encoding Explain the encoding flow briefly A flow diagram of encoding is required! 3. Encoder interface Inputs and outputs of the encoder What kind of data is read in? (format etc.) What is the output data like? 4.Description of algorithms Function prototypes (find from the given C codes) Description of function parameters and return values Description of function behavior and purpose in this design At least DCT, quantization, RLE, and VLC have to be covered here The subsequent sections will be written in exercise 2.
First exercise submission Don t return your unfinished specification What is being specified Instead describe the following things briefly in the submission email: Description of the encoding flow Purpose of the key algorithms in the encoding process: DCT Quantization RLE VLC Deadline is on the exercise page
Links on H.263 related material http://www.itu.int/rec/t-rec-h.263/ ITU-T specification of H.263 http://www.jaxstream.com/products/jaxspeed/wp_m4venc.pdf Basics of MPEG-4 video encoding http://www.ece.purdue.edu/~ace/jpeg-tut/jpegtut1.html JPEG tutorial