RT Digital Media Extended Lab Choose One: Work Alone Work in a Pair Extra Work Required 3 or More Students May Collaborate, but Submissions must be Un

Transcription

1 ECEN 5033 RT Digital Media Systems Lecture 10 Extended Lab Background March 31, 2008 Sam Siewert

2 RT Digital Media Extended Lab Choose One: Work Alone Work in a Pair Extra Work Required 3 or More Students May Collaborate, but Submissions must be Unique and Individual or by Pairs Emphasis on Software Development and Understanding Challenges Related to Video Processing and Scaling Capture and Decode into PPM Frames (using CX18/open source video and ffmpeg) Frame by Frame Enchanement/Transformation Re-encoding encoding into MPEG4 for all final video files Acceleration for Optimum Credit (Compiler, Threading, Vector Processing Instructions, GPU/GP-GPU, GPU, or combinations) Form you own Collaboration Groups and Pairs Individual or Pair Submissions MUST be Unique CREDIT for any help received through Group collaboration outside individual or pairwise work must be CLEARLY NOTED in References Debug Sessions May Be Scheduled to Get Extra Help Sam Siewert 2

3 Digital Video Fair Use and Open Source Open Source Video dia/ ATSC Capture Using CX18 and Hauppauge WinTV- HVR-1600 Once CX18 Driver is installed per Home Lab Instructions, Capture from /dev/video E.g. cat /dev/video to file.mp2 using MythTV to tune to local ATSC broadcast source You may want to remove video from container using ffmpeg or your own parsing software from Lab #2 Sam Siewert 3

4 Practical Tips: Software Design and Revision Control Sam Siewert 4

5 Basic Lifecycle for RT Digital Media System Concept What is it You Want to Do Analysis: Functional Requirements, RT Requirements Planning: Schedule and Resources High Level Design (Modules, Interfaces, Dataflow) Detailed Design (Algorithms, Pseudo-Code, Prototyping) Build and Unit Testing Analysis: Risk Identification and Mitigation Re-planning: Re-schedule and Resource Adjustment System Integration and Test (Verification) System Regression Testing and Optimization (Hardware Optimizations, Scaling, Efficiency) Maintenance Sam Siewert 5

6 Spiral Model for System Development Design Detailed Function Description Analysis Write/review/finalize service functionality Finalize Design Detailed Design Completed Project HLD Define Services And interfaces Project Proposal Write/review Final System Demonstration Finalize Code Modules and Unit Tests Code, Define, Test Reference Projects CMVC, Build and Test Project Build Unit Design and Prototype(s) Completed System Integration and Test Develop and Unit Test Regression Test All Services Complete Sam Siewert And Optimized 6

7 RT Digital Media Development Linux Install with: ffmpeg VLC Development tools including SVN (subversion) for code control GPU for speed-up and to drive display Storage Considerations Single Streaming OK with Direct Attach HDD RAID for multi-stream workloads SSD for high ingest, low latency, and multi-stream IO and Transport Considerations gige or 10G UDP or RTP Video pump for IPTV QAM256 or 8VSB Modulation Card for Digital Cable or Broadcast Fiber channel or iscsi for scalable RAID storage Video Analysis and Post Production Open source approach using ffmpeg, VLC, etc. Professional packages FinalCut Pro, Adobe, etc. Professional Tools MPEG analyzer, Spectrum Analyzer, etc. Sam Siewert 7

8 Revision Control for Software Why Worry about this? Maintain a building and working version of your code Ability to roll back to previously working versions Revision history Open Source Options CVS Subversion Sam Siewert 8

9 Design Methods Sam Siewert 9

10 Hardware Detailed Design - System Rack Diagrams 1U = 1.75 Space, Power, and Cooling Requirements Scaling Considerations Number of Circuits (2 for Resiliency) Capacity in a Single Rack Computing Throughput Networking Required System Network Diagram Processing Nodes Ingest Network or Interfaces Storage Network Distribution Network or Interfaces Sam Siewert 10

11 Software Detailed Design Requirements Analysis Functional Real-Time Dataflow, formats, transport, storage, data-rates UML Methods Visio Add-on UML Templates Dr. Pavel Hruby Deployment, Component Diagrams Static Class Diagrams Sequence Diagrams or Collaboration Diagrams Activity Diagrams or State Machines Structured Analysis and Design Methods Visio Flowchart & Software Category System Data Flow Entity Relationship Diagrams (Like a Class Diagram) Flow Charts Algorithm Descriptions Pseudo-Code Prototype Code English Language Specification Sam Siewert 11

12 Deployment Diagram Shows Installation of Software Modules on HW Servo-Control- Board -Control-Output 1..* -Actuator 1..* Tilt-Pan-Servo- Assembly -Cmd-Handler 1 -Source 1 * Left-Camer VxWorks-Target -xy-location-gen * -xy-location-proc -Cmd-Gen 1 Right-Camera -Source Image-Processing * Target-Tracking * -Frame-Packetizing -Frame-Proc 1 Video-Streaming -Encoder * -Initiator 1 Left-Frame- Grabbe -Frame-Source 1..* -Target -Frame-Source Video-Frame-Driver * 1 1 Network-Transport -Initiator Right-Frame- Grabber -Target * -Stream-Source 1..* 1 -Transporter -Source 1 -Stream-Packets 1 -Encoder 1 Video-Stream- Monitor Local-Area- Network * -Stream-Termination Sam Siewert * -Monitor

13 Class Diagrams Shows Classes, Operators, Data-Types, Methods Driver-Interface +set-brightness() +set-processing-rate() +set-transport-rate() +start-driver() +shutdown-driver() -enables-access-to 1 -supplies 1 1 Frame-Grabber-Driver -executes -runs-prior-to-operation 1 -driven-by 1 1 -sets-up Device-Initializer +start-frame-acquisition() +set-mux() 1 -supports Frame-Event-ISR +interrupt-handler() Frame-Sequencer +frame-rdy-semgive() -provides-frame-services-for Sam Siewert 13 1

14 Sequence Diagrams Shows Collaboration and Function-Call, Message Passing Sequences Between Objects Makes Message Passing or Calling Sequences Obvious Shows How Object Operations Are Used Frame-ISR Frame-Buffer-Sequencer Frame-Converter Edge-Enhancer Find-Centroid Tracking-Control-Function frame-rdy-semgive() convert-to-gs:=convert-to-gs(rgb-frame) enhance-edges() find-center() center-target() Sam Siewert 14

15 Activity Diagrams - State Machines Shows the Active States of an Object Shows Logic for State Transitions Frame-Driver-Initializing when: if(framecnt > LastFrameCnt) Waiting-for-Frame-Rdy when: if(framecnt % FrameProcRate == 0) Do-Frame-Processing when: if(framecnt % FrameTransportRate == 0) Transporting Sam Siewert 15

16 Video Media (A Quick Review) Sam Siewert 16

17 Major Considerations Tools Required (Encode/Decode, Preview, Analyze) Resolution SD, HD? Algorithms Apply To Compressed or Uncompressed Video/Audio? Pixel, Frame, or Sequence Level Processing Pixel - E.g. Color Editing, Enhancement (edge sharpen), Morphing (e.g. adding graphics to video) Fame - Making Cuts Frame by frame or on GoP boundaries? Sequence Encoding or Transcoding, Multi-plexing, Transport Containerizing Number of Streams to Process? Processing Frame Rate Slower than RT, RT, Faster? Hours of Content to Ingest, Store, Distribute? Sam Siewert 17