Interoperable Master Format (IMF) MARCH 10, 2015 SMPTE Standards Update Webcast Series Quarterly Series of 1-hour online, interactive webcasts covering advances in SMPTE Standards development Free of charge Sessions are recorded for member viewing convenience. 2015 Powered by SMPTE Professional Development Academy www.smpte.org 2 1
Your Host Joel E. Welch Director of Education SMPTE 2015 Powered by SMPTE Professional Development Academy www.smpte.org 3 Today s Guest Speaker Annie Chang Vice President of Post- Production Technology Walt Disney Studios Chair, 35PM50 2015 Powered by SMPTE Professional Development Academy www.smpte.org 4 2
Agenda General IMF Background Q&A What the heck is that? Extended There s more? Beyond Q&A 5 What is IMF? Interoperable Master Format Many IMF standards have been published via SMPTE since March 2014! For all post-theatrical masters (HD, 709, etc.) Loosely based on DCinema standards, but with a Broadcast twist Added functionality needed by television systems Interoperability through constrained standards Single, interchangeable master file format Need to store many versions without keeping long play versions True file-based workflow (not just a file equivalent of a tape) 6 3
<?xml version="1.0" encoding="utf-8"?> <CompositionPlaylist ae18-8a51a7dff0f0</id> <Annotation>IMF sample OV Creator> <<ContentTitle>IMF sample OV YUV HPA 2012</ContentTitle> <ContentKind>Test</Content Kind> <?xml version="1.0" encoding="utf-8"?> <CompositionPlaylist xmlns="http://www.smptera.org/schemas/2065-3/xxxx" xmlns:xsi="http://www.w3.org/20 01/XMLSchema-instance"> <Id>urn:uuid:0c265b6a-1123-4649-ae18-8a51a7dff0f0</Id> <Annotation>IMF sample OV YUV HPA 2012</Annotation> <IssueDate>2012-01-19T11:18:12-00:00</IssueDate> <Issuer>DVS Digital Video Systems GmbH</Issuer> <Creator>Clipster 4.7.0.0 Alpha/horst/IMF_HPA2012</Crea tor> <ContentOriginator>Disney</Co ntentoriginator> <ContentTitle>IMF sample OV YUV HPA 2012</ContentTitle> <ContentKind>Test</ContentKin d> <?xml version="1.0" encoding="utf- 8"?> <CompositionPlaylist xmlns="http://www.smptera.org/schemas/2065-3/xxxx" xmlns:xsi="http://www.w3.org/2001/ XMLSchema-instance"> <Id>urn:uuid:0c265b6a-1123-4649- ae18-8a51a7dff0f0</id> <Annotation>IMF sample OV YUV HPA 2012</Annotation> <IssueDate>2012-01-19T11:18:12-00:00</IssueDate> <Issuer>DVS Digital Video Systems GmbH</Issuer> <Creator>Clipster 4.7.0.0 Alpha/horst/IMF_HPA2012</Creator > <ContentOriginator>Disney</Conte ntoriginator> <ContentTitle>IMF sample OV YUV HPA 2012</ContentTitle> <ContentKind>Test</ContentKind> <?xml version="1.0" encoding="utf- 8"?> <CompositionPlaylist xmlns="http://www.smptera.org/schemas/2065-3/xxxx" xmlns:xsi="http://www.w3.org/2001/ XMLSchema-instance"> <Id>urn:uuid:0c265b6a-1123-4649- ae18-8a51a7dff0f0</id> <Annotation>IMF sample OV YUV HPA 2012</Annotation> <IssueDate>2012-01-19T11:18:12-00:00</IssueDate> <Issuer>DVS Digital Video Systems GmbH</Issuer> <Creator>Clipster 4.7.0.0 Alpha/horst/IMF_HPA2012</Creator > <ContentOriginator>Disney</Conte ntoriginator> <ContentTitle>IMF sample OV YUV HPA 2012</ContentTitle> <ContentKind>Test</ContentKind> <?xml version="1.0" encoding="utf- 8"?> <CompositionPlaylist xmlns="http://www.smptera.org/schemas/2065-3/xxxx" xmlns:xsi="http://www.w3.org/2001/ XMLSchema-instance"> <Id>urn:uuid:0c265b6a-1123-4649- ae18-8a51a7dff0f0</id> <Annotation>IMF sample OV YUV HPA 2012</Annotation> <IssueDate>2012-01-19T11:18:12-00:00</IssueDate> <Issuer>DVS Digital Video Systems GmbH</Issuer> <Creator>Clipster 4.7.0.0 Alpha/horst/IMF_HPA2012</Creator > <ContentOriginator>Disney</Conte ntoriginator> <ContentTitle>IMF sample OV YUV HPA 2012</ContentTitle> <ContentKind>Test</ContentKind> SMPTE Standards Update Webcast: Evolution of Being File-Based File XML 7 Summary of IMF Structures Composition Play List Like an Edit Decision List or recipe Points to various Track Files to synchronize and create the proper playback of content Track Files Image, Audio, Subs/CC, Metadata Wrapped in MXF - AS02 Subset (includes metadata) Output Profile List Transcoding/conversion instructions Asset Map Packing List Composition Play List Output Profile List Image Track File Audio Track File Data Essence Track File Metadata Track File Composition Asset Map Packing List Package 8 4
Macro Pipeline SMPTE Standards Update Webcast: Output Profile List An Output Profile List allows the creation of output deliverable for one or more downstream distribution channels over the timeline of a single Composition Preset OPL Simple OPL that plays composition as is Complex OPL manipulates the composition Image scaling or cropping Routing audio Encoding 9 Output Profile List Macro: parameterized processing step Macro Pipeline: ordered sequence of Macro instances Each Macro fully executed in turn Processing can always be parallelized Audio Routing and Mixing Audio Encoding Video Encoder Multiplexer 10 5
Audio Routing and Mixing Macro Extracts source audio channels from Virtual Tracks and combine them into output audio channels OutputChannel L M&E Dialog (en-us) Dialog (fr-ca) Audio Routing and Mixing Macro M&E [L] + Dialog (en-us) [L] OutputChannel R M&E [R] + Dialog (en-us) [R] 11 Image Macros Overall Flow For sub-sampled format, simply upsamples without filtering. Includes adding MXFspecific Metadata, e.g. Display Rectangle Pixel Decoder Image Macro #1 Image Macro #2 Pixel Encoder MXF Picture e.g. from 709 Full Range RGB picture Current Published Image Macros: Pixel Decoder Image Crop Image Scale Pixel Encoder e.g. to YCrCb 601 picture 12 6
Versioning Examples - Title Main Video End Credits Audio & CC Simple OPL Play 5.1 & 2.0 Audio CC IMP Versioning Examples with Subs Title Main Video End Credits 5.1 & 2.0 Audio CC Subtitles Audio & Subs 7
Versioning Examples with Subs IMP Subtitles Audio & Subs Simple OPL Play Audio/ Subs Supplemental Package Versioning Examples Dub Title Main Video End Credits Dub Cards 5.1 & 2.0 Audio 5.1 & 2.0 Audio Titles & Dub CC 8
Versioning Examples Dub Title End Credits Dub Cards 5.1 & 2.0 Audio Titles & Dub Simple OPL Play IMP Dub Supplemental Package Versioning Examples Everything Title Title Main Video End Credits End Credits Dub Cards Audio & CC Simple OPL Play 5.1 & 2.0 Audio 5.1 & 2.0 Audio CC Subtitles Titles & Dub Audio & Subs Complex OPL SD MP4 LC & AAC Full IMP 9
Versioning Examples Airline Edit Title Main Main Video Video Edit Main Video End Credits Airline Version Supplemental IMP Basic Level 5.1 & 2.0 Audio Edit 5.1 & 2.0 Audio Airline Audio Versioning Examples Airline Edit Title Main Main Video Video Edit Main Video End Credits Airline Version Supplemental IMP Basic Level 5.1 & 2.0 Audio Edit 5.1 & 2.0 Audio Airline Audio 10
Versioning Examples Airline Edit IMP Airline Version Supplemental IMP Basic Level X- Fade Airline Audio Simple OPL Play Airline Edit Supplemental Package Example File Savings Three Languages Sofia the First (30 minute episodic) Episodic IMF Example: Total for 3 Languages: ~46GB ProRes-HQ QT: Total: ~130 GB 3 x 43 GB each Frozen (90 minute feature) Feature IMF Example: Total for 3 Languages: ~95 GB ProRes-HQ QT: Total: ~405 GB 3 x 135 GB each Nerd Note: examples use IMF, JPEG2000, 4:2:2 Y C B C R which would be the equivalent to ProRes-HQ 22 11
Modular Approach Core Framework Composition Play Lists MXF Wrapping Major Constraints General Image, Audio and Subtitling Parameters Core Framework Modular Applications Plug-in to framework Allow for specific functionality Higher compressed CODECs Specific resolutions or frame rates Allows for different problems to be solved Interchange within an Application Application #2 JPEG2000 HD Application #3 MPEG-4 SStP 4K Application #n Your App Here 23 IMF Suite of Standards Published documents (Available at https://www.smpte.org/standards) 2067-2 Core Constraints 2067-3 Composition Play List 2067-5 Essence Component (MXF Wrapping) 2067-8 Common Audio Labels 2067-20 IMF : JPEG2000 HD & SD 2067-21 IMF Extended: JPEG2000 UHD 4:2:2 2067-30 IMF Application #3: MPEG-4 SStP 4K & HD 2067-100:2014 IMF Output Profile List 2067-101:2014 IMF Output Profile List Common Image Definitions and Macros 2067-102:2014 IMF Output Profile List - Common Image Pixel Color Schemes 2067-103:2014 IMF Output Profile List Common Audio Definition and Macros 24 12
IMF at SMPTE TC-35PM Technology Committee 35PM50 IMF Working Group Output Profile List Drafting Group Audio Drafting Group Sample Material Interchange Adhoc Group Forum for tracking bugs and resolutions Plugfests Content owners provide various source files Multiple manufacturers attend with their products (creation, encoding, QC tools) Files are created and interchanged Provides a forum for discussion on standards and real time feedback Bugs found are incorporated as fixes in standards documents Next IMF Plugfest March 27 th, Los Angeles, CA USA 25 Current Projects 1 Year Revisions ST 2067-2 IMF Core Constraints ST 2067-3 IMF Composition Play List ST 2067-5 IMF Essence Component ST 2067-20 IMF ST 2067-21 IMF Extended IMF Application #4 Mezzanine Film Format Archive format with JPEG2000 26 13
Cool Things About IMF (that you might not know) Content Version EIDR and/or your own asset identifier Get MXF Metadata into IMF s can reflect the exact metadata contained in the MXF files (resolution, color space, etc...) Open source toolkit (https://github.com/sandflow/regxmllib) converts MXF metadata into XML (based on RegXML) Uses the published SMPTE metadata registers and is automatically updated as new metadata is added to IMF Toolkit generates XML Schemas that can be used to validate IMF s Multi-Channel Audio Labeling Markers F F F F F L F F FHS F F F FEC TXM B D C C T L B B 27 Questions so far? 28 14
IMF Core Audio Essence: Channels Subtitling/CC: Wrapping: Playlist: 24-bit PCM Any number of channels, configurations SMPTE-TT (ST2052) AS-02-constrained MXF Composition Play List (XML) 29 IMF In addition to the IMF Core: Image: JPEG2000 Broadcast Profiles Compression: Lossless, Lossy Resolution: SD, HD Colorimetry: BT.601 (SD), BT.709 (HD) Bit Depth: 8-bit, 10-bit Color Components: R G B, Y C B C R Frame Rates: 23.976, 24, 25, 29.97, 30, 50, 59.94, 60 Stereoscopy: Mono, Stereo Based upon ISO/IEC 15444-1:2004/Amd 3:2010 (JPEG2000 Broadcast Profiles) 30 15
IMF Extended Includes all of plus: Image: JPEG2000 Broadcast Profiles Resolution: UHD Colorimetry: BT.2020, xvycc Bit Depth: 8-bit, 10-bit Color Components: Y C B C R Frame Rates: 23.976, 24, 25, 29.97, 30 Based upon ISO/IEC 15444-1:2004/Amd 3:2010 (JPEG2000 Broadcast Profiles) 31 But What About True 4K HDR The Future RGB 12-bit The Kitchen Sink 8K 16-bit 32 16
Roadmap today time Extended Extended+ Extended++ Mezzanine Film Format SD, HD, 10-bit Extended Adds UHD resolutions in YCbCr Extended+ 2015 Adds true 4096 resolutions in RGB, and 12-bit, 12-bit HDR (ST 2084, 2086) Extended++ Adds HDR support (ST 2084, 2086) Mezzanine Film Format Supports up to 8K, XYZ, 16-bit 33 Roadmap today time Extended Extended 2015 Application #4 Mezzanine Film Format SD, HD, 10-bit Extended Adds UHD resolutions in YCbCr Extended 2015 Adds true 4096 resolutions in RGB, 12-bit, HDR (ST 2084, 2086) ISO added new JPEG2000 Profiles for IMF including up to 8K, HFR, etc. last year 34 17
Beyond (short term) Extended Revision HDR (ST 2084 and ST 2086) 12-bit True 4K (up to 4096 x 3112) and RGB SMPTE-TT Constraint W3C IMSC1 Interoperability! Need Tools to support IMF OPL Testing 35 Beyond (long term) New Applications List of Proponents Create a Project at SMPTE Volunteer to do the work and write the standards! Already have a framework of documents to reference Dynamic Metadata ST 2094: Dynamic Metadata for Color Transforms of HDR and WCG Images Pan-scan metadata Script and other descriptive metadata that follows timeline of program Object-based audio metadata 36 18
Beyond (long term) More Output Profile Lists Compositing / alpha channels Encoding processes 37 Questions Annie Chang Vice President of Post- Production Technology Walt Disney Studios Chair, 35PM50 annie.chang@disney.com Joel E. Welch 2015 Powered by SMPTE Professional Development Academy www.smpte.org 38 19