CinemaDNG Workflow. Using the CinemaDNG File Format in Cinema Workflows. Goals and Benefits TECHNICAL PAPER

Transcription

1 a TECHNICAL PAPER Using the CinemaDNG File Format in Cinema Workflows TABLE OF CONTENTS 1 Goals and Benefits 3 Workflow Schematic 4 Workflow Steps 8 Using Metadata with CinemaDNG 9 Reference Documentation image 1a lost highlight from poor sensor processing. Goals and Benefits When film is used as the capture medium, cinematographers take great care when handling source film footage. This is the original record for the story they are trying to tell. With the move to digital capture by many cinematographers, are there ways to optimize the quality of these original records of scenes? CinemaDNG is providing a solution that will give cinematographers, and those working with digital content in cinema workflows, ways to ensure that their digitally captured scenes can be used most effectively to achieve desired results. Adobe and other industry leaders have produced a file format specification for digitally captured content that addresses problems currently in workflows using digital capture devices. A new file format called CinemaDNG addressed many of these problems. CinemaDNG is similar to the popular DNG format used for digital still photography but adds elements to specifically support cinema workflows. Think of CinemaDNG as the optimal replacement for the original film negative. Why CinemaDNG? In many digital capture workflows content is processed by software/hardware in the camera before it is saved to a storage device. If the wrong assumptions are made during the processing of the camera s sensor data, damage could be done to your saved digital file. This damage may be irreversible or very expensive to fix in post-production. Post-processing of the camera s sensor data allows for access to the wide exposure latitude provided by the camera sensor. Qualitative judgements after the image has been saved to disk puts more power in the hands of the colorist or editor instead of the hardware/software in the camera that processes the sensor data. In the examples on the left, image data has been captured as both rendered RGB data and raw sensor data. The processed image on the left shows lost highlight detail while the raw image on the right still contains highlight information. Information on the left has been lost during the camera processing and can not be recovered. This does not mean that this image on the left could not have been captured more image 1b better highlight detail using post-capture RAW processing. effectively using different exposure settings. However, due to the increased exposure latitude of the raw image data, chances are 1

2 Are Your Fully Connected? Camera 1 Camera 2 Camera 3 Camera 1 Camera 2 Vendor 1 RAW file Vendor 2 RAW file Vendor 3 RAW file Connecting with CinemaDNG Vendor 1 RAW file greater that the operator or editor will be able to recover from this problem using raw image formats like CinemaDNG. Another advantage of the CinemaDNG file is easy integration into visual effects workflows. Because the CinemaDNG data has not been processed into common, gamma-encoded RGB or YUV formats, digital values are still related to the colors and lighting present at the scene. (These values are typically encoded using a linear tone response). Because of this, files can be processed and saved into linear-light formats like OpenEXR for delivery to visual effects designers and more easily matched with other linear-light content used for compositing. X X CinemaDNG X X DPX HD TIFF Non-linear Editor 1 Non-linear Editor 2 Visual Effects DPX HD TIFF Non-linear Editor 1 Non-linear Editor 2 If you are concerned about the storage requirements for captured footage, CinemaDNG provides good news. CinemaDNG files require half the storage space of corresponding DPX files. The RAW image workflow has already become popular with many cinema camera vendors, largely for the reasons mentioned above. However, the RAW file format used by these vendors is usually proprietary. This can cause difficulty in workflows where software tools do not understand or operate well with a wide variety of these proprietary formats. CinemaDNG is meant to solve this problem by offering camera and software vendors an opportunity to use a single RAW format that can be widely understood across workflows. The CinemaDNG file format is an open, standardsbased format relying on currently existing non-proprietary standards like TIFF/EP, DNG, XMP, EXIF, and MXF. There are no licensing fees for it s use. Camera 3 Visual Effects 2

3 On-Set Monitor (usually Rec. 709) Intermediate Color Grade Final Color Grade Mastering Formats (DCDM, DPX) or 7 Camera Calibration Camera Setup Look Preview Camera Capture Using CinemaDNG CinemaDNG Files (Hi-Res or proxies) 4 Non-Linear Editing 2 Transcode RAW formats to CinemaDNG Visual Effects 5 Camera Capture (Proprietary RAW/Data Output) Archive CAPTURE Edit Decision List (EDL) CinemaDNG metadata only CinemaDNG Non-CinemaDNG RAW & look metadata RGB formats 1 - Software is used to pre-visualize camera sensor information before scenes are shot. This allows for correct camera exposure and white point balance. In addition, software controls allow the cinematographer to set preliminary looks to scenes without baking these looks into files saved to disk during capture. Processing parameters are stored as metadata in the CinemaDNG file. 2 - Non-CinemaDNG files can be transcoded to CinemaDNG files using appropriate software. 3 - CinemaDNG files opened for intermediate color grading. Color/tone adjustments are saved in CinemaDNG files or separately using other metadata file formats. 4 - CinemaDNG files are imported into non-linear editing or VFX applications. Prior to the ingestion of these files, CinemaDNG processing software allows CinemaDNG processing metadata to be adjusted. In some cases, proxy files are created when the CinemaDNG files are imported into the non-linear editing software to facilitate faster offline editing. 5 - Compositing/VFX software uses CinemaDNG software to render camera sensor data to linear-light formats for compositing. RGB files are exported and conformed to timelines in editing or grading/finishing software. 6 - (Workflow #1) After off-line editing is complete, an EDL is generated and used by the final grading and finishing applications to acquire original hi-res CinemaDNG files and rendered VFX files. (Workflow #2) After off-line editing is complete, hi-res CinemaDNG files are re-acquired and rendered to an RGB format for delivery to grading and finishing applications. In this case, the best-light processing instructions from the camera capture are baked into the RGB files delivered to the grading application. 7 - RGB files are handed off after final grade and finishing for film printing or transcoding for digital distribution.

4 Workflow Steps Workflow #1 CinemaDNG Capture, Transcode to RGB Proxies for Non-linear Edit, Conform CinemaDNG for Delivery for Final Grading and Finishing, and Render CinemaDNG to Formats for Final Film Printing or Digital Distribution. In this workflow, cinematographers use cameras that are capable of writing CinemaDNG files (or files are transcoded to CinemaDNG from other RAW formats). These CinemaDNG files may be frame sequences or data streams wrapped in the MXF file format. There may be cases were image adjustment is redefined after the CinemaDNG files have been written to disk. Examples of these kinds of adjustments are exposure adjustment and white point color balance. These adjustments can be saved in separate metadata files or in the CinemaDNG file itself. The CinemaDNG files are then ingested into non-linear editing applications. Proxies of original CinemaDNG files are created during ingestion to allow better performance in the non-linear editor. (In some cases, proxies may not be necessary if editing applications can use hi-resolution CinemaDNG data). After editorial work is completed in the non-linear editor, an edit decision list is created and delivered to grading/finishing software. CinemaDNG files are then conformed to the original timeline in the grading software and color edits are performed using the original CinemaDNG files. Previously defined image processing instructions, defined in the CinemaDNG metadata, are available for color correction in the final grading software. After final grading and finishing is complete, files are rendered to appropriate file formats for film printing, digital distribution, or further transcoding. Step 1 Camera Setup Using CinemaDNG Controls. Choose the proper CinemaDNG settings for capture. The CinemaDNG format supports encoding as file sequences or a single MXF file. Some software may only support one of these options. If both options are available, choose the one that best suits your workflow. Other options that should be considered are use of compression (lossless), image capture size, frame rate, audio capture, etc.. Many of these options may be set before the camera is used on set. During camera setup, software used to capture CinemaDNG permits the pre-visualization of the scene before capture. This pre-visualization step allows the cinematographer to preview how the current camera setup will define the sensor data that is encoded in the CinemaDNG file. The pre-visualization step also allows the cinematographer to adjust parameters in the capture software that will affect how the CinemaDNG file is eventually rendered later in post-production. The CinemaDNG software adjustments include exposure control, black and white levels, color balance, etc. It is important to understand that the adjusted parameters will not bake changes into the saved digital file after capture. Instead, they will stay with the CinemaDNG file as metadata and be used when the CinemaDNG file is rendered to RGB for editing later in the workflow. CinemaDNG capture software should provide features that allow the accurate preview of colors and tones using common monitor calibration tools. In addition to the pre-visualization features in the CinemaDNG capture software, there may also be facilities to allow for camera calibration to enable precise decoding of colors captured by the camera sensor. Consult your camera capture software documentation for more information on camera calibration using CinemaDNG. Step 2 Use Capture Software to Include Metadata in CinemaDNG Files. CinemaDNG allows for the storage of metadata in the CinemaDNG file. This metadata can be used later to understand information related to the scene that was captured. Typical information that could be stored in these metadata fields include date/time of capture, camera setup information, actor names, etc. If CinemaDNG files are archived, this data can be a valuable asset when looking for specific footage after the scene has been shot. 4

5 Image 2, on the right, shows the metadata panel in the Adobe Premiere Pro application. For more information on metadata in CinemaDNG, see the section, Using Metadata with CinemaDNG Files, at the end of this document. Step 3 Capture Scenes and Encode Digital Files Using the CinemaDNG File Format. After the proper settings have been made using the CinemaDNG capture software, files are recorded to disk. Step 4 (optional) Footage Processing Using the CinemaDNG Format. There may be situations where adjustments to the CinemaDNG image processing metadata is desirable. This might include exposure or color temperature adjustments to maintain consistency between shots or to achieve a desired look in the captured footage. This step may be necessary if 1) the editing application that renders (demosaic, etc.) the CinemaDNG files lacks the required capabilities to control the CinemaDNG rendering or 2) the editorial process in the non-linear editor requires color that is closer to the desired color in a final print. The nondestructive adjustments made in the CinemaDNG editing application are saved in the CinemaDNG file or external metadata file and delivered to software further on in the workflow (non-linear editor as example). Step 5 (optional) Deliver CinemaDNG Files to Visual Effects Software for Processing. Import CinemaDNG files into visual effects applications. image 2 Premiere Pro metadata panel In many cases, visual effects applications that support CinemaDNG will offer a user interface to allow control over how the sensor data stored in CinemaDNG will be rendered to RGB values used by the visual effects application. Many visual effects applications composite and blend colors using linear light encoding. Importing CinemaDNG files into a visual effects application provides workflow advantages as the CinemaDNG data is already encoded as linear light or log and does not require a conversion from gamma-encoded file formats. Use software controls, if present, to insure that the imported CinemaDNG files are properly adjusted during import. After visual effects work is done, RGB-encoded files are delivered to editing or grading applications for integration into the timeline. Step 6 Deliver CinemaDNG Files to Editorial Software for Processing. Provide CinemaDNG files to non-linear editing (NLE) applications that read CinemaDNG files. CinemaDNG files can be used directly by the NLE or transcoded to a lower-bandwidth codec for faster editing. Proxy files may be created before or during ingestion of the CinemaDNG files into the nonlinear editor. In many cases, editing applications that support CinemaDNG will offer a user interface to allow control over how the sensor data stored in CinemaDNG will be interpreted. Use these controls to insure that the data you work with in editing applications is properly adjusted. 5

6 Step 7 Deliver CinemaDNG files to Grading Software. After editorial work is complete in the non-linear editor, create an edit decision list (EDL) for use by the grading software to conform high resolution files into the timeline. Hi-resolution CinemaDNG files will be acquired and used for color grading. Previously defined processing instructions for the CinemaDNG files will be available to the grading software for the final grade. Step 8 Deliver RGB Files in Appropriate Formats for Film Printing or Digital Distribution. After the color grading and finishing steps are complete, deliver files in appropriate formats like DPX, TIFF, or DCDM for film printing, digital distribution, or additional transcoding. Workflow #2 CinemaDNG Capture, Render to RGB for Non-linear Edit, and Deliver RGB Formats for Final Grading and Finishing. Workflow #2 is similar to workflow #1 in many respects. In workflow #1, CinemaDNG files were handed to applications for final grading. In workflow #2, CinemaDNG files are rendered to RGB/Y'CbCr formats before or during ingestion into the non-linear editor. Non- CinemaDNG files are then passed to applications for final grading and finishing after editorial work is done. In this workflow, cinematographers use cameras that are capable of writing CinemaDNG files (or files transcoded to CinemaDNG from other RAW formats). These CinemaDNG files may be frame sequences or data streams wrapped in the MXF file format. There may be cases were image adjustment is redefined after the CinemaDNG files have been written to disk. Examples of these kinds of adjustments are exposure adjustment and white point color balance. These adjustments can be saved in separate metadata files or in the CinemaDNG file itself. The CinemaDNG files are then ingested into non-linear editing applications. Transcoding/rendering of the CinemaDNG files takes place at this time. After editorial work is completed in the non-linear editor, RGB files (DPX, TIFF, etc.) are delivered to grading and finishing applications. Step 1 Camera Setup Using CinemaDNG Controls. Choose the proper CinemaDNG settings for capture. The CinemaDNG format supports encoding as file sequences or a single MXF file. Some software may only support one of these options. If both options are available, choose the one that best suits your workflow. Other options that should be considered are use of compression (lossless), image capture size, frame rate, audio capture, etc.. Many of these options may be set before the camera is used on set. During camera setup, software used to capture CinemaDNG permits the pre-visualization of the scene before capture. This pre-visualization step allows the cinematographer to preview how the current camera setup will capture data that is encoded in the CinemaDNG file. The pre-visualization step also allows the cinematographer to adjust parameters in the CinemaDNG capture software that will affect how the CinemaDNG file is eventually rendered enabling camera setup based on creative choices made in the CinemaDNG software. The CinemaDNG software adjustments include exposure control, black and white levels, color balance, etc. It is important to understand that the adjusted parameters will not bake changes into the saved digital file after capture. Instead, they will stay with the CinemaDNG file as metadata and be used when the CinemaDNG file is rendered to RGB for editing later in the workflow. CinemaDNG capture software should provide features that allow the accurate preview of colors and tones using common monitor calibration tools. In addition to the pre-visualization features in the CinemaDNG capture software, there may also be facilities to allow for camera calibration to enable precise decoding of colors captured by the camera sensor. In most cases, this type of calibration will be done by the camera manufacturer. In either case, the calibration information will be carried in the CinemaDNG file as non-destructive metadata. 6

7 Consult your camera capture software documentation for more information on camera calibration using CinemaDNG. Step 2 Use Capture Software to Include Metadata in CinemaDNG Files. CinemaDNG allows for the storage of metadata in the CinemaDNG clip. This metadata can be used later to understand information related to the scene that was captured. Typical information that could be stored in these metadata fields include date/time of capture, camera setup information, actor names, etc. If CinemaDNG files are archived, this data can be a valuable asset when looking for specific footage after the scene has been shot. For more information on metadata in CinemaDNG, see the section, Using Metadata with CinemaDNG Files, at the end of this document. Step 3 Capture Scenes and Encode Digital Files Using the CinemaDNG File Format. After the proper settings have been made using the CinemaDNG capture software, files are recorded to disk. Step 4 (optional) Footage Processing Using the CinemaDNG Format. There may be situations where adjustments to the CinemaDNG image processing metadata is desirable. This might include exposure or color temperature adjustments to maintain consistency between shots or to achieve a desired look in the captured footage. This step may be necessary if 1) the editing application that renders (demosaic, etc.) CinemaDNG files lacks the required capabilities to control the CinemaDNG rendering or 2) the editorial process in the non-linear editor requires color that is closer to the desired color in a final print. The adjustments made in the CinemaDNG editing application are saved in the CinemaDNG file or external metadata file and delivered to software further on in the workflow (non-linear editor as example). Step 5 (optional) Deliver CinemaDNG Files to Visual Effects Software for Processing. Import CinemaDNG files into visual effects applications. In many cases, visual effects applications that support CinemaDNG will offer a user interface to allow control over how the sensor data stored in CinemaDNG will be rendered to RGB values used by the visual effects application. Many visual effects applications composite and blend colors using linear light encoding. Importing CinemaDNG files into a visual effects application provides workflow advantages as the CinemaDNG data is already encoded as linear light or log and does not require a conversion from gamma-encoded file formats. Use software controls, if present, to insure that the imported CinemaDNG files are properly adjusted during import. After visual effects work is done, RGB-encoded files are delivered to editing or grading applications for integration into the timeline. Step 6 Deliver CinemaDNG files to Editorial Software for Processing. Provide the CinemaDNG files to non-linear editing (NLE) applications that read CinemaDNG files. In many cases, editing applications that support CinemaDNG will offer a user interface to allow control over how the sensor data stored in CinemaDNG will be transcoded to codecs used by the editing application. Use these controls to insure that the data you work with in editing applications is properly adjusted. After the transcoding step mentioned above, CinemaDNG files are not longer used in this workflow. Future processing steps, like final color grading, will use non-cinemadng formats. 7

8 image 3 Adobe Premiere Pro CinemaDNG Basic Controls Step 7 Deliver RGB Files to Grading Software. After editorial work is complete in the non-linear editor, conform files (if necessary) in the timeline and render to formats (DPX, TIFF, etc.) appropriate for the color grading and finishing software. Step 8 Deliver RGB Files in Appropriate Formats for Film Printing or Digital Distribution. After the color grading and finishing steps are complete, deliver files in appropriate formats like DPX, TIFF, or DCDM for film printing, digital distribution, or additional transcoding. Using Metadata with CinemaDNG Metadata is an essential part of the CinemaDNG workflow. CinemaDNG supports a latebinding workflow where image processing instructions accompany CinemaDNG files through the workflow until files are eventually rendered. In addition to stored image processing parameters, metadata can also contain other important information about the CinemaDNG file. Examples of this type of metadata are camera type, time code, as well as user definable metadata like scene number, GPS location, actor names, editorial comments, etc. You can use popular metadata schemas like EXIF and Dublin Core to define fields for your metadata. CinemaDNG also supports the extensible XMP metadata framework. It is likely that software applications that support reading and writing CinemaDNG files will also support options for defining metadata using these schemas. See the CinemaDNG specification [link will be included] for a partial list of metadata available in the CinemaDNG file format. 8

9 CinemaDNG allows this descriptive metadata to be stored inside the CinemaDNG file or in a separate file. There are pluses and minuses to either approach. Storing the data in the CinemaDNG file makes file management easier as you have fewer files to keep track of. However, storing the metadata in a separate file allows metadata to be saved quickly without the need for image/audio components of the CinemaDNG file to be re-saved along with the metadata. If metadata is stored as a file, this file will use the same name as it s associated CinemaDNG file. These files will have an.xmp extension. In certain workflows, it may be helpful to apply metadata to an individual frame, sets of frames (clips), or both. The CinemaDNG file format has the flexibility to save metadata in clips or individual images and we anticipate most applications that support CinemaDNG will allow for metadata to be saved in these ways. If CinemaDNG format is used for archiving, it is recommended that applicable metadata be used so future analysis and retrieval of footage can be done efficiently. Using XMP with CinemaDNG Most of the metadata you will store in your CinemaDNG files will likely be encoded using XMP (Extensible Metadata Platform), an open-source metadata framework originally developed by Adobe. Common metadata schemas like IPTC and Dublin Core use the XMP framework and provide prescribed fields that can be useful for cinematographers using CinemaDNG. The camera processing instructions are also stored as XMP. The specific schema for this data can be found at: Additional information on XMP can be found at: Reference Documentation CinemaDNG Specification The CinemaDNG specification is available on the Adobe web site at: [http address will be supplied when specification is published] 9