rawtektv, Version 1.0: A File Format For Storing Video Sequences

Transcription

1 rawtektv, Version 1.0: A File Format For Storing Video Sequences Todd Brunhoff, TV Test& Measurement Every image processing system recognizes some file format for storing images and their attributes, such as color space and dimensions; and there are many such formats: for example, JPEG, TIFF and GIF. A sequence of images, such as NTSC video, must include other attributes such as frame rate, number of frames and so on. There are a few such formats: MPEG, AVI, QuickTime and a few others. For the PQA 200 product, we needed a simple, uncompressed format that expresses all of attributes of images and sequences plus the ability to add tags that can express measurements associated with individual fields, frames or the entire sequence. This document describes the format of the file, internally known as rawtektv. 1. Overview This file format is very simple because to consists of merely of three components: an ascii file header, a binary header for each frame, and the raw data associated with each frame; hence, the name rawtektv. It is a descriptive format that details the parameters of the sequence and images, including color space, sample rate, interlace style, dimensions, and so on. Essentially, any continuous sequence of multi-channel images can be stored in this format as long as the distance from pixel to pixel and line to line remains constant. Typically, this is true for all uncompressed sequences. Below is a simple representation of the file layout: file header 8192 bytes frame 1 data w*h*d bytes frame N data... per-frame header 4+12*s bytes w: width h: height d: depth (bytes per pixel) s: # of summary tags Figure 1: Diagram of file layout The size of the file can be calculated from the size of the header (8192 bytes) plus the size of all the frame headers and data, the per-frame headers are calculated from the number of summary tags for the sequence (t); the frame data is calculated from the width and height and depth of the frames. All of these numbers are constant for the entire sequence. The following sections describe each of these parts in detail. rawtektv file format 1 Tektronix Proprietery

2 2. File Header The file header is an ascii, parametric description of the sequence, embedded in an 8K byte block. An (abbreviated) example of the header appears below: rawtektv sample=4:2:2 color=yc pixbytes=1 pixbits=8 size=720x486 frames=60 ^L There are a few observations worth making about this example. Note that the header begins with the ascii string rawtektv. This token must appear alone on the first line of the file. The ^L at the end of the file is an ascii control-l, otherwise known as a form-feed. If you have some unix tools available, then you can look at the contents of the file header with the more command and it will stop when it encounters the ^L. This is more convenient than having a special command whose only purpose is to dissect a binary header and print the contents. Also note that all the other entries in the file appear in the form of keyword assignments to variables, such as size=720x486. Remember that this is an abbreviated example. A complete list of keywords and their meanings appear in section Frame Header The frame header is a binary array of triples that record per-frame information such as measurement results. The structure of the header is equivalent to the C-language declaration of Summary below: struct Cell { int fieldtype; int tagtype; float tagvalue; }; struct Summary { int nsummaries; struct Cell cells[ nsummaries ]; }; An experienced C programmer will recognize that it the array declaration is illegal -- you can t declare an array size with anything but a constant, and nsummaries is not a constant. Nevertheless, this is the structure of every frame header. If the keyword summaries appears in the file header, its value is used for nsummaries. This yields the formula for the size of the header from figure 1: 4+12*s, where s is nsummaries. There is one exception to this formula: if nsummaries is zero or missing from the file header, then the size of the frame header is also zero (not four). The values for fieldtype and tagtype are intended for internal use. However, a few interesting values appear in the table below. Any other values are not documented here and can be ignored. Note that the meaning of the values may change at any time. rawtektv file format 2 Tektronix Proprietery

3 Table 1: Tag and Field Types element value meaning tagtype 0 empty; no meaning 23 frame number; the tagvalue is the frame number for the following frame data 26 sequence ID; the tagvalue is the sequence identifier serial number; for example, 1 is the Cheerleader sequence 27 stripe type; if the tagvalue is zero, the sequence is striped with a header-only stripe; if non-zero, the stripe is a header-plus 28 user defined; if the tagvalue is zero, then the sequence is a Tektronix sequence; otherwise it is a user-defined sequence fieldtype 0 empty; this only occurs when tagtype is zero. 3 all of the non-zero tagtypes listed above will have this value 4. Frame Data Depending on the type of sequence described in the file header, the calculation of the size of each frame can range from simple to complex. In general, the size of the frame data is equal to width*height*depth where depth is number of bytes per pixel.the parameter that makes it complex is the sample rate. An RGB image is always 4:4:4 (or 1:1:1 for the math purist) sample rate, meaning that all three channels have the same number of samples. Or to put it another way, every pixel has one red, one green and one blue sample. If the size of each sample is one byte, then the size of the frame is width*height*3. Chroma subsampling achieves a sort of compression by storing fewer samples in the chroma channels than the luma channel. RGB can t have chroma subsampling because all three channels contain both color and lightness. Y/Cr/Cb is used in most video systems and because the luma channel (Y) is separate from the two chroma channels (Cr and Cb) the latter can be subsampled. If you were to take a close look at a pixel grid in a Y/Cr/Cb image, the storage for 4:2:2 sampling would look like: = Y = Cr and Cb Note that every pixel has a Y sample, but that only every two pixels has a Cr and Cb sample. If you consider just the first four pixels in the first row, there are four Y, two Cr and two Cb samples; hence, the term 4:2:2. If the channels are all one-byte samples, then the size of the image is width*height*( ), or more simply width*height*2. A Y/Cr/Cb image, sometimes called YC, can be sampled at 4:4:4, 4:2:2 or 4:2:0. One final twist to the frame data is the relative positioning of the samples, which is specified by pix- rawtektv file format 3 Tektronix Proprietery

4 elinterleave. Mostly this is a subject for the next section, Frame Data Layout, but can affect the size as well. RGB images sometimes store their pixels on 32-bit boundaries. This requires a zero sample at one end or the other: RGB0 or 0RGB. In this case the size of the image is width*height*4. The table below shows how to calculate the most common frame sizes, assuming all channels are one byte wide.: Table 2: How to Calculate Frame Data Size pixel sampling 4:4:4 4:2:2 4:2:0 ABC, CBA, CAB width*height*3 width*height*2 width*height*3/2 pixel Interleave ABC0, 0ABC, CBA0, 0CBA width*height*4 N/A N/A 5. Frame Data Layout The layout of frame data can be arbitrarily complex, but in practice, occurs in a handful of forms. The parameters that control the layout already discussed are width, height, bytes per channel and sampling. Three others, pixelinterleave, interlace and field1mem also affect the image layout. The parameter pixelinterleave, discussed in the previous section, describes the relative placement of one channel with respect to another. The nomenclature used is a simple ABC, where A corresponds to the first channel, B to the second and C the third. An optional zero can be added on either end to express zero-padding. Although the channels can be arranged in a several different ways, only a few are ever used in practice, and only three are used in a file format: ABC, CBA and CAB. One other form of pixel interleave is none at all. That is, each channel appears in a sub-image by itself. This form of pixelinterleave is called separate.the table below shows what each of the pixelinterleave values mean for different color spaces and sample rates: Table 3: Channel Layout pixelinterleave separate RGB RRRR..., then GGGG..., then BBBB... Y/Cr/Cb 4:4:4 YYYY..., then CrCrCr..., then CbCbCb... Y/Cr/Cb 4:2:2 YYYY..., then CrCrCr..., then CbCbCb... Y/Cr/Cb 4:2:0 YYYY..., then CrCrCr..., then CbCbCb... Gray YYYY... ABC RGBRGB... YCrCbY... YCrYCbYCr... not used not used CAB not used CbYCrCbYCr... CbYCrYCbY... not used not used CBA BGRBGR... not used not used not used not used The interlace keyword specifies the type of interlacing used between two fields of a frame. There are really two dimensions to interlacing and these are both expressed in the value of the keyword. The first is whether the interlacing is done line-by-line or field-by-field. Line-interlacing lays out an image so that it is spatially correct and the lines in the file appear as they would on the screen. Field interlacing has the advantage of presenting each field so that it is temporally correct; the first field appears before the second field. Figure 2 rawtektv file format 4 Tektronix Proprietery

5 illustrates field- and line-interlaced images. Note, that the appearance of a field- or line-interlaced file will be identical, this parameter only describes the layout in the file. field 1 field 2 field 1 field 2 field interlaced line interlaced Figure 2: field- and line-interlaced layout The second dimension specifies which field occupies the top spatial line of the display. This is important only because the two big video standards NTSC and PAL do it differently. In NTSC, the first line of the second field occupies the top line of the display. In PAL, the first line of the first field occupies the top line of the display. These two parameters together yield four values for the interlace parameter: Table 4: Values for interlace value layout which field occupies top line bylinetopfieldfirst line-interlaced first field bylinetopfieldsecond line-interlaced second field bypicttopfieldfirst field-interlaced first field bypicttopfieldsecond field-interlaced second field As a convenience, the keyword standard is abbreviation for the keywords size and fps and provides necessary information for the use of the keyword layout, described below. The values implied by standard are shown in table 5. Table 5: Values Implied by Standard standard value implied values NTSC PAL size 720x x576 fps The keyword layout is an abbreviation for color, field1, interlace, interleave, pixbits, pixbytes and sample. Note that it is dependent on another keyword, standard, which must appear before layout. The values implied by layout are shown in table 6. rawtektv file format 5 Tektronix Proprietery

6 Table 6: Values Implied by Layout layout value implied values color field1 interlace abekas NTSC: bypicttopfieldsecond PAL: bypicttopfieldfirst YC firstinmem interleave CAB pixbits 8 pixbytes 1 sample 4:2:2 rec601 NTSC: bylinetopfieldsecond PAL: bylinetopfieldfirst 6. File Header Details The table below shows each of the keywords that may appear in the ascii file header, their meaning and constraints. Table 7: File Header Keywords, Values and Meaning keyword values meaning byteorder 0x the file was created on a little-endian machine, such as Intel 0x10203 the file was created on a big-endian machine, such as Sparc or Alpha bytesperfield positive integer This is used to describe the number of bytes per field when the size of the frame data cannot be calculated; that is, when the value of type is not public. color gray gray-scale color space, one channel of data per image yc, YC yuv, YUV rgb, RGB color difference, sometimes called luma-chroma another color difference used in the PQR algorithm red, green, blue colorspace comment any string used to store arbitrary comments in the file header field1 firstinmem obsolete; the data for field 1 appears before the data for field 2 secondinmem obsolete; opposite of firstinmem fps real number frame rate expressed in frames per second (not fields per second) frames integer number of frames in the file; it is possible that this number is unreliable if the process that created the file terminated abnormally interlace notinterlaced the sequence is not interlaced bylinetopfieldfirst line interlaced, first field occupies top line bylinetopfieldsecond line interlaced, second field occupies top line bypicttopfieldfirst field interlaced, first field occupies top line bypicttopfieldsecond field interlaced, second field occupies top line interleave separate, ABC, CAB, CBA See Frame Data Layout in section 5 and Channel Layout in table 3 rawtektv file format 6 Tektronix Proprietery

7 Table 7: File Header Keywords, Values and Meaning keyword values meaning layout abekas, rec601 See Values Implied by Layout in table 6 pixbits positive integer number of significant bits in each sample; the value must be 8*pixbytes pixbytes positive integer number of bytes in each sample rawtektv must appear alone, without = or a value, on the first line of the header sample 4:4:4, 4:2:2, 4:2:0 see the discussion on sample rate under Frame Data in section 4 sequence block of values See Sequence Summary Values in section 7 size NxM where N and M are positive integers expressing width and height; the x is literal and there must be no space between the numbers and the x. standard NTSC, PAL See Values Implied by Standard in table 5 summaries positive integer specifies the number of summaries that appear before each frame in the file; see Frame Header in section 3. type public All RGB and YC images are considered type=public, which is the default value; hence, the type keyword does not normally appear in the file header. jndlumaprecompute jndchromaprecompute jndcompositeprecompute These values imply that the image data is opaque, defined only for internal use. The image data is the result of preprocessing a type=public sequence for use in the PQR measurement. The keywords color, field1, interlace, interleave, pixbits, pixbytes, size apply to the original sequence and cannot be used to calculate the size of the frame data. In this case, a value for bytesperfield will be present. 7. Sequence Summary Values Sequence summary values do not pertain to the geometry of a sequence, but fall into three categories: single source measurements, such as detected crop, level, gain and shift impairments; multi-source measurements such as PSNR or PQR measurements; and other non-measured information about sequences, such as a Tektronix serial number that may be used in a measurement. Each of the values in a sequence summary is associated with a field, which can be first, second or even both fields. The sequence summary is optional, and if present, appears as a block enclosed in braces, like this (the... are for illustration only): sequence { field=both sequenceid=1... field=first lumalevel= field=second lumalevel= } rawtektv file format 7 Tektronix Proprietery

8 Note that the appearance of a field keyword applies to all values following until another field keyword is encountered. The keywords and values that can appear in a sequence block are listed in table 8. Table 8: Sequence Summary Keywords, Values and Meaning keyword values meaning positive real number psnrluma psnrcomposite psnrcr psnrcb jndluma jndcomposite xshiftluma xshiftcr xshiftcb yshiftluma yshiftcr yshiftcb x y width height lumalevel crlevel cblevel lumagain crgain cbgain originalwidth originalheight positive real number real number non-negative integer real number real number non-negative integer result from a PSNR measurement, only one of these values will appear result from a PQR measurement, only one of these values will appear Specifies the detected shift in each channel (Luma, Cr, Cb) in both horizontal and vertical (x, y). A positive value means that the detected shift is to the right (x) or down (y). Specifies the detected cropping in whole pixels. The x and y specify the offset from the left and top; the width and height specify the size of the uncropped region. Note that x+width must be the sequence width and y+height must be sequence height. Specifies the detected level change in each channel (Luma, Cr, Cb). A positive value means an increased level. Specifies the detected gain change in each channel (Luma, Cr, Cb). A positive value means an increased gain. Specifies the width and height of the original sequence. This typically appears in a measurement output file (sometimes called a map file), and describes the size of the input sequence before the cropping was applied. The is because the map file is frequently reduced to the size of the cropped image. sequenceid non-negative integer Specifies the sequence serial number, used internally to identify the sequence content headerplusbody 1.0 Specifes the that the type of sequence is one with striping in both the header frames and body frames. userdefinedsequence 1.0 Specifies that the sequence id pertains to a user-defined set, not the those supplied by Tektronix. phase 0, 90, 180, 270 Specifies the detected phase of colorburst of the first frame in the sequence. This only appears if the software is aware that the input signal is analog NTSC or PAL. missingfields field list of non-negative integers first second both Specifies holes or missing fields in the sequence. The form is a list of integers, like missingfields={0,1,2} Keywords following this apply to the first field, second field or both fields, respectively. There must be a field assignment made at the start of a sequence summary block. rawtektv file format 8 Tektronix Proprietery