Understanding Line Data Formats

Size: px

Start display at page:

Download "Understanding Line Data Formats"

Jessie Pitts
5 years ago
Views:

1 BY DAVID STAAL Understanding Line Data Formats Since a large amount of production printing is still based on line data and its hybrids, technical analysts in the print industry need to understand these line data formats, especially when trying to integrate mainframe print data with PC-based solutions. ALTHOUGH laser printers with full graphic capabilities have been available for more than a decade, a large amount of production printing is still based on line data (or hybrids, such as Xerox LCDS). For this reason, technical analysts in the print industry need to thoroughly understand line data formats, especially when trying to integrate mainframe print data with PC-based solutions. As an instructor, I have found that many analysts lack this basic knowledge, largely because it is not available in any simple form. This article is intended to provide a basic and thorough description of the most common line data formats, including mainframe Fixed Block (FBA or FBM), Variable Block (VBA or VBM), and line data. This article will also explain the basic varieties of printer carriage control and provide examples of these formats. Line data, or line printer data, can be defined simply as data formatted for a printer that prints one line at a time. As printers have evolved, more advanced printers (those that could support fonts and graphics) continued to support the data intended for the older line printers, as a backward compatibility feature. Eventually, the expression line data was coined to refer to this older data stream. For a thorough understanding of line data, it is necessary to review the original target of line data, which is, of course, a line printer. LINE PRINTERS Line printers, sometimes called impact printers, have the following characteristics: print one line or less (one character) at a time print text only, no graphics use a fixed font because the characters are physically engraved on the print heads or print band. The font is non-proportional, that is, the characters are all the same width. use continuous feed paper (with holes on the sides), which can include blank paper, green bar paper and special forms, such as labels and pre-printed checks move down the page in only one direction. The paper is moved with a tractor feed and can be advanced, but not backed up. Line data is data that is formatted for line printers. It generally has the following characteristics: is text only is stored as records or lines of data has some kind of carriage control commands that instruct the printer on how to move down the paper, such as Line Feed or Page Break THREE ELEMENTS OF LINE DATA Over the years, and on different computing platforms, many formats evolved for storing line data. The most common of these formats (e.g., text, EBCDIC VBA or VBM and EBCDIC FBA or FBM) will be discussed in detail later in this article. To interpret any line data format, it is necessary to identify the following three basic elements that make up the format: : How are the records separated or delimited? : What type of carriage control is used? : Is the data EBCDIC, or something else?

2 Because line data is fundamentally record-oriented, methods of storing line data must allow for the separating or delimiting of records. This ensures that the program reading the data doesn t accidentally string together two or more records or break a record into pieces. The following three basic schemes are used by different line data formats for record separation: Fixed Length Records: This is the most basic scheme where you arbitrarily decide on a length, such as 80 or 133 1, and all records in the file are made to be the same length. A program processing a fixed-length file of 133-byte records, for example, can then just read 133 bytes into its buffer at a time and assume that it has read in one record. This scheme is used primarily for mainframes, and the most common format for this is called FBA (Fixed Block with ASA/ANSI carriage control) or FBM (Fixed Block with Machine carriage control). Record Delimiters: The record delimiter scheme relies on special characters that are different from normal text to mark the end of a record. The most familiar example of this is text, which uses the Carriage Return (0Dh), Line Feed (0Ah), and Form Feed (0Ch) as record delimiters. A program processing this data would read one byte at a time, and when it came to one of these special characters, it would assume it had reached the end of the record. One flaw in this approach is that something unusual in the data portion of the record (such as an escape code string) might get confused with a record delimiter. Length Byte or Word: The most advanced and flexible scheme for separating records is to use a byte or two (a word) at the beginning of each record to store the length of the following record. This approach provides the benefit of storing variable length records, and allows you to store any type of data, not just text. For example, applications used with Xerox printers can store Xerox Metacode inside of a line data format using this scheme. Two common examples of line data formats FIGURE 1: CARRIAGE CONTROL Machine w/out data Machine with data ASA (ANSI) SCS CC Do not space 03 (no op) 01 0D 0D Space one line 0B09 (space) 0D 15 0D 0A Space two lines D Space three lines 1B19-0D Skip to Channel 1 8B89 1 0D 0C 0C Skip to Channel D Skip to Channel 3 9B99 3 0D Skip to Channel 4 A3 A1 4 0D Skip to Channel 5 ABA9 5 0D Skip to Channel 6 B3 B1 6 0D Skip to Channel 7 BB B9 7 0D Skip to Channel 8 C3 C1 8 0D Skip to Channel 9 CBC9 9 0D Skip to Channel 10 D3 D1 A 0D 04 7A Skip to Channel 11 DBD9 B 0D 04 7B Skip to Channel 12 E3 E1 C 0D 04 7C FIGURE 2: SAMPLE LINE DATA New Page Triple Space. FIGURE 3: LINE DATA ELEMENTS FOR FBA FIGURE 4: RECORD STRUCTURE FOR FBA C Data C Data byte using Length Words are VBA (Variable Blocked with ASA carriage control) or VBM (Variable Blocked with Machine carriage control) and Barr S/370 format. Line data needs to have some kind of carriage control, which are the instructions to the printer on how to move vertically down the page. At a minimum, any type of line Fixed Length Records ASA EBCDIC printer carriage control must have the following two basic commands: move down a line (called Line Feed, New Line or Space One Line) move to top of next page (called Form Feed, Page Break or Skip to Channel 1) Following are the four most common types of carriage control used in the production printing business: 1 The number 133 is derived from the old line printers that could print 132 columns wide. The record would store 1 byte of ASA carriage control followed by 132 bytes of text.

3 : used in the PC, Unix and Internet arenas SCS: used in RJE, 3270, and 5250 print streams ASA or ANSI: used on mainframes and AS/400s, as well as adopted for other platforms 1403/3211 Machine: native carriage control used on mainframe channelattached line printers The table in Figure 1 shows the hexadecimal values for the standard carriage control commands in each type of carriage control. In the case of ASA, it shows the text representation instead, because the hexadecimal value of ASA differs when it is used with or EBCDIC text. The Skip to Channel or Skip to Stop commands are an old line printer standard that instructs the printer to move down the page to a predetermined line on the page. The location of each stop can vary for each print job. For example, on one print job Stop 3 might be defined as line 15 on the page. So, whenever the printer receives the Skip to Channel 3 command, it moves forward to line 15, even if that means is has to move to the next page to get there. On a different print job, Stop 3 might be defined as line 30 or might not be defined at all. The Stops and the lines they represent are coded in a Forms Control Buffer (FCB), which is stored in a library or directory on the host computer and sent to the line printer when needed. Incidentally, the FCB also codes two other pieces of information: the lines-per-inch setting (either 6 or 8), and the form length in number of lines-perform or page. The important difference between ASA and Machine carriage control is that, although both are based on records, they skip at different times. ASA carriage control is executed before the line of data is written. Machine carriage control is executed after the data is written. These two techniques are referred to as Skip Before and Skip After. For example, if the ASA Skip to Channel 1 command (page break) is at the beginning of a record, the printer will issue a page break and print the record on the next page. In contrast, if the Machine Skip to Channel 1 command is at the beginning of a record, the record will be printed at the current location on the current page, and then the printer will move to the next page. This makes Machine carriage control a little FIGURE 5: HEX DUMP OF FBA E C1 40 E3 C5 E2 E3 40-D6 C6 40 C3 C1 D9 D9 C9 A TEST OF CARRI C1 C7 C5 40 C3 D6 D5 E3-D9 D6 D3 4B AGE CONTROL E E _Single Spa B ce A B C D E F F0 C4 96 A Doubl E B e Space F D5 85 A6 40 D New Page A B C D E F FIGURE 6: LINE DATA ELEMENTS FOR VBM FIGURE 7: RECORD STRUCTURE OF VBM Formatted without blocking: harder to read because it seems to occur out of sequence. (EBCDIC or ) Technically, a true line data format should contain only plain text in one of the two basic character sets, EBCDIC or. Record Length Field 3211 Machine EBCDIC (or hidden binary) RL 00 C Data RL 00 C Data Formatted with blocking: byte Record Length (two bytes) BDW BL 00 RL 00 C Data RL 00 C Data byte Record Length (two bytes) Block Length (two bytes) In practice, however, line data formats often contain something besides just text, such as LCDS, Metacode, PCL escape codes, or even raw binary data. Most of the cases that include some kind of binary data are only possible when using the Length Word method of record separation.

There are two basic standards for encoding text characters: EBCDIC: used on mainframes and AS/400s : used by the rest of the computing industry Both EBCDIC and use a single byte of data to represent

4 There are two basic standards for encoding text characters: EBCDIC: used on mainframes and AS/400s : used by the rest of the computing industry Both EBCDIC and use a single byte of data to represent a character. In fact, originally only used the lower 7 bits of the byte. In the days of simple line printers, line data would have only straight EBCDIC or with almost no variations. Sometimes a printer would have a few special characters, such as the cent symbol, which had to be accommodated, but this was easily handled. The examples presented here cover the most common types of line data. Once you have become familiar with these formats and learn to recognize their patterns, you should be able to recognize other variations of line data as well. EXAMPLES OF LINE DATA FORMATS The sample line data shown in Figure 2, with two simple pages of text, will be used in all of the following examples. That way, you can see how different line data formats represent the same printable text. The examples in Figures 3 and 4 (FBA and VBM) originally came from the IBM mainframe environment. The names FBA and VBM are derived from record format designations used on the mainframe, i.e., RECFM=FBA and RECFM=VBM. There are several possible variations of these, so I have chosen the two that I have found to be the most common, FBA and VBM. Once you learn the Fixed and Variable record structures, you will see that they can each be used with ASA or Machine carriage control. The three elements of line data are identified for each example. FIGURE 8: HEX DUMP OF VBM (WITHOUT BLOCKING) C1 40 E3-C5 E2 E3 40 D6 C6 40 C3...A TEST OF C C1 D9 D9 C9 C1 C7 C5 40-C3 D6 D5 E3 D9 D6 D3 4B ARRIAGE CONTROL E E Single Spac B C4-96 A E2 97 e...idouble Sp B 00 0D D5 85 A6 40 D ace...new Pag C1 40-E3 C5 E2 E3 40 D6 C6 40 e...a TEST OF C3 C1 D9 D9 C9 C1 C7 C5-40 C3 D6 D5 E3 D9 D6 D3 CARRIAGE CONTROL B E E Triple Spa B C4 96 A E2 ce...double S B E pace...single A0 40 E B 00-0E D6 A Space...Over B A3 00 0E D6 A print...overpr C A3 int FIGURE 9: LINE DATA ELEMENTS FOR FIGURE 10: RECORD STRUCTURE OF FBA (Fixed Block with ASA ) FBA can be described simply as fixed length records in which the first byte or character of each record is an ASA carriage control character, and the rest of the record is the printable EBCDIC data. In order to read or write fixed length records, you need to know or document the record length for the FBA file, since it can be any arbitrary length. In the mainframe world, a length of 133 is commonly used, because most line printers can print 132 characters across (plus one byte for the ASA carriage control character). Figure 5 shows a hex dump of a typical FBA print file with an EBCDIC interpretation at the right. You will notice a lot of blanks or spaces in the data represented by 40s in the hexadecimal portion of the dump. The hexadecimal value for an EBCDIC space character is 40. Spaces are required in a fixed length format in order to pad out every record to its full length. Clearly, this can result in a lot of wasted storage space, Delimiters (CR, LF, FF) CR, LF, FF Data CR LF Data CR FF FIGURE 11: HEX DUMP OF Delimiter/CC Delimiter/CC F A TEST OF CARRIA F 4E F 4C 2E 0D 0A E GE CONTROL...Sin C E 0D 0A 0D 0A 44 6F gle Space...Do C E 0D 0C 4E uble Space...New D 0A F 46 Page..A TEST OF F 4E F CARRIAGE CONTRO C 2E 0D 0A 0D 0A 0D 0A C L...Triple S E 0D 0A 0D-0A 44 6F C pace...double E 0D 0A E 67 6C Space...Single S E 0D 0A 4F E 74 pace A0 0D 4F E 74 0D 0A... which is a disadvantage of the fixed length record format. The same Fixed Block format can be combined with Machine carriage control, in which case it is called FBM. The dump would look almost the same, with many spaces padding out each record to the same length. Only the carriage control byte at the beginning of each record would look different, containing Machine carriage control commands instead of ASA. One way to recognize Machine carriage control is to look for values of 09h, which is the commonly used write and space one line command. VBM (Variable Block with 3211 Machine ) The VBM format uses a record length field at the beginning of each record to indicate the length of the following record. The length is coded in the field in binary, and it includes itself in the count. See Figure 6. As Figure 7 shows, the IBM variable length record format can be either with or

5 without blocking (V or VB). In the PC environment, there is no need for blocking, so many software packages prefer to use the format without blocking. The block length plus the two trailing nulls are called a BDW or Block Descriptor Word. The record length plus the two trailing nulls are called a RDW or Record Descriptor Word. If you re wondering why there are two null bytes after the record or block length, IBM documentation says they are reserved for possible future system use. The nulls actually turn out to be helpful, because a person or a program can more easily detect this format by looking for the pattern of nulls. Figure 8 shows a hexadecimal dump of a VBM file without blocking, which you could technically call a VM file. Note that you can locate the beginning of each record by just finding two nulls (00h) in a row. As with the Fixed length format in the previous example, the IBM Variable format can be used with either ASA or Machine carriage control (Variable Block with ASA is called VBA). The hex dump of a VBA file would look the same as that for a VBM file, except for the carriage control byte. If you see many values of 40h in the carriage control position, it s ASA, but if you see many values of 09h, it s Machine carriage control. FIGURE 12: LINE DATA ELEMENTS FOR WITH ASA FIGURE 13: RECORD STRUCTURE OF WITH ASA FIGURE 15: TEXT VIEW OF WITH ASA New Page Triple Space. Delimiters (CR, LF) ANSI (ASA) C Data CR LF C Data CR LF Delimiter ASA Character FIGURE 14: HEX DUMP OF WITH ASA Delimiter B F A TEST OF CARRI F 4E F 4C 2E 0D 0A AGE CONTROL..._S E 67 6C E 0D 0A ingle Space...0D F C E 0D 0A 31 4E ouble Space...1N D-0A ew Page.. A TEST F F 4E OF CARRIAGE CON F 4C 2E 0D 0A 2D C TROL...-Triple S E 0D 0A F C pace...0double S E 0D 0A E 67 6C pace... Single S E 0D 0A 20-4F E pace... Overprin A0 74 0D 0A 2B 4F E 74 0D 0A t... is the standard used on the PC platform and on many midrange systems, including Unix systems. uses a very simple form of carriage control, which is made up of just three control characters carriage return (CR or 0Dh), line feed (LF or 0Ah), and form feed (FF or 0Ch). See Figure 9. The Unix flavor of doesn t even use the carriage return, just the line feed and form feed. The control characters also do double duty as delimiters to indicate the end of the record. So, if you are reading an record, and you encounter any one of these three control characters, it means you have reached the end of that record. See Figure 10. Figure 11 shows a hexadecimal dump of an file using the same sample data as all the other cases. In this hexadecimal dump and the one shown in Figure 14, the text representation on the right is based on an interpretation. A plain file will contain nothing but printable text, spaces, and the three control characters. Notice that double space and triple space are represented simply by repeated use of carriage return and line feed. with ANSI (ASA) with ASA is a hybrid format where the ASA carriage control from the mainframe world has been combined with simple text. The delimiters (CR and LF) are still used, but the form feed is dropped. More importantly, the delimiters are not interpreted as carriage control, but simply as record separators. The carriage control is represented by the ASA character at the beginning of each record. See Figures 12 and 13. At first glance, the data in Figure 14 looks similar to ordinary data. You can see the text and the delimiters (0D 0A), and it s hard to tell whether it is ordinary or something more. To make sure, try to view the file as a regular text file using any editor or word processing program. If it is regular, it will look like the original data at the beginning of these examples. If it is with ANSI carriage control, it will look like the sample shown in Figure 15. You should notice that the first character on every line will be a space, a number, or a plus () or minus (-) sign. CONCLUSION The examples presented here cover the most common types of line data. Once you have become familiar with these formats and learn to recognize their patterns, you should be able to recognize other variations of line data as well. David Staal is a professional services supervisor for Barr Systems, Inc., in Gainesville, FL. He can be contacted via at David.Staal@barrsystems.com.

August Line Printer Daemon Protocol. Status of this Memo

August Line Printer Daemon Protocol. Status of this Memo Network Printing Working Group Request for Comments: 1179 L. McLaughlin III, Editor The Wollongong Group August 1990 Line Printer Daemon Protocol Status of this Memo This RFC describes an existing print