The New C Standard (Excerpted material)

1854 preprocessor directives syntax preprocessing-file: group opt group: group-part group group-part group-part: if-section control-line text-line # non-directive if-section: if-group elif-groups opt else-group opt endif-line if-group: # if constant-expression new-line group opt # ifdef identifier new-line group opt # ifndef identifier new-line group opt elif-groups: elif-group elif-groups elif-group elif-group: # elif constant-expression new-line group opt else-group: # else new-line group opt endif-line: # endif new-line control-line: # include pp-tokens new-line # define identifier replacement-list new-line # define identifier lparen identifier-list opt ) replacement-list new-line # define identifier lparen... ) replacement-list new-line # define identifier lparen identifier-list,... ) replacement-list new-line # undef identifier new-line # line pp-tokens new-line # error pp-tokens opt new-line # pragma pp-tokens opt new-line # new-line text-line: pp-tokens opt new-line non-directive: pp-tokens new-line lparen: a ( character not immediately preceded by white-space replacement-list: pp-tokens opt pp-tokens: preprocessing-token pp-tokens preprocessing-token new-line: the new-line character 1854 2 v 1.1 January 30, 2008

1854 It can be seen from the grammar that most terminals do not match any syntax rules in the other parts of the language. The C language essentially contains two independent languages within it. The preprocessor and what might be called the C language proper. A text-line is the sequence of pp-tokens that are scanned for macros to substitute and subsequently processed by the C language syntax. C90 Support for the following syntax is new in C99: group-part: # non-directive control-line: # define identifier lparen... ) replacement-list new-line # define identifier lparen identifier-list,... ) replacement-list new-line The left hand side terms text-line and non-directive were introduced in C99. Their right-hand side occurred directly in the right-hand side of group-part in the C90 Standard. text-line: pp-tokens opt new-line non-directive: pp-tokens new-line The definition of lparen in C90 was: lparen: the left-parentheses character without preceding white-space The Standard specifies the same syntax as the C90 Standard (16p1). Other Languages While most assembly languages support some form of preprocessor, few high-level languages include one (Lisp, PL/1 do). [3] A conditional compilation mechanism was recently added to the Fortran Standard [6] (it is specifically aimed at conditional compilation and does not offer macro processing functionality). While many versions of Unix include a preprocessor that is independent of the C preprocessor (i.e., m4 [10] ) it has never achieved wide usage (this reason was given for its exclusion from the POSIX Standard). A few other stand-alone preprocessors (often called macro processors) have also been written (e.g., ML/I [2] ). Some non-c preprocessors use the character $ or % rather than #. Common Implementations Because of the relative simplicity of the preprocessor grammar most implementations do not use a tabledriven parser approach, but rather process the directives on an ad hoc basis. Some preprocessors operate at a [5, 11] higher level than individual tokens e.g., taking syntax into account. The base document did not support the preprocessing directive #elif or defined. Support for this functionality was added during the early evolution of C. [8] Also early implementations supported trailing tokens on the same line as preprocessing directives #endif MACRO_NAME and some implementations [9] continue to provide an option to support this functionality. Some early translators did not permit white-space between # and define Some also required the # to be the first character on the line gcc supported... in macro definitions prior to C99. The syntax used was to give the varying arguments a (developer chosen) name, for instance in: January 30, 2008 v 1.1 3

1854 1 #define debug(file, format, args...) fprintf(file, format, args) following the parameter args with... specifies that it takes a variable number of arguments. These variable arguments are substituted wherever the named parameter appears in the replacement list (just like other parameters). The Unix System V C compiler [10] (plus gcc and others) supports the creation and use of what it calls assertions. For instance, (the # before an identifier signaling its status as a predicate): 1 /* #assert predicate(token-sequence) */ 2 3 #assert derek (is_male) /* Associate the answer is_male with the predicate derek. */ 4 5 /* #if #predicate(token-sequence) */ 6 7 #if #derek(is_female) /* Evaluates to false. */ 8 #endif 9 /* 10 * Implementations supporting these kinds of assertions usually 11 * define the predicates system (with answers such as svr4, mach, posix, 12 * hpux, etc.), and cpu (with answers such as i386, sparc, m68k, etc.). 13 */ 14 #if #cpu(sparc) 15 /*... */ 16 #elif #cpu(i386) 17 /*... */ 18 #endif The directive: 1 #ident "version X.Y" is used by a number of translators to include version control information in the source code that is also written out to the generated object file. A common technique is to use a sequence of characters, within the string literal, that are recognized by a source code control system (and which can be updated when a new version is checked in) The matching directives #version/#endversion are available in a Bell-Labs translator. It is claimed [1] that developers were 40% more productive when using a version sensitive editor on a multi million line project having many versions (developed over two decades by more than 5,000 developers). The Plan 9 C compiler intentionally lacks support for the #if directive. [7] Coding Guidelines While lines containing non-preprocessing directives are often visually indented (see Figure 1854.1) developers do not generally indent preprocessing directives (see Figure 1854.1). One possible reason for this difference is that indentation of preprocessing directives may not provide a visual edge for readers to run their eyes along (e.g., the visible source between conditional inclusion directives may be statements from the C language whose indentation is driven by factors unrelated to preprocessing; that is there are two indentation schemes operating over the same section of source). In over 90% of preprocessing directives the # character appears at the start of the line. Given that preprocessing directives are usually scattered throughout the source this indentation strategy allows them to generally standout from the surrounding statements. In general both the #include and #define directives appear at the start of a source file (see Figure 1854.2). By their nature the #if directives are localized at the point when a conditional decision needs to be made. 4 v 1.1 January 30, 2008

1855 100,000.c files.h files Visible lines 10,000 1,000 100 10 1 0 4 8 12 16 24 32 40 48 Indentation of # Figure 1854.1: Number of lines containing a preprocessing directive starting at a given indentation from the start of the line (i.e., amount of white space before the first # on a line, with the tab character treated as eight space characters). Based on the visible form of.c and.h files. Occurrences 10,000 1,000 100 10 #define #include 1 0 25 50 75 100 Percentage line count into.c source file 0 25 50 75 100 Percentage line count into.h source file Figure 1854.2: Number of #include and #define directives appearing at a relative location (i.e., 100*line_number/lines_in_file) in the source. Based on the visible form of.c and.h files. Usage A study by Ernst, Badros, and Notkin [4] provides one of the few empirical studies of C preprocessor use. Table 1854.1: Occurrence of preprocessor directive names and preprocessor operators (as a percentage of all directive names and operators). Based on the visible form of the.c and.h files. Directive Name.c file.h file Directive Name.c file.h file #define 19.9 75.0 #if 6.2 1.5 #endif 19.9 7.2 ## 0.3 0.9 #include 28.6 4.1 #elif 0.2 0.2 #ifndef 2.4 3.2 #pragma 0.1 0.1 #ifdef 11.3 2.5 #error 0.2 0.1 #else 4.8 1.7 # 0.0 0.1 defined 3.6 1.7 #line 1.4 0.0 #undef 1.0 1.6 Description 1855 A preprocessing directive preprocessingdirective consists of a sequence of preprocessing tokens that begins preprocessing directive with satisfies the following constraints: consists of January 30, 2008 v 1.1

1858 This defines the term preprocessing directive. Preprocessor directives are line oriented (the second half of the above sentence requires directive to start on a line and finish on the same line). The wording was changed and the sentence split up by the response to DR #303. The first token in the sequence is a # preprocessing token that (at the start of translation phase 4) is either the first character in the source file (optionally after white space containing no new-line characters) or that follows white space containing at least one new-line character,. line splicing It is only after preprocessing that the C language becomes is free format. Line splicing occurs prior to translation phase 4 and was created to enable preprocessor directives to span more than one physical source translation phase line. Because preprocessing directives are identified at the start of translation phase 4 translators do not 4 expanded treat the output from macro expansion (which occurs during translation phase 4) as possible preprocessing token sequence directives (or handle other unusual token sequences as preprocessing directive). not treated as a directive The wording was changed, and this sentence formed, by the response to DR #303. EXAMPLE EMPTY # 1868 C90 The C90 Standard did not contain the words (at the start of translation phase 4), which were added by the response to DR #144. Like C90, the Standard does not specify the start of translation phase 4. and is ended by the next The last token in the sequence is the first new-line character that follows the first token in the sequence. 140) The wording was changed, and this sentence formed, by the response to DR #303. 1856 1857 preprocessing directive ended by macro invocation ) terminates it translation phase 3 A new-line character ends the preprocessing directive even if it occurs within what would otherwise be an invocation of a function-like macro. In the following: 1 #define M_1(a, b) ((a) + (b)) 2 #if M_1(1, /* Syntax violation. */ 3 2) == 3 /* Different logical line unconnected with the previous one. */ 4 #endif 5 6 void f(void) 7 { 8 int loc = M_1(1, 9 2); /* No end-of-line syntax requirements here. */ 10 } the #if preprocessing directive visually spans more than one line. The fact that the macro invocation M_1 is split across two physical lines does not alter the requirements on this preprocessing directive (and a translator required to generate a diagnostic). However, the following example is a rather unusual case: 1 #define M_1(a, b) ((a) + (b)) 2 #if M_1(1, /* Comment about the first argument. 3 Comment about the second argument */ 2) == 3 4 #endif in that the comment is replaced by one space character and the new-line character seen by translation phase 4 is the one that occurs after the preprocessing token 3 (i.e., the code is conforming). 1858 v 1.1 January 30, 2008

1863 C90 This wording was not in the C90 Standard, and was added by the response to DR #017. Like the original C90 Standard, the Standard does not explicitly specify this behavior. However, given that vendors are likely to use a preprocessor that is identical to the one used in their C product (or the one that used to be used, if they nolonger market a C product), it is unlikely that the behaviors seen in practice will be different. 1859 140) Thus, preprocessing directives are commonly called lines. footnote 140 The common developer usage is for the term lines to refer to any sequence of characters terminated by a new-line, i.e., the use is not specific, in the developer community, to preprocessing directives. Coding Guidelines Developers are aware of the line-oriented nature of preprocessing directives. However, no terminology (other than preprocessing directive, or simply directive) is commonly used. 1860 These lines have no other syntactic significance, as all white space is equivalent except in certain situations during preprocessing (see the # character string literal creation operator in 6.10.3.2, for example). All source file lines are processed during translation phase 4, either as directives or as token sequences to examine for possible macro replacement. However, the preprocessing directive lines are deleted at the end of translation phase 4 and play no further role in translation. 1861 A text line shall not begin with a # preprocessing token. This is a requirement on the implementation. Syntactically a text-line can start with a # (which is a preprocessing token). This requirement disambiguates the syntax by specifying that such an interpretation shall not be made by the implementation. preprocessing directives deleted 1862 A non-directive shall not begin with any of the directive names appearing in the syntax. This is a requirement on the implementation. Syntactically any sequence of pp-tokens following a # preprocessing token can be a non-directive. This requirement disambiguates the syntax by specifying that such an interpretation (if one exists for the given sequence of preprocessing tokens) shall not be made by the implementation where there is another interpretation. C90 Explicit support for non-directive is new in C99. Explicit support for non-directive is new in C99 and is not discussed in the Standard. 1863 When in a group that is skipped (6.10.1), the directive syntax is relaxed to allow any sequence of preprocessing tokens to occur between the directive name and the following new-line character. This C sentence introduces the term directive name to describe the preprocessing tokens that occur immediately after the # punctuator (they can be given the status of keywords). footnote 141 January 30, 2008 v 1.1

1865 footnote The Committee recognized that the source code contained within a skipped group might still be under 69 development (which is one reason for it being skipped). Given that comments do not nest, it may not be possible, or desirable, to use a comment to prevent translators analyzing the sequence of characters in detail. directive processing while skipping This relaxation of the syntax allows conditional inclusion directive to be used to bracket a sequence of lines that do not yet form a conforming sequence of pp-tokens. 1 #if WORKING_ON_IT 2 3 #if don t know what condition to put here yet 4 #endif 5 6 #endif The directive name needs to be processed to handle any nested conditional preprocessing directives. C90 The C90 Standard did not explicitly specify this behavior. Like C90, the Standard does not explicitly specify this behavior. Common Implementations Many implementations attempt to process skipped groups as quickly as possible. By default, many implementors only examine the minimum number of preprocessing tokens necessary to work out where the skipped group ends. Constraints white-space The only white-space characters that shall appear between preprocessing tokens within a preprocessing within preprocessing directive directive (from just after the introducing # preprocessing token through just before the terminating new-line character) are space and horizontal-tab (including spaces that have replaced comments or possibly other white-space characters in translation phase 3). white-space characters white-space sequence replaced by one macro object-like The use of other white-space characters, (i.e., vertical-tab or form-feed) within preprocessing directives could cause the visual appearance of the source code to suggest that directives are terminated by new-line when in fact they are not. That is the source would be visually confusing and such usage has no obvious benefit. These white-space characters may still appear within preprocessing tokens, e.g., in a character string literal. It is implementation-defined whether other white-space characters are replaced by one space character in translation phase 3. Common Implementations Some implementations support the appearance of a carriage return character immediately prior to the end of line (when that character is not part of the end of line sequence). This enables them to handle text files written using the MS-DOS end of line conventions (i.e., end of line is represented by the two characters \r\n). Coding Guidelines The presence of white-space within a preprocessing directive may or may not be significant. It is sometimes needed (and invariably appears, whether needed or not) to separate the definition of an object-like macro identifier from its body. For instance, in: 1864 1 #define A (x) 2 #define B-y without the intervening white-space the first definition would be for a function-like macro taking a parameter x. No white-space is used in the second definition, but the source would probably be easier to process if some were used. Semantics v 1.1 January 30, 2008

1867 1865 The implementation can process and skip sections of source files conditionally, include other source files, and replace macros. Preprocessing directives (e.g., #pragma) can also affect how other phases of translation behave. These #pragma directive translation phase operations are performed during translation phase four. 4 Coding Guidelines It is possible to completely change the meaning of some C constructs, compared to the behavior that would be deduced purely from their visual appearance in the source code. Constructs can be added, deleted, and modified. All without any indication that these operations will occur, in the source code (or at least the.c files). While various criticisms are made about the behavior of the preprocessor (e.g., it performs no semantic checks on the substitutions it makes) the actual problem lies with the people who use it. The preprocessor might be likened to a sharp knife (some people might say a large hammer), being sharp makes the job of cutting easy but don t complain if sloppy use results in the user being cut (or objects near to the hammer being crushed through poor aiming). In some cases it is possible to provide guideline recommendations that prevent problems occurring, in other cases complexity is intrinsic to the problem being handled. For instance, the heavy use of #if/#endif may be the result of having a successful product that runs in many different environments. While some coding guideline documents recommend that preprocessor usage be minimized, such recommendations are really a design issue and as such are outside the scope of these coding guidelines. 1866 These capabilities are called preprocessing, because conceptually they occur before translation of the resulting preprocessing translation unit. This defines the term preprocessing (the term macro processing is sometimes used by developers, perhaps because macro substitution is felt to have the largest impact on the visible form of the source). This is also a generally used term within software engineering that refers to any set of operations that are performed on the input prior to what are considered to be the main operations (in C s case the subsequent phases of translation). Any later operations are sometimes referred to as post processing. The component of the translator that performs these operations (independent of whether it is a stand alone program or an integrated part of a larger program) is universally known as the preprocessor. The output from preprocessing is a translation unit. The C preprocessor is sometimes used to provide macro processing and conditional compilation in non-c language contexts (e.g., other languages and text processing tasks). Members of the C committee have at times born this usage in mind when considering how the preprocessor should behave. Common Implementations In many implementations preprocessing is carried out by a stand-alone program, usually known as the preprocessor. Coding Guidelines Developers often think of the output of translation stage four as being the output from the preprocessor. In fact the preprocessor is really only translation phase 4, phases 1 3 would need to be performed even if C did not include a preprocessor (the operations these phases perform are performed by languages that do not include a preprocessor). Many guideline recommendations are driven by how developers visually process source code. The recommendations that involve preprocessor constructs are often made because the processing that occurs (e.g., macro expansion) is not directly visible to readers of the source. translation unit known as footnote 5 January 30, 2008 v 1.1

1868 tokens in directive not expanded unless #if macros expanded #include macros expanded #line macros expanded footnote 146 The preprocessing tokens within a preprocessing directive are not subject to macro expansion unless otherwise stated. Requiring that macro expansion unconditionally occur within preprocessing directives is not necessarily a desirable requirement (it would also be a change to existing translator behavior that could break existing code). Most preprocessing directives have a form in which macro expansion can occur. Example 1867 1 #define defined 1 + 2 3 #if defined(x) /* Not expanded. */ 4 #endif EXAMPLE EMPTY # EXAMPLE In: #define EMPTY EMPTY # include <file.h> 1868 the sequence of preprocessing tokens on the second line is not a preprocessing directive, because it does not begin with a # at the start of translation phase 4, even though it will do so after the macro EMPTY has been replaced. This behavior simplifies the job of tools that scan C source (in that they only need to check the first two preprocessing tokens at the start of a line). However, occurrences of EMPTY in other contexts, within a preprocessing directive, may cause complications. 1 #define EMPTY 2 3 #include EMPTY <stdio.h> 4 #include <stdio.h> EMPTY C90 This example was not in the C90 Standard and was added by the response to DR #144. This example does not appear in the Standard. v 1.1 January 30, 2008

References 1. D. Atkins, T. Ball, T. Graves, and A. Mockus. Using version control data to evaluate the impact of software tools. In Proceedings of the 1999 International Conference on Software Engineering (ICSE 99), pages 324 333, New York, May 1999. Association for Computing Machinery. 2. P. J. Brown. The ML/I macro processor. University of Kent at Canterbury, fourth edition, Aug. 1970. 3. P. J. Brown. Macro Processors and Techniques for Portable Software. John Wiley & Sons, 1974. 4. M. D. Ernst, G. J. Badros, and D. Notkin. An empirical analysis of C preprocessor use. IEEE Transactions on Software Engineering, 28(12):1146 1170, 2002. 5. J. Gosling. Ace: a syntax-driven C preprocessor. In Australian Unix Users Group, July 1989. 6. ISO. ISO/IEC 1539-3:1998 Information technology Programming languages FORTRAN Part 3: Conditional compilation. ISO, 1998. 7. R. Pike. How to use the Plan 9 C compiler. In Plan 9 Programmer s Manual. AT&T Bell Laboratories, 1995. 8. L. Rosler. The evolution of C past and future. AT&T Bell Laboratories Technical Journal, 63(8):1685 1699, 1984. 9. Texas Instruments. TMS370 and TMS370C8 8-Bit Microcontroller Family Optimizing C Compiler Users Guide. Texas Instruments, spnu022c edition, Apr. 1996. 10. Unix System Laboratories, Inc. Programmer s Guide: ANSI C Programming Support Tools. Prentice Hall, Inc, 1990. 11. E. D. Willink and V. B. Muchnick. Preprocessing : Substitution and composition. In Proceedings of the Eastern European Conference on the Technology of Object Oriented Languages and Systems, June 1999. January 30, 2008 v 1.1