flexc++input(7) | flexc++ input file organization | flexc++input(7) |
flexc++input - Organization of flexc++’s input s
Flexc++(1) was designed after flex(1) and flex++(1). Like these two programs flexc++ generates code performing pattern-matching on text, possibly executing actions when certain regular expressions are recognized.
Refer to flexc++(1) for a general overview. This manual page describes how flexc++’s input s should be organized. It contains the following sections:
Starting with version 2.07.00 flexc++ reserved identifiers no longer end in two underscore characters, but in one. This modification was necessary because according to the C++ standard identifiers having two or more consecutive underscore characters are reserved by the language. In practice this could require some minor modifications of existing source files using flexc++’s facilities, most likely limited to changing StartCondition__ into StartCondition_ and changing PostEnum__ into PostEnum_.
The complete list of affected names is:
Flexc++ expects an input file containing directives and the regular expressions that should be recognized by objects of the scanner class generated by flexc++. In this man page the elements and organization of flexc++’s input file is described.
Flexc++’s input file consists of two sections, separated from each other by a line merely containing two consecutive percent characters:
%%The section before this separator contains directives; the section following this separator contains regular expressions and possibly actions to perform when these regular expressions are matched by the object of the scanner class generated by flexc++. If a second line is encountered immediately beginning with two consecutive percent characters then this ends flexc++’s input file processing. See also section 6 (%% SEPARATOR) below.
White space is usually ignored, as is comment, which may be of the traditional C form (i.e., /*, followed by (possibly multi-line) comment text, followed by */, and it may be C++ end-of-line comment: two consecutive slashes (//) start the comment, which continues up to the next newline character.
Flexc++’s input file may be split into multiple files. This allows for the definition of logically separate elements of the specifications in different files. Include directives must be specified on separate lines and may not contain any other information than the (path)name of the file to switch to. File names may be surrounded by double quotes, but these double quotes are optional and are ignored (removed) when encountered. All remaining characters define the name of the subsequently processed file by flexc++. White space characters following //include and preceding the end of the line are ignored. To switch files the following stanza is used:
//include file-location
The //include directive must start in the line’s first column. File locations can be absolute or relative to the location of the file containing the //include directive. Once flexc++ has switched to another file that file’s directory becomes flexc++’s working directory.
Once the end of file of a file has been reached, processing continues at the line beyond the //include directive of the previously scanned file, and flexc++’s working directory is reset to the working directory of the file to which flexc++ returns. The end-of-file of the file that was initially specified when flexc++ started indicates the end of flexc++’s rules specification.
The first section of flexc++’s input file consists of directives. In addition it may associate regular expressions with symbolic names, allowing you to use these identifiers in the rules section. Each directive is defined on a line of its own. When available, directives are overridden by flexc++ command line options.
Some directives require arguments, which are usually provided following separating (but optional) = characters. Arguments of directives are text, surrounded by double quotes (strings), or embedded in raw string literals (rawstrings). Double quotes or backslashes inside strings must themselves be preceded by backslashes; these backslashes are not required when rawstrings are used.
The %s and %x directives are immediately followed by name lists, consisting of identifiers separated by blanks. Here is an example of the definition of a directive:
%class-name = "MyScanner"
Directives accepting a `filename’ do not accept path names, i.e., they cannot contain directory separators (/); options accepting a ’pathname’ may contain directory separators. A ’pathname’ using blank characters should be surrounded by double quotes.
Some directives may generate errors. This happens when a directive conflicts with the contents of an existing file which flexc++ cannot modify (e.g., a scanner class header file exists, but doesn’t define a name space, but a %namespace directive was provided). To solve the error the offending directive could be omitted, the existing file could be removed, or the existing file could be hand-edited according to the directive’s specification. Note that flexc++ currently does not handle the opposite error condition: if a previously used directive is omitted, then flexc++ does not detect the inconsistency. In those cases you may encounter compilation errors.
With a case-insensitive scanner only the first rule can be matched, and flexc++ will issue warnings for the second and third rule about rules that cannot be matched.
First // initial F is transformed to f
first
FIRST // all capitals are transformed to lower case chars
the names of the generated files are, respectively, scanner.h, scanner.ih, and scannerbase.h. Corresponding command-line option: --filenames. The name of the source file (by default lex.cc) is controlled by the %lex-source directive.
%filenames = "scanner"
Mini scanners come in two flavors: inclusive mini scanners and exclusive mini scanners. The rules that apply to an inclusive mini scanner are the mini scanner’s own rules as well as the rules which apply to no mini scanners in particular (i.e., the rules that apply to the default (or INITIAL) mini scanner). Exclusive mini scanners only use the rules that were defined for them.
To define an inclusive mini scanner use %s, followed by one or more identifiers specifying the name(s) of the mini-scanner(s). To define an exclusive mini scanner use %x, followed by or more identifiers specifying the name(s) of the mini-scanner(s). The following example defines the names of two mini scanners: string and comment:
Following this, rules defined in the context of the string mini scanner (see below) will only be used when that mini scanner is active.
%x string comment
A flexc++ input file may contain multiple %s and %x specifications.
Definitions are of the form
identifier regular-expression
Each definition must be entered on a line of its own. Definitions associate identifiers with regular expressions, allowing the use of ${identifier} as synonym for its regular expression in the rules section of flexc++’s input file. One defined, the identifiers representing regular expressions can also be used in subsequent definitions.
Example:
FIRST [A-Za-z_] NAME {FIRST}[-A-Za-z0-9_]*
Following directives and definitions a line merely containing two consecutive % characters is expected. Following this line the rules are defined. Rules consist of regular expressions which should be recognized, possibly followed by actions to be executed once a rule’s regular expression has been matched.
If the rule section contains a line starting with two consecutive % characters, then any remaining input is ignored. Note that this second %% separator does not have to be specified. It is purely optional. To specify a regular expression starting with %% surround the %% with double quotes ("%%") or prefix the %% with a blank space: the %%-characters are only considered a separator if they are encountered at the very beginning of a line.
The regular expressions defined in flexc++’s rules files are matched against the information passed to the scanner’s lex function.
Regular expressions begin as the first non-blank character on a line. Comment is interpreted as comment as long as it isn’t part of the regular expresssion. To define a regular expression starting with two slashes (at least) the first slash can be escaped or double quoted. (E.g., "//".* defines C++ comment to end-of-line).
Regular expressions end at the first blank character (to add a blank character, e.g., a space character, to a regular expression, prefix it by a backslash or put it in a double-quoted string).
Actions may be associated with regular expressions. At a match the action that is associated with the regular expression is executed, after which scanning continues when the lexical scanning function (e.g., lex) is called again. Actions are not required, and regular expressions can be defined without any actions at all. If such action-less regular expressions are matched then the match is performed silently, after which processing continues.
Flexc++ tries to match as many characters of the input file as possible (i.e., it uses `greedy matching’). Non-greedy matching is accomplished by a combination of a scanner and parser and/or by using the `lookahead’ operator (/).
The following regular expression `building blocks’ are available. More complex regular expressions are created by combining them:
By placing the comment
[Warning] input, line 7: null-matching regular expression
on the line just before a regular expression that potentially does not match any text, the warning for that regular expression is suppressed;
//%nowarn
Character classes
Inside a character class all regular expression operators lose their special meanings, except for the escape character (\), the character range operator -, the end of character class operator ], and, at the beginning of the class, ^. All ordinary escape sequences are supported, all other escaped characters are interpreted as literal characters (e.g., \c is a literal c).
To add a closing bracket to a character class use [] or \]. To add a closing bracket to a negated character class use [^] (or use [^ followed by \] somewhere within the character class). Minus characters are used to define character ranges (e.g., [a-d], defining [abcd]) except in the following cases, where flexc++ recognizes a literal minus character: [-, or [^- (a minus at the very beginning of a character class); -] (a minus at the very end of a character class); or \- (an escaped minus character)) Once a character class has started, all subsequent character (ranges) are added to the set, until the final closing bracket (]) has been reached.
Character constants
Character constants are surrounded by single quote characters. They match single characters which, however, can be specified in various ways.
’\n’ return 1;
\n return 2;
Considering the above, to match character (in this example: except for the newline character) including its surrounding quotes a regular expression consisting of an escaped quote character, followed by any character, followed by a quote character can be used:
\’.’ // matches characters surrounded by quotes
Operator precedence
The regular expressions listed above are grouped according to precedence, from highest precedence at the top to lowest at the bottom. From lowest to highest precedence, the operators are:
The lex standard defines concatenation as having a higher precedence than the interval expression. This is different from many other regular expression engines, and flexc++ follows these latter engines, giving all `multiplication operators’ equal priority.
Name expansion has the same precedence as grouping (using parentheses to influence the precedence of the other operators in the regular expression). Since the name expansion is treated as a group in flexc++, it is not allowed to use the lookahead operator in a name definition (a named pattern, defined in the definition section).
Predefined sets of characters
Character classes can also contain character class expressions. These are expressions enclosed inside [: and :] delimiters (which themselves must appear between the [ and ] of the character class. Other elements may occur inside the character class as well). The character class expressions are:
[:alnum:] [:alpha:] [:blank:]
[:cntrl:] [:digit:] [:graph:]
[:lower:] [:print:] [:punct:]
[:space:] [:upper:] [:xdigit:]
Character class expressions designate a set of characters equivalent to the corresponding standard C isXXX function. For example, [:alnum:] designates those characters for which isalnum returns true - i.e., any alphabetic or numeric character. For example, the following character classes are all equivalent:
[[:alnum:]]
[[:alpha:][:digit:]]
[[:alpha:][0-9]]
[a-zA-Z0-9]
A negated character class such as the example [^A-Z] above will match a newline unless \n (or an equivalent escape sequence) is one of the characters explicitly present in the negated character class (e.g., [^A-Z\n]). This differs from the way many other regular expression tools treat negated character classes, but unfortunately the inconsistency is historically entrenched. Matching newlines means that a pattern like [^"]* can match the entire input unless there’s another quote in the input.
Flexc++ allows negation of character class expressions by prepending ^ to the POSIX character class name.
[:^alnum:] [:^alpha:] [:^blank:]
[:^cntrl:] [:^digit:] [:^graph:]
[:^lower:] [:^print:] [:^punct:]
[:^space:] [:^upper:] [:^xdigit:]
Combining character sets
The {-} operator computes the difference of two character classes. For example, [a-c]{-}[b-z] represents all the characters in the class [a-c] that are not in the class [b-z] (which in this case, is just the single character a). The {-} operator is left associative, so [abc]{-}[b]{-}[c] is the same as [a].
The {+} operator computes the union of two character classes. For example, [a-z]{+}[0-9] is the same as [a-z0-9]. This operator is useful when preceded by the result of a difference operation, as in, [[:alpha:]]{-}[[:lower:]]{+}[q], which is equivalent to [A-Zq] in the C locale.
Trailing context
A rule can have at most one instance of trailing context (the / operator or the $ operator). The start condition, ^, and <<EOF>> patterns can only occur at the beginning of a pattern, and cannot be surrounded by parentheses. The characters ^ and $ only have their special properties at, respectively, the beginning and end of regular expressions. In all other cases they are treated as a normal characters.
%option debug %x comment NAME [[:alpha:]][_[:alnum:]]* %% "//".* // ignore "/*" begin(StartCondition_::comment); <comment>.|\n // ignore <comment>"*/" begin(StartCondition_::INITIAL); ^a return 1; a return 2; a$ return 3; {NAME} return 4; .|\n // ignore
)
Flexc++’s default skeleton files are in
/usr/share/flexc++.
By default, flexc++ generates the following files:
flexc++(1), flexc++api(3)
This is free software, distributed under the terms of the GNU General Public License (GPL).
Frank B. Brokken (f.b.brokken@rug.nl),
Jean-Paul van Oosten (j.p.van.oosten@rug.nl),
Richard Berendsen (richardberendsen@xs4all.nl) (until 2010).
2008-2022 | flexc++.2.11.02 |