Path: utzoo!attcan!uunet!husc6!uwvax!oddjob!ncar!ames!lll-tis!helios.ee.lbl.gov!pasteur!ucbvax!hplabs!felix!dhw68k!macintosh
From: earleh@eleazar.dartmouth.edu (Earle R. Horton)
Newsgroups: comp.sources.mac
Subject: Flex for the Mac (part 2 of 7)
Message-ID: <9368@dhw68k.cts.com>
Date: 30 Jun 88 05:00:31 GMT
References: <9364@dhw68k.cts.com>
Sender: macintosh@dhw68k.cts.com
Organization: Dartmouth College, Hanover, NH
Lines: 1049
Approved: bytebug@dhw68k.cts.com (Roger L. Long)

[Flex for the Mac - part 2 of 7]

---
#! /bin/sh
# This is a shell archive, meaning:
# 1. Remove everything above the #! /bin/sh line.
# 2. Save the resulting text in a file.
# 3. Execute the file with /bin/sh (not csh) to create the files:
#	flex/flex.1
#	flex/flex.man.page
# This archive created: Wed Jun 29 07:03:13 1988
# By:	Roger L. Long (bytebug@dhw68k.cts.com)
export PATH; PATH=/bin:$PATH
echo shar: extracting "'flex.1'" '(15558 characters)'
if test -f 'flex.1'
then
	echo shar: will not over-write existing file "'flex.1'"
else
sed 's/^X//' << \SHAR_EOF > 'flex.1'
X.TH FLEX 1 "13 May 1987"
X.SH NAME
Xflex - fast lexical analyzer generator
X.SH SYNOPSIS
X.B flex
X[
X.B -dfirstvFILT -c[efmF] -Sskeleton_file
X] [ 
X.I filename
X]
X.SH DESCRIPTION
X.I flex
Xis a rewrite of
X.I lex
Xintended to right some of that tool's deficiencies: in particular,
X.I flex
Xgenerates lexical analyzers much faster, and the analyzers use
Xsmaller tables and run faster.
X.SH OPTIONS
XIn addition to lex's
X.B -t
Xflag, flex has the following options:
X.TP
X.B -d
Xmakes the generated scanner run in
X.I debug
Xmode.  Whenever a pattern is recognized the scanner will
Xwrite to
X.I stderr
Xa line of the form:
X.nf
X
X    --accepting rule #n
X
X.fi
XRules are numbered sequentially with the first one being 1.
X.TP
X.B -f
Xhas the same effect as lex's -f flag (do not compress the scanner
Xtables); the mnemonic changes from
X.I fast compilation
Xto (take your pick)
X.I full table
Xor
X.I fast scanner.
XThe actual compilation takes
X.I longer,
Xsince flex is I/O bound writing out the big table.
X.IP
XThis option is equivalent to
X.B -cf
X(see below).
X.TP
X.B -i
Xinstructs flex to generate a
X.I case-insensitive
Xscanner.  The case of letters given in the flex input patterns will
Xbe ignored, and the rules will be matched regardless of case.  The
Xmatched text given in
X.I yytext
Xwill have the preserved case (i.e., it will not be folded).
X.TP
X.B -r
Xspecifies that the scanner uses the
X.B REJECT
Xaction.
X.TP
X.B -s
Xcauses the
X.I default rule
X(that unmatched scanner input is echoed to
X.I stdout)
Xto be suppressed.  If the scanner encounters input that does not
Xmatch any of its rules, it aborts with an error.  This option is
Xuseful for finding holes in a scanner's rule set.
X.TP
X.B -v
Xhas the same meaning as for lex (print to
X.I stderr
Xa summary of statistics of the generated scanner).  Many more statistics
Xare printed, though, and the summary spans several lines.  Most
Xof the statistics are meaningless to the casual flex user.
X.TP
X.B -F
Xspecifies that the
X.ul
Xfast
Xscanner table representation should be used.  This representation is
Xabout as fast as the full table representation
X.ul
X(-f),
Xand for some sets of patterns will be considerably smaller (and for
Xothers, larger).  In general, if the pattern set contains both "keywords"
Xand a catch-all, "identifier" rule, such as in the set:
X.nf
X
X	"case"    return ( TOK_CASE );
X	"switch"  return ( TOK_SWITCH );
X	...
X	"default" return ( TOK_DEFAULT );
X	[a-z]+    return ( TOK_ID );
X
X.fi
Xthen you're better off using the full table representation.  If only
Xthe "identifier" rule is present and you then use a hash table or some such
Xto detect the keywords, you're better off using
X.ul
X-F.
X.IP
XThis option is equivalent to
X.B -cF
X(see below).
X.TP
X.B -I
Xinstructs flex to generate an
X.I interactive
Xscanner.  Normally, scanners generated by flex always look ahead one character
Xbefore deciding that a rule has been matched.  At the possible cost of some
Xscanning overhead (it's not clear that more overhead is involved), flex will
Xgenerate a scanner which only looks ahead when needed.  Such scanners are
Xcalled
X.I interactive
Xbecause if you want to write a scanner for an interactive system such
Xas a command shell, you will probably want the user's input to be terminated
Xwith a newline, and without
X.B -I
Xthe user will have to type a character in addition to the newline in order
Xto have the newline recognized.  This leads to dreadful interactive performance.
X.IP
XIf all this seems to confusing, here's the general rule: if a human will
Xbe typing in input to your scanner, use
X.B -I,
Xotherwise don't; if you don't care about how fast your scanners run and
Xdon't want to make any assumptions about the input to your scanner,
Xalways use
X.B -I.
X.IP
XNote,
X.B -I
Xcannot be used in conjunction with
X.I full
Xor
X.I fast tables,
Xi.e., the
X.B -f, -F, -cf,
Xor
X.B -cF
Xflags.
X.TP
X.B -L
Xinstructs flex to not generate
X.B #line
Xdirectives (see below).
X.TP
X.B -T
Xmakes flex run in
X.I trace
Xmode.  It will generate a lot of messages to standard out concerning
Xthe form of the input and the resultant non-deterministic and deterministic
Xfinite automatons.  This option is mostly for use in maintaining flex.
X.TP 
X.B -c[efmF]
Xcontrols the degree of table compression.
X.B -ce
Xdirects flex to construct
X.I equivalence classes,
Xi.e., sets of characters
Xwhich have identical lexical properties (for example, if the only
Xappearance of digits in the flex input is in the character class
X"[0-9]" then the digits '0', '1', ..., '9' will all be put
Xin the same equivalence class).
X.B -cf
Xspecifies that the
X.I full
Xscanner tables should be generated - flex should not compress the
Xtables by taking advantages of similar transition functions for
Xdifferent states.
X.B -cF
Xspecifies that the alternate fast scanner representation (described
Xabove under the
X.B -F
Xflag)
Xshould be used.
X.B -cm
Xdirects flex to construct
X.I meta-equivalence classes,
Xwhich are sets of equivalence classes (or characters, if equivalence
Xclasses are not being used) that are commonly used together.
XA lone
X.B -c
Xspecifies that the scanner tables should be compressed but neither
Xequivalence classes nor meta-equivalence classes should be used.
X.IP
XThe options
X.B -cf
Xor
X.B -cF
Xand
X.B -cm
Xdo not make sense together - there is no opportunity for meta-equivalence
Xclasses if the table is not being compressed.  Otherwise the options
Xmay be freely mixed.
X.IP
XThe default setting is
X.B -cem
Xwhich specifies that flex should generate equivalence classes
Xand meta-equivalence classes.  This setting provides the highest
Xdegree of table compression.  You can trade off
Xfaster-executing scanners at the cost of larger tables with
Xthe following generally being true:
X.nf
X
X    slowest            smallest
X               -cem
X               -ce
X               -cm
X               -c
X               -c{f,F}e
X               -c{f,F}
X    fastest            largest
X
X.fi
X.TP
X.B -Sskeleton_file
Xoverrides the default skeleton file from which flex constructs
Xits scanners.  You'll never need this option unless you are doing
Xflex maintenance or development.
X.SH INCOMPATIBILITIES WITH LEX
X.I flex
Xis fully compatible with
X.I lex
Xwith the following exceptions:
X.IP -
XThere is no run-time library to link with.  You needn't
Xspecify
X.I -ll
Xwhen linking, and you must supply a main program.  (Hacker's note: since
Xthe lex library contains a main() which simply calls yylex(), you actually
X.I can
Xbe lazy and not supply your own main program and link with
X.I -ll.)
X.IP -
Xlex's
X.B %r
X(Ratfor scanners) and
X.B %t
X(translation table) options
Xare not supported.
X.IP -
XThe do-nothing
X.ul
X-n
Xflag is not supported.
X.IP -
XWhen definitions are expanded, flex encloses them in parentheses.
XWith lex, the following
X.nf
X
X    NAME    [A-Z][A-Z0-9]*
X    %%
X    foo{NAME}?      printf( "Found it\\n" );
X    %%
X
X.fi
Xwill not match the string "foo" because when the macro
Xis expanded the rule is equivalent to "foo[A-Z][A-Z0-9]*?"
Xand the precedence is such that the '?' is associated with
X"[A-Z0-9]*".  With flex, the rule will be expanded to
X"foo([A-z][A-Z0-9]*)?" and so the string "foo" will match.
X.IP -
X.B yymore()
Xis not supported.
X.IP -
XThe undocumented lex-scanner internal variable
X.B yylineno
Xis not supported.
X.IP -
XIf your input uses
X.B REJECT,
Xyou must run flex with the
X.B -r
Xflag.  If you leave out the flag, the scanner will abort at run-time
Xwith a message that the scanner was compiled without the flag being
Xspecified.
X.IP -
XThe
X.B input()
Xroutine is not redefinable, though may be called to read characters
Xfollowing whatever has been matched by a rule.  If
X.B input()
Xencounters and end-of-file the normal
X.B yywrap()
Xprocessing is done.  A ``real'' end-of-file is returned as
X.I EOF.
X.IP
XInput can be controlled by redefining the
X.B YY_INPUT
Xmacro.
XYY_INPUT's calling sequence is "YY_INPUT(buf,result,max_size)".  Its
Xaction is to place up to max_size characters in the character buffer "buf"
Xand return in the integer variable "result" either the
Xnumber of characters read or the constant YY_NULL (0 on Unix systems)
Xsystems) to indicate EOF.  The default YY_INPUT reads from the
Xfile-pointer "yyin" (which is by default
X.I stdin),
Xso if you
Xjust want to change the input file, you needn't redefine
XYY_INPUT - just point yyin at the input file.
X.IP
XA sample redefinition of YY_INPUT (in the first section of the input
Xfile):
X.nf
X
X    %{
X    #undef YY_INPUT
X    #define YY_INPUT(buf,result,max_size) \\
X        result = (buf[0] = getchar()) == EOF ? YY_NULL : 1;
X    %}
X
X.fi
XYou also can add in things like counting keeping track of the
Xinput line number this way; but don't expect your scanner to
Xgo very fast.
X.IP -
X.B output()
Xis not supported.
XOutput from the ECHO macro is done to the file-pointer
X"yyout" (default
X.I stdout).
X.IP -
XTrailing context is restricted to patterns which have either
Xa fixed-sized leading part or a fixed-sized trailing part.
XFor example, "a*/b" and "a/b*" are okay, but not "a*/b*".
XThis restriction is due to a bug in the trailing context
Xalgorithm given in
X.I Principles of Compiler Design
X(and
X.I Compilers - Principles, Techniques, and Tools)
Xwhich can result in mismatches.  Try the following lex program
X.nf
X
X    %%
X    x+/xy           printf( "I found \\"%s\\"\\n", yytext );
X
X.fi
Xon the input "xxy".  (If anyone knows of a fast algorithm for
Xfinding the beginning of trailing context for an arbitrary
Xpair of regular expressions, please let me know!)
XIf you must have arbitrary trailing context, you can use
X.B yyless()
Xto effect it.
X.IP -
Xflex reads only one input file, while lex's input is made
Xup of the concatenation of its input files.
X.SH ENHANCEMENTS
X.IP -
X.I Exclusive start-conditions
Xcan be declared by using
X.B %x
Xinstead of
X.B %s.
XThese start-conditions have the property that when they are active,
X.I no other rules are active.
XThus a set of rules governed by the same exclusive start condition
Xdescribe a scanner which is independent of any of the other rules in
Xthe flex input.  This feature makes it easy to specify "mini-scanners"
Xwhich scan portions of the input that are syntactically different
Xfrom the rest (e.g., comments).
X.IP -
Xflex dynamically resizes its internal tables, so directives like "%a 3000"
Xare not needed when specifying large scanners.
X.IP -
XThe scanning routine generated by flex is declared using the macro
X.B YY_DECL.
XBy redefining this macro you can change the routine's name and
Xits calling sequence.  For example, you could use:
X.nf
X
X    #undef YY_DECL
X    #define YY_DECL float lexscan( a, b ) float a, b;
X
X.fi
Xto give it the name
X.I lexscan,
Xreturning a float, and taking two floats as arguments.
X.IP -
Xflex generates
X.B #line
Xdirectives mapping lines in the output to
Xtheir origin in the input file.
X.IP -
XYou can put multiple actions on the same line, separated with
Xsemi-colons.  With lex, the following
X.nf
X
X    foo    handle_foo(); return 1;
X
X.fi
Xis truncated to
X.nf
X
X    foo    handle_foo();
X
X.fi
Xflex does not truncate the action.  Actions that are not enclosed in
Xbraces are terminated at the end of the line.
X.IP -
XActions can be begun with
X.B %{
Xand terminated with
X.B %}.
XIn this case, flex does not count braces to figure out where the
Xaction ends - actions are terminated by the closing
X.B %}.
XThis feature is useful when the enclosed action has extraneous
Xbraces in it (usually in comments or inside inactive #ifdef's)
Xthat throw off the brace-count.
X.IP -
XAll of the scanner actions (e.g.,
X.B ECHO, yywrap ...)
Xexcept the
X.B unput()
Xand
X.B input()
Xroutines,
Xare written as macros, so they can be redefined if necessary
Xwithout requiring a separate library to link to.
X.SH FILES
X.TP
X.I flex.skel
Xskeleton scanner
X.TP
X.I flex.fastskel
Xskeleton scanner for -f and -F
X.TP
X.I flexskelcom.h
Xcommon definitions for skeleton files
X.TP
X.I flexskeldef.h
Xdefinitions for compressed skeleton file
X.TP
X.I fastskeldef.h
Xdefinitions for -f, -F skeleton file
X.SH "SEE ALSO"
X.LP
Xlex(1)
X.LP
XM. E. Lesk and E. Schmidt,
X.I LEX - Lexical Analyzer Generator
X.SH AUTHOR
XVern Paxson, with the help of many ideas and much inspiration from
XVan Jacobson.  Original version by Jef Poskanzer.  Fast table
Xrepresentation is a partial implementation of a design done by Van
XJacobson.  The implementation was done by Kevin Gong and Vern Paxson.
X.LP
XThanks to the many flex beta-testers, especially Casey Leedom,
XNick Christopher, Chris Faylor, Eric Goldman, Craig Leres, Mohamed el Lozy,
XEsmond Pitt, Jef Poskanzer, and Dave Tallman.  Thanks to John Gilmore,
XBob Mulcahy,
XRich Salz, and Richard Stallman for help with various distribution headaches.
X.LP
XSend comments to:
X.nf
X
X     Vern Paxson
X     Real Time Systems
X     Bldg. 46A
X     Lawrence Berkeley Laboratory
X     1 Cyclotron Rd.
X     Berkeley, CA 94720
X
X     (415) 486-6411
X
X     vern@lbl-{csam,rtsg}.arpa
X     ucbvax!lbl-csam.arpa!vern
X
X.fi
X.SH DIAGNOSTICS
X.LP
X.I flex scanner jammed -
Xa scanner compiled with
X.B -s
Xhas encountered an input string which wasn't matched by
Xany of its rules.
X.LP
X.I flex input buffer overflowed -
Xa scanner rule matched a string long enough to overflow the
Xscanner's internal input buffer (as large as
X.B BUFSIZ
Xin "/usr/include/stdio.h").  You can edit
X.I flexskelcom.h
Xand increase
X.B YY_BUF_SIZE
Xand
X.B YY_MAX_LINE
Xto increase this limit.
X.LP
X.I REJECT used and scanner was
X.I not generated using -r -
Xjust like it sounds.  Your scanner uses
X.B REJECT.
XYou must run flex on the scanner description using the
X.B -r
Xflag.
X.LP
X.I old-style lex command ignored -
Xthe flex input contains a lex command (e.g., "%n 1000") which
Xis being ignored.
X.SH BUGS
X.LP
XUse of unput() or input() trashes the current yytext and yyleng.
X.LP
XUse of unput() to push back more text than was matched can
Xresult in the pushed-back text matching a beginning-of-line ('^')
Xrule even though it didn't come at the beginning of the line.
X.LP
XNulls are not allowed in flex inputs or in the inputs to
Xscanners generated by flex.  Their presence generates fatal
Xerrors.
X.LP
XDo not mix trailing context with the '|' operator used to
Xspecify that multiple rules use the same action.  That is,
Xavoid constructs like:
X.nf
X
X        foo/bar      |
X        bletch       |
X        bugprone     { ... }
X
X.fi
XThey can result in subtle mismatches.  This is actually not
Xa problem if there is only one rule
Xusing trailing context and it is the first in the list (so the
Xabove example will actually work okay).  The
Xproblem is due to fall-through in the action switch statement,
Xcausing non-trailing-context rules to execute the
Xtrailing-context code of their fellow rules.  This should
Xbe fixed, as it's a nasty bug and not obvious.  The proper fix is
Xfor flex to spit out a FLEX_TRAILING_CONTEXT_USED #define and then
Xhave the backup logic in a separate table which is consulted for
Xeach rule-match, rather than as part of the rule action.  The
Xplace to do the tweaking is in add_accept() - any kind soul want
Xto be a hero?
X.LP
XThe pattern:
X.nf
X
X	x{3}
X
X.fi
Xis considered to be variable-length for the purposes of trailing
Xcontext, even though it has a clear fixed length.
X.LP
XDue to both buffering of input and read-ahead, you cannot intermix
Xcalls to, for example,
X.B getchar()
Xwith flex rules and expect it to work.  Call
X.B input()
Xinstead.
X.LP
XThe total table entries listed by the
X.B -v
Xflag excludes the number of table entries needed to determine
Xwhat rule has been matched.  The number of entries is equal
Xto the number of DFA states if the scanner was not compiled
Xwith
X.B -r,
Xand greater than the number of states if it was.
X.LP
XThe scanner run-time speeds have not been optimized as much
Xas they deserve.  Van Jacobson's work shows that the can go quite
Xa bit faster still.
SHAR_EOF
if test 15558 -ne "`wc -c < 'flex.1'`"
then
	echo shar: error transmitting "'flex.1'" '(should have been 15558 characters)'
fi
fi # end of overwriting check
echo shar: extracting "'flex.man.page'" '(17746 characters)'
if test -f 'flex.man.page'
then
	echo shar: will not over-write existing file "'flex.man.page'"
else
sed 's/^X//' << \SHAR_EOF > 'flex.man.page'
X
X
X
XFLEX(1)             UNIX Programmer's Manual              FLEX(1)
X
X
X
XNAME
X     flex - fast lexical analyzer generator
X
XSYNOPSIS
X     flex [ -dfirstvFILT -c[efmF] -Sskeleton_file ] [ filename ]
X
XDESCRIPTION
X     flex is a rewrite of lex intended to right some of that
X     tool's deficiencies: in particular, flex generates lexical
X     analyzers much faster, and the analyzers use smaller tables
X     and run faster.
X
XOPTIONS
X     In addition to lex's -t flag, flex has the following
X     options:
X
X     -d   makes the generated scanner run in debug mode.  When-
X          ever a pattern is recognized the scanner will write to
X          stderr a line of the form:
X
X              --accepting rule #n
X
X          Rules are numbered sequentially with the first one
X          being 1.
X
X     -f   has the same effect as lex's -f flag (do not compress
X          the scanner tables); the mnemonic changes from fast
X          compilation to (take your pick) full table or fast
X          scanner. The actual compilation takes longer, since
X          flex is I/O bound writing out the big table.
X
X          This option is equivalent to -cf (see below).
X
X     -i   instructs flex to generate a case-insensitive scanner.
X          The case of letters given in the flex input patterns
X          will be ignored, and the rules will be matched regard-
X          less of case.  The matched text given in yytext will
X          have the preserved case (i.e., it will not be folded).
X
X     -r   specifies that the scanner uses the REJECT action.
X
X     -s   causes the default rule (that unmatched scanner input
X          is echoed to stdout) to be suppressed.  If the scanner
X          encounters input that does not match any of its rules,
X          it aborts with an error.  This option is useful for
X          finding holes in a scanner's rule set.
X
X     -v   has the same meaning as for lex (print to stderr a sum-
X          mary of statistics of the generated scanner).  Many
X          more statistics are printed, though, and the summary
X          spans several lines.  Most of the statistics are mean-
X          ingless to the casual flex user.
X
X     -F   specifies that the fast scanner table representation
X          should be used.  This representation is about as fast
X          as the full table representation (-f), and for some
X          sets of patterns will be considerably smaller (and for
X          others, larger).  In general, if the pattern set con-
X          tains both "keywords" and a catch-all, "identifier"
X          rule, such as in the set:
X
X               "case"    return ( TOK_CASE );
X               "switch"  return ( TOK_SWITCH );
X               ...
X               "default" return ( TOK_DEFAULT );
X               [a-z]+    return ( TOK_ID );
X
X          then you're better off using the full table representa-
X          tion.  If only the "identifier" rule is present and you
X          then use a hash table or some such to detect the key-
X          words, you're better off using -F.
X
X          This option is equivalent to -cF (see below).
X
X     -I   instructs flex to generate an interactive scanner.
X          Normally, scanners generated by flex always look ahead
X          one character before deciding that a rule has been
X          matched.  At the possible cost of some scanning over-
X          head (it's not clear that more overhead is involved),
X          flex will generate a scanner which only looks ahead
X          when needed.  Such scanners are called interactive
X          because if you want to write a scanner for an interac-
X          tive system such as a command shell, you will probably
X          want the user's input to be terminated with a newline,
X          and without -I the user will have to type a character
X          in addition to the newline in order to have the newline
X          recognized.  This leads to dreadful interactive perfor-
X          mance.
X
X          If all this seems to confusing, here's the general
X          rule: if a human will be typing in input to your
X          scanner, use -I, otherwise don't; if you don't care
X          about how fast your scanners run and don't want to make
X          any assumptions about the input to your scanner, always
X          use -I.
X
X          Note, -I cannot be used in conjunction with full or
X          fast tables, i.e., the -f, -F, -cf, or -cF flags.
X
X     -L   instructs flex to not generate #line directives (see
X          below).
X
X     -T   makes flex run in trace mode.  It will generate a lot
X          of messages to standard out concerning the form of the
X          input and the resultant non-deterministic and
X          deterministic finite automatons.  This option is mostly
X          for use in maintaining flex.
X
X     -c[efmF]
X          controls the degree of table compression.  -ce directs
X          flex to construct equivalence classes, i.e., sets of
X          characters which have identical lexical properties (for
X          example, if the only appearance of digits in the flex
X          input is in the character class "[0-9]" then the digits
X          '0', '1', ..., '9' will all be put in the same
X          equivalence class).  -cf specifies that the full
X          scanner tables should be generated - flex should not
X          compress the tables by taking advantages of similar
X          transition functions for different states.  -cF speci-
X          fies that the alternate fast scanner representation
X          (described above under the -F flag) should be used.  -
X          cm directs flex to construct meta-equivalence classes,
X          which are sets of equivalence classes (or characters,
X          if equivalence classes are not being used) that are
X          commonly used together.  A lone -c specifies that the
X          scanner tables should be compressed but neither
X          equivalence classes nor meta-equivalence classes should
X          be used.
X
X          The options -cf or -cF and -cm do not make sense
X          together - there is no opportunity for meta-equivalence
X          classes if the table is not being compressed.  Other-
X          wise the options may be freely mixed.
X
X          The default setting is -cem which specifies that flex
X          should generate equivalence classes and meta-
X          equivalence classes.  This setting provides the highest
X          degree of table compression.  You can trade off
X          faster-executing scanners at the cost of larger tables
X          with the following generally being true:
X
X              slowest            smallest
X                         -cem
X                         -ce
X                         -cm
X                         -c
X                         -c{f,F}e
X                         -c{f,F}
X              fastest            largest
X
X
X     -Sskeleton_file
X          overrides the default skeleton file from which flex
X          constructs its scanners.  You'll never need this option
X          unless you are doing flex maintenance or development.
X
XINCOMPATIBILITIES WITH LEX
X     flex is fully compatible with lex with the following excep-
X     tions:
X
X     -    There is no run-time library to link with.  You needn't
X          specify -ll when linking, and you must supply a main
X          program.  (Hacker's note: since the lex library con-
X          tains a main() which simply calls yylex(), you actually
X          can be lazy and not supply your own main program and
X          link with -ll.)
X
X     -    lex's %r (Ratfor scanners) and %t (translation table)
X          options are not supported.
X
X     -    The do-nothing -n flag is not supported.
X
X     -    When definitions are expanded, flex encloses them in
X          parentheses.  With lex, the following
X
X              NAME    [A-Z][A-Z0-9]*
X              %%
X              foo{NAME}?      printf( "Found it\n" );
X              %%
X
X          will not match the string "foo" because when the macro
X          is expanded the rule is equivalent to "foo[A-Z][A-Z0-
X          9]*?" and the precedence is such that the '?' is asso-
X          ciated with "[A-Z0-9]*".  With flex, the rule will be
X          expanded to "foo([A-z][A-Z0-9]*)?" and so the string
X          "foo" will match.
X
X     -    yymore() is not supported.
X
X     -    The undocumented lex-scanner internal variable yylineno
X          is not supported.
X
X     -    If your input uses REJECT, you must run flex with the
X          -r flag.  If you leave out the flag, the scanner will
X          abort at run-time with a message that the scanner was
X          compiled without the flag being specified.
X
X     -    The input() routine is not redefinable, though may be
X          called to read characters following whatever has been
X          matched by a rule.  If input() encounters and end-of-
X          file the normal yywrap() processing is done.  A
X          ``real'' end-of-file is returned as EOF.
X
X          Input can be controlled by redefining the YY_INPUT
X          macro.  YY_INPUT's calling sequence is
X          "YY_INPUT(buf,result,max_size)".  Its action is to
X          place up to max_size characters in the character buffer
X          "buf" and return in the integer variable "result"
X          either the number of characters read or the constant
X          YY_NULL (0 on Unix systems) systems) to indicate EOF.
X          The default YY_INPUT reads from the file-pointer "yyin"
X          (which is by default stdin), so if you just want to
X          change the input file, you needn't redefine YY_INPUT -
X          just point yyin at the input file.
X
X          A sample redefinition of YY_INPUT (in the first section
X          of the input file):
X
X              %{
X              #undef YY_INPUT
X              #define YY_INPUT(buf,result,max_size) \
X                  result = (buf[0] = getchar()) == EOF ? YY_NULL : 1;
X              %}
X
X          You also can add in things like counting keeping track
X          of the input line number this way; but don't expect
X          your scanner to go very fast.
X
X     -    output() is not supported.  Output from the ECHO macro
X          is done to the file-pointer "yyout" (default stdout).
X
X     -    Trailing context is restricted to patterns which have
X          either a fixed-sized leading part or a fixed-sized
X          trailing part.  For example, "a*/b" and "a/b*" are
X          okay, but not "a*/b*".  This restriction is due to a
X          bug in the trailing context algorithm given in Princi-
X          ples of Compiler Design (and Compilers - Principles,
X          Techniques, and Tools) which can result in mismatches.
X          Try the following lex program
X
X              %%
X              x+/xy           printf( "I found \"%s\"\n", yytext );
X
X          on the input "xxy".  (If anyone knows of a fast algo-
X          rithm for finding the beginning of trailing context for
X          an arbitrary pair of regular expressions, please let me
X          know!) If you must have arbitrary trailing context, you
X          can use yyless() to effect it.
X
X     -    flex reads only one input file, while lex's input is
X          made up of the concatenation of its input files.
X
XENHANCEMENTS
X     -    Exclusive start-conditions can be declared by using %x
X          instead of %s. These start-conditions have the property
X          that when they are active, no other rules are active.
X          Thus a set of rules governed by the same exclusive
X          start condition describe a scanner which is independent
X          of any of the other rules in the flex input.  This
X          feature makes it easy to specify "mini-scanners" which
X          scan portions of the input that are syntactically dif-
X          ferent from the rest (e.g., comments).
X
X     -    flex dynamically resizes its internal tables, so direc-
X          tives like "%a 3000" are not needed when specifying
X          large scanners.
X
X     -    The scanning routine generated by flex is declared
X          using the macro YY_DECL. By redefining this macro you
X          can change the routine's name and its calling sequence.
X          For example, you could use:
X
X              #undef YY_DECL
X              #define YY_DECL float lexscan( a, b ) float a, b;
X
X          to give it the name lexscan, returning a float, and
X          taking two floats as arguments.
X
X     -    flex generates #line directives mapping lines in the
X          output to their origin in the input file.
X
X     -    You can put multiple actions on the same line,
X          separated with semi-colons.  With lex, the following
X
X              foo    handle_foo(); return 1;
X
X          is truncated to
X
X              foo    handle_foo();
X
X          flex does not truncate the action.  Actions that are
X          not enclosed in braces are terminated at the end of the
X          line.
X
X     -    Actions can be begun with %{ and terminated with %}. In
X          this case, flex does not count braces to figure out
X          where the action ends - actions are terminated by the
X          closing %}. This feature is useful when the enclosed
X          action has extraneous braces in it (usually in comments
X          or inside inactive #ifdef's) that throw off the brace-
X          count.
X
X     -    All of the scanner actions (e.g., ECHO, yywrap ...)
X          except the unput() and input() routines, are written as
X          macros, so they can be redefined if necessary without
X          requiring a separate library to link to.
X
XFILES
X     flex.skel
X          skeleton scanner
X
X     flex.fastskel
X          skeleton scanner for -f and -F
X
X     flexskelcom.h
X          common definitions for skeleton files
X
X     flexskeldef.h
X          definitions for compressed skeleton file
X
X     fastskeldef.h
X          definitions for -f, -F skeleton file
X
XSEE ALSO
X     lex(1)
X
X     M. E. Lesk and E. Schmidt, LEX - Lexical Analyzer Generator
X
XAUTHOR
X     Vern Paxson, with the help of many ideas and much inspira-
X     tion from Van Jacobson.  Original version by Jef Poskanzer.
X     Fast table representation is a partial implementation of a
X     design done by Van Jacobson.  The implementation was done by
X     Kevin Gong and Vern Paxson.
X
X     Thanks to the many flex beta-testers, especially Casey Lee-
X     dom, Nick Christopher, Chris Faylor, Eric Goldman, Craig
X     Leres, Mohamed el Lozy, Esmond Pitt, Jef Poskanzer, and Dave
X     Tallman.  Thanks to John Gilmore, Bob Mulcahy, Rich Salz,
X     and Richard Stallman for help with various distribution
X     headaches.
X
X     Send comments to:
X
X          Vern Paxson
X          Real Time Systems
X          Bldg. 46A
X          Lawrence Berkeley Laboratory
X          1 Cyclotron Rd.
X          Berkeley, CA 94720
X
X          (415) 486-6411
X
X          vern@lbl-{csam,rtsg}.arpa
X          ucbvax!lbl-csam.arpa!vern
X
X
XDIAGNOSTICS
X     flex scanner jammed - a scanner compiled with -s has encoun-
X     tered an input string which wasn't matched by any of its
X     rules.
X
X     flex input buffer overflowed - a scanner rule matched a
X     string long enough to overflow the scanner's internal input
X     buffer (as large as BUFSIZ in "/usr/include/stdio.h").  You
X     can edit flexskelcom.h and increase YY_BUF_SIZE and
X     YY_MAX_LINE to increase this limit.
X
X     REJECT used and scanner was not generated using -r - just
X     like it sounds.  Your scanner uses REJECT. You must run flex
X     on the scanner description using the -r flag.
X
X     old-style lex command ignored - the flex input contains a
X     lex command (e.g., "%n 1000") which is being ignored.
X
XBUGS
X     Use of unput() or input() trashes the current yytext and
X     yyleng.
X
X     Use of unput() to push back more text than was matched can
X     result in the pushed-back text matching a beginning-of-line
X     ('^') rule even though it didn't come at the beginning of
X     the line.
X
X     Nulls are not allowed in flex inputs or in the inputs to
X     scanners generated by flex.  Their presence generates fatal
X     errors.
X
X     Do not mix trailing context with the '|' operator used to
X     specify that multiple rules use the same action.  That is,
X     avoid constructs like:
X
X             foo/bar      |
X             bletch       |
X             bugprone     { ... }
X
X     They can result in subtle mismatches.  This is actually not
X     a problem if there is only one rule using trailing context
X     and it is the first in the list (so the above example will
X     actually work okay).  The problem is due to fall-through in
X     the action switch statement, causing non-trailing-context
X     rules to execute the trailing-context code of their fellow
X     rules.  This should be fixed, as it's a nasty bug and not
X     obvious.  The proper fix is for flex to spit out a
X     FLEX_TRAILING_CONTEXT_USED #define and then have the backup
X     logic in a separate table which is consulted for each rule-
X     match, rather than as part of the rule action.  The place to
X     do the tweaking is in add_accept() - any kind soul want to
X     be a hero?
X
X     The pattern:
X
X          x{3}
X
X     is considered to be variable-length for the purposes of
X     trailing context, even though it has a clear fixed length.
X
X     Due to both buffering of input and read-ahead, you cannot
X     intermix calls to, for example, getchar() with flex rules
X     and expect it to work.  Call input() instead.
X
X     The total table entries listed by the -v flag excludes the
X     number of table entries needed to determine what rule has
X     been matched.  The number of entries is equal to the number
X     of DFA states if the scanner was not compiled with -r, and
X     greater than the number of states if it was.
X
X     The scanner run-time speeds have not been optimized as much
X     as they deserve.  Van Jacobson's work shows that they can go
X     quite a bit faster still.
SHAR_EOF
if test 17746 -ne "`wc -c < 'flex.man.page'`"
then
	echo shar: error transmitting "'flex.man.page'" '(should have been 17746 characters)'
fi
fi # end of overwriting check
#	End of shell archive
exit 0
--- end of part 2 ---