Path: utzoo!utgpu!watmath!clyde!att!osu-cis!tut.cis.ohio-state.edu!mailrus!ames!ncar!noao!arizona!naucse!jdc
From: jdc@naucse.UUCP (John Campbell)
Newsgroups: comp.sources.d
Subject: flex/lex and '\0' input
Keywords: lexical scanners, null
Message-ID: <1047@naucse.UUCP>
Date: 30 Nov 88 05:47:44 GMT
Organization: Northern Arizona University, Flagstaff, AZ
Lines: 22

I spent a long time today learning that flex and lex won't deal
with '\0' input.  These nulls are used as padding after line feeds
on the VMS system, or at least they show up in the log files I
wanted to scan.

Can anyone explain why a scanning tool like lex or flex would be
designed to choke on anything in the range of 0-255?  If I can 
find byte input sources that contain these things I'd sure like
to be able to deal with those input streams.

I solved the problem, BTW, by replacing the input routine to lex
and "squeezing" out any '\0's that appear.  This means, however,
that I had to scan the input one extra time before letting the
scanner do it's job.  

If you have a thought on how to change flex (I don't have a source
license to lex) so that it can handle '\0', I'd love to know.  If you
have a rationale regarding the current behavior I'd also like to know.
-- 
	John Campbell               ...!arizona!naucse!jdc
                                    CAMPBELL@NAUVAX.bitnet
	unix?  Sure send me a dozen, all different colors.