Path: utzoo!attcan!uunet!aecom!werner
From: werner@aecom.yu.edu (Craig Werner)
Newsgroups: comp.lang.postscript
Subject: Re: Bug in PostScript math
Summary: A restatement and a fix
Keywords: 200/3
Message-ID: <2398@aecom.yu.edu>
Date: 13 Aug 89 02:23:02 GMT
References: <2395@aecom.yu.edu>
Distribution: na
Organization: Albert Einstein Coll. of Med., NY
Lines: 36

In article <2395@aecom.yu.edu>, werner@aecom.yu.edu (Craig Werner) writes:
> 
> 	I recently encountered a problem that seems to be do to a bug in
> the PostScript arithmetic routines built into an Apple LaserWriter IINT.
> 
> 	Consider the following:
> 	1) divide 200 by 5, save the 40 in a variable
> 	2) add 200, later subtract 200
> 	3) divide the variable byt the current value, the answer is 1.0
> 	4) truncate the 1.0, the answer should still be 1.0.
> 
> 	Now divide 200 by 3, saving 66.6667.
> 	When you truncate the 1.0 in step 4, the result is 0.0
> 

	Let me restate the problem.  I referred to the problem as a bug
when the real problem is that it has been twelve years since I've
encountered such an egregious arithmetic precision error. (The last time
was on a TRS-80 Model I, Level I, system complete with 4K RAM.)  Just
out of curiousity, what is the precision of PostScript's floating point
routines?  It's obviously a lot less than I had expected.
	I'm still confused as to why cvs would report the number as 1.0
instead of what it really was. However, I've used this to my advantage,
and a fix for the particular problem seems to be replacing
	truncate
with	20 string cvs cvr truncate

	(11 string gives me a cvs rangecheck error, and at this point
I'd rather not ask why.)


-- 
	        Craig Werner   (future MD/PhD, 4.5 years down, 2.5 to go)
	     werner@aecom.YU.EDU -- Albert Einstein College of Medicine
              (1935-14E Eastchester Rd., Bronx NY 10461, 212-931-2517)
  "Sometimes you have to run as fast as you can just to stay in the same place."