Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10.2 9/18/84; site peora.UUCP
Path: utzoo!linus!philabs!cmcl2!seismo!lll-crg!dual!qantel!ihnp4!drutx!mtuxo!mtunh!mtung!mtunf!ariel!vax135!petsd!peora!jer
From: jer@peora.UUCP (J. Eric Roskos)
Newsgroups: net.arch
Subject: Re: Floating Point Rounding
Message-ID: <1449@peora.UUCP>
Date: Wed, 7-Aug-85 23:08:14 EDT
Article-I.D.: peora.1449
Posted: Wed Aug  7 23:08:14 1985
Date-Received: Mon, 12-Aug-85 05:11:32 EDT
References: <36900010@ima.UUCP> <1357@peora.UUCP> <1117@ihuxb.UUCP> <2028@ukma.UUCP>
Organization: Perkin-Elmer SDC, Orlando, Fl.
Lines: 47

> Summary: Knuth has some good discussion on rounding
>
> In article <1117@ihuxb.UUCP> wfmans@ihuxb.UUCP (w. mansfield) writes:

Just a little grumble here before I comment... I wrote the text that
appears under this header, not w. mansfield, describing Perkin-Elmer
R* Rounding.

Anyway, to answer the question asked... the radix is indeed 16.  Thus,
according to Knuth's rule,

>    ficantly poor results.... For even radices, there is reason to prefer
>    the following rule: `Round to even when b/2 is odd, round to odd when
>    b/2 is even.'  [B is the radix.]  ... On the other hand, some people
>    prefer rounding to even in all cases, so that the remainder will tend
>    to be 0 more often.  Neither alternative conclusively dominates the
>    other; fortunately the base is usually b = 2 or b = 10, when everyone
>    agrees that round to even is best."

the R* rounding scheme is right, because it is a round-to-odd scheme (if I
understand it all correctly), and 16/2=8, an even number.  This is very
reassuring!  However, I think the person who devised the rounding scheme
may have read Knuth's comments, since I think the floating point unit on
the 3200s was designed just shortly after Knuth's books came out.  I know
he had a substantial theoretical basis for the scheme, since one of our
local historians of sorts here has a large collection of papers in support
of the scheme left over from those days (CACM papers, etc.).

I think the person who said "some numerical analysts put corrections for
a machine's shortcomings into their programs" had a good point, one I hadn't
thought of.  I do know that a numerical analyst (fairly well-known) at
the university where I did my graduate work was originally of the "the
floating point unit is broken" school, then came to like it a great deal
once she sat down and figured out why the results were different, though I
also remember she changed her programs slightly as a result.  But the moral
of the story really was that market forces ultimately seem to determine the
acceptability of a product, much more than its technical correctness, in
many cases.  Though this doesn't mean you shouldn't try...

Disclaimer: I'm just an interested observer of R* rounding, et. al; I had
   nothing to do with inventing or implementing it.  I can't even remember
   the last time I wrote a program that used floating point operations!
-- 
Shyy-Anzr:  J. Eric Roskos
UUCP:       ..!{decvax,ucbvax,ihnp4}!vax135!petsd!peora!jer
US Mail:    MS 795; Perkin-Elmer SDC;
	    2486 Sand Lake Road, Orlando, FL 32809-7642