Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Posting-Version: version B 2.10.1 6/24/83; site lanl-a.UUCP Path: utzoo!watmath!clyde!burl!hou3c!hocda!houxm!vax135!cornell!uw-beaver!tektronix!hplabs!hao!seismo!cmcl2!lanl-a!jlg From: jlg@lanl-a.UUCP Newsgroups: net.micro.68k,net.arch Subject: Re: binary normalization Message-ID: <13545@lanl-a.UUCP> Date: Tue, 18-Sep-84 17:03:05 EDT Article-I.D.: lanl-a.13545 Posted: Tue Sep 18 17:03:05 1984 Date-Received: Tue, 25-Sep-84 09:31:23 EDT References: <1191@rti-sel.UUCP>, <4190@fortune.UUCP>, <420@watdcsu.UUCP>, <1422@wateng.UUCP> Organization: Los Alamos National Laboratory Lines: 32It should be remembered that floating point numbers are a subset of the rationals, not the reals. Therefore, given the real line, only certain points are exactly representable in a give floating point number scheme. If S(b,n,M) is the set of floating point numbers representable with n base b digits between 1/M and M, then |S(b,n,M)| is the size of this set. Note that exponent range limits are excluded from this definition. S can be regarded as a density function, i.e. the number of elements within a given range. A good definition of 'equivalent precision' for two floating point systems would require that the two systems have the same density. So define two systems as giving 'equivalent precision' if: S(d,m,M) lim ---------- = 1 M->inf S(b,m,M) Matula [1] has shown that this is equivalent to: m = n log b + log [d(b-1)/b(d-1)] - log log b d d d d Therefore, 24 bit binary is equivalent to about 6.27 hexadecimal digits. Alternately, 24 bits = 7.49 decimal digits, and 6 digits hex = 7.16 decimal digits. IEEE clearly comes out superior to IBM (for example). Restricting the exponent range has no effect of this comparison, an application which needs a wider range than provided by IEEE single would still be better off with binary normalized arithmetic (use IEEE extended formats). [1] A Formalization of Floating-Point Numeric Base Conversion, IEEE Transactions on Computers, vol. C-19, No. 8, August 1970, pp. 681-692.