Xref: utzoo comp.graphics:3821 comp.windows.x:6492
Path: utzoo!utgpu!watmath!watcgl!srneely
From: srneely@watcgl.waterloo.edu (Shawn Neely)
Newsgroups: comp.graphics,comp.windows.x
Subject: Re: Luminance from RGB
Message-ID: <7162@watcgl.waterloo.edu>
Date: 6 Dec 88 19:57:05 GMT
References: <8811042303.AA21505@dawn.steinmetz.GE.COM> <8811070533.AA08904@vector.sun.com> <8811080030.AA14681@EXPO.LCS.MIT.EDU> <76649@sun.uucp> <6698@watcgl.waterloo.edu> <964@bacchus.dec.com>
Reply-To: srneely@watcgl.waterloo.edu (Shawn Neely)
Organization: U. of Waterloo, Ontario
Lines: 57

In article <964@bacchus.dec.com> karlton@decwrl.dec.com (Philip Karlton) writes:

+I would like now from the experts is what they would do to convert an RGB
+value as specified in the X protocol into a gray level.
+
+The particular problem I have in mind is that of a StaticGray display with N
+equally spaced intensities arranged in a ramp with black at 0 and white at
+N-1. I have access to hardware with N of 2, 4, 16, and 256.
+
+Two different expressions for computing the appropriate pixel value come
+immediately to (my) mind:
+
+For r, g, b in [0..1]
+
+	floor((.299r + .587g + .114b)(n - 1) + 0.5)		(a)
+
+or for r, g, b in [0..1)
+
+	floor((.299r + .587g + .114b)(n))			(b)
+
+ (stuff deleted)
+...Is either (a) or (b) the appropriate choice. Is
+some better function around that I should use?
+
+For the numerically curious: r, g, and b above could be computed using
+
+	r = ((float) screenRed) / maxColor;
+	b = ((float) screenBlue) / maxColor;
+	g = ((float) screenGreen) / maxColor;
+
+where maxColor is dependent upon which of (a) or (b) is chosen:
+
+	float maxColor = (float) (0xFFFF);		/* (a) */
+or
+	float maxColor = (float) (0x10000);		/* (b) */
+
+PK

The correct approach is (a). A strong argument for using
the closed interval [0..1] (and [0..N-1]) is given in
    "Design and Experience with a Generalized Raster Toolkit"
    by Paeth and Booth in Proc. Graphics Interface '86 (Vancouver).

The interval is consistent with the design of a number of colour spaces,
and is correct when data is taken to higher significance. The same is
not true of the wrong (often implicit using bit shifts) use of
the open interval [0..1).

For example, a one-bit image in the [0..1) model allows only
the intensity values 0.0 and 0.5, and not "full on".

The required multiplications and divisions for the correct approach
can often be performed by table lookup.
-- 
(.I.)   "The road of excess leads
 ).(       to the palace of wisdom."
( Y )              -William Blake