Path: utzoo!attcan!uunet!nbires!ico!rcd
From: rcd@ico.ISC.COM (Dick Dunn)
Newsgroups: comp.arch
Subject: RISC bashing at USENIX
Message-ID: <6888@ico.ISC.COM>
Date: 7 Jul 88 19:54:49 GMT
Organization: Interactive Systems Corp, Boulder, CO
Lines: 69


[Better start off by emphasizing that these are my own opinions, not ISC's.]

"Neal Nelson and Associates" had a booth at the USENIX vendor's exhibition
whose sole purpose seemed to be RISC-bashing.  Although they purportedly
have developed a set of tests, which they call their "Business Benchmark
(R)" for helping people make realistic comparisons of machines under
realistic loads, in fact even basic descriptions of the tests cast serious
doubt on their real usefulness.  Some of the tests seem overly simplistic;
others contain obvious biases toward or against certain types of hardware
and/or I/O system software.

A couple of months ago, several trade rags ran articles reporting how Neal
Nelson & Associates (NNA) had shown that CISCs would beat RISCs.  It's
hard to tell just who botched what part of it--whether NNA did a bad job of
reporting it or the trade journals reported what they wanted to hear--but
the articles were abominable.  At USENIX, the NNA booth had great piles of
reprints of several of these RISC-bashing trade-rag articles--and NO OTHER
reports of substantive conclusions.  (Their other literature listed system
configurations they had tested and tried to explain the tests.)  I saw no
attempt to present a balanced picture, nor to compare a representative set
of comparable machines.

For example, the EE Times RISC-bash article (which I happen to have at
hand) shows a Sun 3/260 beating the pants off a Model 25 RT PC and
slightly winning over a MIPS M-500 on the test EE Times chose to show
in detail.  (The Sun 3 was the only CISC design represented; there were
other RISCs.  I'm singling out MIPS and the RT as examples.)  EE Times
chose to report in detail on only one test out of 18 in the benchmark!
The results as reported show some obvious problems:
	- Why were they using the fastest Sun 3 (25 MHz) for comparison
	  against the slowest MIPS machine?
	- If a 25 MHz CISC is only 12% faster than an 8 MHz RISC, how does
	  that make the CISC faster?
	- Although the older RT lost badly, it's not a full RISC design...
	  and one ought perhaps to take into account that the Sun system
	  costs more than 3 times as much.
	- Where are the other CISCs?  It seems very much like only the best
	  CISC they could find was used as the basis for comparison--and
	  certainly the best RISCs were NOT used.
At the point that EE Times published their report, it would have been easy
(for NNA in particular) to say that it was simply a case of EE Times doing
some very selective reporting.  But at USENIX, it was clear that NNA was
quite proud to show off the EE Times article...and it seems clear that NNA
has an ax to grind wrt RISCs even though it's not clear why.  They're
making some pretty strong statements:
	"I'm beginning to believe RISC doesn't belong everywhere, or
	possibly even anywhere..."
	"...we still haven't seen any areas of study that say RISC has been
	implemented and shown a marked improvement..."
...but they're not backing them up.  Both of these are attributed to Neal
Nelson.  Incidentally, the second statement may well be true--but it's sur-
prising that someone working on benchmarks could ignore a decade or so of
work!

(It's interesting that at ASPLOS last fall, the two camps seemed to be
holding viewpoints of "RISC has won" vs "the battle isn't over"--quite a
different story from what Nelson tells.)

Most folks familiar with the RISC-CISC debate have their biases, and in a
moment of passion they may get a little carried away in supporting their
viewpoint.  But what I see in this Neal Nelson stuff goes beyond a
momentary outburst--it's an unsubstantiated, unprofessional reaction to
the substantial real work (and very real, very fast machines) that have
come out of RISC development.  I think it added some heat and absolutely no
light...we need thought more than we need more emotion.
-- 
Dick Dunn      UUCP: {ncar,nbires}!ico!rcd           (303)449-2870
   ...Are you making this up as you go along?