Path: utzoo!utgpu!water!watmath!clyde!bellcore!rutgers!gatech!ncar!ico!rcd
From: rcd@ico.ISC.COM (Dick Dunn)
Newsgroups: comp.arch
Subject: Re: RISC bashing at USENIX
Summary: Look at how it's presented
Message-ID: <6965@ico.ISC.COM>
Date: 12 Jul 88 05:23:47 GMT
References: <6888@ico.ISC.COM> <11496@steinmetz.ge.com>
Organization: Interactive Systems Corp, Boulder, CO
Lines: 61

In response to my griping about Neal Nelson at USENIX, davidsen@
steinmetz.ge.com (William E. Davidsen Jr) writes:
>   Neal Nelson presents the results of his benchmarks and the description
> of each test. You are free to interpret them any way you want. That's
> not a flame, just a reminder that if you choose to interpret his results
> as RISC bashing, then you seem to have decided that RISC is better, and
> that a benchmark which doesn't show that is either biased or useless.

Good grief!  I did NOT interpret NN's results as RISC bashing...I
interpreted the *presentation* of the results as RISC bashing.  The
presentation at USENIX was flamboyantly anti-RISC--meaning that there are
statements about RISCs by NN which are vehemently anti-RISC and not backed
up by fact.  Yes, I am free to interpret the results any way I want--but NN
wants me to interpret them in a particular way, as strongly anti-RISC.  I
don't see that they support that viewpoint at all, which is what I'm com-
plaining about.

There *might* be some useful results and good work behind it all, but I
sure-as-hell can't find it even after trying to peel back the layers of
hype and bad journalism, so I tend to doubt that there's much there.

No, I haven't decided that RISC is better.  The biases in the benchmarks
(more to the point, in the reporting thereof) are evident regardless of
your own biases, if you just look at what's been said.  I pointed out some
of the more obvious problems...so if you think I'm off base, why don't you
take on the *substance* of what I said?  (I.e., if you're trying to say *I'm*
biased, show us how.  For example, do you dispute that they compared the
fastest Sun 3 against the slowest MIPS box?)

Most of my computing is done on CISCs, and they serve well.  But I had a
chance recently to run a couple of problems on a low-end RISC; they ran so
fast that I put in some debugging code to be sure something hadn't gotten
short-circuited!  (It hadn't.)  I'm not taking up the sword to defend RISC,
but I know the RISC guys aren't smoking rope--they're for real.

>   We have looked at the NN benchmarks for a number of machines (I
> obviously can't say which ones), and my personal reaction is that they
> are reasonable and valid for business applications...

OK, so which benchmarks are the good ones?  Note that the one that EE Times
gave such prominent coverage was one of the simplest--a loop with just 4
calculations (+-*/) on 16-bit integers, running 1 to 15 copies at a time.
That has about 0.5 * dsq to do with any real business program.  And, as I
said in the original article, I could have attributed it to EE Times'
sloppiness (the rest of the article was an expository/stylistic/technical
mess) but for the fact that NN was showing it off.  NN has 17 other
benchmarks, and they could have put together a complete presentation of the
benchmarks on comparable machines.  They didn't.  Why not?

> ...why not get a benchmark suite which tests [what concerns you],
> rather than blasting NN?

Done!  I have tests of my own which I run when *I* want to get an idea of
how fast a processor is.  The reason I'm blasting NN is that I see them
misleading people--and using a lot of PR to mislead a lot of people.  It's
that aspect that bothers me--not that it's RISCs per se that they're
bashing, but that they're bashing, instead of testing and reporting
carefully.
-- 
Dick Dunn      UUCP: {ncar,nbires}!ico!rcd           (303)449-2870
   ...Are you making this up as you go along?