Path: utzoo!attcan!uunet!nbires!ico!rcd From: rcd@ico.ISC.COM (Dick Dunn) Newsgroups: comp.arch Subject: RISC bashing at USENIX Message-ID: <6888@ico.ISC.COM> Date: 7 Jul 88 19:54:49 GMT Organization: Interactive Systems Corp, Boulder, CO Lines: 69 [Better start off by emphasizing that these are my own opinions, not ISC's.] "Neal Nelson and Associates" had a booth at the USENIX vendor's exhibition whose sole purpose seemed to be RISC-bashing. Although they purportedly have developed a set of tests, which they call their "Business Benchmark (R)" for helping people make realistic comparisons of machines under realistic loads, in fact even basic descriptions of the tests cast serious doubt on their real usefulness. Some of the tests seem overly simplistic; others contain obvious biases toward or against certain types of hardware and/or I/O system software. A couple of months ago, several trade rags ran articles reporting how Neal Nelson & Associates (NNA) had shown that CISCs would beat RISCs. It's hard to tell just who botched what part of it--whether NNA did a bad job of reporting it or the trade journals reported what they wanted to hear--but the articles were abominable. At USENIX, the NNA booth had great piles of reprints of several of these RISC-bashing trade-rag articles--and NO OTHER reports of substantive conclusions. (Their other literature listed system configurations they had tested and tried to explain the tests.) I saw no attempt to present a balanced picture, nor to compare a representative set of comparable machines. For example, the EE Times RISC-bash article (which I happen to have at hand) shows a Sun 3/260 beating the pants off a Model 25 RT PC and slightly winning over a MIPS M-500 on the test EE Times chose to show in detail. (The Sun 3 was the only CISC design represented; there were other RISCs. I'm singling out MIPS and the RT as examples.) EE Times chose to report in detail on only one test out of 18 in the benchmark! The results as reported show some obvious problems: - Why were they using the fastest Sun 3 (25 MHz) for comparison against the slowest MIPS machine? - If a 25 MHz CISC is only 12% faster than an 8 MHz RISC, how does that make the CISC faster? - Although the older RT lost badly, it's not a full RISC design... and one ought perhaps to take into account that the Sun system costs more than 3 times as much. - Where are the other CISCs? It seems very much like only the best CISC they could find was used as the basis for comparison--and certainly the best RISCs were NOT used. At the point that EE Times published their report, it would have been easy (for NNA in particular) to say that it was simply a case of EE Times doing some very selective reporting. But at USENIX, it was clear that NNA was quite proud to show off the EE Times article...and it seems clear that NNA has an ax to grind wrt RISCs even though it's not clear why. They're making some pretty strong statements: "I'm beginning to believe RISC doesn't belong everywhere, or possibly even anywhere..." "...we still haven't seen any areas of study that say RISC has been implemented and shown a marked improvement..." ...but they're not backing them up. Both of these are attributed to Neal Nelson. Incidentally, the second statement may well be true--but it's sur- prising that someone working on benchmarks could ignore a decade or so of work! (It's interesting that at ASPLOS last fall, the two camps seemed to be holding viewpoints of "RISC has won" vs "the battle isn't over"--quite a different story from what Nelson tells.) Most folks familiar with the RISC-CISC debate have their biases, and in a moment of passion they may get a little carried away in supporting their viewpoint. But what I see in this Neal Nelson stuff goes beyond a momentary outburst--it's an unsubstantiated, unprofessional reaction to the substantial real work (and very real, very fast machines) that have come out of RISC development. I think it added some heat and absolutely no light...we need thought more than we need more emotion. -- Dick Dunn UUCP: {ncar,nbires}!ico!rcd (303)449-2870 ...Are you making this up as you go along?