Path: utzoo!utgpu!water!watmath!clyde!bellcore!rutgers!gatech!ncar!ico!rcd From: rcd@ico.ISC.COM (Dick Dunn) Newsgroups: comp.arch Subject: Re: RISC bashing at USENIX Summary: Look at how it's presented Message-ID: <6965@ico.ISC.COM> Date: 12 Jul 88 05:23:47 GMT References: <6888@ico.ISC.COM> <11496@steinmetz.ge.com> Organization: Interactive Systems Corp, Boulder, CO Lines: 61 In response to my griping about Neal Nelson at USENIX, davidsen@ steinmetz.ge.com (William E. Davidsen Jr) writes: > Neal Nelson presents the results of his benchmarks and the description > of each test. You are free to interpret them any way you want. That's > not a flame, just a reminder that if you choose to interpret his results > as RISC bashing, then you seem to have decided that RISC is better, and > that a benchmark which doesn't show that is either biased or useless. Good grief! I did NOT interpret NN's results as RISC bashing...I interpreted the *presentation* of the results as RISC bashing. The presentation at USENIX was flamboyantly anti-RISC--meaning that there are statements about RISCs by NN which are vehemently anti-RISC and not backed up by fact. Yes, I am free to interpret the results any way I want--but NN wants me to interpret them in a particular way, as strongly anti-RISC. I don't see that they support that viewpoint at all, which is what I'm com- plaining about. There *might* be some useful results and good work behind it all, but I sure-as-hell can't find it even after trying to peel back the layers of hype and bad journalism, so I tend to doubt that there's much there. No, I haven't decided that RISC is better. The biases in the benchmarks (more to the point, in the reporting thereof) are evident regardless of your own biases, if you just look at what's been said. I pointed out some of the more obvious problems...so if you think I'm off base, why don't you take on the *substance* of what I said? (I.e., if you're trying to say *I'm* biased, show us how. For example, do you dispute that they compared the fastest Sun 3 against the slowest MIPS box?) Most of my computing is done on CISCs, and they serve well. But I had a chance recently to run a couple of problems on a low-end RISC; they ran so fast that I put in some debugging code to be sure something hadn't gotten short-circuited! (It hadn't.) I'm not taking up the sword to defend RISC, but I know the RISC guys aren't smoking rope--they're for real. > We have looked at the NN benchmarks for a number of machines (I > obviously can't say which ones), and my personal reaction is that they > are reasonable and valid for business applications... OK, so which benchmarks are the good ones? Note that the one that EE Times gave such prominent coverage was one of the simplest--a loop with just 4 calculations (+-*/) on 16-bit integers, running 1 to 15 copies at a time. That has about 0.5 * dsq to do with any real business program. And, as I said in the original article, I could have attributed it to EE Times' sloppiness (the rest of the article was an expository/stylistic/technical mess) but for the fact that NN was showing it off. NN has 17 other benchmarks, and they could have put together a complete presentation of the benchmarks on comparable machines. They didn't. Why not? > ...why not get a benchmark suite which tests [what concerns you], > rather than blasting NN? Done! I have tests of my own which I run when *I* want to get an idea of how fast a processor is. The reason I'm blasting NN is that I see them misleading people--and using a lot of PR to mislead a lot of people. It's that aspect that bothers me--not that it's RISCs per se that they're bashing, but that they're bashing, instead of testing and reporting carefully. -- Dick Dunn UUCP: {ncar,nbires}!ico!rcd (303)449-2870 ...Are you making this up as you go along?