Path: utzoo!attcan!uunet!yale!mfci!colwell From: colwell@mfci.UUCP (Robert Colwell) Newsgroups: comp.arch Subject: Re: Dhrystone 2.1 Keywords: integer benchmark Message-ID: <465@m3.mfci.UUCP> Date: 13 Jul 88 13:40:25 GMT References: <517@pcrat.UUCP> <2294@sugar.UUCP> <391@attila.weitek.UUCP> Sender: root@mfci.UUCP Reply-To: colwell@mfci.UUCP (Robert Colwell) Organization: Multiflow Computer Inc., Branford Ct. 06405 Lines: 51 In article <391@attila.weitek.UUCP> mahar@attila.UUCP (Mike Mahar) writes: >I've noticed that the way people use the Dhrystone benchmark is very different >than the initial intent. It was supposed to be used to evaluate computer >architectures. Instead, it is used to evaluate C compilers. > >Given this the Dhrystone benchmark is used to evaluate compilers for the >various RISC processors. A very good optimizing compiler will eliminate >all the code in Dhrystone except the calls to times. >... >These three tricks alone can almost double the >Dhrystone rating of a machine. Fast string copy and compare are usefull >library routines but they don't account for 30% of a typical program's time. I think this demonstrates why Dhrystone has been nearly destroyed as a useful benchmark. It's clear at this point that one's Dhrystone rating is at best only weakly correlated to one's realizable performance on interesting systems code. I suspect that it is fast becoming completely uncorrelated, due to the rampant compiler cheating it allows. I don't understand why you think "a very good optimizing compiler will eliminate all the code in Dhrystone". The dead-code removal phase of the compiler isn't really all that important in terms of overall object code performance -- it mostly cleans up conceptual errors on the part of the programmer. Putting a lot of time and energy into that part of the compiler is silly if: a) you want more than just high Dhrystone numbers, and b) you could have put that effort into other optimizations that really meant something. I'm even willing to grant that optimizing away whole routines (or auto-inlining them) is legitimate, if and only if your compiler does that in Dhrystone to the same extent that it does on real code. In fact, if your compiler is pretty good at those kinds of optimizations, the performance measurement on this benchmark would be more accurate rather than less. Under these circumstances, the only issues we'd still have to shake our heads about would be Dhrystone's unrealistic cache utilization and the string manipulation effects you noted above. Using Dhrystone as a measure of compiler cleverness, in my opinion, is very nearly as crazy as using it as an integer throughput predictor. (Sorry, Reinhold, I think it's of great usefulness when evaluating effects on performance while changing architecture, instruction sets, or certain compiler strategies, but in the commercial world it's Toto in a town full of wicked witches). Bob Colwell mfci!colwell@uunet.uucp Multiflow Computer 175 N. Main St. Branford, CT 06405 203-488-6090