Path: utzoo!attcan!uunet!yale!mfci!colwell
From: colwell@mfci.UUCP (Robert Colwell)
Newsgroups: comp.arch
Subject: Re: Dhrystone 2.1
Keywords: integer benchmark
Message-ID: <465@m3.mfci.UUCP>
Date: 13 Jul 88 13:40:25 GMT
References: <517@pcrat.UUCP> <2294@sugar.UUCP> <391@attila.weitek.UUCP>
Sender: root@mfci.UUCP
Reply-To: colwell@mfci.UUCP (Robert Colwell)
Organization: Multiflow Computer Inc., Branford Ct. 06405
Lines: 51

In article <391@attila.weitek.UUCP> mahar@attila.UUCP (Mike Mahar) writes:
>I've noticed that the way people use the Dhrystone benchmark is very different
>than the initial intent.  It was supposed to be used to evaluate computer
>architectures.  Instead, it is used to evaluate C compilers.
>
>Given this the Dhrystone benchmark is used to evaluate compilers for the
>various RISC processors.  A very good optimizing compiler will eliminate
>all the code in Dhrystone except the calls to times.
>...
>These three tricks alone can almost double the
>Dhrystone rating of a machine.  Fast string copy and compare are usefull
>library routines but they don't account for 30% of a typical program's time.

I think this demonstrates why Dhrystone has been nearly destroyed
as a useful benchmark.  It's clear at this point that one's Dhrystone
rating is at best only weakly correlated to one's realizable
performance on interesting systems code.  I suspect that it is fast
becoming completely uncorrelated, due to the rampant compiler
cheating it allows.

I don't understand why you think "a very good optimizing compiler
will eliminate all the code in Dhrystone".  The dead-code removal
phase of the compiler isn't really all that important in terms of
overall object code performance -- it mostly cleans up conceptual
errors on the part of the programmer.  Putting a lot of time and
energy into that part of the compiler is silly if:  a) you want more
than just high Dhrystone numbers, and b) you could have put that
effort into other optimizations that really meant something.

I'm even willing to grant that optimizing away whole routines (or
auto-inlining them) is legitimate, if and only if your compiler does
that in Dhrystone to the same extent that it does on real code.  In
fact, if your compiler is pretty good at those kinds of
optimizations, the performance measurement on this benchmark would be
more accurate rather than less.  Under these circumstances, the only
issues we'd still have to shake our heads about would be Dhrystone's 
unrealistic cache utilization and the string manipulation effects you
noted above.

Using Dhrystone as a measure of compiler cleverness, in my opinion,
is very nearly as crazy as using it as an integer throughput
predictor.  (Sorry, Reinhold, I think it's of great usefulness when
evaluating effects on performance while changing architecture,
instruction sets, or certain compiler strategies, but in the
commercial world it's Toto in a town full of wicked witches).


Bob Colwell            mfci!colwell@uunet.uucp
Multiflow Computer
175 N. Main St.
Branford, CT 06405     203-488-6090