Path: utzoo!attcan!uunet!yale!husc6!babbage!reiter
From: reiter@babbage.harvard.edu (Ehud Reiter)
Newsgroups: comp.arch
Subject: Re: Dhrystone 2.1
Keywords: integer benchmark
Message-ID: <4936@husc6.harvard.edu>
Date: 13 Jul 88 18:19:09 GMT
References: <517@pcrat.UUCP> <2294@sugar.UUCP> <391@attila.weitek.UUCP>
Sender: news@husc6.harvard.edu
Reply-To: reiter@harvard.UUCP (Ehud Reiter)
Organization: Aiken Computation Lab Harvard, Cambridge, MA
Lines: 31

We seem to be starting up another argument about how accurately
Dhrystone serves as a predictor of performance of real programs.  I
think, though, that the main benefit of Dhrystone and other benchmarks
is not to predict performance (that's impossible, since no single
figure of merit can predict the performance of widely differing
application programs), but rather to help architecture designers design
good computers, and compiler writers design good compilers, by giving
them some real code which is at least slightly representative to play
with.  I think this point was perhaps made most strongly by Richard
Gabriel, in his book on LISP benchmarks, where he claimed that the
availability of a good benchmark suite enabled some compiler writers to
improve the execution speed of their output by a factor of two or more.

So, I think it is unfair to criticize Dhrystone on the basis that
compiler writers have optimized their compilers to produce fast
Dhrystone code, which means that Dhrystone performance is not
representative of real application performance.  This is true, but
misses the point that the purpose of Dhrystone is to encourage compiler
writers to build good optimizations into their systems.  If the
optimizations required for good Dhrystone performance include
optimizations that are not relevant to real application programs, who
cares - it doesn't do any harm to include rarely-used optimizations in
a compiler.  If optimizations which would be very useful in general are
not required for good Dhrystone performance, this is a problem, but our
response should not be to throw out Dhrystone, but rather to add to it
a new section (or perhaps a whole new benchmark) which does require
these optimizations to obtain good performance.

					Ehud Reiter
					reiter@harvard	(ARPA,BITNET,UUCP)
					reiter@harvard.harvard.EDU  (new ARPA)