Path: utzoo!attcan!uunet!husc6!uwvax!oddjob!ncar!ames!pasteur!ucbvax!CORY.BERKELEY.EDU!dillon
From: dillon@CORY.BERKELEY.EDU (Matt Dillon)
Newsgroups: comp.sys.amiga
Subject: Re: Dhrystone
Message-ID: <8808180050.AA05343@cory.Berkeley.EDU>
Date: 18 Aug 88 00:50:29 GMT
Sender: daemon@ucbvax.BERKELEY.EDU
Lines: 39


:In article <8808150554.AA14630@cory.Berkeley.EDU> dillon@CORY.BERKELEY.EDU (Matt Dillon) writes:
:>
:>	This is why such benchmarks are ludicrous, when people fine-tune
:>the benchmark and/or compiler to make the benchmark look better.  I won't
:
:I think your comment is the ludicrous thing here.  Think about it and tell
:me you don't think in-line string handling improves performance on ANY
:program.  Many of the in-line string routines in Lattice are both smaller
:AND faster than pushing parameters on the stack, branching, returning and
:cleaning up the stack.  Benchmarks need never be considered.  Once the
:feature is in, of course it should be taken into account when the benchmarks
:are determined.  You are tarring Lattive and Manx with Intel's
:brush, when there is no evidence whatsoever that either is guilty.

	Uh huh, you haven't thought the thing through have you?  Let me
explain it more carefully:  How large a percentage increase in speed do you
think you will get by replacing a subroutine-strcpy() with an inline-strcpy()
(etc...) ???

	Now, does the benchmark give a 'faster' value that agrees with the
relative speed increase of your program?

	Properly, the idea is to repeat the test on a whole shitload of
programs (that were not designed specifically to make a benchmark look good),
get the mean/average/whatever, and compare that relative speed increase to
the relative speed increase in the benchmark by the inline code.

				--

	Does that answer you question?  I am not tarring either Lattice or 
Manx, but pointing out two things people do not seem to understand about
benchmarks.  (1) Never fine tune a benchmark, and (2) Benchmarks are 
incredibly difficult to write if written properly.

	Going a little deeper:  A properly written benchmark should be 
difficult to fine-tune (anybody else catch that inference?)

						-Matt