Path: utzoo!attcan!uunet!husc6!uwvax!oddjob!ncar!ames!pasteur!ucbvax!CORY.BERKELEY.EDU!dillon From: dillon@CORY.BERKELEY.EDU (Matt Dillon) Newsgroups: comp.sys.amiga Subject: Re: Dhrystone Message-ID: <8808180050.AA05343@cory.Berkeley.EDU> Date: 18 Aug 88 00:50:29 GMT Sender: daemon@ucbvax.BERKELEY.EDU Lines: 39 :In article <8808150554.AA14630@cory.Berkeley.EDU> dillon@CORY.BERKELEY.EDU (Matt Dillon) writes: :> :> This is why such benchmarks are ludicrous, when people fine-tune :>the benchmark and/or compiler to make the benchmark look better. I won't : :I think your comment is the ludicrous thing here. Think about it and tell :me you don't think in-line string handling improves performance on ANY :program. Many of the in-line string routines in Lattice are both smaller :AND faster than pushing parameters on the stack, branching, returning and :cleaning up the stack. Benchmarks need never be considered. Once the :feature is in, of course it should be taken into account when the benchmarks :are determined. You are tarring Lattive and Manx with Intel's :brush, when there is no evidence whatsoever that either is guilty. Uh huh, you haven't thought the thing through have you? Let me explain it more carefully: How large a percentage increase in speed do you think you will get by replacing a subroutine-strcpy() with an inline-strcpy() (etc...) ??? Now, does the benchmark give a 'faster' value that agrees with the relative speed increase of your program? Properly, the idea is to repeat the test on a whole shitload of programs (that were not designed specifically to make a benchmark look good), get the mean/average/whatever, and compare that relative speed increase to the relative speed increase in the benchmark by the inline code. -- Does that answer you question? I am not tarring either Lattice or Manx, but pointing out two things people do not seem to understand about benchmarks. (1) Never fine tune a benchmark, and (2) Benchmarks are incredibly difficult to write if written properly. Going a little deeper: A properly written benchmark should be difficult to fine-tune (anybody else catch that inference?) -Matt