Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10.2 9/18/84; site peora.UUCP
Path: utzoo!watmath!clyde!burl!ulysses!mhuxr!mhuxt!houxm!vax135!petsd!peora!jer
From: jer@peora.UUCP (J. Eric Roskos)
Newsgroups: net.arch
Subject: Re: RISC and MIPS
Message-ID: <1512@peora.UUCP>
Date: Thu, 22-Aug-85 09:00:19 EDT
Article-I.D.: peora.1512
Posted: Thu Aug 22 09:00:19 1985
Date-Received: Sun, 25-Aug-85 04:27:28 EDT
References: <419@kontron.UUCP> <2300001@uicsl> <1093@ames.UUCP>
Organization: Perkin-Elmer SDC, Orlando, Fl.
Lines: 69

> Whetstones were mentioned in another letter, but the only people who use
> these are computer manufacturers.

This statement isn't true.  For example, back when I was a graduate-student
researcher in computer architectures, we used the Whetstones to test our
vertical-migration software.

> What qualities do our performance metrics need to have?

I think you need to make your performance measurements in such a way that
you get a set of distinct numbers which can be used analytically to determine
performance for a given program if you know certain properties of the
program.  For example:

1) The rate of execution of each member of the set of arithmetic operations
provided by the machine's instruction set, assuming the operands are all
in registers, with cache disabled.  This would give you an approximation of
the execution time of the algorithms for the arithmetic operations,
along with the instruction-fetch times, when not aided by caching.

2) The rate of execution of 1-word memory-to-memory moves, with cache
disabled.  This gives you the word-sized operand fetch and store times,
along with (again) the instruction-fetch times for these instructions.

3) The rate of execution of a tight loop performing only register-to-register
moves, with cache disabled.

4) The rate of execution of a tight loop performing only register-to-register
moves, with cache enabled.

5) The rate of execution of a tight loop performing (same word size as #3
and #4 above) memory-to-memory moves that produce all cache "hits", with
cache enabled.  Note that this gives you two properties of your cache: your
speedup for operand fetch and store resulting from caching, and any
performance penalties resulting from a write-through vs. write-back cache.

6) Specifications such as the number of registers available to the user,
the size of the cache, etc.

Well, you get the idea, anyway... personally I tend to feel that statistical
performance measurements are not nearly as useful as analytical ones; I
would rather see a list of fairly distinct performance properties of a pro-
cessor anytime, since I think you can do more with them in terms of
saying how the machine will perform for a given application that way.
To do this, you do need to understand your application, though.

I separated out the various forms of caching (operations in registers, and
use of a cache between the CPU and the primary memory) because so many
people "fudge" their results that way without giving any information from
which you can determine real performance.  The above list is just meant to
suggest "qualities" rather than being an exhaustive list; i.e., that the
performance metrics should reveal (rather than hide) the set of factors
that actually influence performance. [Unfortunately, this would never suit
most marketing organizations nor customers, since they want an all-
encompassing number.]

The metrics should also be compiler-independent, especially if you are
making measurements on microcomputers, since the majority of microcomputer
compilers today generate terrible object code (see my posting awhile back
of a "hand-compiled" 68000 program for the Macintosh in net.micro.mac for
an example of how bad this can be (and how little the significance of this
was understood!)).
-- 
Shyy-Anzr:  J. Eric Roskos
UUCP:       ..!{decvax,ucbvax,ihnp4}!vax135!petsd!peora!jer
US Mail:    MS 795; Perkin-Elmer SDC;
	    2486 Sand Lake Road, Orlando, FL 32809-7642

	"Gurl ubyq gur fxl/Ba gur bgure fvqr/Bs obeqreyvarf..."