Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10.1 6/24/83; site utah-cs.UUCP
Path: utzoo!watmath!clyde!burl!ulysses!unc!mcnc!decvax!decwrl!greipa!pesnta!amd!amdcad!lll-crg!seismo!utah-cs!shebs
From: shebs@utah-cs.UUCP (Stanley Shebs)
Newsgroups: net.ai
Subject: Re: Any data on programmer productivity
Message-ID: <3511@utah-cs.UUCP>
Date: Wed, 30-Oct-85 18:08:29 EST
Article-I.D.: utah-cs.3511
Posted: Wed Oct 30 18:08:29 1985
Date-Received: Sun, 3-Nov-85 05:30:39 EST
References: <526@ihwpt.UUCP> <3500007@uiucdcsp>
Reply-To: shebs@utah-cs.UUCP (Stanley shebs)
Organization: Univ of Utah CS Dept
Lines: 42

In article <3500007@uiucdcsp> forbus@uiucdcsp.CS.UIUC.EDU writes:

>Your best source for raw performance data is
>	"Performance and Evaluation of Lisp Systems"
>	by Richard P. Gabriel, $25 from MIT press
>
>This is the book form of Gabriel's DARPA-funded study.  One of the amazing
>regularities is that performance of specialized lisp machines is vastly
>better than any mainframe run as a STAND-ALONE machine, much less
>time-shared.  The only exception is running a Cray stand-alone; no
>measurements are available for a time-shared Cray.

Are we looking at the same study?  The numbers on my crummy old xerox
copies of the results don't show that at all.  Admittedly they are
incomplete compared to the book (which I've not yet purchased a copy
of), but I don't believe the numbers have changed any.

On the average, the 3600 does the best of any lisp machine.
The 3600 gets 12.0 secs for the boyer benchmark, while PSL on a DEC-20
(a mainframe presumably) gets 11.96 secs.  Not "vast" in my book.
PSL on a 68K (HP200) gets 25.52 secs, which is half as fast as the
3600.  I suppose that might be considered "vast", but then the 68K
*is* a microprocessor...

For other benchmarks, the 3600 (and the other lisp machines) fare worse.
Running "browse", the 3600 gets 32.1 secs against the HP200's 39.84;
a 20 running PSL gets 23.81 secs.  Running "puzzle", the 3600 gets
14.21 secs, and the HP200 zips along at 10.85 secs!  That's the
best result; the worst puts the 68K PSL about 6 times slower.
The mainframes range all over from substantially better to substantially
worse.

My reading overall is that the 3600 is on the average comparable
to a VAX 780 running PSL, and faster than a 68K Unix box, but not
impressively so.  The true moral of the study is that no Lisp
can be judged "best" or even "fastest" on the basis of benchmarks;
each implementation has its strengths and weaknesses.  In fact,
the proper audience of this book is the implementors themselves;
we now have a basis for deciding what aspects of our Lisps need work,
and which implementations we ought to look at to improve those aspects.

							stan shebs