Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10.2 9/18/84; site amdahl.UUCP
Path: utzoo!utcs!lsuc!pesnta!amdcad!amdahl!sjl
From: sjl@amdahl.UUCP (Steve Langdon)
Newsgroups: net.micro.mac
Subject: Re: C Compilers:  A Comparison (long)
Message-ID: <1231@amdahl.UUCP>
Date: Mon, 4-Mar-85 23:13:10 EST
Article-I.D.: amdahl.1231
Posted: Mon Mar  4 23:13:10 1985
Date-Received: Wed, 6-Mar-85 00:43:57 EST
References: <5056@ucbvax.ARPA>
Organization: Amdahl Corp, Sunnyvale CA
Lines: 158

Gordon Hamachi's comparison of C compilers at the Macworld Expo was certain
to influence the buying decisions of many readers of net.micro.mac.  While
he was careful to disclaim his tests as a definitive comparison, people are
likely to use it to guide their buying decisions.  As someone who attended
the Expo, I realize that it was difficult to perform any tests.  Gordon is
to be congratulated for getting any comparative results.  However, when
posting this type of commentary, care is needed.  Gordon noted the differences
in hardware configuration, but did not provide some other important information.

He has already posted a note on the differences in default int size between
Megamax and Mac C (from Consulair).  There are other differences that will
affect the times he quoted.  For example, "Load+Run" time can be significantly
affected by the version of the Finder used.  Anyone who has used one of the 
2.xX series Finders knows that they launch applications quicker than 1.1g.
I own a copy of FastFinder (TM) which launches much faster than any standard
finder.  Other important speed differences that can occur (particularly on
floppies) relate to file layout on disk.  If a file is fragmented it can
substantially increase the time spent on disk I/O.

All of my comments above are a long-winded way of saying that you should be
very cautious about using a single benchmark to select an important tool
like a compiler.  Many other factors need considered, for example, support
may be important to you (it is to me). As someone who has used Mac C since
last October I would recommend it to other buyers even if the code was 
substantially slower than an alternative.  I would do this because Consulair
have been consistently helpful when I ran into problems.  No compiler is
free from bugs.  One just has to read net.unix.whatever to read about PCC
problems.  Fixes and reasonable means of avoiding existing bugs are important.

Those who are reading comparisons of compilers should be careful to consider
what they are buying when comparing compiler prices.  I cannot describe the
contents of packages other than Mac C.  With Mac C you currently get the C
compiler itself, a pre-release version of the Apple MDS system, and the standard
libraries in relocatable form.  If you buy the Mac C Toolkit you receive some
additional useful library routines in both source and relocatable form, you
also get the source of the standard libraries.  Mac C is available without copy
protection for an extra $25 when you sign a further license agreement.  In
either case a reasonable amount of telephone support is included.  The policy
of other companies is likely to differ, so check what they charge for library
source and support.

By now you are probably bored with reading advice on how to buy a compiler so
I will present some entertainment in the form of further benchmark information.
As people insist on using the sieve, I will go along with the crowd.  However,
we should agree on a common version of the code to insure that results are
comparable.  I have therefore included (below) source that I propose we use for
future comparisons of C compilers for the Mac.  Changes from the original are
timing based on the Mac's tick counter, pauses before and after the run, and
provision for register variables and different integer sizes.

Results on my Mac using a version of Mac C that supports register variables
(semantically as well as syntactically) are as follows:

	*** WARNING THIS VERSION IS NOT AND NEVER WILL BE RELEASED ***
	*** IT IS AN INTERIM VERSION OF THE RELEASE SCHEDULED FOR  ***
	*** APRIL 15.  CONSULAIR'S CURRENT DEVELOPMENT VERSION IS  ***
	*** FASTER.						   ***

			32 bit integers		16 bit integers

vanilla			7.75			6.2
regvars			4.73			3.92
regvars + regpointer	4.45			3.67

One interesting discovery that I made while generating these numbers was that
Bill Duval's Hyperdrive equipped Mac is a lot faster than a standard Mac.  When
I was picking up the compiler used to generate the results above I saw him run
the sieve on his machine.  When I ran the same program on my machine   it was
about 25% slower.  He has verified the speed difference on his machines.  It
appears that there are more reasons to buy a Hyperdrive than just the I/O
improvements.  More on this subject when GCC provides additional information.

Standard disclaimer.  I have no connection with Consulair other than as a
satisfied customer (Who paid full price for his compiler).

#include 

#define regvars
#define regpointer
#define int16bit

#define SIZE 8190
#define FALSE 0
#define TRUE 1
#define NTIMES 10
#define tickCount *((long *)0x16A)

#ifdef int16bit
#Options -I
/*Mac C way to specify 16 bit ints.  Replace as needed for other compilers*/
#endif	/* int16bit */

char flags[SIZE+1];

main() /* compute primes using sieve of Eratosthenes */
{
	long ticks;	/* added to allow timing */
	
#ifdef regvars

	register int i, prime, k, count, iter;
	
#ifdef regpointer
	register char *flag;
	flag = flags;
#else	/* regpointer */
#define flag flags
#endif	/* regpointer */
#else	/* regvars */
#define flag flags
	int i, prime, k, count, iter;
#endif	/* regvars */
	getchar();	/* added to allow disk activity to cease */
	printf("10 iterations: ");
	ticks = -tickCount;
	for(iter = 1; iter <= NTIMES; iter++) {
		count = 0;
		for(i=0; i<=SIZE; i++)
			flag[i]=TRUE;
		for(i=0; i<=SIZE; i++) {
			if(flag[i]) {
				prime = i + i + 3;
				/*printf("\n%d",prime);*/
				for(k = i + prime; k <= SIZE; k += prime)
					flag[k]=FALSE; /* discard multiples */
				count++;
			}
		}
	}
	
	ticks += tickCount;
	printf("%d primes in %d.%d Seconds", count, ticks/60,
	       (((ticks % 60) * 100)+30)/60);
	getchar();	/* added to allow output to be read */
	exit(0);
}

For those without the original posting at hand, Gordon's results are presented
below.  He used default int size, and no register variables.  I do not have
the actual source used.

> 
> Vendor	  Memory  Disk        Compile  Link   Load+Run Run   Size   Small
> ===========================================================================
> Megamax   512K    Floppy       7.9      71    13.0     6.6   5484    5276
> Megamax   512K    Corvus Omni  2.7      23     9.2     6.6   5484    5276
> 
> Softworks 128K    Floppy      77.4     111.9  15.8     8.8  34560   26368
> 
> Consulair 512K    HyperDrive  19.2      14.5  15.2     8.0  10496   10240
> 
> Hippo 2   512K    Ram Disk    11.6       8.8  14.1     ---  13864     ---

-- 
Stephen J. Langdon                  ...!{ihnp4,hplabs,sun,nsc}!amdahl!sjl

[ The article above is not an official statement from any organization
  in the known universe. ]