Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10.2.fluke 9/24/84; site vax1.fluke.UUCP
Path: utzoo!watmath!clyde!cbosgd!ihnp4!houxm!vax135!cornell!uw-beaver!fluke!witters
From: witters@fluke.UUCP (John Witters)
Newsgroups: net.micro.68k,net.arch
Subject: Re: Multiple 68020's on VME ?
Message-ID: <1192@vax1.fluke.UUCP>
Date: Tue, 1-Oct-85 12:37:10 EDT
Article-I.D.: vax1.1192
Posted: Tue Oct  1 12:37:10 1985
Date-Received: Thu, 3-Oct-85 04:18:40 EDT
References: <442@rna.UUCP>
Organization: John Fluke Mfg. Co., Inc., Everett, WA
Lines: 89
Xref: watmath net.micro.68k:1178 net.arch:1842

> 
> 	We would like to hear from people who know about or who have
> used multiple 680XX's on a bus. We are tentatively considering a ~10
> processor machine using 68020's on VME for a particular real-time
> data collection, analysis and display application.
> 

I'd suggest reading the August 1st 1985 issue of Computer Design before you
rush off and do this.  The article of interest is titled "Metastability haunts
VME bus and Multibus II system designers" on page 29.  I'll quote the relevant
section below.  I haven't looked too closely at this, but it seems to me that
the interrupt daisy chain scheme should suffer from the same problem.  If you
can't overcome the metastability problems, maybe you could loosely couple the
processors using the VMSbus instead of the VMEbus.  The VMSbus is a synchronous
high speed serial bus, so by definition you shouldn't have metastability
problems.  Another solution is to use a different bus request level for each
board in your system, but this limits one to only four bus masters since the
VMEbus has only four bus request levels.

Multiprocessor system fails

An early victim of metastability in a VMEbus product was John Willis, head of
the Rapid Bus multiprocessor project at Carnegie Mellon's Robotics Laboratory
(Pittsburg, PA).  Willis had planned to use Motorola's VM02 cards as
68000-based CPU nodes in a multiprocessor design, but eventually discarded the
VM02 parts.  According to Willis, a VMEbus-based system using as few as two
8-MHz VM02 cards would lock up within 4 to 10 minutes.  The Motorola
specification states that up to 16 VM02 boards can operate reliably in a
multimaster configuration.

Willis was able to isolate the failure to the VM02 card's dual-port arbiter,
bus request arbiter, and bus requester.  His biggest source of trouble was the
dual-port arbiter, which controls access to each VM02 card's dual-port memory.
The dual-port arbiter decides whether the VM02's onboard 68000 processor will
access memory, or whether a processor on another board will access the memory
via the VMEbus.

Metastability problems arose in the arbiter's synchronization device, which was
responsible for synchronizing the two bus-request sources with its own clock.
(The arbiter's clock is synchronous with the 68000's clock.)  Because the
arbiter makes its arbitration decisions in about 20ns, the output of its
synchronizer has only 20 ns to settle to a stable state, but needs at least 50
ns to ensure reliable operation.  The VM02's bus-grant arbiter proved unreliable
for the same reason -- it was trying to make dual-port arbitration decisions in
only 20 ns.

The bus requester's function is to issue bus requests and sample the bus-grant
signal.  Each master on the VMEbus has a requester, and the requesters are
arranged in a daisy chain fashion, such that the bus-grant signal issued by the
arbiter passes serially through each requester.  Each requester decides whether
to intercept the bus-grant signal and access the bus, or pass the signal on to
the next requester on the daisy chain.  If the master associated with a
particular requester has a request pending, the requester will intercept the
bus-grant signal.  If the master has made no request, it will pass the grant
signal along to the next requester.

This scheme is unreliable, Willis claims, because of the asynchronous
relationships between the bus-grant and bus-request signals.  He points out
that the device in each requester that synchonizes bus grants and bus requests
is not allowed enough time to settle.  If the rising edge of a request signal
at one synchonizer's input coincides with the rising edge of a grant signal at
the other synchronizer's input, the requester will behave unpredictably, and
may grant the bus to two masters at once.

Support for Wills' argument comes from Dave Barr of Indocomp (Drayton Plains,
MI), the designer of that company's line of VMEbus-based multiprocessor
systems.  During a test in which two masters on the VMEbus periodically wrote
data to a global memory (also on the VMEbus), Barr found that the bus
collisions between requesting masters caused incorrect data to be written to
memory.  To avoid bus-collision problems on the VMEbus, Barr scrapped the
VMEbus' arbitration strategy in favor of a synchronous arbitration scheme
which uses a single 4-MHz clock to coordinate bus requests and bus grants.
This solution places an upper limit on arbitration speed, but guarantees
system reliability.

Barr could have modified the VMEbus' arbitration logic, but admits that his
lack of familiarity with the problem of metastability led him to avoid it
altogether.  A potentially simple fix was not implemented because of a lack of
understanding.
							Ken Marrin
							Senior Editor
-- 

						John Witters
						John Fluke Mfg. Co.  Inc.
						P.O.B. C9090 M/S 243F
						Everett, Washington  98206

						(206) 356-5274