Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10.2 9/18/84; site loral.UUCP
Path: utzoo!watmath!clyde!burl!ulysses!allegra!bellcore!decvax!ittvax!dcdwest!sdcsvax!sdcc3!sdcc6!loral!ian
From: ian@loral.UUCP (Ian Kaplan)
Newsgroups: net.arch
Subject: Cube designs
Message-ID: <798@loral.UUCP>
Date: Wed, 27-Feb-85 22:42:52 EST
Article-I.D.: loral.798
Posted: Wed Feb 27 22:42:52 1985
Date-Received: Sat, 2-Mar-85 04:05:12 EST
References: <7306@watrose.UUCP>
Reply-To: ian@loral.UUCP (Ian Kaplan)
Organization: Loral Instrumentation, San Diego
Lines: 94
Keywords: Cosmic cube, parallel processing
Summary: Networks are not as simple as some suggest.

In article <7306@watrose.UUCP> cdshaw@watrose.UUCP (Chris Shaw) writes:
>
>The average communication length is 3, with no bus contention along the way.
>The contention is really hidden in the point-to-point waiting for receive and
>transmit.
>

  I think that things are a little more complex than this.  Contention is
  not entirely nonexistent.  Two processors can contend for the bus which
  connects them.  This is probably a minor issue, but it could be a problem
  in applications with heavy nearest neighbor communication.

  Other Problems with Networked Parallel Processors

  Before you start to plan your network connected parallel processor which
  will threaten Cray's dominance of the super-computer market here are a
  few things to consider.

  1. Network routing

  One problem which has not been addressed in the discussion of the cube
  architectures is the complexity of managing communication in the cube 
  network.  I have not seem a description of the devices which handle 
  the interprocessor busses, but I assume that they simply handle access 
  and contention.  Once a message arrives, it must be routed to another 
  I/O channel if it is addressed to that processor.  This seems to 
  require processor intervention.  After all, routing tables must be 
  examined to determine the port through which the message should be sent.  
  This processing will introduce some lag at each stop in the network.  
  If a separate network processor is not provided to handle network routing, 
  it must be done by the CPU which is executing the application code.  
  This will be overhead that will slow down application execution.

  Another area where lag can be introduced is when several processors route
  their messages through the same intermediate processor.  Presumably the
  network router can not handle them all at once and they will have to be 
  queued.

  2. Broadcasting

  Broadcasting in a network system is difficult.  There must be a special
  message header for messages which are broadcast.  When a processor gets
  this message it broadcasts it to all of the processors it is connected
  to.   
  
  All cube processors are connected to an ethernet, but I wonder if
  multicasting is supported.  Even if multicasting is supported, this
  divides communication between two media, one where multicasting is
  supported and one where it is not.  This adds still more complexity to
  the system.

  4. Software
  
  People have been building parallel processors for some time and it is now
  widely realized that the hard part is not constructing the machine, but
  programming it.  Below are listed several areas which must be addressed
  in the software for a network system.

  A. Associating Logical Communication Channels with Physical Communication
     Channels.

  I have not yet received anything from iNTEL on the software for the iNTEL
  cube, so I am still uncertain about how they program it.  According to the
  ACM article on the Cosmic Cube, Cal. Tech. uses communicating parallel
  processes.  I assume that iNTEL does the same.  When process A
  communicates with process B, it does so via a logical communication
  channel.  No processor address is specified.  When the program is loaded
  onto the cube there must be a mapping between the logical communication
  paths and the physical processor address.

  B. Programming Parallel Processes is Difficult

  A number of people have commented that programming applications composed
  of parallel processes is more like writing operating systems than
  applications as we usually regard them.  A particularly good discussion
  of the problems encounted in programs composed of parallel processes is 
  provided in [1].  In the Cosmic Cube article no example was given of an 
  application which had heavy interprocess communication.   By not
  discussing this, the author side stepped an important issue.  If the 
  iNTEL cube follows the Cal. Tech. model I think that it will be very
  hard to program.

  [1] "Exploiting Multiprocessors: Issues and Options" by James McGraw and
      Timothy S. Axelrod, Preprint - Lawrence Livermore National Labs.  
      UCRL-91734 Available from - Technical Information Dept. LLNL, 
      Livermore, CA 94550

  
			 Ian Kaplan
			 Loral Instrumentation
			 (619) 560-5888 x4812
		 USENET: {decvax, ucbvax,ihnp4}!sdcsvax!sdcc6!loral!ian
		 ARPA:   sdcc6!loral!ian@UCSD
		 USPS:   8401 Aero Dr. San Diego, CA 92123