Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!seismo!vrdxhq!verdix!ogcvax!dinucci
From: dinucci@ogcvax.UUCP (David C. DiNucci)
Newsgroups: comp.arch
Subject: Re: parallel computing
Message-ID: <1350@ogcvax.UUCP>
Date: Sat, 11-Jul-87 20:01:20 EDT
Article-I.D.: ogcvax.1350
Posted: Sat Jul 11 20:01:20 1987
Date-Received: Mon, 13-Jul-87 00:44:33 EDT
References: <8270@amdahl.amdahl.com> 
Reply-To: dinucci@ogcvax.UUCP (David C. DiNucci)
Organization: Oregon Graduate Center, Beaverton, OR
Lines: 68
Summary: Portable parallel programming is getting closer

In article  henry@utzoo.UUCP (Henry Spencer) writes:
>But *which* *flavor* of parallelism?  And how are they going to do this
>when the languages don't support it in a standard way?  And how are the
>languages going to support it in a standard way when the current machines
>are all so different from each other that's there practically no common
>ground?  Parallelism is never going to get really popular in programming
>until there is some vaguely portable way of expressing it, or some utterly
>overwhelming consensus on the preferred architecture of parallel machines.
>I am not holding my breath.

Some ideas on this subject are starting to emerge.  I refer you to a
book just being published by MIT Press called "The Characteristics of
Parallel Algorithms" where a number of them are discussed by their
proponents -  Lusk and Overbeek's monitors, Harry Jordan's "The Force",
and Robert Babb's Large Grain Dataflow (LGDF), to name a few.  I believe
Lusk and Overbeek are writing another book on their approach
with a number of other people.  With luck, a paper
by Babb and myself will be presented at the HICSS-21 conference in
Hawaii this Winter on the new version of LGDF.

There are lots of similarities in these models.  Most all use some
common sequential language as a basis, then supplement it (usually
with macros) with new constructs which are implemented in a machine-
specific way.  They also each have one-flavor for shared-memory machines
and another flavor (or perhaps no flavor) for distributed memory
machines.

LGDF, which I am most familiar with, is becoming what you might call a
very-high-level graphical language.  That is, an LGDF program IS a
network, and the nodes in it, which would be analogous to
statements in other languages, are actually sequential processes
in LGDF.  These processes can be written in any supported high-level
language (currently C and Fortran), augmented by two "special"
statements (macro calls); "grant and "receive".

The idea here is to let the high-level language (C or Fortran)
abstract away the low-level machine-specific details of the sequential
instruction set of the machine (as they have always done), while the
LGDF language abstracts away the low-level machine-specific details
of the inter-process interactions (including operating-system interface,
memory hierarchy, and interconnection structure).  The new version of
LGDF does this well enough that the same (source) program should run
well on shared- and distributed-memory machines.  We have implemented
it on the Sequent with good results.  With luck, the Intel hypercube
will be soon.

There are other ways to go, such as writing programs in declarative or
dataflow languages or CSP in which sequential execution is not implied
by statement ordering, but rather by other constructs and only when the
programmer or compiler deems it absolutely necessary.  Mastering these
new techniques will not happen overnight.  Another way, of
course, is to hope that vectorizing and parallelizing compilers will
come to the rescue.  All of these approaches have their drawbacks and
advantages.

In any case, the software development cycle, especially debugging,
should not be forgotten when considering a solution to this problem.
If one were watching the execution of a parallel program using a
debugger, would its actions ever surprise the user or be hard to follow,
even if the user had a source listing handy or other tools handy?  Is it
even feasible to write a debugger?  My own view is that too many people
are offering new parallel constructs and models without considering
these issues.

Dave DiNucci    dinucci@Oregon-Grad     ..!sequent!ogcvax!dinucci

Disclaimer:  I co-authored the two works on LGDF cited here, but will
  not benefit financially from either one (to the best of my knowledge!)