Path: utzoo!attcan!utgpu!watmath!iuvax!cica!gatech!hubcap!dinucci
From: dinucci@cse.ogc.edu (David C. DiNucci)
Newsgroups: comp.parallel
Subject: Re: Determinacy Issues in Parallel Programs
Keywords: determinacy parallel programming
Message-ID: <6280@hubcap.clemson.edu>
Date: 18 Aug 89 13:19:24 GMT
Lines: 104
Approved: parallel@hubcap.clemson.edu

In article <6243@hubcap.clemson.edu> boulder!boulder!adamb@ncar.UCAR.EDU (Adam 'Velvis' Beguelin) writes:
>If you knew a parallel program was determinate or nondeterminate
>before you ran it, then it would make writing and 
>debugging the program easier.

(Hi Adam!)  I'm not sure what you mean by easier to write, since you
would do that before you ran it, but since most of your other comments
address debugging, so will I.  

>If the program was determinate you would know that errors produced by
>the program were not an artifacts of race conditions in your program.

You would also know that you could rerun the program within a debugger
and observe it closely without altering its behavior.

>If you knew that the program may be nondeterminate, you could look
>to those portions of the program which may cause nondeterminacy
>to see if these sections of the code contribute to the incorrectness
>of the computation.

...and if a run-time system knew which portions of the program could
allow nondeterminacy, just the results of those non-determinate choices
could be traced, yielding enough information to replay the entire
execution within a debugger while forcing the same behavior.

>If you expected the program to be determinate 
>and were told that it was nondeterminate, this would also be 
>useful information.

This is the real heart of the matter.  In most languages and models,
non-determinacy is an emergent phenomenon:  it's not specified
by the programmer in any one place, but rather creeps in based on how
everything works together, usually unnoticed by the programmer.  Yet,
the programmer *knows* whether the program *should* be determinate or
not, based on the problem specification.

>Also related to determinacy, some languages (i.e. functional languages)
>enforce determinacy, while most parallel languages do not.   Enforcing
>determinacy restricts the number of computations one may easily express.

A non-deterministic choice is one where the program does not specify
which choice is to be made.  (Thus, in my opinion, the concept of "fair
non-determinism" is confused.)  It does not mean that a choice will be
made in such a complex, pseudo-random way that the user will not be able
to guess how it was made, but this is the best that a functional language
can do when the problem statement includes non-determinacy.

There is an implicit assumption that the computer will make the choice
which can be acted on the soonest, even though this is not really part
of the definition of non-determinacy.)

>Does the average Joe parallel programmer need the ability of writing
>nondeterministic programs?  Would he be better off with a language
>that encouraged or enforced determinacy, even though the set of computations
>expressible in such a language may be smaller?

He/She needs nondeterminism if maximum performance is desired from a parallel
computer, and most people would agree that this is kind of important.

It comes down to the notion that there is often a set of acceptable
answers to a computation or sub-computation, each element of the set
representing a different function evaluated on the computation's inputs.
The elements may be simple permutations of each other, may have
different types of roundoff, may be different local minima in a state
space, etc., but they are all acceptable answers.  The goal is to
compute the one that can be computed the soonest, subject to the availability
of the input data, the temperature of the machine room, the activity on
the disk drive, etc.  Unless you wish to build all these parameters into
a function, you are going to need the ability to write non-deterministic
programs.

>Finally, does anyone have references to papers that discuss such topics?
>I know of a papers on the Blaze and Linda languages which touch on these
>issues but I'm not aware of anything that explicitly discusses the pros
>and cons of determinacy in parallel programming.

Try Chandy, Misra's book on Unity.

M. J. Manthey, "Hierarchy in Sequential and Concurrent Systems", in
"Characteristics of Parallel Algorithms", ed. Jamieson, Gannon,
Douglass, MIT Press, 1987.

T. J. LeBlanc, J Mellor-Crummey, "Debugging parallel programs
with instant replay", IEEE Trans. on Computers, April 87

You might also look at the LGDF2 (F-Nets) stuff we're working on,
especially in the HICSS-21 (1988), and COMPCON Spring '89 proceedings,
as well as tech reports 88-005 (Design of a Debugger based on LGDF2),
and 88-006 (A Formal model of computation for LGDF2).

In my soon-(I hope ;-)-to-be-finished dissertation, I provide a form
for expressing any determinate parallel program (shared memory or
message passing) so that it can be known determinate by inspection,
without restricting communication-control patterns between processes.

>
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>Adam Beguelin 			Computer Science Department Box 430 
>adamb@boulder.Colorado.Edu		     University of Colorado
>303/492-7906				      Boulder, CO 80309-430
-- 
David C. DiNucci                UUCP:  ..ucbvax!tektronix!ogccse!dinucci
Oregon Graduate Center          CSNET: dinucci@cse.ogc.edu
Beaverton, Oregon