Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!seismo!rutgers!topaz.rutgers.edu!klinzhai.rutgers.edu!webber
From: webber@klinzhai.rutgers.edu (Webber)
Newsgroups: news.admin,news.misc,news.groups,news.sysadmin
Subject: The Requested Presentation of Quota Based News Control
Message-ID: <285@klinzhai.rutgers.edu>
Date: Sun, 5-Jul-87 07:43:57 EDT
Article-I.D.: klinzhai.285
Posted: Sun Jul  5 07:43:57 1987
Date-Received: Sun, 5-Jul-87 18:40:59 EDT
References: <266@brandx.rutgers.edu> <8225@utzoo.UUCP> <272@brandx.rutgers.edu> <153@tmsoft.UUCP>
Organization: Rutgers Univ., New Brunswick, N.J.
Lines: 210
Summary: please consider that the author of this message is probably more interested in your response than the rest of the net, so send mail.
Xref: mnetor news.admin:613 news.misc:691 news.groups:1166 news.sysadmin:270
To: utzoo!utgpu!tmsoft!mason, webber@aramis.rutgers.edu


In article <153@tmsoft.UUCP>, mason@tmsoft.UUCP writes:
> ...                                I would suggest you explain again your
> quota idea; then let it lie fallow.  Someone may pick up on it & it may
> save the net (if you want to get credited with saving the net, you should
> probably re-post every 6 months or so, so everyone knows it's your idea).

Ok.  Below I present how the quota based system would work and the
answers to those objections that I am aware of.  Anyone who gives much 
thought to it will realize that the net is not in eminent danger of
following my plan, so it should be sufficient to send me your objections
via mail.  If I find one overwhelming, I will gladly post to the net a
message giving you credit for straightening me out and apologizing for
the confusion that my ignorance caused.  I will keep an updated copy of this
message available for request by mail and for occasional reposting until
it no longer seems relevant to the net.  It is because of the existance
of this alternative that I believe one can legitimately oppose the conversion
of the net to all-moderation in the face of the net's limited bandwidth.

I view the entire net flow as one continuous stream of messages.  So far,
nearly every aspect of the message has been used as a basis for determining
whether or not that message will be passed on or taken except for
whether or not there is room for it.  On the other hand, when calculating the
expense of handling the net flow for any given site, whether it is disk
storage, communication bandwidth, or cpu that is the bottleneck, the
expense is always a function of the sum of the sizes of the messages
regardless of the other aspects of the message.

The closest the net has come so far to using quotas has been that some
systems impose limits on the size of messages due to problems uucp has/had
with large individual messages.  Since that quota arose at a time when 
nearly everyone had the same problem, it worked out rather well.

There are two places one could contemplate putting quotas, either on
the originator of the message or on the channel through which the
message flows.  I believe that placing a quota on the channel is the
simplest way to handle the problem.  Indeed, once a quota is imposed
on the channel, the technical problem of too much flow is solved and
the question of the nature of the flow can be handled at a pace that
is appropriate to the evolution of a new media.

Currently there is already a quota on all the channels, in that there
is a maximum flow that each can handle.  However, since the channels
have other uses than news, news must be prevented from dominating each
of these channels.  The amount of news flow that each channel (link between
neighboring nodes) can handle comfortably is differs from channel to channel.

One question that arises is how do you find out the size of a message.
There are two places that that can be done.  It can be done by remotely
querying the site from which the message is to be transfered or it
can be done by keeping a running total of the size of the messages transferred
so far and stop the transfer once they reach some cutoff (this total
could be calculated either locally or remotely).  Clearly it is not necessary
for both sites on the link to agree on the quota for that channel, but it
would certainly make it easier to handle for both of them.

The question arises of what will happen to the net when sites start refusing
to carry more than x bytes of news per day.

If communication bandwidth were all of the problem, then news would simply
pile up at various sites until it eventually got tranfered.  However,
at the the current rate of 2 meg per day, it is not difficult to imagine
that limitations on disk space would quickly dominate the situation.

Clearly if there is no room for something where it is and no place to put
it, it is not going to last long.  It is interesting to note that what
happens happens on the remote machine and not on the local machine.  Thus
it is the remote machine that will set policy on which of the unsent messages
will be expired (unless someone wants to take on a considerably larger
implementation task than just the one I am advocating).  

There are two options here: 1) delete some of the messages already in the
queue and 2) stop adding new messages to the queue.

Both the first and the second option mean that some people will miss some 
messages since some sites will be able to circulate much more news than
others.  However,  given the number of messages available and the bandwidth 
restrictions of the links in the net, this is inevitable and already the case.

Currently those people who are choosing to receive only a subset of the net 
are doing so based on group name.  This means that site administrators must
take responsibility for decisions such as whether or not it is a proper
utilization of their resources to carry a group whose discussion topic is
a computer that they don't and never plan to own or a hobby such a 
birdwatching, or job offerings from competitors, or the pros and cons of
abortion, or the philosophical aspects of the sciences.  If one attempted
to justify the groups one was transmitting on the basis of their content,
I doubt if there would be more than 4 groups that could manage country wide
distribution.  However, there is another aspect to these groups beside their
content and that is the morale of the participants in the various discussion
groups.  From a morale point of view, each of the groups is justified (and
many more groups as well).

Let us look at the various kinds of posting and consider the significance of
them getting `dropped on the floor.'

A request for information:  almost inevitably, the queries being
handled on the net are answerable elsewhere.  The public libraries
in the United States are adequate for handling many of them.
Contacting neighboring universities and colleges would result in
finding people who could handle most of the rest.  It is also worth
noting that if the questioner had been watching the traffic of the
group much, it would not be difficult to identify a few experts to 
whom a direct computer mail query could be sent.  Of course, most people
have anecdotes of questions that only the net could answer; these
are interesting because it is so rare and notable.  Certainly, the 
worst that would happen from dropping most queries for information on
the floor is that some people would learn how to use libraries and
browse professional journals.

Answer to the request for information:  clearly anyone seriously interested
in helping the person making the request would send them direct mail
to make sure that they noticed the answer.  However, most people like to
use queries for information as an opportunity to place that information
before a larger audience (this message is an example).  Thus, it doesn't
matter to the sender who specifically recieves the message just as long
as it gets wide enough distribution so that it has some possibility of
generating some action somewhere.

Blanket postings of information:  as noted above, no one is really maintaining
that every person who logs into a unix system should be given an opportunity
to see their message, just that the message is of general interest and
so they saw no reason to restrict its distribution anymore than the
net already does.  Thus if it doesn't make it to some places, the world will
not come to an end.

Part n of m:  this kind of message is rare, i.e., most people expect their
messages to be able to stand on their own basis as opposed to being
one of a series that is useless unless you collect the whole series.
However, there is one notable exception: large source postings.

One question of interest is just how big does a source posting have to be
before it will cause more trouble by trying to be sent as a single message
than it will cause by getting separated from its other pieces.  It would
seem to me that communication has improved enough over the past few years
that it would be worthwhile investigating the question of how long different
links would take to transfer an x byte file (not due to the rate of
the link but due to the strategy the link uses to handle errors in 
transmission).

Another question is: just what is happening when someone tries to post
a very large source or binary to the net?  Clearly such sources and
binaries were not meant to be read and hence are not intended as communication
between people.  Instead, they are intended to be used.  If a neighboring
node has a program that my node can use, then it makes sense to
establish an ad hoc link to transfer it (perhaps, even in rare cases 
transferring it piecewise via direct mail).  Thus, one could imagine
an announcement of an 800k source being made on the net and then it
actually move through the net as a chain letter (site to site NOT user
to user).  For that matter, floppy disks and mag tapes through the regular
post have been underutilized.  A source that isn't worth the postage and
media to the reciever probably isn't worth posting to the net either.

Rather than writing monolithic programs that do only one thing, it would
make more sense to post to the net small general purpose utilities that
other people could read and use within their own code.

While admittedly, the above is from the viewpoint of someone who can
write their own sources, I would maintain that it is less than totally
clever for a person who cannot program to use a source or binary that
they recieved off the net.  

In summary, you could say that I see the difference between mail and
news to be in the case of mail you know exactly who you want to send
to and the system tries to offer you as much support as possible in
getting your message to the recipient, whereas in news you really
don't know if there is anyone out there interested in your message and
the system distributes your message according to what links are interested
in transferring how many blind messages.  In neither case do the links
attempt to intrude by judging the contents of the messages, although the
systems have quite different behavior based on the intended usage.

Thus, I do not see any problem being generated from the use of quotas
to manage net news due to occasional loss of messages.  Indeed, I
see it as actually encouraging a more responsible usage of the media
in conjunction with making joining news less of a problem for individual
site managers.  Neither do I see quotas as causing any implementation
problems.  I await enlightenment.

> OR
> You can program your quota system and get Rutgers to use it, then
> on the basis of whatever is wonderful about it at Rutgers, convince your
> net neighbours, then the state, then the east coast, and by then you can
> probably convince the rest of us.

I by far think that this would be the best way to handle things, but
Rutgers is not a site where the flow has gotten so bad that people are
advocating the closure of unmoderated groups and the general control of
the flow through moderation.  Thus, implementing it at Rutgers would 
show that the code works, but would not address the advisability of actually
using it.  No one has so far maintained that the idea would be difficult to 
code in a manner that would integrate with the rest of the net.  Instead,
the discussion has always rested on the advisability of using it even
if it already existed.  

I believe I have adequately addressed all the issues that have formed
a basis for objection in the past as summarized above.  I have been
addressing this issue off and on since February and over that time
my understanding of the problems of implementing a quota based system
system within the structure of Usenet has grown.  In the past I have
stressed the notion that the net would adapt to the bandwidth it found
by reducing the number of postings and that this would occur by having
quotas push further and further back into the system until individual
sites were rationing the postings from their own users.  It now strikes
me that there would be little motivation for this since once the quotas
are in place, the pressure for changing the Usenet setup will be decentralized
and take a wholely unpredictable course (although I have not yet extrapolated
a future that would be worse than the currently expected one of increasing
use of moderated groups and group-name based all or nothing flow decisions).

----- BOB (webber@aramis.rutgers.edu ; rutgers!aramis.rutgers.edu!webber)