Path: utzoo!utgpu!water!watmath!clyde!bellcore!faline!thumper!ulysses!andante!princeton!njin!aramis.rutgers.edu!webber
From: webber@aramis.rutgers.edu.UUCP
Newsgroups: news.misc
Subject: Re: "NNTP has had a number of very bad effects on the net..."
Message-ID: 
Date: 10 Jul 88 02:51:57 GMT
References:  <4277@pasteur.Berkeley.Edu>
Organization: Rutgers Univ., New Brunswick, N.J.
Lines: 270
To: fair@ucbarpa.Berkeley.EDU

In article <4277@pasteur.Berkeley.Edu>, fair@ucbarpa.Berkeley.EDU (Erik E. Fair) writes:
< I concede point 1 about speed of article propagation. This was, in
< fact, one of my goals. Netnews propagation across the Internet is
< nearly as fast as an Internet mailing list.
<
< You assert that this is bad, because it has increased the level of
< idle chit-chat in netnews. It is not clear to me at all that the
< effects are related. The ARPANET (and the Internet) have had much
< faster message propagation in their mailing lists for many years,
< and those mailing lists haven't suffered this effect, with the
< exception of those mailing lists whose purpose is idle chit-chat
< in the first place. 

Do you count the AI digest, RISKS, and SF-LOVERS as mailing lists whose
purpose was idle chit-chat in the first place.  For the most part, ARPANET
mailing lists are a real looser.  Generally someone has a neat idea of what
they would like to see a discussion of and a bunch of people subscribe
and then find out they are all waiting for someone else to shed wisdom.
When this fails to happen, they revert to discussing how neat it would
be if they knew something about the topic at hand.  After a few months
this generally dies down and then a year later you get a message in you
box saying ``hey is anyone out there?''

The only exceptions to this have been groups specifically dedicated to
the maintence of some specific software, e.g., the networking
discussion lists and the gnu list.

<                                Also, is there any particular newsgroup or
< class of newsgroups that exemplify the effect you cite?

No -- the effect is rather evenly spread throughout the net.  If you supply
me with the last 5 years archives in a handful of technical groups such
as comp.arch, I will be more than glad to give you a detailed analysis
demonstrating the thesis.

< The secondary effect of people using mail in place of netnews is
< not related to NNTP either; that has been going for a very long
< time, because the UUCP network doesn't have a universally distributed
< reliable mail system (Thanks AT&T!). 

Well, after all, they made money on both the sending and bouncing so you
really can't look to them for help.  

Six or seven years ago, mail worked very reliably.  Not all mail paths
would work, but those that did tended to continue to.  So while contacting
random people on the net might have required more effort, once contacted
it was easier to maintain the link.  Merging the UUCP network into the
ARPA network has been a mixed blessing headed toward unmixed regret of
which nntp is just one more nail in the coffin.  Sticking to the
original technology, by now UUCP could have been entirely free of
institutional connection, instead being run by unix hackers on their
home machines.

< on the UUCP network. I assert that this is because the Internet
< has a real, working electronic mail system.

Giggle.  Where have you been for the last few years as the Internet changed
the way it handled address lookup.

< How has increased speed of article propagation reduced the
< effectiveness of the cancel control message?

The farther a message gets before the cancel message is issued, the
less likely the cancel message will find all the places it went.
Alot of this has to do with most of the news flowing over slower links
and nntp increasing the number of points at which the original message
might enter hence the possibility of a cancel message issued 15 minutes
after the original ending up traveling a day or more behind the original
in some portions of the net.

< In the second point, I quarrel with your wording; NNTP has to some
< extent *changed* the economics of the network, but I don't think
< they're messed up, although I'm sure that the Europeans would say
< that the U.S. USENET has had its economics screwed up since the
< word "go."

Well, the Europeans are just as messed up, so there is no problem
there.  Originally the net was small enough that although it was
``payed for'' by institutions, they seldom noticed it.  Now it has
grown big enough that it is coming more and more to the attention of
these institutions and so things are getting shakier.  The problem is
more that the original economic structure just doesn't scale up
well and that nntp has hastened the attempt to scale it up while
decreasing the ability to convert to something more solid.

< I was the one who suggested that the NNTP distribution network
< should be set up to strictly reflect the physical connectivity of
< the ARPANET; in this way you can make the guarantee that no article
< will cross a physical link more than once, modulo IMP routing
< irregularities. If one were then to convert all the public Internet
< mailing lists to netnews newsgroups transmitted by NNTP, there
< would be a number of very interesting effects, not the least of
< which would be lower agreggate ARPANET traffic.
<
< Unfortunately, I don't have the power to declare my views as
< regulation; all I can do is persuade, cajole, and jaw-bone, and
< my efforts in this regard have had little effect. It seems that
< netnews links are built more on personal contacts than anything
< else, and most of the system administrators I know are very reluctant
< to take the initiative to contact an obvious neighbor and set up
< the links on a more rational basis if they don't already know the
< individual on the other side. Part of this is a trust problem -
< can I trust the guy on the other end to provide reliable service,
< and respond promptly to problem reports?

If you had set up nntp so that it dynamically figured out who its optimal
news neighbors were instead of relying on a fixed file that an
administrator would have to continually monitor, you might have had
a chance.  But leaving it up to hundreds of new news administrators to
worry about this along with all the other problems of bring up new 
software was obviously [20-20 hindsight] too much to expect.  Of course,
once something is set up, only breaking it will get it to change.

< Also, be aware that Internet links cost too, just in different
< geld.

But not in a manner where the person using them feels that they
personally have any influence over the availability of the service.
I believe in economics this is referred to as the paradox of the Commons
(after similar problems with shared grazing pastures in the British Isles).

<...
< gateways to other organizational networks, and the offered load
< from these other networks to the ARPANET is staggering. No wonder
< the ARPANET is staggering under it. It also doesn't help that the
< LANs connected to the ARPANET are primarily 10Mbit/sec Ethernets,
< and the ARPANET IMP-IMP trunks are all 56Kbaud DDS circuits.

And to this you thought it wise to add NNTP???????????

< Prior to the release of NNTP, a number of organizations were
< exchanging netnews over the Internet, either by batching with SMTP,
< or by using UUCP over IP/TCP, and the numbers were increasing.
< NNTP's release accelerated the exchange of netnews over the Internet
< enormously, by using a much more efficient protocol, and by solving
< the problem of local netnews access for sites without distributed
< filesystems or the resources to run netnews on all their systems.

So, what is the bottom line?  Was the efficiency enough to compensate for
the extra usage or did it just dig the grave faster?

< NNTP changed the economics from one of communications cost, to one
< of CPU and disk cost. The limits are still there, they're just
< different for the Internet sites. NNTP hasn't changed the economics
< for a UUCP site, save that they might be able, due to the relative
< ubiquitousness of the Internet, to get a full netnews feed physically
< closer to where they are (and thus cheaper in phone bills). These
< changes would have happened anyway for all organizations that have
< internal networks or access to the Internet; NNTP just speeded it up.

Hardly.  The major economic significance of NNTP communications is that
it combines cheapness with institutionalism.  While it may increase the
number of readers, all of the new readers are dependent on the old
institutions because the old institutions interconnect whereas the new
readers just connect up to the old institutions.  

CPU and disk cost have ALWAYS been the bottleneck in effective usage of
the net.  Why do you think there are no archives?  Why do you think most
people can afford to keyword search incoming news?

< I didn't understand what you meant in point three, can you expand it?
< Particularly the part about backbone sites handling more "local"
< traffic?

I think I just did above (as will as in my reply to Joe Buck).

< the agreggate duplicate rejection rate under 40%). Granted that
< the new backbone may not be using the bandwidth of the underlying
< network as well as it should be; this is not something I can fix.

Actually it is something you COULD fix, but I am sure you have other
things to do.  For that matter, it is something I COULD fix as well --
although it is not clear how many people would be interested in me
mucking around with their net connections.

< Some people are probably asking themselves what the Internet is
< getting out the deal. Answer: a computer conferencing system that
< beats all hell out of mailing lists. When a mailing list converts
< completely to a netnews newsgroup, you get:
< 
< 1. reliability - no central distribution point because postings
< 	radiate out from their origin. Also, netnews revels in

Hmmm, I guess you haven't noticed the tendency toward moderated lists.

< 2. simplicity of administration - once netnews is set up, you have a
< 	distribution channel, and new newsgroups are very easy to
< 	add. The software automatically notifies everyone reading
< 	netnews that there is a new newsgroup, and would they like
< 	to read it?

At which point it becomes a major administrative decision.  Whereas,
a mailing list anyone can anytime decide to ask to be added without
getting it cleared thru the site admin who may or may not have any
interest in the topic of the particular group.

< 	No more mailing list administration - readers of a newsgroup

Granted it is easier on the mailing list maintainers to not have to
maintain a mailing list, but it is not clear that it is easier on
the random user to have to deal with their local news administrator
and those they connect to than to just send a note to the mailing list
maintainer.

< 	No more cryptic errors for the users to interpret - did that
< 	last mailing get to everyone on the list? What does this
< 	bounce mean?

Well, if they are going to be using mail, it is best they learn
sometime -  I particularly love the AT&T messages that don't include
any indentification of the mail message but just tell you some uux
process failed.

< 	centralized information access - can you find every mailing
< 	list on the Internet?

Well, actually if I wanted to I could (it would be a rare mailing list
that didn't send traffic thru rutgers).

< 3. efficiency - netnews by NNTP is delivered on a per-network basis,
< 	not a per-host basis, which means fewer long-haul connections
< 	to make in distributing the messages.

Actually, alog the the newer mail software bundles messages too
cutting down on this problem.  

< The only thing we can't really give people is privacy - so private
< membership mailing lists will continue to exist.

Oh, privacy is no problem.  You can forge messages to protect your
identity (e.g., the recent friend of Mel's who posted the sendsys
request) and you can always post encrypted messages to binary groups.
Rot13 is just the tip of the iceberg.

< In summary, I fail to understand your objections to the effects of
< NNTP, given that you are a proponent of the free flow of information.

Does it make more sense to you now?

< NNTP has made it possible for the network to grow much more quickly
< and painlessly than it otherwise might have, and I think it has
< staved off (albeit temporarily) the coming balkanization of the
< USENET. There is more information flow through USENET because of
< NNTP than there would have been without it.

Currently our library has a shelving problem.  In order to take in new
books they have to get rid of old books.  Similarly, from the very beginning
the net has had to throw-away old information to give new information
a chance.  For the most part, this has not increased the amount of available
information, it has just made for more confusion.  If there were solid
archives and constant access and review of all this information, then
I would say sure go ahead and increase things as much as the system will
bear, but given that any new information comes at the expense of old
(and although there are a few private treasure collections).

< In some sense, by turning up the volume, I'm hoping that I've made
< the problem of information overload much more obvious to the software
< writers of the network, and that they will come up with some attacks
< on the problem that I haven't thought of yet.

I assure you it was quite obvious 7 years ago.  It is amusing that while
everyone agrees that the net exists solely on the basis of trust, here
you are purposely stress testing it.  Perhaps as your next act you should
take nntp away and see how they handle that.

< 	your rebuttal, sir?

Ditto?

---- BOB (webber@athos.rutgers.edu ; rutgers!athos.rutgers.edu!webber)