Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Path: utzoo!mnetor!seismo!rutgers!topaz.rutgers.edu!klinzhai.rutgers.edu!webber From: webber@klinzhai.rutgers.edu (Webber) Newsgroups: news.admin,news.misc,news.groups,news.sysadmin Subject: The Requested Presentation of Quota Based News Control Message-ID: <285@klinzhai.rutgers.edu> Date: Sun, 5-Jul-87 07:43:57 EDT Article-I.D.: klinzhai.285 Posted: Sun Jul 5 07:43:57 1987 Date-Received: Sun, 5-Jul-87 18:40:59 EDT References: <266@brandx.rutgers.edu> <8225@utzoo.UUCP> <272@brandx.rutgers.edu> <153@tmsoft.UUCP> Organization: Rutgers Univ., New Brunswick, N.J. Lines: 210 Summary: please consider that the author of this message is probably more interested in your response than the rest of the net, so send mail. Xref: mnetor news.admin:613 news.misc:691 news.groups:1166 news.sysadmin:270 To: utzoo!utgpu!tmsoft!mason, webber@aramis.rutgers.edu In article <153@tmsoft.UUCP>, mason@tmsoft.UUCP writes: > ... I would suggest you explain again your > quota idea; then let it lie fallow. Someone may pick up on it & it may > save the net (if you want to get credited with saving the net, you should > probably re-post every 6 months or so, so everyone knows it's your idea). Ok. Below I present how the quota based system would work and the answers to those objections that I am aware of. Anyone who gives much thought to it will realize that the net is not in eminent danger of following my plan, so it should be sufficient to send me your objections via mail. If I find one overwhelming, I will gladly post to the net a message giving you credit for straightening me out and apologizing for the confusion that my ignorance caused. I will keep an updated copy of this message available for request by mail and for occasional reposting until it no longer seems relevant to the net. It is because of the existance of this alternative that I believe one can legitimately oppose the conversion of the net to all-moderation in the face of the net's limited bandwidth. I view the entire net flow as one continuous stream of messages. So far, nearly every aspect of the message has been used as a basis for determining whether or not that message will be passed on or taken except for whether or not there is room for it. On the other hand, when calculating the expense of handling the net flow for any given site, whether it is disk storage, communication bandwidth, or cpu that is the bottleneck, the expense is always a function of the sum of the sizes of the messages regardless of the other aspects of the message. The closest the net has come so far to using quotas has been that some systems impose limits on the size of messages due to problems uucp has/had with large individual messages. Since that quota arose at a time when nearly everyone had the same problem, it worked out rather well. There are two places one could contemplate putting quotas, either on the originator of the message or on the channel through which the message flows. I believe that placing a quota on the channel is the simplest way to handle the problem. Indeed, once a quota is imposed on the channel, the technical problem of too much flow is solved and the question of the nature of the flow can be handled at a pace that is appropriate to the evolution of a new media. Currently there is already a quota on all the channels, in that there is a maximum flow that each can handle. However, since the channels have other uses than news, news must be prevented from dominating each of these channels. The amount of news flow that each channel (link between neighboring nodes) can handle comfortably is differs from channel to channel. One question that arises is how do you find out the size of a message. There are two places that that can be done. It can be done by remotely querying the site from which the message is to be transfered or it can be done by keeping a running total of the size of the messages transferred so far and stop the transfer once they reach some cutoff (this total could be calculated either locally or remotely). Clearly it is not necessary for both sites on the link to agree on the quota for that channel, but it would certainly make it easier to handle for both of them. The question arises of what will happen to the net when sites start refusing to carry more than x bytes of news per day. If communication bandwidth were all of the problem, then news would simply pile up at various sites until it eventually got tranfered. However, at the the current rate of 2 meg per day, it is not difficult to imagine that limitations on disk space would quickly dominate the situation. Clearly if there is no room for something where it is and no place to put it, it is not going to last long. It is interesting to note that what happens happens on the remote machine and not on the local machine. Thus it is the remote machine that will set policy on which of the unsent messages will be expired (unless someone wants to take on a considerably larger implementation task than just the one I am advocating). There are two options here: 1) delete some of the messages already in the queue and 2) stop adding new messages to the queue. Both the first and the second option mean that some people will miss some messages since some sites will be able to circulate much more news than others. However, given the number of messages available and the bandwidth restrictions of the links in the net, this is inevitable and already the case. Currently those people who are choosing to receive only a subset of the net are doing so based on group name. This means that site administrators must take responsibility for decisions such as whether or not it is a proper utilization of their resources to carry a group whose discussion topic is a computer that they don't and never plan to own or a hobby such a birdwatching, or job offerings from competitors, or the pros and cons of abortion, or the philosophical aspects of the sciences. If one attempted to justify the groups one was transmitting on the basis of their content, I doubt if there would be more than 4 groups that could manage country wide distribution. However, there is another aspect to these groups beside their content and that is the morale of the participants in the various discussion groups. From a morale point of view, each of the groups is justified (and many more groups as well). Let us look at the various kinds of posting and consider the significance of them getting `dropped on the floor.' A request for information: almost inevitably, the queries being handled on the net are answerable elsewhere. The public libraries in the United States are adequate for handling many of them. Contacting neighboring universities and colleges would result in finding people who could handle most of the rest. It is also worth noting that if the questioner had been watching the traffic of the group much, it would not be difficult to identify a few experts to whom a direct computer mail query could be sent. Of course, most people have anecdotes of questions that only the net could answer; these are interesting because it is so rare and notable. Certainly, the worst that would happen from dropping most queries for information on the floor is that some people would learn how to use libraries and browse professional journals. Answer to the request for information: clearly anyone seriously interested in helping the person making the request would send them direct mail to make sure that they noticed the answer. However, most people like to use queries for information as an opportunity to place that information before a larger audience (this message is an example). Thus, it doesn't matter to the sender who specifically recieves the message just as long as it gets wide enough distribution so that it has some possibility of generating some action somewhere. Blanket postings of information: as noted above, no one is really maintaining that every person who logs into a unix system should be given an opportunity to see their message, just that the message is of general interest and so they saw no reason to restrict its distribution anymore than the net already does. Thus if it doesn't make it to some places, the world will not come to an end. Part n of m: this kind of message is rare, i.e., most people expect their messages to be able to stand on their own basis as opposed to being one of a series that is useless unless you collect the whole series. However, there is one notable exception: large source postings. One question of interest is just how big does a source posting have to be before it will cause more trouble by trying to be sent as a single message than it will cause by getting separated from its other pieces. It would seem to me that communication has improved enough over the past few years that it would be worthwhile investigating the question of how long different links would take to transfer an x byte file (not due to the rate of the link but due to the strategy the link uses to handle errors in transmission). Another question is: just what is happening when someone tries to post a very large source or binary to the net? Clearly such sources and binaries were not meant to be read and hence are not intended as communication between people. Instead, they are intended to be used. If a neighboring node has a program that my node can use, then it makes sense to establish an ad hoc link to transfer it (perhaps, even in rare cases transferring it piecewise via direct mail). Thus, one could imagine an announcement of an 800k source being made on the net and then it actually move through the net as a chain letter (site to site NOT user to user). For that matter, floppy disks and mag tapes through the regular post have been underutilized. A source that isn't worth the postage and media to the reciever probably isn't worth posting to the net either. Rather than writing monolithic programs that do only one thing, it would make more sense to post to the net small general purpose utilities that other people could read and use within their own code. While admittedly, the above is from the viewpoint of someone who can write their own sources, I would maintain that it is less than totally clever for a person who cannot program to use a source or binary that they recieved off the net. In summary, you could say that I see the difference between mail and news to be in the case of mail you know exactly who you want to send to and the system tries to offer you as much support as possible in getting your message to the recipient, whereas in news you really don't know if there is anyone out there interested in your message and the system distributes your message according to what links are interested in transferring how many blind messages. In neither case do the links attempt to intrude by judging the contents of the messages, although the systems have quite different behavior based on the intended usage. Thus, I do not see any problem being generated from the use of quotas to manage net news due to occasional loss of messages. Indeed, I see it as actually encouraging a more responsible usage of the media in conjunction with making joining news less of a problem for individual site managers. Neither do I see quotas as causing any implementation problems. I await enlightenment. > OR > You can program your quota system and get Rutgers to use it, then > on the basis of whatever is wonderful about it at Rutgers, convince your > net neighbours, then the state, then the east coast, and by then you can > probably convince the rest of us. I by far think that this would be the best way to handle things, but Rutgers is not a site where the flow has gotten so bad that people are advocating the closure of unmoderated groups and the general control of the flow through moderation. Thus, implementing it at Rutgers would show that the code works, but would not address the advisability of actually using it. No one has so far maintained that the idea would be difficult to code in a manner that would integrate with the rest of the net. Instead, the discussion has always rested on the advisability of using it even if it already existed. I believe I have adequately addressed all the issues that have formed a basis for objection in the past as summarized above. I have been addressing this issue off and on since February and over that time my understanding of the problems of implementing a quota based system system within the structure of Usenet has grown. In the past I have stressed the notion that the net would adapt to the bandwidth it found by reducing the number of postings and that this would occur by having quotas push further and further back into the system until individual sites were rationing the postings from their own users. It now strikes me that there would be little motivation for this since once the quotas are in place, the pressure for changing the Usenet setup will be decentralized and take a wholely unpredictable course (although I have not yet extrapolated a future that would be worse than the currently expected one of increasing use of moderated groups and group-name based all or nothing flow decisions). ----- BOB (webber@aramis.rutgers.edu ; rutgers!aramis.rutgers.edu!webber)