Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!husc6!spdcc!m2c!ulowell!page
From: page@ulowell.cs.ulowell.edu (Bob Page)
Newsgroups: comp.sys.amiga
Subject: The secrets of the file system (was FillDisk)
Message-ID: <2080@ulowell.cs.ulowell.edu>
Date: Tue, 1-Dec-87 11:31:44 EST
Article-I.D.: ulowell.2080
Posted: Tue Dec  1 11:31:44 1987
Date-Received: Fri, 4-Dec-87 07:11:14 EST
References: <6098@jade.BERKELEY.EDU>
Reply-To: page@swan.ulowell.edu (Bob Page)
Organization: University of Lowell, Computer Science Dept.
Lines: 119

[This is a bit long, but I think it's reasonable. ..Bob]

spencer@eris () wrote:
>For those who fear using Disk Doctor on their hard disk,

I certainly do.  I think the Amiga community was spoiled by DiskSalv,
and would like to see CBM somehow buy it from Dave H for inclusion
in the standard WB distribution.

>take a look at Fill Disk by FNF on one of the Fish Disks.  It writes
>a file to the disk until the disk is full.  This means that Disk
>Doctor will no longer recover bits and pieces of deleted files.

This is great, as long as you aren't trying to recover a deleted or
otherwise broken file (simple things like invalid keys are OK though).
Also, there is a LARGE danger in using Fill Disk if you have a
scrambled disk (i.e. NOT A DOS DISK).  Since the root block is
probably mangled, you can have an INVALID BITMAP, but have it
incorrectly marked as valid.  FillDisk will gladly allocate all those
sectors, and diskdoctor naturally won't work as you think it should.

Also note of course FillDisk is useless on NOT A DOS DISKs, since you
can't use DOS to grab sectors until the disk is validated.

For "normal" problems with disks, like invalid keys and directory
loops, FillDisk/Diskdoctor is a much better solution than Diskdoctor
alone.  But then, for "normal" problems, a disk editor is an even
better (faster) solution - after you learned most of the inner
workings of the file system, which would take you a couple of days...
;-)

I found, after 4 days unscrambling my hard disk (DiskDoctor would have
thrown away most of the files, by the way), many things about DOS I've
never seen documented anywhere before (I don't claim to have seen
everything published, either .. in fact, I haven't seen the Transactor
article, which supposedly was quite good).  I got bits and pieces from
all the PD disk utilities around, and still am not through fixing my
drive, although it's now a DOS disk, and I have most of my files back!
I think I can eventually get the rest back, too.

>I had a validating hard disk, and when that was done (20 minutes later),
>Then I had a working hard disk.

I always wondered what the disk validator did ... like HOW it worked.
You can force a disk validation by running DiskChange DH0: (or
whatever the name of your disk is).  The steps are (this is from
memory, I'm not at home), ignoring FLUSH, MOTOR and CLEAR requests:

	Get Boot Block (block 0).  First four bytes should say "DOS0",
		where '0' is a NULL character, not the digit.
		The rest of the block is ignored, but is usually filled
		with "DOSx" in each longword, where x is the offset
		into the block.
	Get Root block (block maxblock/2).  Should check out as a root
		block - first longword is 0x00000002, last (127) longword
		is 0x00000001.  There are other sanity checks, and
		info for things like disk name and the bitmaps are
		received from the root block.  Also there is a longword
		that shows if the bitmap is valid or not.  It looks
		like it's INVALID if 0, and valid any other way.  I've
		seen disks with values of -1 and 1, so I guess either
		mean OK.
	If the bitmap is OK, for each block that holds the bitmap, get the
		bitmap block.  There is only one block for a floppy; 10
		for a 20MB disk, Maximum of 26 bitmap blocks per partition.
	If the bitmap is invalid, manually traverse the disk looking for
		valid blocks (files, directories, file lists) and add each
		one found to the bitmap.

There must be more the validator does but I haven't needed to know
much more than this.  The process of searching every block for valid
blocks (to rebuild the bitmap) seems to take a lot longer than it
should.  It also of course updates info in the DOS tables.

I like and dislike the file system - the redundancy it provides is
invaluable when trying to put damaged files and disks back together.
DOS provides headers for every block, for easy identification, hashing
for quick lookups, block checksumming, parent block pointers, block
sequence numbers, etc.  It's really tough to lose a file if you know
the details of the FS and know how to reconstruct it from the
information provided.

On the other hand, every block now has non-data that has to be
stripped out - slowing down bulk-DMA reads, since you can't just DMA
it into a buffer without processing it first.  Also, the bitmap
sectors are not allocated adjacent to each other (or even in the
middle of the disk) or even spread evenly throughout the disk.  The
bitmap is too small and has NO redundancy, although it could be
reconstructed (witness the Disk Validator).  Maybe the thing I like
the least about the [potential] reliability of the FS is there is only
one root block -- no backup.  If you trash it, you either are left to
the whims of DiskDoctor of have to learn the FS the hard way.  You
can't even use DiskEd; it only works on floppies.

I have heard there is a new FS being beta tested addressing some of
these concerns, especially the "dos info in every data block" problem
so DMA hard disks will really fly.  Also the FS layout is different,
so opening a file is a little slower but directory listings are really
quick.  Good news from the user's standpoint.  How about redundant
root blocks?  How much info will be provided to reconstruct damaged
disks?

I noted with interest Dave Haynie's statement that a HD DiskSalv
program would not surface (from him) until around the time 1.3 came
out.  He said it was because he was busy (which I don't doubt), but I
suspect another reason is he doesn't want to work on another version
when he knows it wouldn't work with the new FS, part of 1.3...

I thought about putting together a "Definitive Inside Look at the
AmigaDOS Filing System" but don't think it's worthwhile if we're on
the verge of a new FS.  Then again, if it can help one person
restore her data from a trashed disk, it would be worth it.  Can
anyone at CBM discuss the time frame for the new FS appearing?

..Bob
-- 
Bob Page, U of Lowell CS Dept.  page@ulowell.edu  ulowell!page
"I've never liked reality all that much, but I haven't found a
better solution."		--Dave Haynie, Commodore-Amiga