Path: utzoo!attcan!uunet!mcvax!ukc!eagle!icdoc!qmc-cs!liam From: liam@cs.qmc.ac.uk (William Roberts) Newsgroups: comp.unix.aux Subject: HFS library for AUX (part 1 of 2) Message-ID: <584@sequent.cs.qmc.ac.uk> Date: 15 Aug 88 16:55:50 GMT Reply-To: liam@cs.qmc.ac.uk (William Roberts) Organization: CS Dept, Queen Mary College, University of London, UK. Lines: 996 # This is a shar archive. # Remove everything above and including the cut line. # Then run the rest of the file through sh. #--------cut--------cut--------cut--------cut--------cut-------- #! /bin/sh # shar: Shell Archiver # Run the following with /bin/sh to create: # README # README.ptab # Makefile # copyright # document # maccopy.1 # macls.c.patch # This archive created: Mon Aug 15 17:54:16 WET DST 1988 echo shar: extracting "README" '('1999 chars')' if test -f README then echo shar: will not overwrite existing file "README" else cat << \SHAR_EOF > README This is the first version of an A/UX library for reading HFS disk partitions. It was written by Matthew Kempthorne-Ley-Edwards for the Department of Computer Science at Queen Mary College, and is supplied as a library of routines plus some include files. There are two example programs: maccopy - copies a MacOS file from the HFS disk to A/UX macls - like ls but operates on the HFS partition Unfortunately macls is essentially just the Berkeley "ls" program and therefore cannot be distributed to non-source licencees: it is therefore included as a context-diff file suitable for use with patch. Fortunately, the library works but providing a Mac equivalent of all of the read-only file system calls, namely open read write lseek stat lstat close opendir telldir seekdir rewinddir closedir readdir and the stdio routines fopen fseek fread fwrite fclose rewind ftell together with .h files that redefine the world so that your program uses the Mac versions rather than the AUX versions. If you look at the patches for macls then you will see that it basically cuts out some things we didn't know that A/UX could do (isatty) and sticks in the right include files, amongst others "macdir.h". Everything else is completely standard. For those that want both HFS and AUX, there is an include file "undodefs.h" which removes these helpful #defines: it is then up to you to call Mac_X for each routine. For an example, see the maccopy routine. THIS IS A PRELIMINARY VERSION - more will be done. However, you can help us by sending any bug {reports,fixes} to aux@qmc-cs.UUCP together with any suggestions for improvements or additional information that is needed so that we can WRITE to HFS. The file "document" contains the information we currently have about the HFS disk data structures, together with a summary of what else we need to know... Hope this is useful to you - please feel free to pass it on (see the copyright file though). SHAR_EOF if test 1999 -ne `wc -c < README` then echo shar: error transmitting "README" '('should be 1999 chars')' else echo README fi fi echo shar: extracting "README.ptab" '('358 chars')' if test -f README.ptab then echo shar: will not overwrite existing file "README.ptab" else cat << \SHAR_EOF > README.ptab In order to use the maccopy and macls utilities, the MacOS partition on the hard disk must be accessible as a device. A suitable ptab entry is MacOS:Apple_HFS:0:0:16 which associates the MacOS partition with /dev/dsk/c0d0s16 Such an entry can be generated by /etc/pname -c 0 -d 0 -s 16 -t Apple_HFS MacOS (Or at least, I think that was what we did...) SHAR_EOF if test 358 -ne `wc -c < README.ptab` then echo shar: error transmitting "README.ptab" '('should be 358 chars')' else echo README.ptab fi fi echo shar: extracting "Makefile" '('1205 chars')' if test -f Makefile then echo shar: will not overwrite existing file "Makefile" else cat << \SHAR_EOF > Makefile # # # Makefile for Apple mac - unix filestores # # programs # macls, maccopy # # # # Written by # # Matthew Kempthorne-Ley-Edwards # # Devonshire House # 34c Gyllyng St. # Falmouth # Cornwall # TR11 3EL # # # (c) 1988 Queen Mary College, University of London # CFLAGS = -g GOBJS = extents.o fcb.o rootblock.o bstar.o unixdir.o stat.o \ files.o GSRCS = extents.c fcb.c rootblock.c bstar.c unixdir.c stat.c \ files.c SRCS = $(GSRCS) macls.c maccopy.c OBJS = $(GOBJS) macls.o maccopy.o LIBS = all : libhfs.a macls maccopy extents.o fcb.o rootblock.o bstar.o : disc.h unixdir.o stat.o : macdir.h disc.h files.o : macfile.h macdir.h disc.h macls.o : macfile.h macdir.h disc.h maccopy.o : macfile.h macdir.h disc.h $(GOBJS) : $(GSRCS) libhfs.a: ${GOBJS} ar ru libhfs.a $? macls: macls.o libhfs.a cc ${CFLAGS} -o macls macls.o libhfs.a ${LIBS} maccopy: maccopy.o libhfs.a cc ${CFLAGS} -o maccopy maccopy.o libhfs.a ${LIBS} SHAR_EOF if test 1205 -ne `wc -c < Makefile` then echo shar: error transmitting "Makefile" '('should be 1205 chars')' else echo Makefile fi fi echo shar: extracting "copyright" '('515 chars')' if test -f copyright then echo shar: will not overwrite existing file "copyright" else cat << \SHAR_EOF > copyright /* * A/UX HFS library. Version 1.0 * * Copyright: Matthew Kempthorne-Ley-Edwards, August 1988 * * for Department of Computer Science * Queen Mary College * London E1 4NS * UK * * This software is provisional and carries no guarantee of fitness * for any purpose. It may be freely copied provided this copyright * message included in any such distribution. It may not be sold or * incorporated into any product without prior permission of both * the author and Queen Mary College. */ SHAR_EOF if test 515 -ne `wc -c < copyright` then echo shar: error transmitting "copyright" '('should be 515 chars')' else echo copyright fi fi echo shar: extracting "document" '('19867 chars')' if test -f document then echo shar: will not overwrite existing file "document" else cat << \SHAR_EOF > document The Macintosh Filestore Layout. This document is split into 3 sections, and is basically a reference document. Section 1 Disc layout ( of sectors ) Section 2 How to read files Section 3 About BStar trees Section 4 The Dir tree file & entries Section 5 Things not yet known The information herein was derived from pages IV-160 to IV-174 of the Inside Macintosh books ( Volume IV ). ( With some calculated guess work ) ############################## SECTION 1 ################################ Disc Layout 1.1 - What sectors hold what info. 1.2 - What info is stored about the disc ( in sector block 2 ) NOTE : The normal discs ( floppys and hard disc ) use 512 byte sectors and 1 sector per allocation block ( ie the allocation size is the same as the sector size of 512 bytes ) #1.1 The disc is split into 4 sections as shown below. +---------------+ Block 0 | | | System Start | +- -+ Used for mac boot etc. Block 1 | info | | | +---------------+ Block 2 | Volume info | Information about the disc (See 1.2) | - - - - - - - | +---------------+ Block 3 | | Bit map holding 1 for used allocation . | Disc Usage | 0 if unused | Bit map | a-1 | - - - - - - - | +---------------+ a | Allocation | The disc is then considered as . | 0 | Allocation blocks of 'n' sectors . +---------------+ each, starting at sector 'a', a+n | Allocation | and ending at block 'max' . | 1 | . +---------------+ a+2n | Allocation | . | 2 | 'a' is 'drAlBlSt' . +---------------+ ( see 1.2 ) a+3n | Allocation | 'max' is 'drNmAlBlks' . | 3 | . +---------------+ 'n' is 'drAlBlkSiz'/SECTORSIZE . | | . | | . | | . | | . | | . | Last Allocatn | | Block (max) | +---------------+ ###################################################### #1.2 Volume information sector This is a sector which holds information used in reading and writing to/from the disc. Byte in sector (Length) Name Use 0 2 drSigWord always $4244 ( check bytes ) 2 4 drCrDate time of formatting 6 4 drLsMod time of last write 10 2 drAtrb Disc Locked flags 12 2 drNmFls* Number of files in dir 14 2 drVBMSt First block in bit map 16 2 drAllocPtr* 18 2 drNmAlBlks Number of allocation blocks 20 4 drAlBlkSiz Allocation Block Size 24 4 drClpSiz Default clump size 28 2 drAlBlst First block in bit map 30 4 drNxtCNID next ID number for file/dir 34 2 drFreeBks Free Allocation Block count 36 1 drVN Volume name length 37 - ---- Volume name 64 4 drVolBkUp time of last backup 68 2 drVSeqNum* 70 4 drWrCnt Volume Write Count 74 4 drXTClpSiz clump size for extents file 78 4 drCTClpSiz clump size for Dir file 82 2 drNmRtDirs Number of dir's in root 84 4 drFilCnt Number of files on disc 88 4 drDirCnt Number of dir's on disc 92 32 drFndrInfo* 124 2 drVCSize* 126 2 drVCBMSize* 128 2 drCtlCSize* 130 4 drXTFlSize Length of extents tree file 134 12 drXTExtRec Extents tree extents 0,1,2 146 4 drCTFlSize Length of CAT tree file 150 12 drCTExtRec Dir tree extents 0,1,2 ( Any Entries marked with a '*' have no known details as yet ) Bytes 10/11 ( drAtrb ) has two single bit flags in it. Bit 7 - set if disc locked by hardware ( Read Only ) Bit 15- set if disc locked by software A clump is the amount of bytes added to a file space when it is extended. This clump is contiguous on the disc, and the larger it is, the less the file gets fragmented ( split up on the disc ) - see file storeage. ################################ SECTION 2 ########################### How to read files NOTE: There are 2 special files, called the Dir and the Extent files, these hold information about all other files on the disc. Files can be thought of in 4 different levels, 1 - The whole file 2 - A continuous section of the disc ( can be many for one file ) ( This is called an extent ) 3 - Allocation Blocks ( This is a group of continuous sectors on the disc ) 4 - Sectors ( a 512 byte chunk of the disc ) File Extents Allocations Sectors +-------+ +-------+ +-------+ +-------+ | | | | | | | | | | | | |+ + + +| |+ + + +| | | |Extent | | | | | | | | 0 | |+ + + +| |+ + + +| | | | | | | | | | | |+ + + +| |+++++++| |+++++++| | | | | | | | | | | | | |+ + + +| |+ + + +| | | |Extent | | | | | | | | 1 | |+ + + +| |+ + + +| | | | | | | | | | | |+ + + +| |+++++++| |+++++++| | | | | | | | | | | | | |+ + + +| |+ + + +| | | | | | | | | | | | | |+ + + +| |+ + + +| | | |Extent | | | | | | | | 2 | |+ + + +| |+ + + +| | | | | | | | | | | |+++++++| |+++++++| |+++++++| | | | | | | | | | | | | | | | | | | | etc | | | | | | | | | | | | | | | | | | | | | +-------+ +-------+ +-------+ +-------+ + + + + indicates continuous blocks +++++++ indicates continued else where An Extent is a Start-Allocation-Block and a Number-of-Allocation- Blocks, So ideally, the file would be in one extent. But if there is not a single space that will take it, then another extent can be used to hold the second section ( and so on ). This extent gives a range of Allocation blocks ( by start number and number of blocks ), which then can be mapped on to a range of sectors on the disc. The directory and extent files have their first 3 extents ( Numbers 0,1 & 2 ) stored in the disc volume information. The normal files have their first 3 extents stored in with the file information in the directory entry for that file. This directory entry is held in the Dir file. If any file has more that 3 extents, then the others must be looked up in the extents tree file. - This is not normally required as most files are only 1 or 2 extents long. Each extent gives an area on the disc in which the file is stored, There is usually some left over bytes at the end, which are not used. A files physical length is the total bytes used on the disc, so when reading any file, the 'Logical length' must be used to count to the end of the file, so any remaining unused bytes will also be copied. To read the file itself, The first extent must be read. The first sector of the file can then be calculated from the following. drAlBlSt+ (drAlSize/SECTORLENGTH)*first_allocation_block Then a number of sectors can be read in, which is Number_of_allocation_blocks_in_Extent*drAlSize/SECTORLENGTH If the number of allocation blocks in the extent is zero, then it is unused ( and so are any futher extent records ) This should be done for each extent, until the file is completely read. ################################# Section 3 ############################# About BStar trees BStar trees are a tree structure used to locate infomation keeping ALL branches to the SAME optimum length. Entries are searched by reference to a key, which is in effect a sequence of bytes used to identify a particular piece of information. ( Such as a directory number & file name ) The tree is kept sorted into key order, so that the lowest key is at the Left hand side, and the largest key is at the right hand side. There are 2 diferent types of node in this tree, these are the leaf node, and the junction node. In the junction node, there is a group of records, each record has a key and a pointer to another node. In the leaf nodes, there are keys, and asociated data for the entries. ( All nodes have sideway links to other nodes on the same level ) To search for a particular entry, the root node must be found. ( it is the only node where both sideway links are 0 and the level>0 ). Then a loop can be entered doing the following If the node is a leaf search each record in node until key matches or key>record key if match then FOUND else NOTFOUND the node is a junction, so... search each record in the node untill one that is greater than the search key is found ( or it matches ) Go to the node pointed to by the pointer after the entry before, ie the one just_LESS_than ( or equal to ) the search key. The stucture of a node is as follows byte length name use 0 4 ndFLink sideways forward link 4 4 ndBLink sideways backward link 8 1 ndType #FF=leaf, #00=junction 9 1 ndLevel 0=not used, 1=leaf, 2,3,4.. junction levels 10 2 ndNRecs Number of records in this node 12..end record area working backwards from the end of the node, there are a set of double byte numbers that give the distance through the node of each record, and a double byte number that gives the position of any free space. So the record looks something like this... byte offset use 0 +---------------+ | ndFLink | 4 +---------------+ | ndBLink | 8 +---------------+ | ndType | 9 +---------------+ | ndLevel | 10 +---------------+ | ndNRecs | 12 +---------------+ | | normally 14 +---------------+ <--------+ | | | | Record 0 | | +---------------+ <------+ | Note that the records can | | | | be of any different | Record 1 | | | lengths. +---------------+ <----+ | | | | | | | | Record 2 | | | | +---------------+ <--+ | | | | | | | | | | Record 3 | | | | | Free Space +---------------+ <+ | | | | . . | | | | | . . | | | | | . . | | | | | FS pointer 502 +---------------+ | | | | | | Offset |--+ | | | | - Free Space pointer 504 +---------------+ | | | | | Offset |----+ | | | - Record 3 pointer 506 +---------------+ | | | | Offset |------+ | | - Recore 2 pointer 508 +---------------+ | | | Offset |--------+ | - Record 1 pointer 510 +---------------+ | | Offset 000E |----------+ - Record 0 pointer end of node +---------------+ ####### The junction node records of the Dir tree ####### These seem to be accessed by a key made up from the directory number that an entry is in, and its name. Byte example Use 0 #25 Length of key 1 #00 Not used 2 #00000002 Dir Number 6 #03 name length 7 "tmp" name in ascii 38(#25) #00000004 node number where entry found followed by next record ( if any ) When searching and comparing keys, the 4 bytes of the Dir number can be compared first, to give <,>, or =. If '=', then the name can be checked, but when doing this you must note that it is not case dependent, so 'c' is less than 'D' and more than 'A'. Ie case is ignored. All names seem to be padded out to 32 chars with character zero's. ########## SECTION 4 - Dir tree entries ########### These are in the leaf nodes of the Dir file BSTAR tree. These hold directory entries and links, and these are stored as follows. byte name use 0 ckrKeyLen Key length 1 ckrResrv1* reserved 2 ckrParID Owning directory ID number ( 4 bytes ) 6 ckrCName length of name 7 name ( in asci bytes ) in here there is an extra byte of #00 if it is not currently on a word boundary, ie... The store is padded to an even number of bytes with zeros, so that the following entry starts on an even byte boundary. There now follows a directory entry of one of three types. TYPE 1 0 cdrType =1 ( Entry is a subdirectory entry ) 1 cdrResrv2* reserved 2 dirFlags* flags 4 dirVal* Valence 6 dirDirID ID of sub dir ( 4 byte number ) 10 dirCrDat creation date 14 dirMdDat modification date 18 dirBkDat backup date 22 dirUsrInfo* 38 dirFndrInfo* 54 dirResrv* TYPE 2 0 cdrType =2 ( Entry is a file entry ) 1 cdrResrv2* reserved 2 filFlags* flags ( bit 0=1 if file locked ) 4 filUsrWds* 20 filFlNum FileNumber 24 filStBlk* First allocation block of data fork 26 filLgLen Logical length of data fork 30 filPyLen Length of data fork ( rounded up to sector ) 34 filRStBlk* First allocation block of resource fork 36 filRLgLen Logical length of resource fork 40 filRPyLen length of resource fork to end of allocation 44 filCrDat creation date 48 filMdDat modification date 52 filBkDat date of last backup 56 filFndrInfo* 72 filClpSize default file clump size 74 filExtRec first 3 extents of data fork 86 filRExtRec first 3 extents of resource fork 98 filResrv* TYPE 3 0 cdrType =3 ( Entry is a link to parent ) 1 cdrResrv2* reserverd 2 thdResrv* reserved 10 thdParId 4 byte number of parent directory 14 thdCName name length 15 name of this directory ( Any Entries marked with a '*' have no known details as yet ) ######################## SECTION 5 #################### Unknown details There are a few details that I do not fully understand yet These are The finder info & reserved bytes in the directory entries If the dir tree nodes are 1 sector or 1 allocation_block in size - ( It is more likely to be allocation blocks but I am not certain ) Exactly what needs updating when a new file is added to the disc, or removed ( erased ). How to find the root node of the bstar tree. ( At the moment, a scan of all nodes is made ) The main points are, all of the fields marked with a '*' in this document, as these are created for a new file, and If are not set properly, then unknown things may happen. The inner workings of the extent file is not known. though its whereabouts is known. ( From volume info ). - This was not looked at in great detail as most files can be accessed without looking at this file. ( Unless a file write is done, of it is longer than 3 extents in length ). ######################################################################### SHAR_EOF if test 19867 -ne `wc -c < document` then echo shar: error transmitting "document" '('should be 19867 chars')' else echo document fi fi echo shar: extracting "maccopy.1" '('1233 chars')' if test -f maccopy.1 then echo shar: will not overwrite existing file "maccopy.1" else cat << \SHAR_EOF > maccopy.1 .TH MACCOPY 1 "Aug 1988" QMC .SH NAME maccopy \- copy files from the mac partition to unix .SH SYNOPSIS .B maccopy [ .B macfile ] [ .B unixfile ] .SH DESCRIPTION .I Macfile is copied onto .IR unixfile . Only one HFS file can be copied at a time, and it must be named in full. If it does not live on the default HFS partition, then .I macfile should be the device name, a colon, and then the full path name with folder names separated by "/", the UNIX directory separator character. Filename parsing is generous, i.e. the longest matching valid prefix is taken to be the directory name, so names such as .I "A/UX\0Startup" will work correctly as folder names. .PP If any non alpha character occurs in the name, a `?' wildcard can be used, which will match any character in that position. Note that the .I macfile argument normally needs to be enclosed in single or double quotation marks to force the correct interpretation. .SH FILES /dev/dsk/c0d0s16 \(en default HFS partition. .SH "SEE ALSO" macls(1) .SH BUGS Won't take `*' wildcards, or any name completely made up from from `?'s. Can't copy from A/UX to HFS. .SH AUTHOR .nf Matthew Kempthorne-Ley-Edwards Department of Computer Science Queen Mary College London E1 4NS UK .fi SHAR_EOF if test 1233 -ne `wc -c < maccopy.1` then echo shar: error transmitting "maccopy.1" '('should be 1233 chars')' else echo maccopy.1 fi fi echo shar: extracting "macls.c.patch" '('4107 chars')' if test -f macls.c.patch then echo shar: will not overwrite existing file "macls.c.patch" else cat << \SHAR_EOF > macls.c.patch *** ls.c Mon Aug 15 15:24:56 1988 --- macls.c Mon Aug 15 15:09:14 1988 *************** *** 8,13 * 4.2bsd version for symbolic links, variable length * directory entries, block size in the inode, etc. */ #include#include #include --- 8,20 ----- * 4.2bsd version for symbolic links, variable length * directory entries, block size in the inode, etc. */ + #ifndef BSD + /* apple only defns */ + #include + #include + #define dbtob(x) dtob(x) /* BSD -> SYS V naming */ + #endif + #include #include *************** *** 9,15 * directory entries, block size in the inode, etc. */ #include - #include #include #include #include --- 16,21 ----- #endif #include #include /* ******************************************* */ *************** *** 11,16 #include #include #include #include #include --- 17,31 ----- #include #include + + /* ******************************************* */ + #ifndef BSD + #include "macdir.h" + #else + #include + #endif + /* ******************************************* */ + #include #include *************** *** 14,19 #include #include #define kbytes(size) (((size) + 1023) / 1024) struct afile { --- 29,36 ----- #include #include + + #define kbytes(size) (((size) + 1023) / 1024) struct afile { *************** *** 40,46 int aflg, dflg, gflg, lflg, sflg, tflg, uflg, iflg, fflg, cflg, rflg = 1; int qflg, Aflg, Cflg, Fflg, Lflg, Rflg; ! int usetabs; time_t now, sixmonthsago, onehourfromnow; --- 57,63 ----- int aflg, dflg, gflg, lflg, sflg, tflg, uflg, iflg, fflg, cflg, rflg = 1; int qflg, Aflg, Cflg, Fflg, Lflg, Rflg; ! int usetabs = 1; time_t now, sixmonthsago, onehourfromnow; *************** *** 63,69 int i; struct afile *fp0, *fplast; register struct afile *fp; - struct sgttyb sgbuf; argc--, argv++; if (getuid() == 0) --- 80,85 ----- int i; struct afile *fp0, *fplast; register struct afile *fp; argc--, argv++; if (getuid() == 0) *************** *** 72,77 sixmonthsago = now - 6L*30L*24L*60L*60L; onehourfromnow = now + 60L*60L; now += 60; if (isatty(1)) { qflg = Cflg = 1; (void) gtty(1, &sgbuf); --- 88,94 ----- sixmonthsago = now - 6L*30L*24L*60L*60L; onehourfromnow = now + 60L*60L; now += 60; + #ifdef BSD if (isatty(1)) { struct sgtty sgbuf; *************** *** 73,78 onehourfromnow = now + 60L*60L; now += 60; if (isatty(1)) { qflg = Cflg = 1; (void) gtty(1, &sgbuf); if ((sgbuf.sg_flags & XTABS) == 0) --- 90,97 ----- now += 60; #ifdef BSD if (isatty(1)) { + struct sgtty sgbuf; + qflg = Cflg = 1; (void) gtty(1, &sgbuf); if ((sgbuf.sg_flags & XTABS) == 0) *************** *** 79,84 usetabs = 1; } else usetabs = 1; while (argc > 0 && **argv == '-') { (*argv)++; while (**argv) switch (*(*argv)++) { --- 98,104 ----- usetabs = 1; } else usetabs = 1; + #endif while (argc > 0 && **argv == '-') { (*argv)++; while (**argv) switch (*(*argv)++) { *************** *** 624,629 #define NMAX (sizeof (utmp.ut_name)) #define SCPYN(a, b) strncpy(a, b, NMAX) #define MAXUID 2048 #define MINUID -2 /* for nfs */ #define MAXGID 300 --- 644,650 ----- #define NMAX (sizeof (utmp.ut_name)) #define SCPYN(a, b) strncpy(a, b, NMAX) + #ifndef MAXUID #define MAXUID 2048 #endif #define MINUID -2 /* for nfs */ *************** *** 625,630 #define SCPYN(a, b) strncpy(a, b, NMAX) #define MAXUID 2048 #define MINUID -2 /* for nfs */ #define MAXGID 300 --- 646,652 ----- #ifndef MAXUID #define MAXUID 2048 + #endif #define MINUID -2 /* for nfs */ #define MAXGID 300 SHAR_EOF if test 4107 -ne `wc -c < macls.c.patch` then echo shar: error transmitting "macls.c.patch" '('should be 4107 chars')' else echo macls.c.patch fi fi # End of shar archive exit 0 -- William Roberts ARPA: liam@cs.qmc.ac.uk (gw: cs.ucl.edu) Queen Mary College UUCP: liam@qmc-cs.UUCP LONDON, UK Tel: 01-975 5250