Path: utzoo!attcan!uunet!seismo!sundc!pitstop!sun!amdcad!ames!lll-tis!helios.ee.lbl.gov!nosc!ucsd!ucsdhub!jack!crash!pnet01!haitex From: haitex@pnet01.cts.com (Wade Bickel) Newsgroups: comp.sys.amiga.tech Subject: Re: New IFF format details (long). Message-ID: <3470@crash.cts.com> Date: 26 Sep 88 11:56:28 GMT Sender: news@crash.cts.com Organization: People-Net [pnet01], El Cajon CA Lines: 244 Thanks Stu, for the explanation of IFF '85. I had not looked at the sub specifier in that light, and now see that it is indeed a good idea. I still think it would be nice to incorporate the individual BAD chunk handling and Dirty/Modified chunk bits (please remember this was to be a user/program option), but not sufficeint to warrent re-doing IFF. A MAP chunk is of course one alternate solution, but seems to me to be clumsy and inefficient (what happens when parallel tracks of audio data are being stored?). >But the crux of Wade's proposal is this: > >| What I really want to do is create a purely Data driven mechanism, as >| opposed to the Code driven one in the current IFF. Rather than having to >| write code to handle each type of occurance, a structure would be initialized >| at run time, and this would be passed to the Reader or Writer parser to be >| handled. In this way it would never be necessary to update the Library(s). > >He's primarily interested in the mechanism for reading/writing such >files, and writes about it to great length in his article. If this >mechanism were useful for reading and writing "IFF 88" files, then it >would be equally applicable to existing IFF files just by changing the >file grammar slightly, as I did above. > Quite so. I had originally intended to apply this to the IFF '85 standard. Then, in my telephone discussion with you I had mistakenly come to the conclusion that nesting control was limited in IFF '85, and thus the first half of my proposal. >Wade has used the phrase "data-driven" several times, but I don't see >what he's talking about. Simple. In the current IFF system the knowledge to read/write IFF files is contained in the IFF code. Thus it is "code-driven". To extend the systems knowledge you must modify the code. In the "data-driven" system I propose the knowledge is contained in data (which may include code). Thus the data, not the code, is modified to extend the systems knowledge. >| The Writer Mechanism >| ------------------------ >What Wade describes here is basically a tree-like data structure to >control writing an IFF file. The tree would presumably be traversed by >the writer library code and user functions would be called to write the >actual bytes of data. Wade refers to this as "data driven." > >| Whereas IFF '85 reader/writers' require re-compilation of the >| source to accomodate format updates, IFF '88 will not. > >But I don't get it. Sure, the data structure controls the chunk >nesting, but the actual business of writing bytes gets handled by user >code, so where's the extendability? I still have to have the code to >write the chunks in my program which means re-compilation when something >changes. Not so. Only if you use some format which has not yet become part of the standard library. Under normal circumstances you would call routines found in the standard support librarys. Thus, a developer of a new format includes a library of private support routines to be used with the system. It would also be possible (though a bit cludgy) to include routines that are part of your active code (ie: not in a library). An AddFunc() type routine to add these calls to a table and assign an index might be desirable. >However, since the structure of the file and the code to write the >actual bytes are both provided by the client program, I fail to see how >creating this structure and passing it to a generic writer is any >different from just having the following piece of code in the client >program: > > /* WriteILBM: bitmap, colormap */ > > PushChunk (iff, ID_FORM, ID_ILBM); > > PushChunk (iff, ID_BMHD, 0); > WriteBMHD (bitmap); > PopChunk (iff); > > PushChunk (iff, ID_CMAP, 0); > WriteCMAP (colormap); > PopChunk (iff); > > PushChunk (iff, ID_BODY, 0); > WriteBODY (bitmap); > PopChunk (iff); > > PopChunk (iff); > >(This is an actual example of the use of the iff.library. RSN!) It >seems that for either method I need to have the writing code in my >program, and I need to know the structure of the file I want. If >anything, I would think that constructing a large tree data structure >would be more difficult than just having code to write the file >directly. What's the advantage? > Think about the control structure from the point of view of the file reader. It contains the sum of what the library and your code know about IFF files. Also, how about the following minor change in the way you handle file writing: PushChunk (iff, ID_FORM, ID_ILBM); PushChunk (iff, ID_BMHD, 0); WriteBMHD (bitmap); PopChunk (iff); PushChunk (iff, ID_CMAP, 0); WriteBytes (colortable,size); (* colortable is a ptr *) PopChunk (iff); PushChunk (iff, ID_BODY, 0); WriteBODY (bitmap); PopChunk (iff); PopChunk (iff); Of course, these are supported types. Lets pretend they're not. In this case it would look something like this: PushChunk (iff, ID_FORM, ID_ILBM); PushChunk (iff, ID_BMHD, 0); bmhd := MakeBMHD(bitmap); WriteBytes (bmhd, sizeofBMHD); PopChunk (iff); PushChunk (iff, ID_CMAP, 0); WriteBytes (colortable,sizeofCT); (* colortable is a ptr *) PopChunk (iff); PushChunk (iff, ID_BODY, 0); WriteCustom (rtn, bitmap); (* rtn is a ptr to code *) PopChunk (iff); PopChunk (iff); The chunks are now all generic. The library supports them even though they may not exist at the time of libary compilation. "rtn" is the address of a routine expecting one parameter on the stack, in this case a BitMap pointer. In this instance, it would be a routine in the library, but it could be any routine. I have not decided on the best way to handle custom routines, and would like suggestions. One parameter on the stack, push everything but D0, seems good to me. In practice it would probably be best to reference such routines from a jumptable. >I'm genuinly interested in this *mechanism* for reading and writing >files since it should work equally well for real IFF files. If there >are real advantages to this, Wade, I missed them. Could you provide an >example of a file format changing and user programs not needing to be >recompiled? I had not pictured the file format changing. Rather, I had been thinking of how to handle new IFF data types. With EA's code, I found it somewhat frustrating to have to go in, modify parts of the code, and re-compile it to add new chunks. What I want to be able to do is inform the system of a new data type (and the rules for reading it), without touching the original source. Why? Well, for one thing, having myriads of modified IFF reader code all based on the same originals, but containing different mods, is a mess. More importantly, library management, should your IFF library be made available, will be a mess if the library is constantly being updated to support new data types. Your example writer code looks superior to the writer structure method I described. The structure really comes into use for the reading of files, but since describing the writer was simpler than describing the reader, and we have to have a format before we can discuss how to read it, I chose to describe the writer. Please take a look at the writer system I described, and imagine using the same method for reading the files. Some method of creating "groups" of items on the same level would be desireable, which complicates the format, but is irrelevant as far a theory goes. Basically, a system of nodes is created which describes readable files, and how to read them. These nodes reference *knowledge*, which can be in the IFF library(s) or elsewhere. In this way, programs using new IFF data types can describe only the new knowledge to the system. For instance, lets suppose the library understands ILBMs but not ANIM's. The program defines ANIM by first calling library routines to generate a structure which reads ILBMs, then adds nodes which describe ANIM's and how to read them at the appropriate points. Now the system understands and can read ANIM. Periodically new types would be added to the library, as circumstances warrant. Standard calls to write/read standard types would also be in the library, to make things easy for the novice, and act as jumping off points for the designer of new IFF data types. ---------------------------------------- >On separate issue, Wade talks about "dirty" chunks. >While this is an interesting and valid idea, it really makes life >miserable for programmers. They have to retain chunks they don't need, >don't understand and can't use, just so they can write them out again >trying to preserve the original IFF file as much as possible only to >fail much of the time. It also means that all programs need to fully >support standard chunks so that standard chunks will never be marked as >dirty. It also means that programs that use "non-standard" chunks need >to make some intelligent decisions about whether a chunk marked as >dirty" is good within the context of a specific file. It might be >possible, but it could also be a real headache. I'm just not convinced >that the advantages are great enough to want to provide such a >mechanism. Simple, just turn it off. It should be a switch in the library, If the switch is on when the library reads a file, it tracks dirty chunks. If a file containing dirty chunks is read, it looks at the reader control structure (inited by the useing program) and determines which chunks are bad. If the switch is off unrecognized chunks or dirty chunks are ignored on reads. >This facility can be provided for any new IFF formtypes, however, by >equiping them with a "MAP" chunk (or some such, but it should be >consistent across FORMs) which contains a list of the chunks in the >file and their status. It is not possible or even desireable to retro- >fit this capability into existing formats. True. However, I think it was a mistake in the format not to leave a WORD for this. Not leaving a word for the future is shortsightedness. Using a Map chunk will be messy, making dirtychunk tracking harder than it would otherwise be. Also, what about data correction for chunks? It won't be that much longer and musicians will be using IFF to store CD quality sound. We can't be throughing away a file just because one bit has gone bad in a chunk. Thanks, Wade. UUCP: {cbosgd, hplabs!hp-sdd, sdcsvax, nosc}!crash!pnet01!haitex ARPA: crash!pnet01!haitex@nosc.mil INET: haitex@pnet01.CTS.COM Opionions expressed are mine, and not necessarily those of my employer.