| CiderPress request - HFS large files [message #392410] |
Wed, 25 March 2020 12:53  |
|
|
Originally posted by: fatdog projects
Hi Andy! I use CiderPress all the time for manipulating tons of files with various little fun IIGS projects. Never once have I had an issue with using it to get what I wanted done. 10's of thousands of files transferred without a hitch. It's such a great tool and so valuable for everyone in the community!
Today I've stumbled into a fringe case that I don't think anyone ever expected. I don't think anyone has ever had a reason to want to test something like this.
So I want to put a 45 MB file on a .PO image. I understand that ProDOS disks do not allow for files > 16 MB, and that the maximum RAM size on a physical GS is 8MB. I would like to be able to put the file on an HFS .PO image anyway.
Today I discovered that the paint program DreamGrafix on the GS supports virtual memory. I would like to test it with a very large animation (30 FPS full screen video). If it breaks, that's ok :) I'd still like to try. I'm currently attempting to push the machine as far as I possibly can. Did you know it can handle 44.1 KHz streaming sound from RAM? Also, if the resolution is small enough, the GS can push actual 60 FPS video, and a decent sized 30FPS? Aaannnd, that it can do both of those things at the same time if you are patient enough to allow everything to load!
Do you think it is possible to allow CiderPress to import files >16MB to an HFS image? Thanks for your continued work, and for all of the cool Apple stuff you've made through the years!
|
|
|
|
|
|
|
|
| Re: CiderPress request - HFS large files [message #392418 is a reply to message #392415] |
Wed, 25 March 2020 18:35  |
|
|
Originally posted by: fatdog projects
On Wednesday, March 25, 2020 at 4:14:56 PM UTC-4, I am Rob wrote:
> Probably speaking out of turn since you specifically requested Andy, but just a thought.
>
> Does the video have to be one complete file or can you break it into 3x 15 Mb segments?
Not at all Rob, thanks for your suggestion. The consensus of the people I asked was that I need to split it up, put the smaller files on a disk image, and then use GNO/ME and the HFS.FST on the GS to concatenate the files.
So I need to learn me some GNO/ME
|
|
|
|
| Re: CiderPress request - HFS large files [message #392421 is a reply to message #392418] |
Wed, 25 March 2020 19:47 |
gids.rs
Messages: 1395
Registered: October 2012
Karma: 0
|
Senior Member |
|
|
I don’t know if this helps, but I had uploaded a cd disk image of an HFS .iso disk image of 640 Mb. I see it is on Asimov.
The zip file is only 640 kb for download and when unzipped found that Ciderpress was able to read and write to it.
My impression though was that even with HFS.FST the max size of a file was still 16 Mb even though the volume can be as big as 2 gig.
Was this ability to have recognize files larger than 16 Mb even programmed into the FST or any IIGS programs?
|
|
|
|
| Re: CiderPress request - HFS large files [message #392424 is a reply to message #392421] |
Wed, 25 March 2020 22:43 |
|
|
Originally posted by: fatdog projects
On Wednesday, March 25, 2020 at 7:47:24 PM UTC-4, I am Rob wrote:
> I don’t know if this helps, but I had uploaded a cd disk image of an HFS .iso disk image of 640 Mb. I see it is on Asimov.
>
> The zip file is only 640 kb for download and when unzipped found that Ciderpress was able to read and write to it.
>
> My impression though was that even with HFS.FST the max size of a file was still 16 Mb even though the volume can be as big as 2 gig.
>
> Was this ability to have recognize files larger than 16 Mb even programmed into the FST or any IIGS programs?
I do not believe that ability was programmed into any GS apps. ProDOS maximum file size is 16 MB. At the time of this post I am unsure what the file size cap is with HFS.FST. I had assumed it would be 2GB like the standard, but now I am doubtful of that and tend to think, like you, that it is 16 MB by design with the FST.
The only IIGS program I have seen that can access files above the 8 MB RAM limit (but below the 16 MB file cap) is DreamGrafix. I am under the impression that it handles the large file in chunks to avoid overflowing the RAM space. It refers to the technique as virtual memory, but I do not think it is the same concept as traditional VMem. There may be other IIGS programs that do something similar, but all that I have tested just refuse to put the file into memory if there is not enough room.
My test is going to be what happens when DreamGrafix attempts to handle a 45 MB file from an HFS disk, since it seems like it will attempt to try. The limit for files with the HFS format is 2 GB, but with HFS.FST that is the limit for volumes. I think you are correct when you say the limit for files is 16 MB with HFS.FST, but I don't want to believe that.
|
|
|
|
| Re: CiderPress request - HFS large files [message #392428 is a reply to message #392410] |
Thu, 26 March 2020 01:06 |
|
|
Originally posted by: fadden
On Wednesday, March 25, 2020 at 9:53:10 AM UTC-7, fatdog projects wrote:
> Do you think it is possible to allow CiderPress to import files >16MB to an HFS image?
Yes. The 16MB file limit test is ignoring the filesystem type. That's a bug.
I commented out the check and was able to add a 200MB file to a 1GB HFS disk image. So the underlying libhfs code is capable.
I think the file size limit for HFS is 2GB. I don't know what the limits on the GS/OS HFS FST are.
I'll try to throw something together tomorrow morning. The hard part is remembering what steps are required to build the EXE and generate an installer. I should probably write them down. :-)
|
|
|
|
|
|
| Re: CiderPress request - HFS large files [message #392438 is a reply to message #392435] |
Thu, 26 March 2020 09:15 |
Steven Hirsch
Messages: 798
Registered: October 2012
Karma: 0
|
Senior Member |
|
|
On 3/26/20 5:56 AM, Bill Buckels wrote:
> "fadden" <thefadden@gmail.com> wrote:
>> The hard part is remembering what steps are required to build the EXE and
>> generate an installer. I should probably write them down. :-)
>
> It gets worse over time. Write them down Professor :)
<sigh...> You're not kidding. From time to time I unearth or revisit some
old software project from way-back-when and discover that I failed to document
the environment, tool chain and/or commands. My attitude at the time was like
Dr. McCoy on Star Trek in that famous old episode "Spok's Brain": "..why it's
so simple, Jim, a child could do it..".
|
|
|
|
| Re: CiderPress request - HFS large files [message #392442 is a reply to message #392438] |
Thu, 26 March 2020 09:52 |
D Finnigan
Messages: 1154
Registered: October 2012
Karma: 0
|
Senior Member |
|
|
Steven Hirsch wrote:
> On 3/26/20 5:56 AM, Bill Buckels wrote:
>> "fadden" <thefadden@gmail.com> wrote:
>>> The hard part is remembering what steps are required to build the EXE
>>> and
>>> generate an installer. I should probably write them down. :-)
>>
>> It gets worse over time. Write them down Professor :)
>
> <sigh...> You're not kidding. From time to time I unearth or revisit
> some
>
> old software project from way-back-when and discover that I failed to
> document
> the environment, tool chain and/or commands. My attitude at the time was
> like
> Dr. McCoy on Star Trek in that famous old episode "Spok's Brain": "..why
> it's
> so simple, Jim, a child could do it..".
>
Or my favorite, "Easier done than said." So why the need to explain it? :-P
I had the same problem two weeks ago, again using one of my own tool chains,
but I hadn't used it in about 4 years. :-/
--
]DF$
The New Apple II User's Guide:
https://macgui.com/newa2guide/
|
|
|
|
| Re: CiderPress request - HFS large files [message #392447 is a reply to message #392428] |
Thu, 26 March 2020 12:45 |
|
|
Originally posted by: fatdog projects
On Thursday, March 26, 2020 at 1:06:30 AM UTC-4, fadden wrote:
> On Wednesday, March 25, 2020 at 9:53:10 AM UTC-7, fatdog projects wrote:
>> Do you think it is possible to allow CiderPress to import files >16MB to an HFS image?
>
> Yes. The 16MB file limit test is ignoring the filesystem type. That's a bug.
>
> I commented out the check and was able to add a 200MB file to a 1GB HFS disk image. So the underlying libhfs code is capable.
>
> I think the file size limit for HFS is 2GB. I don't know what the limits on the GS/OS HFS FST are.
>
> I'll try to throw something together tomorrow morning. The hard part is remembering what steps are required to build the EXE and generate an installer. I should probably write them down. :-)
Thank you! I will keep you guys posted on some of the videos if you are interested.
Bill, I use your A2B tool all the time to convert from BMP -> C1. It is extremely quick for what I need to do, processing thousands of files at a time.. I want to thank you for all of the work you've done with your various projects. For me, BMP2DHR has been very useful and educational. Many others have been trying to take your ideas farther, try to find us!
Are you guys working on any current Apple projects? :)
|
|
|
|
| Re: CiderPress request - HFS large files [message #392448 is a reply to message #392424] |
Thu, 26 March 2020 12:49 |
gids.rs
Messages: 1395
Registered: October 2012
Karma: 0
|
Senior Member |
|
|
> . I do not believe that ability was programmed into any GS apps. ProDOS maximum file size is 16 MB. At the time of this post I am unsure what the file size cap is with HFS.FST. I had assumed it would be 2GB like the standard, but now I am doubtful of that and tend to think, like you, that it is 16 MB by design with the FST.
Usually some apps request an end-of-file limit in bytes to know when reading a file has come to an end. You wouldn’t even need to copy a 45 Mb file to an hfs volume to test. All you would need to do is create a sparse file with a file size > 16 Mb by just storing a few bytes above the 16 Mb range. This should be able to be done with just a few bytes of code using GSOS .
|
|
|
|
| Re: CiderPress request - HFS large files [message #392451 is a reply to message #392428] |
Thu, 26 March 2020 14:16 |
|
|
Originally posted by: fadden
On Wednesday, March 25, 2020 at 10:06:30 PM UTC-7, fadden wrote:
> On Wednesday, March 25, 2020 at 9:53:10 AM UTC-7, fatdog projects wrote:
>> Do you think it is possible to allow CiderPress to import files >16MB to an HFS image?
> I'll try to throw something together tomorrow morning.
I've posted CiderPress v4.0.4-d1, which allows files larger than 16MB to be added to HFS volumes. It also has a fix for an Apple /// BASIC token that I checked in a while back.
CiderPress was originally written without HFS support, so some of the code makes simplifying assumptions, like the idea that any Apple II file will fit comfortably in memory on a Windows system. This might not be the case with a max-sized HFS file in a 32-bit app. Apparently I also got lazy and didn't bother driving the progress bar while adding files, so the app will appear to hang while adding something large.
https://github.com/fadden/ciderpress/releases
|
|
|
|
| Re: CiderPress request - HFS large files [message #392452 is a reply to message #392451] |
Thu, 26 March 2020 14:55 |
|
|
Originally posted by: fatdog projects
On Thursday, March 26, 2020 at 2:16:44 PM UTC-4, fadden wrote:
> On Wednesday, March 25, 2020 at 10:06:30 PM UTC-7, fadden wrote:
>> On Wednesday, March 25, 2020 at 9:53:10 AM UTC-7, fatdog projects wrote:
>>> Do you think it is possible to allow CiderPress to import files >16MB to an HFS image?
>> I'll try to throw something together tomorrow morning.
>
> I've posted CiderPress v4.0.4-d1, which allows files larger than 16MB to be added to HFS volumes. It also has a fix for an Apple /// BASIC token that I checked in a while back.
>
> CiderPress was originally written without HFS support, so some of the code makes simplifying assumptions, like the idea that any Apple II file will fit comfortably in memory on a Windows system. This might not be the case with a max-sized HFS file in a 32-bit app. Apparently I also got lazy and didn't bother driving the progress bar while adding files, so the app will appear to hang while adding something large.
>
> https://github.com/fadden/ciderpress/releases
That was fast, thank you again! DreamGrafix chewed through that 45 MB file like it was nothing at all. CiderPress imported it correctly, and I barely noticed the transfer time.
The 45 MB file is a full screen (320x180) 30 FPS video with 2700 frames of animation, and it played from beginning to end with GSPlus. This is very encouraging for my project :)
|
|
|
|
|
|
| Re: CiderPress request - HFS large files [message #392460 is a reply to message #392447] |
Thu, 26 March 2020 20:47 |
Bill Buckels
Messages: 1418
Registered: November 2012
Karma: 0
|
Senior Member |
|
|
"fatdog projects" <fatdogprojects@gmail.com> wrote:
> I will keep you guys posted on some of the videos if you are interested.
Of course...
> Bill, I use your A2B tool all the time to convert from BMP -> C1. It is
> extremely quick for what I need to do, processing thousands of files at a
> time. I want to thank you for all of the work you've done with your various
> projects. For me, BMP2DHR has been very useful and educational. Many others
> have been trying to take your ideas farther, try to find us!
Ok... I will... and yes A2B and whatever else it drags along is pretty quick
and pretty simple once a person does things with it a time or two. I too
processed thousands at a time back then... but to no real purpose except to
test the idea of the thing.
> Are you guys working on any current Apple projects? :)
I am just slowly getting back here again... I need to reach out to Jonas
(STYNX) and see what's new... but I am setting-up my working environment and
have some small projects in mind before trying to improve on video
conversions... I guess I'd better try to find out what's out there right
now.
That's true...
Regards,
Bill
|
|
|
|
Members
Search
Help
Register
Login
Home
