Hi Tim, Hi Dave,
> On Mar 31, 2024, at 10:15 AM, Tim Rowledge <tim@rowledge.org> wrote:
>
>
>
>> On 2024-03-31, at 9:04 AM, lewis@mail.msen.com wrote:
>> I'm also interested in the idea of doing the saving work in a separate forkSqueak OSProcess to let it run at low priority, and with little or no performance impact on multi-core platforms.
>
> This is an idea I like for quite a lot of long-running operations, and this particular case seems ideal.
>
> I can't remember any detail of how the squeaksource save is done, but forking the entire image usually takes close to zero time and then the forked image can take its time to complete the job. The possible complication I can imagine (and that may not apply, see above memory issue) would be if multiple save requests are made during the time a forked save is running. I *think* there can Only Be One data.obj and if we get multiple concurrent requests to save it we could probably just kill any running fork and let a new one replace whatever had been already written?
The problems with the fork-and-save design are that
1. it makes it harder to elide unneeded saves when the saving processes get behind. Keeping things in one image makes it easier to discard saves that can’t be started and/or completed by the time a new version of the graph is ready to save
2. whether the save processes take time from the server process depends at least somewhat on the OS. Given that the save processes are doing file i/o, the OS may prioritize the save processes above the server process
This can be handled like a batch job, all started saves finish. If you make the batch in time it gets saved, if not you wait till the next batch, add a delay after save to collect more items in the batch. The guarantee is each item is saved after commit time + delay time. Delay could be 24 hours, or maybe wait till 3am. Because the save is async anyway, you are just adding some time to the backup which isn't used (much/at all) anyway.
Ron
>
> Oh, and this reminds me that the ARM64 VM still explodes if you try to do the fork; IIRC we concluded it was something to do with executable permission pages and resetting the phase of the chocolate clock, something like that.
Submit an issue on github?
>
>
> tim
> --
> tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim
> The severity of the itch is proportional to the reach.
>
>
>