In recent pharo thread [Pharo-project] Save and quit vs. Save - a clue to performance??? it is said that saveAndQuit is amazingly fast, while save is damned slow... ... because it goes through all image startup. Cloning the vm/image could be more efficient:
primFork "return true if we are in the forked image, false in the original"
<primitive: XXX> self error: 'fork failed'
snapshot self primFork ifTrue: [ self aboutToSnapshot. self primSaveAndQuit. self returnFromSnapshot].
Apart being dependent on another feature of the OS, do you know any reason why this would not work?
PS: sorry if double posting, I clicked on wrong vm-dev address first time
Nicolas Cellier wrote:
Apart being dependent on another feature of the OS, do you know any reason why this would not work?
Well, there is the small matter that fork() doesn't exist on Windows. And no equivalent functionality either. Cygwin might emulate this but I have severe doubts as to whether this would work reliably in non-trivial cases (i.e., threads using sockets etc) and I can almost guarantee that it's going to be very, very slow if it worked at all.
Cheers, - Andreas
2009/7/17 Andreas Raab andreas.raab@gmx.de:
Nicolas Cellier wrote:
Apart being dependent on another feature of the OS, do you know any reason why this would not work?
Well, there is the small matter that fork() doesn't exist on Windows. And no equivalent functionality either. Cygwin might emulate this but I have severe doubts as to whether this would work reliably in non-trivial cases (i.e., threads using sockets etc) and I can almost guarantee that it's going to be very, very slow if it worked at all.
Yeah, and instead of this, i'd rather look for a ways to optimize cold/warm startup code. Many classes ignoring this difference and doing all things from scratch.
Cheers, - Andreas
On Fri, Jul 17, 2009 at 01:36:45AM +0200, Nicolas Cellier wrote:
In recent pharo thread [Pharo-project] Save and quit vs. Save - a clue to performance??? it is said that saveAndQuit is amazingly fast, while save is damned slow... ... because it goes through all image startup. Cloning the vm/image could be more efficient:
primFork "return true if we are in the forked image, false in the original"
<primitive: XXX> self error: 'fork failed'
snapshot self primFork ifTrue: [ self aboutToSnapshot. self primSaveAndQuit. self returnFromSnapshot].
The primFork that you describe has been part of OSProcess for a long time. The #primitiveForkSqueakprimitiveForkSqueak primitive is in the unix flavor of OSProcessPlugin, so it's probably already in whatever unix VM you are running today. For what you describe, you would probably want to invoke it with UnixProcess>>forkHeadlessSqueakAndDoThenQuit:
So you can just try it and see if it gives the performance you want.
Apart being dependent on another feature of the OS, do you know any reason why this would not work?
It definitely does work, but it is very OS dependent, and that's an extremely good reason for not doing it as a general solution.
I don't understand why save should be "amazingly slow". What's going on in the image startup that would cause this be become an issue? It sounds to me like somebody is not checking the resuming flag in a #startup: method, hence running a bunch of initialization code that is not required after an image save.
Dave
I'll wager a guess on the "amazingly slow" startup: Freetype.
Cheers, - Andreas
David T. Lewis wrote:
On Fri, Jul 17, 2009 at 01:36:45AM +0200, Nicolas Cellier wrote:
In recent pharo thread [Pharo-project] Save and quit vs. Save - a clue to performance??? it is said that saveAndQuit is amazingly fast, while save is damned slow... ... because it goes through all image startup. Cloning the vm/image could be more efficient:
primFork "return true if we are in the forked image, false in the original"
<primitive: XXX> self error: 'fork failed'
snapshot self primFork ifTrue: [ self aboutToSnapshot. self primSaveAndQuit. self returnFromSnapshot].
The primFork that you describe has been part of OSProcess for a long time. The #primitiveForkSqueakprimitiveForkSqueak primitive is in the unix flavor of OSProcessPlugin, so it's probably already in whatever unix VM you are running today. For what you describe, you would probably want to invoke it with UnixProcess>>forkHeadlessSqueakAndDoThenQuit:
So you can just try it and see if it gives the performance you want.
Apart being dependent on another feature of the OS, do you know any reason why this would not work?
It definitely does work, but it is very OS dependent, and that's an extremely good reason for not doing it as a general solution.
I don't understand why save should be "amazingly slow". What's going on in the image startup that would cause this be become an issue? It sounds to me like somebody is not checking the resuming flag in a #startup: method, hence running a bunch of initialization code that is not required after an image save.
Dave
OK, thanks to all, OS dependency is a good reason, we definitely prefer the more portable Smalltalk way :) Still, cloning an image with a HydraVM would be a portable option :) (Efficient if pages can be optimized for copy-on-write which is again an OS-dependent feature I presume...).
Maybe I'll give a try in Unix just for fun, or I'll have a deeper look on what's going on exactly.
Cheers
Nicolas
2009/7/17 David T. Lewis lewis@mail.msen.com:
On Fri, Jul 17, 2009 at 01:36:45AM +0200, Nicolas Cellier wrote:
In recent pharo thread [Pharo-project] Save and quit vs. Save - a clue to performance??? it is said that saveAndQuit is amazingly fast, while save is damned slow... ... because it goes through all image startup. Cloning the vm/image could be more efficient:
primFork "return true if we are in the forked image, false in the original"
<primitive: XXX> self error: 'fork failed'
snapshot self primFork ifTrue: [ self aboutToSnapshot. self primSaveAndQuit. self returnFromSnapshot].
The primFork that you describe has been part of OSProcess for a long time. The #primitiveForkSqueakprimitiveForkSqueak primitive is in the unix flavor of OSProcessPlugin, so it's probably already in whatever unix VM you are running today. For what you describe, you would probably want to invoke it with UnixProcess>>forkHeadlessSqueakAndDoThenQuit:
So you can just try it and see if it gives the performance you want.
Apart being dependent on another feature of the OS, do you know any reason why this would not work?
It definitely does work, but it is very OS dependent, and that's an extremely good reason for not doing it as a general solution.
I don't understand why save should be "amazingly slow". What's going on in the image startup that would cause this be become an issue? It sounds to me like somebody is not checking the resuming flag in a #startup: method, hence running a bunch of initialization code that is not required after an image save.
Dave
2009/7/17 Nicolas Cellier nicolas.cellier.aka.nice@gmail.com:
OK, thanks to all, OS dependency is a good reason, we definitely prefer the more portable Smalltalk way :) Still, cloning an image with a HydraVM would be a portable option :) (Efficient if pages can be optimized for copy-on-write which is again an OS-dependent feature I presume...).
Maybe I'll give a try in Unix just for fun, or I'll have a deeper look on what's going on exactly.
Nicolas, if your image is a complete sandbox, which using no any external resources , then a proposed approach makes sense. But in most cases it is not, and i don't see how external resources, allocated by different primitives/plugins can be safely 'cloned'/forked. So, i think, we should focus on improving snapshot/startup code on image side, not in VM.
Cheers
Nicolas
2009/7/17 David T. Lewis lewis@mail.msen.com:
On Fri, Jul 17, 2009 at 01:36:45AM +0200, Nicolas Cellier wrote:
In recent pharo thread [Pharo-project] Save and quit vs. Save - a clue to performance??? it is said that saveAndQuit is amazingly fast, while save is damned slow... ... because it goes through all image startup. Cloning the vm/image could be more efficient:
primFork "return true if we are in the forked image, false in the original"
<primitive: XXX> self error: 'fork failed'
snapshot self primFork ifTrue: [ self aboutToSnapshot. self primSaveAndQuit. self returnFromSnapshot].
The primFork that you describe has been part of OSProcess for a long time. The #primitiveForkSqueakprimitiveForkSqueak primitive is in the unix flavor of OSProcessPlugin, so it's probably already in whatever unix VM you are running today. For what you describe, you would probably want to invoke it with UnixProcess>>forkHeadlessSqueakAndDoThenQuit:
So you can just try it and see if it gives the performance you want.
Apart being dependent on another feature of the OS, do you know any reason why this would not work?
It definitely does work, but it is very OS dependent, and that's an extremely good reason for not doing it as a general solution.
I don't understand why save should be "amazingly slow". What's going on in the image startup that would cause this be become an issue? It sounds to me like somebody is not checking the resuming flag in a #startup: method, hence running a bunch of initialization code that is not required after an image save.
Dave
Igor, agree, cloning an image is functionnally the same as returning from snapshot... The difference here is that we haven't performed the cleanup before snapshot :(
So my scheme was more or less to release all external pointers in the clone before snapshoting, without performing any OS action (like flushing stream buffers etc...). But I should check what fork() really do exactly about allocated ressources, because it's a long time since I last use this kind of magic...
The only interest of this approach was to lazily rely on already optimized OS features, but you already convinced me this was not portable, so beat it :)
cheers
Nicolas
2009/7/17 Igor Stasenko siguctua@gmail.com:
2009/7/17 Nicolas Cellier nicolas.cellier.aka.nice@gmail.com:
OK, thanks to all, OS dependency is a good reason, we definitely prefer the more portable Smalltalk way :) Still, cloning an image with a HydraVM would be a portable option :) (Efficient if pages can be optimized for copy-on-write which is again an OS-dependent feature I presume...).
Maybe I'll give a try in Unix just for fun, or I'll have a deeper look on what's going on exactly.
Nicolas, if your image is a complete sandbox, which using no any external resources , then a proposed approach makes sense. But in most cases it is not, and i don't see how external resources, allocated by different primitives/plugins can be safely 'cloned'/forked. So, i think, we should focus on improving snapshot/startup code on image side, not in VM.
Cheers
Nicolas
2009/7/17 David T. Lewis lewis@mail.msen.com:
On Fri, Jul 17, 2009 at 01:36:45AM +0200, Nicolas Cellier wrote:
In recent pharo thread [Pharo-project] Save and quit vs. Save - a clue to performance??? it is said that saveAndQuit is amazingly fast, while save is damned slow... ... because it goes through all image startup. Cloning the vm/image could be more efficient:
primFork "return true if we are in the forked image, false in the original"
<primitive: XXX> self error: 'fork failed'
snapshot self primFork ifTrue: [ self aboutToSnapshot. self primSaveAndQuit. self returnFromSnapshot].
The primFork that you describe has been part of OSProcess for a long time. The #primitiveForkSqueakprimitiveForkSqueak primitive is in the unix flavor of OSProcessPlugin, so it's probably already in whatever unix VM you are running today. For what you describe, you would probably want to invoke it with UnixProcess>>forkHeadlessSqueakAndDoThenQuit:
So you can just try it and see if it gives the performance you want.
Apart being dependent on another feature of the OS, do you know any reason why this would not work?
It definitely does work, but it is very OS dependent, and that's an extremely good reason for not doing it as a general solution.
I don't understand why save should be "amazingly slow". What's going on in the image startup that would cause this be become an issue? It sounds to me like somebody is not checking the resuming flag in a #startup: method, hence running a bunch of initialization code that is not required after an image save.
Dave
-- Best regards, Igor Stasenko AKA sig.
vm-dev@lists.squeakfoundation.org