These change sets are updates to the VM and to the low space watcher that fix low space interrupt handling and user interrupt handling in Morphic.
Under certain conditions the low space watcher was unable to determine the correct process to suspend following a low space signal. These changes permit the VM to remember the identity of the process that caused the low space condition, and to report it to the image through a primitive.
The low space watcher changes are backward compatible for use with VMs that do not yet have the new primitive (i.e. broken the same way as was the case in Squeak 3.7).
Caveats: - This is done with VMMaker 3-8b2, hence is not updated for 64 bit VM. - I added a numbered primitive, which is probably a bad thing (but I'm not sure where best to put it, possibly MiscPrimitivePlugin?). - There may be a simpler way to accomplish this, but at least this is a working implementation. - I have not tested the interpreter simulator, still need to make sure I did not break it.
Cross posted to vm-dev and squeak-dev. Follow-ups should go to vm-dev (unless someone can think of a way to handle this properly without touching the VM).
Dave
Thank you for your report. I have transferred your report to Squeak's Mantis Database and you can followup on the issue if desired by going to http://bugs.impara.de/view.php?id=1041 .
In the future please report new issues on Squeak's Mantis Database at http://bugs.impara.de/ .
Thanks!
On Sun, 2005-04-03 at 17:15 -0400, David T. Lewis wrote:
These change sets are updates to the VM and to the low space watcher that fix low space interrupt handling and user interrupt handling in Morphic.
Under certain conditions the low space watcher was unable to determine the correct process to suspend following a low space signal. These changes permit the VM to remember the identity of the process that caused the low space condition, and to report it to the image through a primitive.
The low space watcher changes are backward compatible for use with VMs that do not yet have the new primitive (i.e. broken the same way as was the case in Squeak 3.7).
Caveats:
- This is done with VMMaker 3-8b2, hence is not updated for 64 bit VM.
- I added a numbered primitive, which is probably a bad thing (but I'm not sure where best to put it, possibly MiscPrimitivePlugin?).
- There may be a simpler way to accomplish this, but at least this is a working implementation.
- I have not tested the interpreter simulator, still need to make sure I did not break it.
Cross posted to vm-dev and squeak-dev. Follow-ups should go to vm-dev (unless someone can think of a way to handle this properly without touching the VM).
Dave
Hi David -
Thanks for the code but let's not get too fast with this - I am still trying to understand why you could make the crash happen reliably... in some ways this doesn't make sense (since the failure should be triggered statistically) and I'd rather see that we either have situations in which we do not fail or that we understand why exactly we can repeat this behavior with 100% reliability (my explanation does not explain that).
Cheers, - Andreas
David T. Lewis wrote:
These change sets are updates to the VM and to the low space watcher that fix low space interrupt handling and user interrupt handling in Morphic.
Under certain conditions the low space watcher was unable to determine the correct process to suspend following a low space signal. These changes permit the VM to remember the identity of the process that caused the low space condition, and to report it to the image through a primitive.
The low space watcher changes are backward compatible for use with VMs that do not yet have the new primitive (i.e. broken the same way as was the case in Squeak 3.7).
Caveats:
- This is done with VMMaker 3-8b2, hence is not updated for 64 bit VM.
- I added a numbered primitive, which is probably a bad thing (but I'm not sure where best to put it, possibly MiscPrimitivePlugin?).
- There may be a simpler way to accomplish this, but at least this is a working implementation.
- I have not tested the interpreter simulator, still need to make sure I did not break it.
Cross posted to vm-dev and squeak-dev. Follow-ups should go to vm-dev (unless someone can think of a way to handle this properly without touching the VM).
Dave
On Mon, Apr 04, 2005 at 09:50:13AM -0700, Andreas Raab wrote:
Thanks for the code but let's not get too fast with this - I am still trying to understand why you could make the crash happen reliably... in some ways this doesn't make sense (since the failure should be triggered statistically) and I'd rather see that we either have situations in which we do not fail or that we understand why exactly we can repeat this behavior with 100% reliability (my explanation does not explain that).
It depends on how much time elapses between the low space condition and the arrival of the signal to the semaphore. If more than 500ms is consistently being used for garbage collection, then your explanation holds up very well. (Note: I am using a relatively slow 850MHz Pentium).
Dave
p.s. Speaking of going to fast, there is now a Mantis report corresponding to this at http://bugs.impara.de/view.php?id=1041. My posting was intended only as a draft proposal; I should not have put the [ENH] tag on the message.
On Mon, Apr 04, 2005 at 02:14:06PM -0400, David T. Lewis wrote:
On Mon, Apr 04, 2005 at 09:50:13AM -0700, Andreas Raab wrote:
Thanks for the code but let's not get too fast with this - I am still trying to understand why you could make the crash happen reliably... in some ways this doesn't make sense (since the failure should be triggered statistically) and I'd rather see that we either have situations in which we do not fail or that we understand why exactly we can repeat this behavior with 100% reliability (my explanation does not explain that).
It depends on how much time elapses between the low space condition and the arrival of the signal to the semaphore. If more than 500ms is consistently being used for garbage collection, then your explanation holds up very well. (Note: I am using a relatively slow 850MHz Pentium).
I ran some additional tests this evening and confirmed that, on my machine, the GC at the time of low space detection is consistently taking more than 500ms. Under these conditions, we should see a 100% failure rate, which is what I am observing. This is with a 9M fixed memory size, an 850MHz Pentium, and Unix VM.
The debugging output from five repeated tests is attached, showing console output produced with #createStackOverflow. The numbers to the left of each line are millisecond clock values, the first three of them done with printf in the VM, and the fourth with an OSProcess trace from the image. They show about 600ms elapsed time between entering allocateChunk() and signaling the low space semaphore from checkForInterrupts().
Dave
==== 28230 allocateChunk() entered 28823 allocateChunk() detected low memory condition 28824 checkForInterrupts() about to signal the low space semaphore 28825:2390:3864:SystemDictionary>>lowSpaceWatcher:low space semaphore signal received, preempted process is [] in EventSensor>>eventTickler {[delay wait. delta := Time millisecondClockValue - lastEventPoll. (delta <...]} BlockContext>>on:do: [] in EventSensor>>installEventTickler {[self eventTickler]} [] in BlockContext>>newProcess {[self value. Processor terminateActive]} ==== 27584 allocateChunk() entered 28176 allocateChunk() detected low memory condition 28177 checkForInterrupts() about to signal the low space semaphore 28178:1185:3864:SystemDictionary>>lowSpaceWatcher:low space semaphore signal received, preempted process is [] in EventSensor>>eventTickler {[delay wait. delta := Time millisecondClockValue - lastEventPoll. (delta <...]} BlockContext>>on:do: [] in EventSensor>>installEventTickler {[self eventTickler]} [] in BlockContext>>newProcess {[self value. Processor terminateActive]} ==== 73783 allocateChunk() entered 74374 allocateChunk() detected low memory condition 74375 checkForInterrupts() about to signal the low space semaphore 74377:3662:3864:SystemDictionary>>lowSpaceWatcher:low space semaphore signal received, preempted process is [] in EventSensor>>eventTickler {[delay wait. delta := Time millisecondClockValue - lastEventPoll. (delta <...]} BlockContext>>on:do: [] in EventSensor>>installEventTickler {[self eventTickler]} [] in BlockContext>>newProcess {[self value. Processor terminateActive]} ==== 121860 allocateChunk() entered 122451 allocateChunk() detected low memory condition 122452 checkForInterrupts() about to signal the low space semaphore 122453:792:3864:SystemDictionary>>lowSpaceWatcher:low space semaphore signal received, preempted process is [] in EventSensor>>eventTickler {[delay wait. delta := Time millisecondClockValue - lastEventPoll. (delta <...]} BlockContext>>on:do: [] in EventSensor>>installEventTickler {[self eventTickler]} [] in BlockContext>>newProcess {[self value. Processor terminateActive]} ==== 166147 allocateChunk() entered 166743 allocateChunk() detected low memory condition 166744 checkForInterrupts() about to signal the low space semaphore 166745:3674:3864:SystemDictionary>>lowSpaceWatcher:low space semaphore signal received, preempted process is [] in EventSensor>>eventTickler {[delay wait. delta := Time millisecondClockValue - lastEventPoll. (delta <...]} BlockContext>>on:do: [] in EventSensor>>installEventTickler {[self eventTickler]} [] in BlockContext>>newProcess {[self value. Processor terminateActive]} ====
This is an update for MVC.
LowSpaceAndInterruptHandlerMVC-dtl.cs.gz adds low space and interrupt key handling for MVC. Load this after LowSpaceAnddInterruptHandler-dtl, and load VMMLowSpaceAndInterruptHander-dtl for the VM changes.
With these change sets loaded (and with the VM rebuilt), Squeak MVC passes the four Andreas tests for interruptability:
* "[true] whileTrue" * "[[true] whileTrue] forkAt: Processor userSchedulingPriority + 1" * "Smalltalk createStackOverflow" * "[Smalltalk createStackOverflow] forkAt: Processor userSchedulingPriority + 1"
These changes also correct the long-standing (unreported) problem of not being able to interrupt a high priority background process in MVC (test 2 above). That problem goes back to at least Squeak 3.6.
(I've attached all three change sets so folks don't have to dig through the mail to find them, but LowSpaceAndInterruptHandlerMVC-dtl.cs.gz is the only thing that is new.)
Dave
On Sun, Apr 03, 2005 at 05:15:54PM -0400, David T. Lewis wrote:
These change sets are updates to the VM and to the low space watcher that fix low space interrupt handling and user interrupt handling in Morphic.
Under certain conditions the low space watcher was unable to determine the correct process to suspend following a low space signal. These changes permit the VM to remember the identity of the process that caused the low space condition, and to report it to the image through a primitive.
The low space watcher changes are backward compatible for use with VMs that do not yet have the new primitive (i.e. broken the same way as was the case in Squeak 3.7).
Caveats:
- This is done with VMMaker 3-8b2, hence is not updated for 64 bit VM.
- I added a numbered primitive, which is probably a bad thing (but I'm not sure where best to put it, possibly MiscPrimitivePlugin?).
- There may be a simpler way to accomplish this, but at least this is a working implementation.
- I have not tested the interpreter simulator, still need to make sure I did not break it.
Cross posted to vm-dev and squeak-dev. Follow-ups should go to vm-dev (unless someone can think of a way to handle this properly without touching the VM).
On Sun, Apr 03, 2005 at 05:15:54PM -0400, David T. Lewis wrote:
These change sets are updates to the VM and to the low space watcher that fix low space interrupt handling and user interrupt handling in Morphic.
Under certain conditions the low space watcher was unable to determine the correct process to suspend following a low space signal. These changes permit the VM to remember the identity of the process that caused the low space condition, and to report it to the image through a primitive.
The low space watcher changes are backward compatible for use with VMs that do not yet have the new primitive (i.e. broken the same way as was the case in Squeak 3.7).
Caveats:
- This is done with VMMaker 3-8b2, hence is not updated for 64 bit VM.
- I added a numbered primitive, which is probably a bad thing (but I'm not sure where best to put it, possibly MiscPrimitivePlugin?).
- There may be a simpler way to accomplish this, but at least this is a working implementation.
- I have not tested the interpreter simulator, still need to make sure I did not break it.
Tim suggested (on Mantis) that the process causing the low space condition could be passed back to the image in the special objects array, rather than passing the identity hash via a primitive. The implemention is attached.
LowSpaceAndInterruptHandler-2-dtl - image side patches for Morphic and MVC VMMLowSpaceAndInterruptHandler-2-dtl - VM patches
Dave
vm-dev@lists.squeakfoundation.org
-
Andreas Raab -
David T. Lewis -
Ken Causey