Leon Matthes uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.threaded-LM.3344.mcz
==================== Summary ====================
Name: VMMaker.threaded-LM.3344
Author: LM
Time: 9 November 2023, 5:33:02.730756 pm
UUID: c6953f3e-c636-40dd-b367-66cdfd88d1f7
Ancestors: VMMaker.threaded-LM.3343
Fix multiple issues with process switching
1. Processes became re-activated when suspended
2. AWOL processes were garbage-collected
3. AWOL processes were stored in the CogVMThread itself, causing realloc to move the CogVMThread struct. This invalidated the previously created pointers, including the handles returned from disownVM!
=============== Diff against VMMaker.threaded-LM.3343 ===============
Item was changed:
----- Method: CoInterpreterMT>>ownVM: (in category 'vm scheduling') -----
ownVM: vmThreadHandle
<public>
<inline: false>
<var: #vmThreadHandle type: #'void *'>
<var: #vmThread type: #'CogVMThread *'>
"This is the entry-point for plugins and primitives that wish to reacquire the VM after having
released it via disownVM or callbacks that want to acquire it without knowing their ownership
status. This call will block until the VM is owned by the current thread or an error occurs.
The argument should be the value answered by disownVM, or 0 for callbacks that don't know
if they have disowned or not. This is both an optimization to avoid having to query thread-
local storage for the current thread's index (since it can easily keep it in some local variable),
and a record of when an unbound process becomes affined to a thread for the dynamic
extent of some operation.
Answer 0 if the current thread is known to the VM (and on return owns the VM).
Answer 1 if the current thread is unknown to the VM and takes ownership.
Answer -1 if the current thread is unknown to the VM and fails to take ownership."
| flags vmThread |
vmThread := self cCoerce: vmThreadHandle to: #'CogVMThread *'.
vmThread ifNil:
[^self ownVMFromUnidentifiedThread].
+
+ self assert: vmThread = (cogThreadManager vmThreadAt: vmThread index).
flags := vmThread disownFlags.
(flags anyMask: DisownVMForProcessorRelinquish) ifTrue:
["Presumably we have nothing to do; this primitive is typically called from the
background process. So we should /not/ try and activate any threads in the
pool; they will waste cycles finding there is no runnable process, and will
cause a VM abort if no runnable process is found. But we /do/ want to allow
FFI calls that have completed, or callbacks a chance to get into the VM; they
do have something to do. DisownVMForProcessorRelinquish indicates this."
relinquishing := false.
self sqLowLevelMFence].
vmThread := cogThreadManager acquireVMFor: vmThread.
disownCount := disownCount - 1.
disowningVMThread ifNotNil:
[vmThread = disowningVMThread ifTrue:
[self assert: (vmThread cFramePointer isNil
or: [CFramePointer = vmThread cFramePointer and: [CStackPointer = vmThread cStackPointer]]).
self assert: self successful.
self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject.
disowningVMThread := nil.
cogit recordEventTrace ifTrue:
[self recordTrace: TraceOwnVM thing: ConstOne source: 0].
^0]. "if not preempted we're done."
self preemptDisowningThread].
"We've been preempted; we must restore state and update the threadId
in our process, and may have to put the active process to sleep."
self restoreVMStateFor: vmThread andFlags: flags.
cogit recordEventTrace ifTrue:
[self recordTrace: TraceOwnVM thing: ConstTwo source: 0].
^flags bitAnd: OwnVMForeignThreadFlag!
Item was changed:
----- Method: CoInterpreterMT>>preemptDisowningThread (in category 'vm scheduling') -----
preemptDisowningThread
"Set the relevant state for disowningVMThread so that it can resume after
being preempted and set disowningVMThread to nil to indicate preemption.
N.B. This should only be sent from checkPreemptionOfDisowningThread.
There are essentially four things to do.
a) save the VM's notion of the current C stack pointers; these are pointers
into a thread's stack and must be saved and restored in thread switch.
b) save the VM's notion of the current Smalltalk execution point. This is
simply the suspend half of a process switch that saves the current context
in the current process.
c) add the process to the thread's set of AWOL processes so that the scheduler
won't try to run the process while the thread has disowned the VM.
d) save the in-primitive VM state, newMethod and argumentCount
ownVM: will restore the VM context as of disownVM: from the above when it
finds it has been preempted."
| activeProc activeContext preemptedThread |
<var: #preemptedThread type: #'CogVMThread *'>
<inline: false>
self assert: disowningVMThread notNil.
self assert: (disowningVMThread vmThreadState = CTMUnavailable
or: [disowningVMThread vmThreadState = CTMWantingOwnership]).
self assertCStackPointersBelongToDisowningThread.
cogit recordEventTrace ifTrue:
[self recordTrace: TracePreemptDisowningThread
thing: (objectMemory integerObjectOf: disowningVMThread index)
source: 0].
disowningVMThread cStackPointer: CStackPointer.
disowningVMThread cFramePointer: CFramePointer.
activeProc := self activeProcess.
+ "To make sure the process isn't garbage collected, add the Process to the list of processes that are in external code.
+ This should also be helpful if we want to fail any processes that are AWOL when restarting an image."
+ self addFirstLink: activeProc toList: (objectMemory splObj: ProcessInExternalCodeTag).
+
- self assert: (objectMemory fetchPointer: MyListIndex ofObject: activeProc) = objectMemory nilObject.
- objectMemory
- storePointer: MyListIndex
- ofObject: activeProc
- withValue: (objectMemory splObj: ProcessInExternalCodeTag).
activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer.
objectMemory
storePointer: SuspendedContextIndex
ofObject: activeProc
withValue: activeContext.
"The instructionPointer must be pushed because the convention for inactive stack pages is that the
instructionPointer is top of stack. We need to know if this primitive is called from machine code
because the invariant that the return pc of an interpreter callee calling a machine code caller is
ceReturnToInterpreterPC must be maintained."
self push: instructionPointer.
self externalWriteBackHeadFramePointers.
"Since pushing the awol process may realloc disowningVMThread we need to reassign.
But since we're going to nil disowningVMThread anyway we can assign to a local."
+ cogThreadManager pushAWOLProcess: activeProc on: disowningVMThread.
+ preemptedThread := disowningVMThread.
- preemptedThread := cogThreadManager pushAWOLProcess: activeProc on: disowningVMThread.
disowningVMThread := nil.
(self threadAffinityOfProcess: activeProc) = 0 ifTrue:
[self setTemporaryThreadAffinityOfProcess: activeProc to: preemptedThread index bind: false].
preemptedThread
newMethodOrNull: newMethod;
argumentCount: argumentCount;
inMachineCode: instructionPointer <= objectMemory startOfMemory!
Item was changed:
----- Method: CoInterpreterMT>>restoreVMStateFor:andFlags: (in category 'vm scheduling') -----
restoreVMStateFor: vmThread andFlags: flags
"We've been preempted; we must restore state and update the threadId
in our process, and may have to put the active process to sleep."
+ | sched activeProc myProc removedFromList |
- | sched activeProc myProc |
sched := self schedulerPointer.
activeProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched.
(flags anyMask: OwnVMForeignThreadFlag)
ifTrue:
[self assert: foreignCallbackProcessSlot == ForeignCallbackProcess.
myProc := objectMemory splObj: foreignCallbackProcessSlot.
self assert: myProc ~= objectMemory nilObject.
objectMemory splObj: foreignCallbackProcessSlot put: objectMemory nilObject]
ifFalse: [myProc := cogThreadManager popAWOLProcess: vmThread].
self assert: activeProc ~= myProc.
(activeProc ~= objectMemory nilObject
and: [(objectMemory fetchPointer: MyListIndex ofObject: activeProc) = objectMemory nilObject]) ifTrue:
[self putToSleep: activeProc yieldingIf: preemptionYields].
+
+ "Make sure to remove the Process from the list of processes that are in external code, such that it may be GCed in future."
self assert: (objectMemory fetchPointer: MyListIndex ofObject: myProc) = (objectMemory splObj: ProcessInExternalCodeTag).
+ removedFromList := self removeProcess: myProc fromList: (objectMemory splObj: ProcessInExternalCodeTag).
+ self assert: removedFromList. "We shouldn't put code that does important things into an assert, so save result and check that."
objectMemory
+ storePointerUnchecked: MyListIndex ofObject: myProc withValue: objectMemory nilObject;
+ storePointer: ActiveProcessIndex ofObject: sched withValue: myProc.
+
+
- storePointer: ActiveProcessIndex ofObject: sched withValue: myProc;
- storePointerUnchecked: MyListIndex ofObject: myProc withValue: objectMemory nilObject.
"Only unaffine if the process was affined at this level and did not become bound in the interim."
((flags anyMask: ProcessUnaffinedOnDisown)
and: [(self isBoundProcess: myProc) not]) ifTrue:
[self setTemporaryThreadAffinityOfProcess: myProc to: 0 bind: false].
self initPrimCall.
self cCode:
[self externalSetStackPageAndPointersForSuspendedContextOfProcess: myProc]
inSmalltalk:
["Bypass the no-offset stack depth check in the simulator's externalSetStackPageAndPointersForSuspendedContextOfProcess:"
super externalSetStackPageAndPointersForSuspendedContextOfProcess: myProc.
"We're in ownVM:, hence in a primitive, hence need to include the argument count"
(self isMachineCodeFrame: framePointer) ifTrue:
[self maybeCheckStackDepth: vmThread argumentCount
sp: stackPointer
pc: instructionPointer]].
"If this primitive is called from machine code maintain the invariant that the return pc
of an interpreter callee calling a machine code caller is ceReturnToInterpreterPC."
(vmThread inMachineCode
and: [instructionPointer >= objectMemory startOfMemory]) ifTrue:
[self iframeSavedIP: framePointer put: instructionPointer.
instructionPointer := cogit ceReturnToInterpreterPC].
newMethod := vmThread newMethodOrNull.
argumentCount := vmThread argumentCount.
vmThread newMethodOrNull: nil.
self cCode: '' inSmalltalk:
[| range |
range := self cStackRangeForThreadIndex: vmThread index.
self assert: ((range includes: vmThread cStackPointer) and: [range includes: vmThread cFramePointer])].
self setCFramePointer: vmThread cFramePointer setCStackPointer: vmThread cStackPointer.
self assert: newMethod notNil
!
Item was changed:
----- Method: CoInterpreterMT>>transferTo:from: (in category 'process primitive support') -----
transferTo: newProc from: sourceCode
"Record a process to be awoken on the next interpreter cycle. Override to
potentially switch threads either if the new process is bound to another thread,
or if there is no runnable process but there is a waiting thread. Note that the
abort on no runnable process has beeen moved here from wakeHighestPriority."
| sched oldProc activeContext |
<inline: false>
statProcessSwitch := statProcessSwitch + 1.
self push: instructionPointer.
self externalWriteBackHeadFramePointers.
self assertValidExecutionPointe: instructionPointer r: framePointer s: stackPointer.
"ensureMethodIsCogged: in makeBaseFrameFor: in
externalSetStackPageAndPointersForSuspendedContextOfProcess:
below may do a code compaction. Nil instructionPointer to avoid it getting pushed twice."
instructionPointer := 0.
sched := self schedulerPointer.
oldProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched.
self recordContextSwitchFrom: oldProc in: sourceCode.
activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer + objectMemory wordSize.
objectMemory storePointer: SuspendedContextIndex ofObject: oldProc withValue: activeContext.
newProc ifNil:
["Two possibilities. One, there is at least one thread waiting to own the VM in which
+ case it should be activated. Two, there are no processes to run and so abort. This
+ is new in the MT VM, and only happens when the primitiveRelinquishProcessor has been
+ preempted. In that case the idle Process is not runnable and there is no Process to return to.
+ By setting the activeProcess to nilObject, any threads woken by the heartbeat don't actually
+ start running Smalltalk. This is then fixed when an AWOL thread comes back and restores its
+ previous state."
+ objectMemory
+ storePointer: ActiveProcessIndex ofObject: sched withValue: objectMemory nilObject.
+
- case it should be activated. Two, there are no processes to run and so abort."
cogThreadManager willingVMThread ifNotNil:
[:vmThread|
vmThread vmThreadState = CTMWantingOwnership ifTrue:
[self returnToSchedulingLoopAndReleaseVMOrWakeThread: vmThread source: sourceCode]].
"self error: 'scheduler could not find a runnable process'"
self returnToSchedulingLoopAndReleaseVMOrWakeThread: nil source: sourceCode].
"Switch to the new process"
objectMemory
storePointer: ActiveProcessIndex ofObject: sched withValue: newProc;
storePointerUnchecked: MyListIndex ofObject: newProc withValue: objectMemory nilObject.
self externalSetStackPageAndPointersForSuspendedContextOfProcess: newProc.
"Finally thread switch if required"
self threadSwitchIfNecessary: newProc from: sourceCode!
Item was added:
+ ----- Method: CogThreadManager>>growAWOLProcessesOf: (in category 'public api') -----
+ growAWOLProcessesOf: vmThread
+ <var: #vmThread type: #'CogVMThread*'>
+ self
+ cCode: [vmThread awolProcesses: (self realloc: vmThread awolProcesses _: vmThread awolProcLength + AWOLProcessesIncrement * (self sizeof: #sqInt))]
+ inSmalltalk: [vmThread awolProcesses setObject: vmThread awolProcesses object, (Array new: AWOLProcessesIncrement)].
+ vmThread awolProcLength: vmThread awolProcLength + AWOLProcessesIncrement.!
Item was changed:
----- Method: CogThreadManager>>growThreadInfosToAtLeast: (in category 'thread set') -----
growThreadInfosToAtLeast: index
"Grow the thread infos to at least index in numThreadsIncrement quanta."
| newThreads newNumThreads |
<var: #newThreads type: #'CogVMThread **'>
<inline: false>
memoryIsScarce ifTrue:
[^false].
newNumThreads := index + numThreadsIncrement - 1 // numThreadsIncrement * numThreadsIncrement.
newNumThreads >= self maxNumThreads ifTrue:
[^false].
"Since 0 is not a valid index we allocate one extra CogVMThread and use 1-relative indices."
newThreads := self cCode: 'realloc(GIV(threads), (newNumThreads + 1) * sizeof(CogVMThread *))'
inSmalltalk: [(Array new: newNumThreads)
replaceFrom: 1 to: numThreads
with: threads startingAt: 1].
+
+ newThreads ifNil: [memoryIsScarce := true. ^ false].
- (newThreads notNil
- and: [self populate: newThreads from: numThreads + 1 to: newNumThreads]) ifFalse:
- [
- "TODO: This cannot free 'newThreads', as that's going to mean 'threads' is freed as well."
- self abort.
- self free: newThreads.
- memoryIsScarce := true.
- ^false].
threads := newThreads.
+
+ (self populate: newThreads from: numThreads + 1 to: newNumThreads)
+ ifTrue: [numThreads := newNumThreads.
+ ^true]
+ ifFalse: ["Allocation of new threads may fail, even after the array has been moved.
+ If this is the case, simply do not increase the number of useable threads.
+ The old ones will still point to the right addresses, they'll just be in a new list
+ which technically has too much space, but that doesn't hurt anything."
+ memoryIsScarce := true.
+ ^false].
+ !
- numThreads := newNumThreads.
- ^true!
Item was changed:
----- Method: CogThreadManager>>populate:from:to: (in category 'thread set') -----
populate: vmThreadPointers from: start to: finish
"Populate vmThreadPointers with vmThreads over the given range."
<var: #vmThreadPointers type: #'CogVMThread **'>
| nThreads vmThreads |
<var: #vmThreads type: #'CogVMThread *'>
<var: #vmThread type: #'CogVMThread *'>
<inline: true>
nThreads := finish - start + 1.
vmThreads := self cCode: [self calloc: nThreads _: (self sizeof: CogVMThread)]
inSmalltalk: [CArrayAccessor on: ((1 to: nThreads) collect: [:ign| CogVMThread new])].
vmThreads ifNil:
[^false].
"Since 0 is not a valid index, in C we allocate one extra CogVMThread and use 1-relative indices."
self cCode: [start = 1 ifTrue: [vmThreadPointers at: 0 put: nil]]
inSmalltalk: [].
start to: finish do:
[:i| | vmThread |
vmThread := self addressOf: (vmThreads at: i - start).
vmThread initializeThreadState.
(self ioNewOSSemaphore: (self addressOf: vmThread osSemaphore put: [:sem| vmThread osSemaphore: sem])) ~= 0 ifTrue:
[start to: i - 1 do:
[:j|
vmThread := self addressOf: (vmThreads at: j - start).
self ioDestroyOSSemaphore: (self addressOf: vmThread osSemaphore)].
self free: vmThreads.
^false].
vmThreadPointers at: i put: vmThread.
- vmThread awolProcLength: AWOLProcessesIncrement.
vmThread index: i.
self cCode: [] inSmalltalk: [vmThread reenterThreadSchedulingLoop: ReenterThreadSchedulingLoop new]].
^true!
Item was changed:
----- Method: CogThreadManager>>pushAWOLProcess:on: (in category 'public api') -----
pushAWOLProcess: awolProcess on: vmThread
<var: #vmThread type: #'CogVMThread *'>
+ self assert: (vmThread awolProcIndex between: 0 and: vmThread awolProcLength).
+ vmThread awolProcIndex >= vmThread awolProcLength
+ ifTrue: [self growAWOLProcessesOf: vmThread].
+
+ self assert: vmThread awolProcLength > vmThread awolProcIndex.
+ vmThread awolProcesses at: vmThread awolProcIndex put: awolProcess.
+ vmThread awolProcIndex: vmThread awolProcIndex + 1.!
- <returnTypeC: #'CogVMThread *'>
- | cvt |
- <var: #cvt type: #'CogVMThread *'>
- cvt := vmThread.
- self assert: (cvt awolProcIndex between: 0 and: cvt awolProcLength).
- cvt awolProcIndex >= cvt awolProcLength ifTrue:
- ["The realloc doesn't look like it grows but it does so by AWOLProcessesIncrement
- entries because sizeof(CogVMThread) includes room for that many entries."
- cvt := self cCode: 'realloc(cvt,sizeof(CogVMThread) + (sizeof(sqInt) * cvt->awolProcLength))'
- inSmalltalk: [cvt growAWOLProcesses].
- threads at: vmThread index put: cvt.
- cvt awolProcLength: cvt awolProcLength + AWOLProcessesIncrement].
- cvt awolProcesses at: cvt awolProcIndex put: awolProcess.
- cvt awolProcIndex: cvt awolProcIndex + 1.
- ^cvt!
Item was changed:
VMStructType subclass: #CogVMThread
+ instanceVariableNames: 'index state priority osSemaphore osThread disownFlags newMethodOrNull argumentCount inMachineCode cStackPointer cFramePointer reenterThreadSchedulingLoop awolProcIndex awolProcLength awolProcesses'
- instanceVariableNames: 'index state priority osSemaphore osThread disownFlags newMethodOrNull argumentCount inMachineCode cStackPointer cFramePointer awolProcIndex awolProcLength awolProcesses reenterThreadSchedulingLoop'
classVariableNames: ''
poolDictionaries: 'VMThreadingConstants'
category: 'VMMaker-Multithreading'!
!CogVMThread commentStamp: '<historical>' prior: 0!
Instances of this class represent control blocks for native threads that cooperatively schedule the VM. See the class comment of CogThreadManager for full documentation.
N.B. awolProcesses must be the last inst var.!
Item was changed:
----- Method: CogVMThread class>>instVarNamesAndTypesForTranslationDo: (in category 'translation') -----
instVarNamesAndTypesForTranslationDo: aBinaryBlock
"enumerate aBinaryBlock with the names and C type strings for the inst vars to include in a CogVMThread struct."
self allInstVarNames do:
[:ivn|
aBinaryBlock
value: ivn
value: (ivn caseOf: {
+ ['awolProcesses'] -> [#'sqInt *'].
- ['awolProcesses'] -> [{#sqInt. '[', CogThreadManager awolProcessesIncrement printString, ']'}].
['cStackPointer'] -> [#usqIntptr_t].
['cFramePointer'] -> [#usqIntptr_t].
['osSemaphore'] -> ['sqOSSemaphore'].
['osThread'] -> ['sqOSThread'].
['reenterThreadSchedulingLoop'] -> ['jmp_buf'].
['state'] -> ['volatile atomic_int'] }
otherwise:
[#sqInt])]!
Item was removed:
- ----- Method: CogVMThread>>growAWOLProcesses (in category 'simulation only') -----
- growAWOLProcesses
- <doNotGenerate>
- awolProcesses setObject: awolProcesses object, (Array new: CogThreadManager awolProcessesIncrement)!
Item was changed:
----- Method: CogVMThread>>initializeThreadState (in category 'initialize-release') -----
initializeThreadState
"Unfortunately this cannot be inlined as Slang otherwise screws up the generation of the `atomic_store` call."
<inline: false>
"In comparision to #initialize, this is also called in C code to initialize the VMThread, not just in the Smalltalk simulation."
self cCode: [] inSmalltalk: [state := AtomicValue new].
+ self atomic_store: (self addressOf: self state) _: CTMUninitialized.
+
+ self
+ cCode: [awolProcesses := self malloc: AWOLProcessesIncrement * (self sizeof: #sqInt)]
+ inSmalltalk: [awolProcesses := CArrayAccessor on: (Array new: AWOLProcessesIncrement)].
+ awolProcIndex := 0.
+ awolProcLength := AWOLProcessesIncrement.!
- self atomic_store: (self addressOf: self state) _: CTMUninitialized.!
Branch: refs/heads/virtend
Home: https://github.com/OpenSmalltalk/opensmalltalk-vm
Commit: a3a3105fcf19ee3b51cd7f5ffb2e5334f8421a9f
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a3a3105fcf19ee3b51…
Author: Marcel Taeumel <marcel.taeumel(a)hpi.de>
Date: 2023-11-03 (Fri, 03 Nov 2023)
Changed paths:
A .github/workflows.ignore/extra-arm.yml
A .github/workflows.ignore/extra.yml
A .github/workflows.ignore/linux-arm.yml
A .github/workflows.ignore/linux.yml
A .github/workflows.ignore/macos-arm.yml
A .github/workflows.ignore/macos.yml
A .github/workflows.ignore/win.yml
R .github/workflows/extra-arm.yml
R .github/workflows/extra.yml
R .github/workflows/linux-arm.yml
R .github/workflows/linux.yml
R .github/workflows/macos-arm.yml
R .github/workflows/macos.yml
R .github/workflows/win.yml
Log Message:
-----------
Skip GitHub workflows in "virtend" branch for now because free macOS CI minutes are too precious.