Vm-dev February 2024

vm-dev@lists.squeakfoundation.org

8 participants
13 discussions

[OpenSmalltalk/opensmalltalk-vm] use --whole-archive option instead of -whole-archive for solaris ld (PR #673)
by dcstes 19 Apr '24

19 Apr '24

I believe that the GNU link editor supports both single hyphen and double hyphen options -whole-archive and --whole-archive However the SUN (Solaris and SunOS and Illumos) link editor uses different names like "-z weakextract" Three is a GNU compatibility layer but the SUN link editor insists for GNU compatibility on the double hyphen format -whole-archive and --no-whole-archive David Stes You can view, comment on, or merge this pull request online at: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/673 -- Commit Summary -- * use --whole-archive option instead of -whole-archive for solaris ld -- File Changes -- M platforms/unix/config/Makefile.in (2) -- Patch Links -- https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/673.patch https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/673.diff -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/673 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/pull/673(a)github.com>

2 4

[OpenSmalltalk/opensmalltalk-vm] mmap() sqUnixSpurMemory patch (Issue #665)
by David Stes 03 Mar '24

03 Mar '24

Hi, I have sent in the past email to Eliot Miranda about an issue which I discovered in August 2021 (two years ago), and I thought to raise an issue here. It is a Linux/UNIX patch issue with the behavior of the mmap() system call or library call as used by the OpenSmalltalk COG vm. When way back in 2021 some changes were made for Linux ARM8 to the file saUnixSpurMemory.c I reported that this was breaking the Solaris 11.3 build. Curiously the Solaris 11.4 build was still working fine. I think (I don't know) that perhaps the behavior of mmap() on Solaris 11.4 is slightly diferent than on 11.3, to match (perhaps) the Linux kernel behavior of mmap(). An idea that I have is to ask some UNIX guru's whether there exist a autoconf-archive (configure script) test or script to test the behavior of mmap() for portability. However here I will just report what the issue is : since august 2021 I am building succesfully on Solaris with the following patch ``` --- opensmalltalk-vm-sun-v5.0.34/platforms/unix/vm/sqUnixSpurMemory.c Fri Aug 13 19:54:42 2021 +++ p0/opensmalltalk-vm-sun-v5.0.34/platforms/unix/vm/sqUnixSpurMemory.c Sun Aug 15 14:11:58 2021 @@ -277,16 +277,21 @@ { void *hint = sbrk(0); // a hint of the lowest possible address for mmap void *result; + char *address; + unsigned long alignment; pageSize = getpagesize(); pageMask = ~(pageSize - 1); + alignment = max(pageSize,1024*1024); + address = (char *)(((usqInt)hint + alignment - 1) & ~(alignment - 1)); + #if !defined(MAP_JIT) # define MAP_JIT 0 #endif *desiredSize = roundUpToPage(*desiredSize); - result = mmap(hint, *desiredSize, + result = mmap(address, *desiredSize, #if DUAL_MAPPED_CODE_ZONE PROT_READ | PROT_EXEC, #else ``` I really need Eliot's help on this. Eliot : what I did was try to understand your code and you seem to make some alignment calculations on the desired memory location that you request through mmap(). I believe that the traditional UNIX mmap() or the Solaris 10 and Solaris 11.3 mmap() is reacting slightly differently to the requested memory "hint". By some experimentatoin and debugging I found the above patch, which I wrote based on looking at your code and try to mirror the calculations that you are doing, which are not fully symmetric. By copying some code, I discovered that with the above patch the build for both Solaris 11.3 and Solaris 11.4 are working fine again. I think this is a general UNIX issue that could be of interest to all UNIX software. If an autoconfigure autoconf-archive test would exist to test the behavior of mmap() then that could be dealt in all software that uses mmap() with anonymous memory mapping. As far as I know the history of the mmap() call is that it originated in the world of Sun and Solaris and is since a very long time present in Linux as well, and I think some improvements in Linux where then backported to Solaris 11.4. That would explain why the current OpenSmalltalk-VM code works fine on Solaris 11.4. For reasons of portability to all flavors of UNIX and Linux, I think a configure build test for the code could benefit portability of your Cog VM. Let us know please what you think. I could simply submit my patch as a patch to mainstream upstream OpenSmalltalk but I don't know your opinion about it. And the way I have been using it as a platform specific patch (that I apply prior to the build on Solaris) is fine for me as well. Regards, David Stes -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/665 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/665(a)github.com>

4 19

[OpenSmalltalk/opensmalltalk-vm] align return value of sbrk() (PR #672)
by dcstes 02 Mar '24

02 Mar '24

Possible fix for issue mmap() sqUnixSpurMemory patch #665 David Stes You can view, comment on, or merge this pull request online at: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/672 -- Commit Summary -- * align return value of sbrk() -- File Changes -- M platforms/unix/vm/sqUnixSpurMemory.c (7) -- Patch Links -- https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/672.patch https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/672.diff -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/672 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/pull/672(a)github.com>

3 8

[OpenSmalltalk/opensmalltalk-vm] c1cd03: CogVM source as per VMMaker.oscog-eem.3349
by Eliot Miranda 16 Feb '24

16 Feb '24

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: c1cd0354069516bfdecd939710f81c8ecdd49f7e https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/c1cd0354069516bfde… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2024-02-13 (Tue, 13 Feb 2024) Changed paths: M platforms/iOS/plugins/ClipboardExtendedPlugin/sqMacExtendedClipboard.m M scripts/revertIfEssentiallyUnchanged M src/plugins/FilePlugin/FilePlugin.c M src/plugins/Squeak3D/Squeak3D.c M src/plugins/SqueakSSL/SqueakSSL.c M src/spur32.cog.lowcode/cointerp.c M src/spur32.cog.lowcode/gcc3x-cointerp.c M src/spur32.cog.newspeak/cointerp.c M src/spur32.cog.newspeak/gcc3x-cointerp.c M src/spur32.cog/cointerp.c M src/spur32.cog/cointerpmt.c M src/spur32.cog/gcc3x-cointerp.c M src/spur32.cog/gcc3x-cointerpmt.c M src/spur32.sista/cointerp.c M src/spur32.sista/gcc3x-cointerp.c M src/spur64.cog.lowcode/cointerp.c M src/spur64.cog.lowcode/gcc3x-cointerp.c M src/spur64.cog.newspeak/cointerp.c M src/spur64.cog.newspeak/gcc3x-cointerp.c M src/spur64.cog/cogitARMv8.c M src/spur64.cog/cointerp.c M src/spur64.cog/cointerpmt.c M src/spur64.cog/gcc3x-cointerp.c M src/spur64.cog/gcc3x-cointerpmt.c M src/spur64.sista/cogitARMv8.c M src/spur64.sista/cointerp.c M src/spur64.sista/gcc3x-cointerp.c M src/spur64.stack.newspeak/gcc3x-interp.c M src/spur64.stack.newspeak/interp.c M src/v3.cog/cointerp.c M src/v3.cog/gcc3x-cointerp.c Log Message: ----------- CogVM source as per VMMaker.oscog-eem.3349 Integrate VMMaker.threaded-LM.3339, 3341, 3342 & 3343. Make vmThread>>#state an atomic_int Allow -1 to be set as vmOwner. This is currently used by the heartbeat thread. In future this will be used by the ownVMFromUnidentifiedThread function. This combines well with the additional change that makes disownVM return a CogVMThread* disguised as void*. We can then ensure that the threads variable is only ever accessed by the vm owner. Fix multiple issues regarding thread safety 1. vmOwner -1 can be used to lock the VM to a thread that is not a classical "VM Thread" (e.g. the heartbeat thread). 2. maxWaitingPriority is now an atomic. Fix thread switch accidentally disowning the VM which causes an incorrect preemption. Allow thread switching during the SqueakSSL primitiveConnect. Rename ownerIndex to threadAffinity. And update revertIfEssentiallyUnchanged to cope with the CryptographyPlugins.

3 3

[OpenSmalltalk/opensmalltalk-vm] 48ac7b: CogVM source as per VMMaker.oscog-eem.3351
by Eliot Miranda 15 Feb '24

15 Feb '24

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 48ac7b50956a6d3fb6be3a8abb3af488a2ac35ce https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/48ac7b50956a6d3fb6… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2024-02-15 (Thu, 15 Feb 2024) Changed paths: M src/spur32.cog.lowcode/cointerp.c M src/spur32.cog.lowcode/cointerp.h M src/spur32.cog.lowcode/gcc3x-cointerp.c M src/spur32.cog/cointerp.c M src/spur32.cog/cointerp.h M src/spur32.cog/gcc3x-cointerp.c M src/spur32.sista/cointerp.c M src/spur32.sista/cointerp.h M src/spur32.sista/gcc3x-cointerp.c M src/spur32.stack.lowcode/gcc3x-interp.c M src/spur32.stack.lowcode/interp.c M src/spur32.stack/gcc3x-interp.c M src/spur32.stack/interp.c M src/spur64.cog.lowcode/cointerp.c M src/spur64.cog.lowcode/cointerp.h M src/spur64.cog.lowcode/gcc3x-cointerp.c M src/spur64.cog/cointerp.c M src/spur64.cog/cointerp.h M src/spur64.cog/gcc3x-cointerp.c M src/spur64.sista/cointerp.c M src/spur64.sista/cointerp.h M src/spur64.sista/gcc3x-cointerp.c M src/spur64.stack.lowcode/gcc3x-interp.c M src/spur64.stack.lowcode/interp.c M src/spur64.stack/gcc3x-interp.c M src/spur64.stack/interp.c M src/v3.cog/cointerp.c M src/v3.cog/cointerp.h M src/v3.cog/gcc3x-cointerp.c M src/v3.stack/gcc3x-interp.c M src/v3.stack/interp.c Log Message: ----------- CogVM source as per VMMaker.oscog-eem.3351 Oops; not so fast! The fallback disown/own code needs to be a little more careful to avoid failing the sender.

1 0

VM Maker: VMMaker.oscog-eem.3351.mcz
by commits＠source.squeak.org 15 Feb '24

15 Feb '24

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3351.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.3351 Author: eem Time: 15 February 2024, 2:08:43.718855 pm UUID: ee0e1c44-4495-48cb-aa70-ca668d8bd20d Ancestors: VMMaker.oscog-eem.3350 Oops; not so fast! The fallback disown/own code needs to be a little more careful to avoid failing the sender. =============== Diff against VMMaker.oscog-eem.3350 =============== Item was changed: ----- Method: StackInterpreter>>disownVM: (in category 'vm scheduling') ----- disownVM: flags <public> <inline: false> "Release the VM to other threads and answer the current thread's index. Currently valid flags for the non-threaded VM are: DisownVMForThreading - allow the VM to thread-switch; this is ignored DisownVMLockOutFullGC - prevent fullGCs while this thread disowns the VM DisownVMForFFICall - informs the VM that it is entering an FFI call This is the entry-point for plugins and primitives that wish to release the VM while performing some operation that may potentially block, and for callbacks returning back to some blocking operation. While this exists for the threaded FFI VM we use it to reset newMethod and the argumentCount after a callback." self assert: primFailCode = 0. flags = DisownVMForThreading ifTrue: + [^flags]. - [^0]. "Hack encodings of client state. We use non-immediates (bottom three bits clear) for FFI/Plugin doing save := self disownVM: FLAGS. ... callout ... self ownVM: save. We use immediate integer (bottom bit 1) for callbacks doing save := self ownVM: 0. ... callback ... self disownVM: save. return to C" self assert: ((objectMemory isImmediate: flags) ifFalse: [flags = (flags bitAnd: DisownVMForFFICall+DisownVMForThreading) and: [flags anyMask: DisownVMForFFICall]] ifTrue: [(objectMemory isIntegerObject: flags) and: [(objectMemory integerValueOf: flags) between: 0 and: (self argumentCountOfMethodHeader: -1)]]). "If DisownVMForFFICall this is from the FFI plugin and we're making a callout; remember the fact." (((objectMemory isImmediate: flags)) not and: [flags anyMask: DisownVMForFFICall]) ifTrue: [self assert: ((objectMemory isOopCompiledMethod: newMethod) and: [(self argumentCountOf: newMethod) = argumentCount]). inFFIFlags := DisownVMForFFICall. ^flags]. self assert: ((objectMemory isIntegerObject: flags) and: [(objectMemory integerValueOf: flags) between: 0 and: (self argumentCountOfMethodHeader: -1)]). "Otherwise this is a callback return; restore argumentCount and newMethod as per the ownVM: on callback." argumentCount := objectMemory integerValueOf: flags. newMethod := self popStack. self assert: ((objectMemory isOopCompiledMethod: newMethod) and: [(self argumentCountOf: newMethod) = argumentCount]). ^0! Item was changed: ----- Method: StackInterpreter>>ownVM: (in category 'vm scheduling') ----- ownVM: threadIndexAndFlags <public> <inline: false> "This is the entry-point for plugins and primitives that wish to reacquire the VM after having released it via disownVM or callbacks that want to acquire it without knowing their ownership status. While this exists for the threaded FFI VM we use it to reset newMethod and the argumentCount after a callback. Answer -1 if the current thread is unknown to the VM and fails to take ownership." <var: 'amInVMThread' declareC: 'extern sqInt amInVMThread(void)'> self cppIf: COGMTVM ifTrue: [self amInVMThread ifFalse: [^-1]]. self assert: ((objectMemory isOopCompiledMethod: newMethod) and: [(self argumentCountOf: newMethod) = argumentCount]). + threadIndexAndFlags = DisownVMForThreading ifTrue: + [^threadIndexAndFlags]. + "Hack encodings of client state. We use non-immediates (bottom three bits clear) for FFI/Plugin doing save := self disownVM: FLAGS. ... callout ... self ownVM: save. We use immediate integer (bottom bit 1) for callbacks doing save := self ownVM: 0. ... callback ... self disownVM: save. return to C" "If DisownVMForFFICall this is from the FFI plugin and we're returning from a callout." (threadIndexAndFlags anyMask: DisownVMForFFICall) ifTrue: [inFFIFlags := 0. ^threadIndexAndFlags]. "Otherwise this is a callback; stash newMethod on the stack and encode argumentCount in the flags retrieved when the calback calls disownVM:." self assert: primFailCode = 0. self push: newMethod. ^objectMemory integerObjectOf: argumentCount!

1 0

[OpenSmalltalk/opensmalltalk-vm] 172df7: CogVM source as per VMMaker.oscog-eem.3350
by Eliot Miranda 15 Feb '24

15 Feb '24

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 172df70d3c8942aefa3ab42768573beee68070f3 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/172df70d3c8942aefa… Author: Eliot Miranda <eliot.miranda(a)gmail.com> Date: 2024-02-15 (Thu, 15 Feb 2024) Changed paths: M src/spur32.cog.lowcode/cointerp.c M src/spur32.cog.lowcode/cointerp.h M src/spur32.cog.lowcode/gcc3x-cointerp.c M src/spur32.cog/cointerp.c M src/spur32.cog/cointerp.h M src/spur32.cog/gcc3x-cointerp.c M src/spur32.sista/cointerp.c M src/spur32.sista/cointerp.h M src/spur32.sista/gcc3x-cointerp.c M src/spur32.stack.lowcode/gcc3x-interp.c M src/spur32.stack.lowcode/interp.c M src/spur32.stack/gcc3x-interp.c M src/spur32.stack/interp.c M src/spur64.cog.lowcode/cointerp.c M src/spur64.cog.lowcode/cointerp.h M src/spur64.cog.lowcode/gcc3x-cointerp.c M src/spur64.cog/cointerp.c M src/spur64.cog/cointerp.h M src/spur64.cog/gcc3x-cointerp.c M src/spur64.sista/cointerp.c M src/spur64.sista/cointerp.h M src/spur64.sista/gcc3x-cointerp.c M src/spur64.stack.lowcode/gcc3x-interp.c M src/spur64.stack.lowcode/interp.c M src/spur64.stack/gcc3x-interp.c M src/spur64.stack/interp.c M src/v3.cog/cointerp.c M src/v3.cog/cointerp.h M src/v3.cog/gcc3x-cointerp.c M src/v3.stack/gcc3x-interp.c M src/v3.stack/interp.c Log Message: ----------- CogVM source as per VMMaker.oscog-eem.3350 Fix regression introduced in VMMaker.oscog-eem.3349 by adding a disownVM:/ownVM: pair to the SSL connnect primitive.

1 0

VM Maker: VMMaker.oscog-eem.3350.mcz
by commits＠source.squeak.org 15 Feb '24

15 Feb '24

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3350.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.3350 Author: eem Time: 15 February 2024, 1:18:23.815793 pm UUID: b14d789f-780b-49f2-91aa-d50085ca62e6 Ancestors: VMMaker.oscog-eem.3349 Fix regression introduced in VMMaker.oscog-eem.3349 by adding a disownVM:/ownVM: pair to the SSL connnect primitive. =============== Diff against VMMaker.oscog-eem.3349 =============== Item was changed: ----- Method: StackInterpreter>>disownVM: (in category 'vm scheduling') ----- disownVM: flags <public> <inline: false> "Release the VM to other threads and answer the current thread's index. Currently valid flags for the non-threaded VM are: + DisownVMForThreading - allow the VM to thread-switch; this is ignored DisownVMLockOutFullGC - prevent fullGCs while this thread disowns the VM + DisownVMForFFICall - informs the VM that it is entering an FFI call - DisownVMForFFICall - informs the VM that it is entering an FFI call This is the entry-point for plugins and primitives that wish to release the VM while performing some operation that may potentially block, and for callbacks returning back to some blocking operation. While this exists for the threaded FFI VM we use it to reset newMethod and the argumentCount after a callback." self assert: primFailCode = 0. + flags = DisownVMForThreading ifTrue: + [^0]. + "Hack encodings of client state. We use non-immediates (bottom three bits clear) for FFI/Plugin doing save := self disownVM: FLAGS. ... callout ... self ownVM: save. We use immediate integer (bottom bit 1) for callbacks doing save := self ownVM: 0. ... callback ... self disownVM: save. return to C" self assert: ((objectMemory isImmediate: flags) ifFalse: [flags = (flags bitAnd: DisownVMForFFICall+DisownVMForThreading) and: [flags anyMask: DisownVMForFFICall]] ifTrue: [(objectMemory isIntegerObject: flags) and: [(objectMemory integerValueOf: flags) between: 0 and: (self argumentCountOfMethodHeader: -1)]]). "If DisownVMForFFICall this is from the FFI plugin and we're making a callout; remember the fact." (((objectMemory isImmediate: flags)) not and: [flags anyMask: DisownVMForFFICall]) ifTrue: [self assert: ((objectMemory isOopCompiledMethod: newMethod) and: [(self argumentCountOf: newMethod) = argumentCount]). inFFIFlags := DisownVMForFFICall. ^flags]. self assert: ((objectMemory isIntegerObject: flags) and: [(objectMemory integerValueOf: flags) between: 0 and: (self argumentCountOfMethodHeader: -1)]). "Otherwise this is a callback return; restore argumentCount and newMethod as per the ownVM: on callback." argumentCount := objectMemory integerValueOf: flags. newMethod := self popStack. self assert: ((objectMemory isOopCompiledMethod: newMethod) and: [(self argumentCountOf: newMethod) = argumentCount]). ^0!

1 0

[OpenSmalltalk/opensmalltalk-vm] Verify Y2k38 issue, 64-bit time (Issue #676)
by Marcel Taeumel 14 Feb '24

14 Feb '24

https://lwn.net/ml/debian-devel-announce/Zb0WpSukajgythGe@homer.dodds.net/ We might want to think about a strategy für 32-bit VM flavors and how OSVM (and Squeak, Cuis, ... for that matter) would be affected by this issue. Let's discuss this here. -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/676 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/676(a)github.com>

3 2

VM Maker: VMMaker.oscog-eem.3349.mcz
by commits＠source.squeak.org 13 Feb '24

13 Feb '24

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.3349.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.3349 Author: eem Time: 13 February 2024, 3:11:47.759154 pm UUID: 09001547-129a-4b6b-9fed-ccdd6790c8f4 Ancestors: VMMaker.oscog-eem.3348 Integrate VMMaker.threaded-LM.3339, 3341, 3342 & 3343. Make vmThread>>#state an atomic_int Allow -1 to be set as vmOwner. This is currently used by the heartbeat thread. In future this will be used by the ownVMFromUnidentifiedThread function. This combines well with the additional change that makes disownVM return a CogVMThread* disguised as void*. We can then ensure that the threads variable is only ever accessed by the vm owner. Fix multiple issues regarding thread safety 1. vmOwner -1 can be used to lock the VM to a thread that is not a classical "VM Thread" (e.g. the heartbeat thread). 2. maxWaitingPriority is now an atomic. Fix thread switch accidentally disowning the VM which causes an incorrect preemption. Allow thread switching during the SqueakSSL primitiveConnect. Rename ownerIndex to threadAffinity. =============== Diff against VMMaker.oscog-eem.3348 =============== Item was added: + ----- Method: AtomicValue>>printOn: (in category 'as yet unclassified') ----- + printOn: aStream + + aStream nextPutAll: 'Atomic: '. + self value printOn: aStream.! Item was changed: StackInterpreterPrimitives subclass: #CoInterpreter instanceVariableNames: 'cogit cogMethodZone gcMode cogCodeSize desiredCogCodeSize heapBase primitiveMetadataTable lastCoggableInterpretedBlockMethod deferSmash deferredSmash primTraceLog primTraceLogIndex traceLog traceLogIndex traceSources cogCompiledCodeCompactionCalledFor statCodeCompactionCount statCodeCompactionUsecs lastUncoggableInterpretedBlockMethod flagInterpretedMethods maxLiteralCountForCompile minBackwardJumpCountForCompile CFramePointer CStackPointer CReturnAddress primTracePluginName primCalloutIsExternal' + classVariableNames: 'CSCallbackEnter CSCallbackLeave CSCheckEvents CSEnterCriticalSection CSExitCriticalSection CSOwnVM CSResume CSSignal CSSuspend CSSwitchIfNeccessary CSThreadBind CSThreadSchedulingLoop CSTryToExecuteSmalltalk CSWait CSYield HasBeenReturnedFromMCPC HasBeenReturnedFromMCPCOop MFMethodFlagFrameIsMarkedFlag MinBackwardJumpCountForCompile PrimTraceLogSize RumpCStackSize TraceBlockActivation TraceBlockCreation TraceBufferSize TraceCodeCompaction TraceContextSwitch TraceDisownVM TraceFullGC TraceIncrementalGC TraceIsFromInterpreter TraceIsFromMachineCode TraceOwnVM TracePreemptDisowningThread TracePrimitiveFailure TracePrimitiveRetry TraceSources TraceStackOverflow TraceThreadSwitch TraceVMCallback TraceVMCallbackReturn' - classVariableNames: 'CSCallbackEnter CSCallbackLeave CSCheckEvents CSEnterCriticalSection CSExitCriticalSection CSOwnVM CSResume CSSignal CSSuspend CSSwitchIfNeccessary CSThreadBind CSThreadSchedulingLoop CSWait CSYield HasBeenReturnedFromMCPC HasBeenReturnedFromMCPCOop MFMethodFlagFrameIsMarkedFlag MinBackwardJumpCountForCompile PrimTraceLogSize RumpCStackSize TraceBlockActivation TraceBlockCreation TraceBufferSize TraceCodeCompaction TraceContextSwitch TraceDisownVM TraceFullGC TraceIncrementalGC TraceIsFromInterpreter TraceIsFromMachineCode TraceOwnVM TracePreemptDisowningThread TracePrimitiveFailure TracePrimitiveRetry TraceSources TraceStackOverflow TraceThreadSwitch TraceVMCallback TraceVMCallbackReturn' poolDictionaries: 'CogMethodConstants VMStackFrameOffsets' category: 'VMMaker-JIT'! !CoInterpreter commentStamp: 'eem 3/31/2020 18:56' prior: 0! I am a variant of the StackInterpreter that can co-exist with the Cog JIT. I interpret unjitted methods, either because they have been found for the first time or because they are judged to be too big to JIT. See CogMethod class's comment for method interoperability. cogCodeSize - the current size of the machine code zone cogCompiledCodeCompactionCalledFor - a variable set when the machine code zone runs out of space, causing a machine code zone compaction at the next available opportunity cogMethodZone - the manager for the machine code zone (instance of CogMethodZone) cogit - the JIT (co-jit) (instance of SimpleStackBasedCogit, StackToRegisterMappoingCogit, etc) deferSmash - a flag causing deferral of smashes of the stackLimit around the call of functionSymbol (for assert checks) deferredSmash - a flag noting deferral of smashes of the stackLimit around the call of functionSymbol (for assert checks) desiredCogCodeSize - the desred size of the machine code zone, set at startup or via primitiveVMParameter to be written at snapshot time flagInterpretedMethods - true if methods that are interpreted shoudl have their flag bit set (used to identity methods that are interpreted because they're unjittable for some reason) gcMode - the variable holding the gcMode, used to inform the cogit of how to scan the machine code zone for oops on GC heapBase - the address in memory of the base of the objectMemory's heap, which is immediately above the machine code zone lastCoggableInterpretedBlockMethod - a variable used to invoke the cogit for a block mehtod being invoked repeatedly in the interpreter lastUncoggableInterpretedBlockMethod - a variable used to avoid invoking the cogit for an unjittable method encountered on block evaluation maxLiteralCountForCompile - the variable controlling which methods to jit. methods with a literal count above this value will not be jitted (on the grounds that large methods are typically used for initialization, and take up a lot of space in the code zone) minBackwardJumpCountForCompile - the variable controlling when to attempt to jit a method being interpreted. If as many backward jumps as this occur, the current method will be jitted primTraceLog - a small array implementing a crcular buffer logging the last N primitive invocations, GCs, code compactions, etc used for crash reporting primTraceLogIndex - the index into primTraceLog of the next entry reenterInterpreter - the jmpbuf used to jmp back into the interpreter when transitioning from machine code to the interpreter statCodeCompactionCount - the count of machine code zone compactions statCodeCompactionUsecs - the total microseconds spent in machine code zone compactions traceLog - a log of various events, used in debugging traceLogIndex - the index into traceLog of the next entry traceSources - the names associated with the codes of events in traceLog CFramePointer - if in use, the value of the C frame pointer on most recent entry to the interpreter after start-up or a callback. Used to establish the C stack when calling the run-time from generated machine code. CStackPointer - the value of the C stack pointer on most recent entry to the interpreter after start-up or a callback. Used to establish the C stack when calling the run-time from generated machine code. CReturnAddress - the return address for the function call which invoked the interpreter at start-up. Using this as teh return address when entering the interpreter via ceInvokeInterpeter maintains a valid stack. Since this is effevtively a constant it does not need to be saved and restored once set.! Item was changed: ----- Method: CoInterpreter class>>initializeMiscConstants (in category 'initialization') ----- initializeMiscConstants super initializeMiscConstants. COGVM := true. MinBackwardJumpCountForCompile := 40. MaxNumArgs := 15. + "These flags communicate primitive attributes to the Cogit so it can generate appropriate, often better, code." PrimCallOnSmalltalkStack := 1. "Speed up simple external prims by avoiding stack switch" PrimCallOnSmalltalkStackAlign2x := 2. "Align stack to a 2 x word size boundary, e.g. for MMX instructions etc" PrimCallNeedsNewMethod := 4. "e.g. primitiveExternalCall and primitiveCalloutToFFI extract info from newMethod's first literal" PrimCallMayEndureCodeCompaction := 8. "primitiveExternalCall and primitiveCalloutToFFI may invoke callbacks, hence may experience code compaction." PrimCallCollectsProfileSamples := 16. "tells JIT to compile support for profiling primitives" PrimCallIsExternalCall := 32. "Whether a primitive is not included in the VM, but loaded dynamically. Hence it can only be called through a CallFullRT." "Flags for use in primitiveMetadata: in external primitives, overlap with the PrimCallXXX flags above" FastCPrimitiveFlag := 1. "a.k.a. PrimCallOnSmalltalkStack" FastCPrimitiveAlignForFloatsFlag := 2. "a.k.a. PrimCallOnSmalltalkStackAlign2x" "And to shift away the flags, to compute the accessor depth, use... c.f. NullSpurMetadata in sq.h" SpurPrimitiveAccessorDepthShift := 8. SpurPrimitiveFlagsMask := 1 << SpurPrimitiveAccessorDepthShift - 1. "the primitive trace log; a record of the last 256 named/external primitives or significant events invoked." PrimTraceLogSize := 256. "Room for 256 selectors. Must be 256 because we use a byte to hold the index" TraceBufferSize := 256 * 3. "Room for 256 events" TraceContextSwitch := self objectMemoryClass basicNew integerObjectOf: 1. TraceBlockActivation := self objectMemoryClass basicNew integerObjectOf: 2. TraceBlockCreation := self objectMemoryClass basicNew integerObjectOf: 3. TraceIncrementalGC := self objectMemoryClass basicNew integerObjectOf: 4. TraceFullGC := self objectMemoryClass basicNew integerObjectOf: 5. TraceCodeCompaction := self objectMemoryClass basicNew integerObjectOf: 6. TraceOwnVM := self objectMemoryClass basicNew integerObjectOf: 7. TraceDisownVM := self objectMemoryClass basicNew integerObjectOf: 8. TraceThreadSwitch := self objectMemoryClass basicNew integerObjectOf: 9. TracePreemptDisowningThread := self objectMemoryClass basicNew integerObjectOf: 10. TraceVMCallback := self objectMemoryClass basicNew integerObjectOf: 11. TraceVMCallbackReturn := self objectMemoryClass basicNew integerObjectOf: 12. TraceStackOverflow := self objectMemoryClass basicNew integerObjectOf: 13. TracePrimitiveFailure := self objectMemoryClass basicNew integerObjectOf: 14. TracePrimitiveRetry := self objectMemoryClass basicNew integerObjectOf: 15. TraceIsFromMachineCode := 1. TraceIsFromInterpreter := 2. + "A list of sources for the trace log. SOme of these are specific to the threaded VM (CoInterpreterMT) but are declared here to keep related definitions together." CSCallbackEnter := 3. CSCallbackLeave := 4. CSEnterCriticalSection := 5. CSExitCriticalSection := 6. CSResume := 7. CSSignal := 8. CSSuspend := 9. CSWait := 10. CSYield := 11. CSCheckEvents := 12. CSThreadSchedulingLoop := 13. CSOwnVM := 14. CSThreadBind := 15. CSSwitchIfNeccessary := 16. + CSTryToExecuteSmalltalk := 17. + TraceSources := CArrayAccessor on: #('?' 'm' 'i' 'callbackEnter' 'callbackLeave' 'enterCritical' 'exitCritical' 'resume' 'signal' 'suspend' 'wait' 'yield' 'eventcheck' 'threadsched' 'ownVM' 'bindToThread' 'switchIfNecessary' 'tryToExecteSmalltalk'). - TraceSources := CArrayAccessor on: #('?' 'm' 'i' 'callbackEnter' 'callbackLeave' 'enterCritical' 'exitCritical' 'resume' 'signal' 'suspend' 'wait' 'yield' 'eventcheck' 'threadsched' 'ownVM' 'bindToThread' 'switchIfNecessary'). "this is simulation only" RumpCStackSize := 4096! Item was changed: CoInterpreterPrimitives subclass: #CoInterpreterMT instanceVariableNames: 'cogThreadManager checkThreadActivation maxWaitingPriority foreignCallbackPriority deferThreadSwitch disowningVMThread disownCount foreignCallbackProcessSlot activeProcessAffined relinquishing processHasThreadAffinity willNotThreadWarnCount' + classVariableNames: 'DisownVMForProcessorRelinquish OwnVMForeignThreadFlag PerThreadRumpCStackSize PrimNumberRelinquishProcessor ProcessUnaffinedOnDisown ReturnToThreadSchedulingLoop VMAlreadyOwnedHenceDoNotDisown' - classVariableNames: 'DisownFlagsShift DisownVMForProcessorRelinquish OwnVMForeignThreadFlag PerThreadRumpCStackSize PrimNumberRelinquishProcessor ProcessUnaffinedOnDisown ReturnToThreadSchedulingLoop VMAlreadyOwnedHenceDoNotDisown' poolDictionaries: 'VMThreadingConstants' category: 'VMMaker-Multithreading'! Item was changed: ----- Method: CoInterpreterMT class>>initializeSchedulerIndices (in category 'initialization') ----- initializeSchedulerIndices super initializeSchedulerIndices. "Class Process" "The thread id of a process is either nil or a SmallInteger that defines how a process binds to threads. If nil, the process may run on any thread. The least significant bit of threadId is a flag. The most significant bits are a threadId. If the threadId is nil the process can run on any thread. If the flag (least significant bit) is set then If the threadId is positive the process can only be run on the thread with that thread Id. If the threadId is negative the process must not be run on the thread with that thread Id. If the flag (least significant bit) is not set then the thread id will not be negative and if non-zero is the id of the thread the process is currenty running on The flag is probably a mistake..." "In part, what's really going on here is an attempt to deal with threading on Mac. Events can only be delivered on the GUi thread. To avoid blocking and/or crashing the VM by making a blocking (e.g. FFI) call on the GUI thread we want a nice way of preventing a process from running on the GUI thread. We can then use a process whose threadId precludes running on the GUI thread to make blocking calls. The alternative, of arranging that events are delivered to a thread the VM does not run on, is problematic; it cannot support event callbacks." "So if we simplify, and threadId is only instructive to the VM, (i.e. can say ''run on a given thread'', or ''don't run on a given thread'', and ''don't care; run on any thread'') and does not necessarily hold the threadId of the thread the process is currenty bound to, how do we locate the threadId for a process temporarily affined to a thread for the duration of an FFI call? Note that a process *must* be bound to a thread for the duration of a threaded call (or foreign callback) so that return occurs on the correct thread. We can use the least significant bit to mean ''temporarily affined'', but we need to support ''don't run on this thread''. We could use bit fields (yuck); we could allocate a two field object and assign it to threadId when setting a process to not run on a given thread. This isn't so bad; use negative values to mean ''don't run on this thread'', and positive values to mean ''run on this thread''. Split the smallest SmallInteger (32-bit, 1 bit sign, 2-bit tags, leaving 29//2) into two 14 bit fields. The least significant 14 bits are the thread id the receiver is temporarily affined to. The most significant 14 bits are the thread id of the thread the proess is either bound to or excluded from. If zero, the process is agnostic. See CogThreadManager>>#maxNumThreads" ThreadIdIndex := 4. ThreadIdShift := 14. "could be 30 in 64-bits" "disown result/own argument flags" OwnVMForeignThreadFlag := 1. VMAlreadyOwnedHenceDoNotDisown := 2. ProcessUnaffinedOnDisown := 4. "& defined in StackInterpreter are..." DisownVMForFFICall := 16. DisownVMForThreading := 32. "N.B. some of these DisownFlags are replicated in platforms/Cross/vm/sqVirtualMachine.h. Hence they should always be initialized." + DisownVMForProcessorRelinquish := 64.! - DisownVMForProcessorRelinquish := 64. - - DisownFlagsShift := DisownVMForProcessorRelinquish highBit! Item was changed: ----- Method: CoInterpreterMT>>cedeToHigherPriorityThreads (in category 'process primitive support') ----- cedeToHigherPriorityThreads "Invoked from checkForEventsMayContextSwitch: to switch threads if a thread wanting to acquire the VM has higher priority than the active process." + | activeProc processAffinity activeContext activePriority activeThread vmThread waitingPriority | - | activeProc processAffinity activeContext activePriority activeThread vmThread | <var: #activeThread type: #'CogVMThread *'> <var: #vmThread type: #'CogVMThread *'> <inline: false> activeProc := self activeProcess. activePriority := self quickFetchInteger: PriorityIndex ofObject: activeProc. + processAffinity := self threadAffinityOfProcess: activeProc. - processAffinity := self ownerIndexOfProcess: activeProc. activeThread := cogThreadManager currentVMThread. self assert: (cogThreadManager threadIndex: activeThread index isCompatibleWith: processAffinity). + waitingPriority := self getMaxWaitingPriority. activeThread priority: activePriority. vmThread := cogThreadManager highestPriorityThreadIfHigherThan: activePriority + expectedMax: waitingPriority. + - expectedMax: maxWaitingPriority. (vmThread isNil "no waiting thread of sufficiently high priority. Do not switch." or: [vmThread = activeThread]) "The activeProcess needs to run on a different thread. Leave this to threadSwitchIfNecessary:from: in checkForEventsMayContextSwitch:" ifTrue: + [waitingPriority > activePriority ifTrue: + ["We found no thread of sufficiently high priority, even though waitingPriority indicated there should be one. + So reduce the waiting priority back to the priority of the currently active process." + self reduceWaitingPriorityFrom: waitingPriority to: activePriority]. - [maxWaitingPriority > activePriority ifTrue: - [maxWaitingPriority := activePriority]. ^self]. self assert: vmThread priority > activePriority. self assert: vmThread ~= cogThreadManager currentVMThread. self assertValidExecutionPointe: instructionPointer r: framePointer s: stackPointer. + waitingPriority > vmThread priority ifTrue: + [self reduceWaitingPriorityFrom: waitingPriority to: vmThread priority]. - maxWaitingPriority > vmThread priority ifTrue: - [maxWaitingPriority := vmThread priority]. statProcessSwitch := statProcessSwitch + 1. activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer. objectMemory storePointer: SuspendedContextIndex ofObject: activeProc withValue: activeContext. self ensurePushedInstructionPointer. self externalWriteBackHeadFramePointers. self putToSleep: activeProc yieldingIf: preemptionYields. "Transcript cr; print: #cedeToHighestPriorityThreadIfHigherThan:; cr. self printExternalHeadFrame. self print: 'ip: '; printHex: self instructionPointer. Transcript cr; flush." self returnToSchedulingLoopAndReleaseVMOrWakeThread: vmThread source: CSCheckEvents! Item was changed: ----- Method: CoInterpreterMT>>checkVMOwnershipFromHeartbeat (in category 'process primitive support') ----- checkVMOwnershipFromHeartbeat "Check whether the VM is unowned and needs to set a thread running to try and own it. Do not attempt this if the image doesn't have a threadAffinity inst var in Process; the VM can't thread these images." <inline: false> self sqLowLevelMFence. (processHasThreadAffinity + and: [cogThreadManager doTryLockVMOwnerTo: CTMUnknownOwner]) ifTrue: - and: [cogThreadManager vmIsUnowned]) ifTrue: [cogThreadManager ensureRunningVMThread: relinquishing]! Item was changed: ----- Method: CoInterpreterMT>>disownVM: (in category 'vm scheduling') ----- disownVM: flags "Release the VM to other threads and answer the current thread's index. Currently valid flags: DisownVMForFFICall - informs the VM that it is entering an FFI call DisownVMForThreading - informs the VM that it is entering code during which threading should be permitted OwnVMForeignThreadFlag - indicates lowest-level entry from a foreign thread - not to be used explicitly by clients - only set by ownVMFromUnidentifiedThread VMAlreadyOwnedHenceDoNotDisown - indicates an ownVM from a callback was made when the vm was still owned. - not to be used explicitly by clients - only set by ownVMFromUnidentifiedThread This is the entry-point for plugins and primitives that wish to release the VM while performing some operation that may potentially block, and for callbacks returning back to some blocking operation. If this thread does not reclaim the VM before- hand then when the next heartbeat occurs the thread manager will schedule a thread to acquire the VM which may start running the VM in place of this thread. N.B. Most of the state needed to resume after preemption is set in preemptDisowningThread." <public> <inline: false> + <returnTypeC: #'void *'> + | vmThread activeProc | + self assert: flags >= 0. - | vmThread result | - self assert: (flags >= 0 and: [flags < (1 bitShift: DisownFlagsShift)]). self assert: self successful. + self assert: (cogThreadManager vmOwnerIs: cogThreadManager ioGetThreadLocalThreadIndex). + cogit recordEventTrace ifTrue: [self recordTrace: TraceDisownVM thing: (objectMemory integerObjectOf: flags) source: 0]. processHasThreadAffinity ifFalse: [willNotThreadWarnCount < 10 ifTrue: [self print: 'warning: VM parameter 48 indicates Process doesn''t have threadId; VM will not thread'; cr. willNotThreadWarnCount := willNotThreadWarnCount + 1]]. vmThread := cogThreadManager currentVMThread. (flags anyMask: VMAlreadyOwnedHenceDoNotDisown) ifTrue: [disowningVMThread := vmThread. vmThread setVmThreadState: CTMUnavailable. + ^nil]. - ^0]. self assertCStackPointersBelongToCurrentThread. self assertValidNewMethodPropertyFlags. self cCode: '' inSmalltalk: [cogThreadManager saveRegisterStateForCurrentProcess. cogThreadManager clearRegisterStates.]. (flags anyMask: DisownVMForProcessorRelinquish) ifTrue: [| proc | (proc := objectMemory splObj: foreignCallbackProcessSlot) ~= objectMemory nilObject ifTrue: [foreignCallbackPriority := self quickFetchInteger: PriorityIndex ofObject: proc]. relinquishing := true. self sqLowLevelMFence]. disownCount := disownCount + 1. "If we're disowning the VM because there's no active process to run, there's nothing to preempt later, so don't indicate that there's a disowningVMThread that needs to be restored later." + activeProc := self activeProcess. + activeProc ~= objectMemory nilObject + ifTrue: [disowningVMThread := vmThread. + vmThread priority: (self quickFetchInteger: PriorityIndex ofObject: activeProc).]. - self activeProcess ~= objectMemory nilObject - ifTrue: [disowningVMThread := vmThread]. "OwnVMForeignThreadFlag indicates lowest-level of entry by a foreign thread. If that's where we are then release the vmThread. Otherwise indicate the vmThread is off doing something outside of the VM." (flags anyMask: OwnVMForeignThreadFlag) ifTrue: ["I don't think this is quite right. Josh's use case is creating some foreign thread and then registering it with the VM. That's not the same as binding a process to a foreign thread given that the foreign callback process is about to terminate anyway (it is returning from a callback here). So do we need an additional concept, that of a vmThread being either of the set known to the VM or floating?" self flag: 'issue with registering foreign threads with the VM'. (self isBoundProcess: self activeProcess) ifFalse: [cogThreadManager unregisterVMThread: vmThread]] ifFalse: [vmThread setVmThreadState: CTMUnavailable]. + vmThread disownFlags: (flags bitOr: (activeProcessAffined ifTrue: [0] ifFalse: [ProcessUnaffinedOnDisown])). - result := ((vmThread index bitShift: DisownFlagsShift) - bitOr: (activeProcessAffined ifTrue: [0] ifFalse: [ProcessUnaffinedOnDisown])) - bitOr: flags. cogThreadManager releaseVM. + ^vmThread! - ^result! Item was changed: ----- Method: CoInterpreterMT>>getImageHeaderFlags (in category 'image save/restore') ----- getImageHeaderFlags "Answer the flags that are contained in the 7th long of the image header." ^fullScreenFlag "0 or 1" + (VMBIGENDIAN ifTrue: [0] ifFalse: [2]) "this is the imageFloatsLittleEndian flag" + (processHasThreadAffinity ifTrue: [4] ifFalse: [0]) + (flagInterpretedMethods ifTrue: [8] ifFalse: [0]) + (preemptionYields ifTrue: [0] ifFalse: [16r10]) + "was: noThreadingOfGUIThread ifTrue: [16r20] ifFalse: [0]); a broken idea" - "+ 16r20 -- unassigned" + (newFinalization ifTrue: [16r40] ifFalse: [0]) + (sendWheelEvents ifTrue: [16r80] ifFalse: [0]) + (primitiveDoMixedArithmetic ifTrue: [0] ifFalse: [16r100]) "N.B. flag mask 16r200 is fileTimesInUTC, responded to by the FilePlugin & FileAttributesPlugin" + (upscaleDisplayIfHighDPI ifTrue: [0] ifFalse: [16r400]) + + (imageHeaderFlags bitClear: 16r7FF) "these are any flags we do not recognize"! - + (imageHeaderFlags bitClear: 1+2+4+8+16r10"+16r20"+16r40+16r80+16r100"+16r200"+16r400) "these are any flags we do not recognize"! Item was added: + ----- Method: CoInterpreterMT>>getMaxWaitingPriority (in category 'accessing') ----- + getMaxWaitingPriority + + ^ self atomic_load: (self addressOf: maxWaitingPriority)! Item was changed: ----- Method: CoInterpreterMT>>initialize (in category 'initialization') ----- initialize super initialize. relinquishing := checkThreadActivation := deferThreadSwitch := false. + foreignCallbackPriority := disownCount := willNotThreadWarnCount := 0. + maxWaitingPriority := AtomicValue new.! - foreignCallbackPriority := maxWaitingPriority := disownCount := willNotThreadWarnCount := 0! Item was changed: ----- Method: CoInterpreterMT>>initializeInterpreter: (in category 'initialization') ----- initializeInterpreter: bytesToShift super initializeInterpreter: bytesToShift. foreignCallbackProcessSlot := (objectMemory lengthOf: objectMemory specialObjectsOop) > ForeignCallbackProcess ifTrue: [ForeignCallbackProcess] ifFalse: [NilObject]. + + self atomic_store: (self addressOf: maxWaitingPriority) _: 0.! - ! Item was changed: ----- Method: CoInterpreterMT>>isBoundProcess: (in category 'process primitive support') ----- isBoundProcess: aProcess + ^self isBoundThreadId: (self threadAffinityOfProcess: aProcess)! - ^self isBoundThreadId: (self ownerIndexOfProcess: aProcess)! Item was changed: ----- Method: CoInterpreterMT>>loadInitialContext (in category 'initialization') ----- loadInitialContext | activeProc | super loadInitialContext. activeProc := self activeProcess. + self assert: (self threadAffinityOfProcess: activeProc) = 0. + activeProcessAffined := (self threadAffinityOfProcess: activeProc) ~= 0! - self assert: (self ownerIndexOfProcess: activeProc) = 0. - activeProcessAffined := (self ownerIndexOfProcess: activeProc) ~= 0! Item was changed: ----- Method: CoInterpreterMT>>ownVM: (in category 'vm scheduling') ----- + ownVM: vmThreadHandle - ownVM: threadIndexAndFlags <public> <inline: false> + <var: #vmThreadHandle type: #'void *'> + <var: #vmThread type: #'CogVMThread *'> "This is the entry-point for plugins and primitives that wish to reacquire the VM after having released it via disownVM or callbacks that want to acquire it without knowing their ownership status. This call will block until the VM is owned by the current thread or an error occurs. The argument should be the value answered by disownVM, or 0 for callbacks that don't know if they have disowned or not. This is both an optimization to avoid having to query thread- local storage for the current thread's index (since it can easily keep it in some local variable), and a record of when an unbound process becomes affined to a thread for the dynamic extent of some operation. Answer 0 if the current thread is known to the VM (and on return owns the VM). Answer 1 if the current thread is unknown to the VM and takes ownership. Answer -1 if the current thread is unknown to the VM and fails to take ownership." + | flags vmThread | + vmThread := self cCoerce: vmThreadHandle to: #'CogVMThread *'. + vmThread ifNil: - | threadIndex flags vmThread | - threadIndexAndFlags = 0 ifTrue: [^self ownVMFromUnidentifiedThread]. + flags := vmThread disownFlags. - threadIndex := threadIndexAndFlags bitShift: DisownFlagsShift negated. - flags := threadIndexAndFlags bitAnd: (1 bitShift: DisownFlagsShift) - 1. (flags anyMask: DisownVMForProcessorRelinquish) ifTrue: ["Presumably we have nothing to do; this primitive is typically called from the background process. So we should /not/ try and activate any threads in the pool; they will waste cycles finding there is no runnable process, and will cause a VM abort if no runnable process is found. But we /do/ want to allow FFI calls that have completed, or callbacks a chance to get into the VM; they do have something to do. DisownVMForProcessorRelinquish indicates this." relinquishing := false. self sqLowLevelMFence]. + vmThread := cogThreadManager acquireVMFor: vmThread. - vmThread := cogThreadManager acquireVMFor: threadIndex. disownCount := disownCount - 1. disowningVMThread ifNotNil: [vmThread = disowningVMThread ifTrue: [self assert: (vmThread cFramePointer isNil or: [CFramePointer = vmThread cFramePointer and: [CStackPointer = vmThread cStackPointer]]). self assert: self successful. self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject. disowningVMThread := nil. cogit recordEventTrace ifTrue: [self recordTrace: TraceOwnVM thing: ConstOne source: 0]. ^0]. "if not preempted we're done." self preemptDisowningThread]. "We've been preempted; we must restore state and update the threadId in our process, and may have to put the active process to sleep." + self restoreVMStateFor: vmThread andFlags: flags. - self restoreVMStateFor: vmThread threadIndexAndFlags: threadIndexAndFlags. cogit recordEventTrace ifTrue: [self recordTrace: TraceOwnVM thing: ConstTwo source: 0]. + ^flags bitAnd: OwnVMForeignThreadFlag! - ^threadIndexAndFlags bitAnd: OwnVMForeignThreadFlag! Item was changed: ----- Method: CoInterpreterMT>>ownVMFromUnidentifiedThread (in category 'vm scheduling') ----- ownVMFromUnidentifiedThread "Attempt to take ownership from a thread that as yet doesn't know its index. This supports callbacks where the callback could originate from any thread. Answer 0 if the owning thread is known to the VM. Answer 1 if the owning thread is unknown to the VM and now owns the VM. Answer -1 if the owning thread is unknown to the VM and fails to own the VM. Answer -2 if the owning thread is unknown to the VM and there is no foreign callback process installed." | count threadIndex vmThread | <var: #vmThread type: #'CogVMThread *'> <inline: false> self cCode: [] inSmalltalk: [self halt: 'TODO: Implement processor register switching']. (threadIndex := cogThreadManager ioGetThreadLocalThreadIndex) ~= 0 ifTrue: [ "this is a callback from a known thread" (cogThreadManager vmOwnerIs: threadIndex) ifTrue: "the VM has not been disowned" [self assert: (disowningVMThread isNil or: [disowningVMThread = self currentVMThread]). disowningVMThread := nil. self currentVMThread setVmThreadState: CTMAssignableOrInVM. + ^VMAlreadyOwnedHenceDoNotDisown]]. + + (threadIndex = 0 and: [foreignCallbackPriority = 0]) ifTrue: - ^VMAlreadyOwnedHenceDoNotDisown]. - ^self ownVM: threadIndex]. - foreignCallbackPriority = 0 ifTrue: [^-2]. count := 0. + + "Before we can proceed, we need to temporarily lock the vm, so we can either find our CogVMThread struct or allocate a new one." + cogThreadManager acquireVMForIndex: threadIndex withPriority: foreignCallbackPriority. + + threadIndex ~= 0 + ifTrue: ["this is a callback from a known thread. Simply own the VM for that thread." + vmThread := cogThreadManager vmThreadAt: threadIndex. + ^ self ownVM: vmThread]. + "If the current thread doesn't have an index it's new to the vm and we need to allocate a new threadInfo, failing if we can't. We also need a process in the foreignCallbackProcessSlot upon + which to run the thread's eventual callback." + [(objectMemory splObj: foreignCallbackProcessSlot) = objectMemory nilObject] whileTrue: - which to run the thread's eventual callback." - [[cogThreadManager tryLockVMOwnerTo: cogThreadManager ioCurrentOSThread asUnsignedInteger] whileFalse: - [self waitingPriorityIsAtLeast: foreignCallbackPriority. - cogThreadManager ioTransferTimeslice]. - (objectMemory splObj: foreignCallbackProcessSlot) ~= objectMemory nilObject] whileFalse: [cogThreadManager releaseVM. (count := count + 1) > 1000 ifTrue: [^-2]. + cogThreadManager ioMilliSleep: 1. + cogThreadManager acquireVMForIndex: threadIndex withPriority: foreignCallbackPriority]. - cogThreadManager ioMilliSleep: 1]. vmThread := cogThreadManager unusedThreadInfo. "N.B. Keep the VM locked anonymously so that we reserve the non-nil ForeignCallbackProcess for this thread, avoiding the race between competing foreign callbacks. The acquireVMFor: in ownVM: will set the vmOwner to the actual index. So only unlock on failure." vmThread ifNil: [cogThreadManager releaseVM. ^-1]. cogThreadManager setVMOwner: vmThread index. vmThread + priority: foreignCallbackPriority; + disownFlags: OwnVMForeignThreadFlag; + setVmThreadState: CTMWantingOwnership. - setVmThreadState: CTMWantingOwnership; - priority: foreignCallbackPriority. cogThreadManager registerVMThread: vmThread. + ^self ownVM: vmThread! - ^self ownVM: vmThread index + OwnVMForeignThreadFlag! Item was removed: - ----- Method: CoInterpreterMT>>ownerIndexOfProcess: (in category 'process primitive support') ----- - ownerIndexOfProcess: aProcess - ^self ownerIndexOfThreadId: (self threadAffinityFieldOf: aProcess)! Item was changed: ----- Method: CoInterpreterMT>>preemptDisowningThread (in category 'vm scheduling') ----- preemptDisowningThread "Set the relevant state for disowningVMThread so that it can resume after being preempted and set disowningVMThread to nil to indicate preemption. N.B. This should only be sent from checkPreemptionOfDisowningThread. There are essentially four things to do. a) save the VM's notion of the current C stack pointers; these are pointers into a thread's stack and must be saved and restored in thread switch. b) save the VM's notion of the current Smalltalk execution point. This is simply the suspend half of a process switch that saves the current context in the current process. c) add the process to the thread's set of AWOL processes so that the scheduler won't try to run the process while the thread has disowned the VM. d) save the in-primitive VM state, newMethod and argumentCount ownVM: will restore the VM context as of disownVM: from the above when it finds it has been preempted." | activeProc activeContext preemptedThread | <var: #preemptedThread type: #'CogVMThread *'> <inline: false> self assert: disowningVMThread notNil. self assert: (disowningVMThread vmThreadState = CTMUnavailable or: [disowningVMThread vmThreadState = CTMWantingOwnership]). self assertCStackPointersBelongToDisowningThread. cogit recordEventTrace ifTrue: [self recordTrace: TracePreemptDisowningThread thing: (objectMemory integerObjectOf: disowningVMThread index) source: 0]. disowningVMThread cStackPointer: CStackPointer. disowningVMThread cFramePointer: CFramePointer. activeProc := self activeProcess. self assert: (objectMemory fetchPointer: MyListIndex ofObject: activeProc) = objectMemory nilObject. objectMemory storePointer: MyListIndex ofObject: activeProc withValue: (objectMemory splObj: ProcessInExternalCodeTag). activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer. objectMemory storePointer: SuspendedContextIndex ofObject: activeProc withValue: activeContext. "The instructionPointer must be pushed because the convention for inactive stack pages is that the instructionPointer is top of stack. We need to know if this primitive is called from machine code because the invariant that the return pc of an interpreter callee calling a machine code caller is ceReturnToInterpreterPC must be maintained." self push: instructionPointer. self externalWriteBackHeadFramePointers. "Since pushing the awol process may realloc disowningVMThread we need to reassign. But since we're going to nil disowningVMThread anyway we can assign to a local." preemptedThread := cogThreadManager pushAWOLProcess: activeProc on: disowningVMThread. disowningVMThread := nil. + (self threadAffinityOfProcess: activeProc) = 0 ifTrue: + [self setTemporaryThreadAffinityOfProcess: activeProc to: preemptedThread index bind: false]. - preemptedThread priority: (self quickFetchInteger: PriorityIndex ofObject: activeProc). - (self ownerIndexOfProcess: activeProc) = 0 ifTrue: - [self setOwnerIndexOfProcess: activeProc to: preemptedThread index bind: false]. preemptedThread newMethodOrNull: newMethod; argumentCount: argumentCount; inMachineCode: instructionPointer <= objectMemory startOfMemory! Item was changed: ----- Method: CoInterpreterMT>>primitiveProcessBindToThreadId (in category 'process primitives') ----- primitiveProcessBindToThreadId "Attempt to bind the receiver to the thread with the id of the argument or nil, where the receiver is a Process. If successful the VM will ensure that there are at least id many threads active." + | aProcess id waitingPriority activePriority | - | aProcess id | <export: true> self cCode: [] inSmalltalk: [cogThreadManager isNil ifTrue: [^self primitiveFail]]. processHasThreadAffinity ifFalse: [^self primitiveFailFor: PrimErrUnsupported]. id := self stackTop. aProcess := self stackValue: 1. ((id = objectMemory nilObject or: [(objectMemory isIntegerObject: id) and: [id ~= (objectMemory integerObjectOf: 0)]]) and: [(objectMemory isPointers: aProcess) and: [(objectMemory slotSizeOf: aProcess) >= (ThreadIdIndex + 1)]]) ifFalse: [^self primitiveFailFor: PrimErrBadArgument]. id := id = objectMemory nilObject ifTrue: [0] ifFalse: [objectMemory integerValueOf: id]. id abs >= cogThreadManager maxNumThreads ifTrue: [^self primitiveFailFor: PrimErrLimitExceeded]. (self bindProcess: aProcess toId: id) ifNotNil: [:ec| ^self primitiveFailFor: ec]. self methodReturnReceiver. + waitingPriority := self getMaxWaitingPriority. + activePriority := self quickFetchInteger: PriorityIndex ofObject: aProcess. + id := self threadAffinityOfProcess: aProcess. - id := self ownerIndexOfProcess: aProcess. (aProcess = self activeProcess and: [(activeProcessAffined := id ~= 0) and: [(cogThreadManager vmOwnerIsCompatibleWith: id) not]]) ifTrue: + [activePriority < waitingPriority ifTrue: + [self reduceWaitingPriorityFrom: waitingPriority to: activePriority "TODO: Check if this is correct?"]. - [(self quickFetchInteger: PriorityIndex ofObject: aProcess) < maxWaitingPriority ifTrue: - [maxWaitingPriority = self quickFetchInteger: PriorityIndex ofObject: aProcess]. self threadSwitchIfNecessary: aProcess from: CSThreadBind]! Item was changed: ----- Method: CoInterpreterMT>>primitiveProcessBoundThreadId (in category 'process primitives') ----- primitiveProcessBoundThreadId "Answer the receiver's current threadAffinity or nil, where the receiver is a Process. If the threadAffinity is positive then the receiver is bound to the thread with that id. If the threadAffinity is negative then the receiver is excluded from running on the thread with that id." | aProcess id | <export: true> self cCode: [] inSmalltalk: [cogThreadManager isNil ifTrue: [^self primitiveFail]]. processHasThreadAffinity ifFalse: [^self primitiveFailFor: PrimErrUnsupported]. aProcess := self stackTop. + id := self threadAffinityOfProcess: aProcess. - id := self ownerIndexOfProcess: aProcess. self methodReturnValue: (id = 0 ifTrue: [objectMemory nilObject] ifFalse: [objectMemory integerObjectOf: id])! Item was changed: ----- Method: CoInterpreterMT>>primitiveRelinquishProcessor (in category 'I/O primitives') ----- primitiveRelinquishProcessor "Relinquish the processor for up to the given number of microseconds. The exact behavior of this primitive is platform dependent. Override to check for waiting threads." + | microSecs vmHandle currentCStackPointer currentCFramePointer | - | microSecs threadIndexAndFlags currentCStackPointer currentCFramePointer | <var: #currentCStackPointer type: #'volatile usqIntptr_t'> <var: #currentCFramePointer type: #'volatile usqIntptr_t'> microSecs := self stackTop. (objectMemory isIntegerObject: microSecs) ifFalse: [^self primitiveFail]. self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject. self assert: relinquishing not. "DO NOT allow relinquishing the processor while we are profiling since this may skew the time base for our measures (it may reduce processor speed etc). Instead we go full speed, therefore measuring the precise time we spend in the inner idle loop as a busy loop." nextProfileTick = 0 ifTrue: "Presumably we have nothing to do; this primitive is typically called from the background process. So we should /not/ try and activate any threads in the pool; they will waste cycles finding there is no runnable process, and will cause a VM abort if no runnable process is found. But we /do/ want to allow FFI calls that have completed, or callbacks a chance to get into the VM; they do have something to do. DisownVMForProcessorRelinquish indicates this." [currentCStackPointer := CStackPointer. currentCFramePointer := CFramePointer. + vmHandle := self disownVM: DisownVMForProcessorRelinquish. - threadIndexAndFlags := self disownVM: DisownVMForProcessorRelinquish. self assert: relinquishing. self ioRelinquishProcessorForMicroseconds: (objectMemory integerValueOf: microSecs). self assert: relinquishing. + self ownVM: vmHandle. - self ownVM: threadIndexAndFlags. self assert: relinquishing not. self assert: cogThreadManager currentVMThread vmThreadState = CTMAssignableOrInVM. self assert: currentCStackPointer = CStackPointer. self assert: currentCFramePointer = CFramePointer. "In simulation we allow ioRelinquishProcessorForMicroseconds: to fail so that we can arrange that the simulator responds to input events promptly. This *DOES NOT HAPPEN* in the real vm." self cCode: [] inSmalltalk: [primFailCode ~= 0 ifTrue: [^self]]]. self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject. self pop: 1 "microSecs; leave rcvr on stack"! Item was added: + ----- Method: CoInterpreterMT>>reduceWaitingPriorityFrom:to: (in category 'accessing') ----- + reduceWaitingPriorityFrom: existingWaitingPriority to: newMaxPriority + <var: #existing type: #int> + | existing | + self cCode: [existing := existingWaitingPriority] + inSmalltalk: [existing := AtomicValue new. + existing value: existingWaitingPriority.]. + "This CPXCHG may fail, that's fine though, as there may have been + another thread that increased the priority in the meantime. + In that case that threads priority is the correct one to use." + ^ self atomic: (self addressOf: maxWaitingPriority) + _compare: (self addressOf: existing) + _exchange_strong: newMaxPriority! Item was added: + ----- Method: CoInterpreterMT>>restoreVMStateFor:andFlags: (in category 'vm scheduling') ----- + restoreVMStateFor: vmThread andFlags: flags + "We've been preempted; we must restore state and update the threadId + in our process, and may have to put the active process to sleep." + | sched activeProc myProc | + sched := self schedulerPointer. + activeProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched. + (flags anyMask: OwnVMForeignThreadFlag) + ifTrue: + [self assert: foreignCallbackProcessSlot == ForeignCallbackProcess. + myProc := objectMemory splObj: foreignCallbackProcessSlot. + self assert: myProc ~= objectMemory nilObject. + objectMemory splObj: foreignCallbackProcessSlot put: objectMemory nilObject] + ifFalse: [myProc := cogThreadManager popAWOLProcess: vmThread]. + self assert: activeProc ~= myProc. + (activeProc ~= objectMemory nilObject + and: [(objectMemory fetchPointer: MyListIndex ofObject: activeProc) = objectMemory nilObject]) ifTrue: + [self putToSleep: activeProc yieldingIf: preemptionYields]. + self assert: (objectMemory fetchPointer: MyListIndex ofObject: myProc) = (objectMemory splObj: ProcessInExternalCodeTag). + objectMemory + storePointer: ActiveProcessIndex ofObject: sched withValue: myProc; + storePointerUnchecked: MyListIndex ofObject: myProc withValue: objectMemory nilObject. + "Only unaffine if the process was affined at this level and did not become bound in the interim." + ((flags anyMask: ProcessUnaffinedOnDisown) + and: [(self isBoundProcess: myProc) not]) ifTrue: + [self setTemporaryThreadAffinityOfProcess: myProc to: 0 bind: false]. + self initPrimCall. + self cCode: + [self externalSetStackPageAndPointersForSuspendedContextOfProcess: myProc] + inSmalltalk: + ["Bypass the no-offset stack depth check in the simulator's externalSetStackPageAndPointersForSuspendedContextOfProcess:" + super externalSetStackPageAndPointersForSuspendedContextOfProcess: myProc. + "We're in ownVM:, hence in a primitive, hence need to include the argument count" + (self isMachineCodeFrame: framePointer) ifTrue: + [self maybeCheckStackDepth: vmThread argumentCount + sp: stackPointer + pc: instructionPointer]]. + "If this primitive is called from machine code maintain the invariant that the return pc + of an interpreter callee calling a machine code caller is ceReturnToInterpreterPC." + (vmThread inMachineCode + and: [instructionPointer >= objectMemory startOfMemory]) ifTrue: + [self iframeSavedIP: framePointer put: instructionPointer. + instructionPointer := cogit ceReturnToInterpreterPC]. + newMethod := vmThread newMethodOrNull. + argumentCount := vmThread argumentCount. + vmThread newMethodOrNull: nil. + self cCode: '' inSmalltalk: + [| range | + range := self cStackRangeForThreadIndex: vmThread index. + self assert: ((range includes: vmThread cStackPointer) and: [range includes: vmThread cFramePointer])]. + self setCFramePointer: vmThread cFramePointer setCStackPointer: vmThread cStackPointer. + self assert: newMethod notNil + ! Item was removed: - ----- Method: CoInterpreterMT>>restoreVMStateFor:threadIndexAndFlags: (in category 'vm scheduling') ----- - restoreVMStateFor: vmThread threadIndexAndFlags: threadIndexAndFlags - "We've been preempted; we must restore state and update the threadId - in our process, and may have to put the active process to sleep." - | sched activeProc myProc | - sched := self schedulerPointer. - activeProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched. - (threadIndexAndFlags anyMask: OwnVMForeignThreadFlag) - ifTrue: - [self assert: foreignCallbackProcessSlot == ForeignCallbackProcess. - myProc := objectMemory splObj: foreignCallbackProcessSlot. - self assert: myProc ~= objectMemory nilObject. - objectMemory splObj: foreignCallbackProcessSlot put: objectMemory nilObject] - ifFalse: [myProc := cogThreadManager popAWOLProcess: vmThread]. - self assert: activeProc ~= myProc. - (activeProc ~= objectMemory nilObject - and: [(objectMemory fetchPointer: MyListIndex ofObject: activeProc) = objectMemory nilObject]) ifTrue: - [self putToSleep: activeProc yieldingIf: preemptionYields]. - self assert: (objectMemory fetchPointer: MyListIndex ofObject: myProc) = (objectMemory splObj: ProcessInExternalCodeTag). - objectMemory - storePointer: ActiveProcessIndex ofObject: sched withValue: myProc; - storePointerUnchecked: MyListIndex ofObject: myProc withValue: objectMemory nilObject. - "Only unaffine if the process was affined at this level and did not become bound in the interim." - ((threadIndexAndFlags anyMask: ProcessUnaffinedOnDisown) - and: [(self isBoundProcess: myProc) not]) ifTrue: - [self setOwnerIndexOfProcess: myProc to: 0 bind: false]. - self initPrimCall. - self cCode: - [self externalSetStackPageAndPointersForSuspendedContextOfProcess: myProc] - inSmalltalk: - ["Bypass the no-offset stack depth check in the simulator's externalSetStackPageAndPointersForSuspendedContextOfProcess:" - super externalSetStackPageAndPointersForSuspendedContextOfProcess: myProc. - "We're in ownVM:, hence in a primitive, hence need to include the argument count" - (self isMachineCodeFrame: framePointer) ifTrue: - [self maybeCheckStackDepth: vmThread argumentCount - sp: stackPointer - pc: instructionPointer]]. - "If this primitive is called from machine code maintain the invariant that the return pc - of an interpreter callee calling a machine code caller is ceReturnToInterpreterPC." - (vmThread inMachineCode - and: [instructionPointer >= objectMemory startOfMemory]) ifTrue: - [self iframeSavedIP: framePointer put: instructionPointer. - instructionPointer := cogit ceReturnToInterpreterPC]. - newMethod := vmThread newMethodOrNull. - argumentCount := vmThread argumentCount. - vmThread newMethodOrNull: nil. - self cCode: '' inSmalltalk: - [| range | - range := self cStackRangeForThreadIndex: vmThread index. - self assert: ((range includes: vmThread cStackPointer) and: [range includes: vmThread cFramePointer])]. - self setCFramePointer: vmThread cFramePointer setCStackPointer: vmThread cStackPointer. - self assert: newMethod notNil - ! Item was changed: ----- Method: CoInterpreterMT>>returnToSchedulingLoopAndReleaseVMOrWakeThread:source: (in category 'process primitive support') ----- returnToSchedulingLoopAndReleaseVMOrWakeThread: vmThread source: source <var: #vmThread type: #'CogVMThread *'> <inline: false> | activeThread | activeThread := cogThreadManager currentVMThread. self recordThreadSwitchTo: (vmThread ifNotNil: [vmThread index] ifNil: [0]) source: source. vmThread + ifNotNil: [cogThreadManager wakeVMThread: vmThread] + ifNil: [cogit releaseVM "TODO: Do we need to saveRegisterStateForCurrentProcess here?"]. - ifNotNil: [cogThreadManager wakeVMThreadFor: vmThread index] - ifNil: [cogit disownVM: DisownVMForThreading]. "I am not frightened of flying. Any value will do. I don't mind. Why should I be frightened of flying? There's no reason for it." self _longjmp: activeThread reenterThreadSchedulingLoop _: 1 ! Item was changed: ----- Method: CoInterpreterMT>>setImageHeaderFlags: (in category 'internal interpreter access') ----- setImageHeaderFlags: flags "Set an array of flags indicating various properties of the saved image, responded to on image load. These are the same as the image header flags shifted right two bits, omitting the fullScreenFlag and float byte order flag. Bit 0: if set, implies the image's Process class has threadAffinity as its 3rd inst var (zero relative) Bit 1: if set, methods that are interpreted will have the flag bit set in their header Bit 2: if set, implies preempting a process does not put it to the back of its run queue + Bit 3: unassigned; used to mean if set, implies a threaded VM will not disown the VM if owned by the GUI thread; a broken idea - Bit 3: (unassigned) Bit 4: if set, implies the new finalization scheme where WeakArrays are queued Bit 5: if set, implies wheel events will be delivered as such and not mapped to arrow key events Bit 6: if set, implies arithmetic primitives will fail if given arguments of different types (float vs int) Bit 7: if set, implies file primitives (FilePlugin, FileAttributesPlugin) will answer file times in UTC not local times Bit 8: if set, implies the VM will not upscale the display on high DPI monitors; older VMs did this by default." flags asUnsignedInteger > 511 ifTrue: [^self primitiveFailFor: PrimErrUnsupported]. (flags anyMask: 8) ifTrue: [^self primitiveFailFor: PrimErrInappropriate]. processHasThreadAffinity := flags anyMask: 1. flagInterpretedMethods := flags anyMask: 2. preemptionYields := flags noMask: 4. "was: noThreadingOfGUIThread := flags anyMask: 8. a broken idea" - imageHeaderFlags := (flags anyMask: 8) - ifTrue: [imageHeaderFlags bitOr: 32] - ifFalse: [imageHeaderFlags bitClear: 32]. newFinalization := flags anyMask: 16. sendWheelEvents := flags anyMask: 32. primitiveDoMixedArithmetic := flags noMask: 64. - "fileTimesInUTC := flags anyMask: 128. see FilePlugin & FileAttributesPlugin" imageHeaderFlags := (flags anyMask: 128) ifTrue: [imageHeaderFlags bitOr: 512] ifFalse: [imageHeaderFlags bitClear: 512]. upscaleDisplayIfHighDPI := flags noMask: 256! Item was removed: - ----- Method: CoInterpreterMT>>setMaxWaitingPriorityTo: (in category 'process primitive support') ----- - setMaxWaitingPriorityTo: minPriority - maxWaitingPriority := minPriority! Item was removed: - ----- Method: CoInterpreterMT>>setOwnerIndexOfProcess:to:bind: (in category 'process primitive support') ----- - setOwnerIndexOfProcess: aProcess to: anIndex bind: bind - | threadId | - threadId := anIndex = 0 - ifTrue: [objectMemory nilObject] - ifFalse: [objectMemory integerObjectOf: (anIndex << 1) + (bind ifTrue: [1] ifFalse: [0])]. - objectMemory storePointerUnchecked: ThreadIdIndex ofObject: aProcess withValue: threadId! Item was added: + ----- Method: CoInterpreterMT>>setTemporaryThreadAffinityOfProcess:to:bind: (in category 'process primitive support') ----- + setTemporaryThreadAffinityOfProcess: aProcess to: anIndex bind: bind + | threadId | + threadId := anIndex = 0 + ifTrue: [objectMemory nilObject] + ifFalse: [objectMemory integerObjectOf: (anIndex << 1) + (bind ifTrue: [1] ifFalse: [0])]. + objectMemory storePointerUnchecked: ThreadIdIndex ofObject: aProcess withValue: threadId! Item was added: + ----- Method: CoInterpreterMT>>setThreadAffinityOfProcess:to:bind: (in category 'process primitive support') ----- + setThreadAffinityOfProcess: aProcess to: anIndex bind: bind + | threadId | + threadId := anIndex = 0 + ifTrue: [objectMemory nilObject] + ifFalse: [objectMemory integerObjectOf: (anIndex << 1) + (bind ifTrue: [1] ifFalse: [0])]. + objectMemory storePointerUnchecked: ThreadIdIndex ofObject: aProcess withValue: threadId! Item was changed: ----- Method: CoInterpreterMT>>threadAffinityFieldValueOf: (in category 'process primitive support') ----- threadAffinityFieldValueOf: aProcess ^processHasThreadAffinity ifTrue: [| field | field := objectMemory fetchPointer: ThreadIdIndex ofObject: aProcess. + field = objectMemory nilObject + ifTrue: [0] + ifFalse: [objectMemory integerValueOf: field]] - (objectMemory isIntegerObject: field) - ifTrue: [objectMemory integerValueOf: field] - ifFalse: [0]] ifFalse: [0]! Item was added: + ----- Method: CoInterpreterMT>>threadAffinityOfProcess: (in category 'process primitive support') ----- + threadAffinityOfProcess: aProcess + ^self ownerIndexOfThreadId: (self threadAffinityFieldOf: aProcess)! Item was changed: ----- Method: CoInterpreterMT>>threadSchedulingLoopImplementation: (in category 'vm scheduling') ----- threadSchedulingLoopImplementation: vmThread "Enter a loop attempting to run the VM with the highest priority process and blocking on the thread's OS semaphore when unable to run that process. We will return to this via threadSwitchIfNecessary:from: which is called in the middle of transferTo:from: once the active process has been stored in the scheduler." <var: #vmThread type: #'CogVMThread *'> <inline: false> self _setjmp: vmThread reenterThreadSchedulingLoop. [self assert: vmThread vmThreadState = CTMAssignableOrInVM. (cogThreadManager tryLockVMOwnerTo: vmThread index) ifTrue: ["Yay, we're the VM owner!!" "If relinquishing is true, then primitiveRelinquishProcessor has disowned the VM and only a returning call or callback should take ownership in that case." relinquishing ifFalse: [self tryToExecuteSmalltalk: vmThread]. + "tryToExecuteSmalltalk: may return if there's no runnable process. + Usually it doesn't return, but jumps straight back to the _setjmp at the top of this function, + so this is only reached in case there's no runnable process." + "TODO: Do we need to saveRegisterStateForCurrentProcess here?" + self releaseVM]. - self disownVM: DisownVMForThreading.]. cogThreadManager waitForWork: vmThread. true] whileTrue! Item was changed: ----- Method: CoInterpreterMT>>threadSwitchIfNecessary:from: (in category 'process primitive support') ----- threadSwitchIfNecessary: newProc from: sourceCode "Invoked from transferTo:from: or primitiveProcessBindToThreadId to switch threads if the new process is bound or affined to some other thread." + | newProcThreadAffinity vmThread activeContext | - | newProcOwnerIndex vmThread activeContext | self assert: (cogThreadManager vmOwnerIs: cogThreadManager ioGetThreadLocalThreadIndex). deferThreadSwitch ifTrue: [^self]. cogThreadManager assertValidProcessorStackPointersForIndex: cogThreadManager getVMOwner. "If the current process is unaffined or it is affined to the current thread we're ok to run, but we should yield asap if a higher-priority thread wants the VM." + newProcThreadAffinity := self threadAffinityOfProcess: newProc. + ((activeProcessAffined := newProcThreadAffinity ~= 0) + and: [(cogThreadManager vmOwnerIsCompatibleWith: newProcThreadAffinity) not]) ifFalse: + [(self quickFetchInteger: PriorityIndex ofObject: newProc) < self getMaxWaitingPriority ifTrue: - newProcOwnerIndex := self ownerIndexOfProcess: newProc. - ((activeProcessAffined := newProcOwnerIndex ~= 0) - and: [(cogThreadManager vmOwnerIsCompatibleWith: newProcOwnerIndex) not]) ifFalse: - [(self quickFetchInteger: PriorityIndex ofObject: newProc) < maxWaitingPriority ifTrue: [checkThreadActivation := true. self forceInterruptCheck]. ^self]. "The current process is affined to a thread, but not to the current owner. So switch to that owner." self cCode: '' inSmalltalk: [transcript ensureCr; f: 'threadSwitchIfNecessary: %08x from: %s(%d) owner %d -> %d\n' + printf: { newProc. TraceSources at: sourceCode. sourceCode. cogThreadManager getVMOwner. newProcThreadAffinity }]. - printf: { newProc. TraceSources at: sourceCode. sourceCode. cogThreadManager getVMOwner. newProcOwnerIndex }]. "We at least need to externalize the stack pointers to enable a thread switch..." (objectMemory fetchPointer: SuspendedContextIndex ofObject: newProc) = objectMemory nilObject ifTrue: [self assert: newProc = self activeProcess. self push: instructionPointer. self externalWriteBackHeadFramePointers. false ifTrue: "If the activeProcess doesn't have a context yet, it needs one from which the new thread can resume execution." [activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer. objectMemory storePointer: SuspendedContextIndex ofObject: newProc withValue: activeContext]]. + newProcThreadAffinity < 0 - newProcOwnerIndex < 0 ifTrue: + [self assert: newProcThreadAffinity negated = cogThreadManager getVMOwner. - [self assert: newProcOwnerIndex negated = cogThreadManager getVMOwner. vmThread := cogThreadManager ensureWillingThread. self deny: vmThread index = cogThreadManager getVMOwner. + self assert: (cogThreadManager threadIndex: vmThread index isCompatibleWith: newProcThreadAffinity)] - self assert: (cogThreadManager threadIndex: vmThread index isCompatibleWith: newProcOwnerIndex)] ifFalse: + [vmThread := cogThreadManager vmThreadAt: newProcThreadAffinity. - [vmThread := cogThreadManager vmThreadAt: newProcOwnerIndex. vmThread priority: (self quickFetchInteger: PriorityIndex ofObject: newProc). vmThread vmThreadState = CTMUnavailable ifTrue: [vmThread setVmThreadState: CTMWantingOwnership]]. self returnToSchedulingLoopAndReleaseVMOrWakeThread: vmThread source: CSSwitchIfNeccessary! Item was changed: ----- Method: CoInterpreterMT>>transferTo:from: (in category 'process primitive support') ----- transferTo: newProc from: sourceCode "Record a process to be awoken on the next interpreter cycle. Override to potentially switch threads either if the new process is bound to another thread, or if there is no runnable process but there is a waiting thread. Note that the abort on no runnable process has beeen moved here from wakeHighestPriority." | sched oldProc activeContext | <inline: false> statProcessSwitch := statProcessSwitch + 1. self push: instructionPointer. self externalWriteBackHeadFramePointers. self assertValidExecutionPointe: instructionPointer r: framePointer s: stackPointer. "ensureMethodIsCogged: in makeBaseFrameFor: in externalSetStackPageAndPointersForSuspendedContextOfProcess: below may do a code compaction. Nil instructionPointer to avoid it getting pushed twice." instructionPointer := 0. sched := self schedulerPointer. oldProc := objectMemory fetchPointer: ActiveProcessIndex ofObject: sched. self recordContextSwitchFrom: oldProc in: sourceCode. activeContext := self ensureFrameIsMarried: framePointer SP: stackPointer + objectMemory wordSize. objectMemory storePointer: SuspendedContextIndex ofObject: oldProc withValue: activeContext. newProc ifNil: ["Two possibilities. One, there is at least one thread waiting to own the VM in which case it should be activated. Two, there are no processes to run and so abort." cogThreadManager willingVMThread ifNotNil: [:vmThread| vmThread vmThreadState = CTMWantingOwnership ifTrue: [self returnToSchedulingLoopAndReleaseVMOrWakeThread: vmThread source: sourceCode]]. + "self error: 'scheduler could not find a runnable process'" + self returnToSchedulingLoopAndReleaseVMOrWakeThread: nil source: sourceCode]. - self error: 'scheduler could not find a runnable process']. "Switch to the new process" objectMemory storePointer: ActiveProcessIndex ofObject: sched withValue: newProc; storePointerUnchecked: MyListIndex ofObject: newProc withValue: objectMemory nilObject. self externalSetStackPageAndPointersForSuspendedContextOfProcess: newProc. "Finally thread switch if required" self threadSwitchIfNecessary: newProc from: sourceCode! Item was changed: ----- Method: CoInterpreterMT>>tryToExecuteSmalltalk: (in category 'vm scheduling') ----- tryToExecuteSmalltalk: vmThread "Attempt to run the current process, if it exists, on the given vmThread." <var: #vmThread type: #'CogVMThread *'> + | activeProc threadAffinity | - | dvmt activeProc ownerIndex | - <var: #dvmt type: #'CogVMThread *'> self assert: (cogThreadManager vmOwnerIs: vmThread index). self assert: cogThreadManager ioGetThreadLocalThreadIndex = vmThread index. + - dvmt := disowningVMThread. disowningVMThread ifNil: [activeProc := self activeProcess] ifNotNil: [self preemptDisowningThread. activeProc := self wakeHighestPriority. activeProc ifNil: [activeProc := objectMemory nilObject] ifNotNil: [objectMemory storePointerUnchecked: MyListIndex ofObject: activeProc withValue: objectMemory nilObject]. objectMemory storePointer: ActiveProcessIndex ofObject: self schedulerPointer withValue: activeProc]. activeProc = objectMemory nilObject ifTrue:[^nil]. + threadAffinity := self threadAffinityOfProcess: activeProc. + (cogThreadManager vmOwnerIsCompatibleWith: threadAffinity) ifTrue: - ownerIndex := self ownerIndexOfProcess: activeProc. - (ownerIndex = 0 or: [cogThreadManager vmOwnerIsCompatibleWith: ownerIndex]) ifTrue: [self assert: (objectMemory fetchPointer: MyListIndex ofObject: self activeProcess) = objectMemory nilObject. (objectMemory fetchPointer: SuspendedContextIndex ofObject: activeProc) ~= objectMemory nilObject ifTrue: [self externalSetStackPageAndPointersForSuspendedContextOfProcess: activeProc]. instructionPointer = cogit ceReturnToInterpreterPC ifTrue: [self deny: (self isMachineCodeFrame: framePointer). instructionPointer := self iframeSavedIP: framePointer]. self enterSmalltalkExecutive. "When we return here we should have already given up the VM and so we cannot touch any interpreter state." + self error: 'NOTREACHED'.]. + cogThreadManager returnToSchedulingLoopAndWakeThreadFor: threadAffinity source: CSTryToExecuteSmalltalk. + "This is only reached if the above call has failed, then ownership has not been transferred and we still need to release the VM."! - "NOTREACHED"]. - cogThreadManager wakeVMThreadFor: ownerIndex! Item was changed: + ----- Method: CoInterpreterMT>>waitingPriorityIsAtLeast: (in category 'accessing') ----- - ----- Method: CoInterpreterMT>>waitingPriorityIsAtLeast: (in category 'process primitive support') ----- waitingPriorityIsAtLeast: minPriority "Set the maxWaitingPriority to at least minPriority on behalf of a thread wanting to acquire the VM. If maxWaitingPriority is increased, schedule a thread activation check asap." + <var: #currentWaitingPriority type: #int> + | currentWaitingPriority didIncrease | + self cCode: [currentWaitingPriority := self getMaxWaitingPriority.] + inSmalltalk: [currentWaitingPriority := AtomicValue new. + currentWaitingPriority value: self getMaxWaitingPriority]. + + didIncrease := false. + [(self cCode: [currentWaitingPriority] inSmalltalk: [currentWaitingPriority value]) >= minPriority + or: [didIncrease := self atomic: (self addressOf: maxWaitingPriority) + _compare: (self addressOf: currentWaitingPriority) + _exchange_strong: minPriority]] whileFalse. + didIncrease + ifTrue: [ self assert: (self cCode: [currentWaitingPriority] inSmalltalk: [currentWaitingPriority value]) < minPriority. + checkThreadActivation := true. + self forceInterruptCheck]! - maxWaitingPriority < minPriority ifTrue: - [maxWaitingPriority := minPriority. - checkThreadActivation := true. - self forceInterruptCheck]! Item was changed: ----- Method: CogThreadManager class>>declareCVarsIn: (in category 'translation') ----- declareCVarsIn: cCodeGen cCodeGen removeVariable: 'coInterpreter'; removeVariable: 'cogit'; removeVariable: 'threadLocalStorage'; removeVariable: 'processorOwner'; removeVariable: 'registerStates'. cCodeGen var: #threads type: #'CogVMThread **'; var: #vmOSThread type: #sqOSThread; + var: #vmOwner type: #'volatile atomic_int'; + var: #maxWaitingPriority type: #'volatile atomic_int'. - var: #vmOwner type: #'volatile atomic_int'. cCodeGen addHeaderFile: '<stdatomic.h>'! Item was changed: ----- Method: CogThreadManager class>>initialize (in category 'class initialization') ----- initialize "CogThreadManager initialize" CTMUninitialized := 0. CTMInitializing := 1. CTMUnavailable := 2. "off doing its own thing and not available to run the VM, e.g. calling-out." CTMAssignableOrInVM := 3. "either owning the VM or blocked on its osSemaphore available for VM work" CTMWantingOwnership := 4. "with something specific to do in the VM (e.g. return a result from a call-out, make a call-back)" + + CTMUnknownOwner := -1. "Define the size of the stack of processes at time of disown/own." AWOLProcessesIncrement := 4! Item was changed: ----- Method: CogThreadManager>>acquireVMFor: (in category 'public api') ----- + acquireVMFor: vmThread - acquireVMFor: threadIndex "Attempt to acquire the VM, eventually blocking until it becomes available. Spin until the maxWaitingPriority has been updated if it is lower than this thread's priority." <returnTypeC: #'CogVMThread *'> - | vmThread | <var: #vmThread type: #'CogVMThread *'> + self assert: vmThread index = self ioGetThreadLocalThreadIndex. - self assert: threadIndex = self ioGetThreadLocalThreadIndex. - vmThread := self vmThreadAt: threadIndex. self assert: (vmThread vmThreadState = CTMUnavailable or: [vmThread vmThreadState = CTMWantingOwnership]). + (self tryLockVMOwnerTo: vmThread index) - (self tryLockVMOwnerTo: threadIndex) ifTrue: [vmThread setVmThreadState: CTMAssignableOrInVM] ifFalse: [vmThread setVmThreadState: CTMWantingOwnership. + [self tryLockVMOwnerTo: vmThread index] whileFalse: - [(self vmOwnerIs: threadIndex) - or: [self tryLockVMOwnerTo: threadIndex]] whileFalse: [vmThread priority ifNotNil: [coInterpreter waitingPriorityIsAtLeast: vmThread priority]. + (self vmOwnerIs: vmThread index) ifFalse: - (self vmOwnerIs: threadIndex) ifFalse: [self ioWaitOnOSSemaphore: (self addressOf: vmThread osSemaphore)]]]. coInterpreter assertProcessorStackPointersBelongToCurrentThread. vmOSThread := vmThread osThread. vmThread setVmThreadState: CTMAssignableOrInVM. ^vmThread! Item was added: + ----- Method: CogThreadManager>>acquireVMForIndex:withPriority: (in category 'public api') ----- + acquireVMForIndex: threadIndex withPriority: aPriority + + [self tryLockVMOwnerTo: (threadIndex > 0 ifTrue: [threadIndex] ifFalse: [CTMUnknownOwner])] whileFalse: + [self waitingPriorityIsAtLeast: aPriority. + self ioTransferTimeslice].! Item was changed: ----- Method: CogThreadManager>>assertValidProcessorStackPointersForIndex: (in category 'simulation') ----- assertValidProcessorStackPointersForIndex: threadIndex <cmacro: '(ignored) 0'> "simulation only" | time range | + threadIndex = CTMUnknownOwner + ifTrue: [^ self assertEmptyRegisterStates: cogit processor registerState]. time := Time utcMicrosecondClock. range := coInterpreter cStackRangeForThreadIndex: threadIndex. self assert: ((range includes: cogit processor fp) and: [range includes: cogit processor sp]) "(0 to: numThreads + numThreadsIncrement) detect: [:i| | range | range := coInterpreter cStackRangeForThreadIndex: threadIndex. ((range includes: cogit processor fp) and: [range includes: cogit processor sp])] ifNone: []" "{ coInterpreter whereIs: cogit processor fp. coInterpreter whereIs: cogit processor sp }"! Item was changed: + ----- Method: CogThreadManager>>doTryLockVMOwnerTo: (in category 'Cogit lock implementation') ----- - ----- Method: CogThreadManager>>doTryLockVMOwnerTo: (in category 'simulation') ----- doTryLockVMOwnerTo: threadIndex "In the simulation, this is being called by #simulateTryLockVMOwnerTo:, in C this method will just be called directly. Returns true if the vmOwner has been successfully set to the given thread index." <inline: #always> <var: #expected type: #int> | expected | expected := self cCode: 0 inSmalltalk: [AtomicValue newFrom: 0]. ^ (self atomic: (self addressOf: vmOwner) _compare: (self addressOf: expected) _exchange_strong: threadIndex) + or: ["We may already be vmOwner. The current vmOwner will be stored in expected. + However, if an unknown owner is present, we cannot assume that's us!!" + expected = threadIndex and: [threadIndex ~= CTMUnknownOwner]]! - or: ["We may already be vmOwner. The current vmOwner will be stored in expected" expected = threadIndex]! Item was changed: ----- Method: CogThreadManager>>ensureRunningVMThread: (in category 'public api') ----- ensureRunningVMThread: vmIsRelinquishing + "Called from checkVMOwnershipFromHeartbeat if the heartbeat thread manages to lock the vmOwner to CTMUnknownOwner. + Hence we are the vmOwner and are in the heartbeat thread. + We must therefore ensure that when we leave this method either: + 1. Some other thread is running and vmOwner + 2. We've released ownership of the VM" - "Called from checkVMOwnershipFromHeartbeat if the VM is unowned. - Hence we are in the heartbeat thread. The race is against that thread - owning the VM and against foreign callbacks." <returnTypeC: #void> <var: #vmThread type: #'CogVMThread *'> self willingVMThread ifNotNil: [:vmThread| | threadState | threadState := vmThread vmThreadState. "If the VM is relinquishing the processor then only schedule a thread if it has work to do." (vmIsRelinquishing and: [threadState ~= CTMWantingOwnership]) ifTrue: + [^self releaseVM]. - [^self]. self assert: ((threadState = CTMAssignableOrInVM or: [threadState = CTMInitializing]) or: [threadState = CTMWantingOwnership]). + "Ownership will be transferred to vmThread, no need to release the VM." + ^ self wakeVMThread: vmThread]. - (self tryLockVMOwnerTo: vmThread index) ifFalse: "someone beat us to it..." - [^self]. - vmOSThread := vmThread osThread. - "release the thread from its blocking loop" - self ioSignalOSSemaphore: (self addressOf: vmThread osSemaphore). - self ioTransferTimeslice. - "self cCode: [coInterpreter print: 'ERVT signalled '; printNum: vmThread index; cr]." - ^self]. "If the VM is relinquishing the processor then only schedule a thread if it has work to do (willingVMThread not nil above). If we have failed to allocate thread storage before there is no point continuing to try to do so. By this time we should have quite a few threads in the pool." (vmIsRelinquishing or: [memoryIsScarce]) ifTrue: + [^self releaseVM]. + self unusedThreadInfo ifNotNil: [:vmThread| ^ self wakeVMThread: vmThread]. + ^ self releaseVM! - [^self]. - self unusedThreadInfo ifNotNil: - [:vmThread| - (self tryLockVMOwnerTo: vmThread index) ifTrue: - [(self startThreadForThreadInfo: vmThread) ifFalse: - [self releaseVM]]]! Item was changed: ----- Method: CogThreadManager>>highestPriorityThreadIfHigherThan:expectedMax: (in category 'public api') ----- highestPriorityThreadIfHigherThan: activePriority expectedMax: maxPriority "Answer the first vmThread waiting to acquire the VM that is of higher priority than activePriority, or answer nil if none. If there is a higher priority thread then set the coInterpreter's maxWaitingPriority to either the priority of the next highest priority vmThread, or to 0 if none is waiting." <returnTypeC: #'CogVMThread *'> | vmThread highest nextHighest | <var: #vmThread type: #'CogVMThread *'> <var: #highest type: #'CogVMThread *'> <var: #nextHighest type: #'CogVMThread *'> highest := nextHighest := nil. "To make this fair we could remember the last index at which we found the highest and start the search at the following index." 1 to: numThreads do: [:i| vmThread := threads at: i. vmThread vmThreadState = CTMWantingOwnership ifTrue: [self assert: vmThread priority <= maxPriority. highest isNil ifTrue: [highest := vmThread] ifFalse: [vmThread priority > highest priority ifTrue: [nextHighest := highest. highest := vmThread] ifFalse: [nextHighest isNil ifTrue: [nextHighest := vmThread] ifFalse: [vmThread priority > nextHighest priority ifTrue: [nextHighest := vmThread]]]]]]. highest isNil ifTrue: [^nil]. highest priority <= activePriority ifTrue: [^nil]. + coInterpreter waitingPriorityIsAtLeast: (nextHighest isNil - coInterpreter setMaxWaitingPriorityTo: (nextHighest isNil ifTrue: [0] ifFalse: [nextHighest priority]). ^highest! Item was changed: ----- Method: CogThreadManager>>loadOrInitializeRegisterStateFor: (in category 'simulation') ----- loadOrInitializeRegisterStateFor: threadIndex <doNotGenerate> + |processor| + threadIndex = CTMUnknownOwner ifTrue: [^ self halt]. + - |fakeThreadIndex processor| - "The heartbeat thread will lock the VM to -1, so generate a fake processor data for this." - fakeThreadIndex := threadIndex == -1 ifTrue: [self maxNumThreads] ifFalse: [threadIndex]. processor := cogit processor. processor setRegisterState: (registerStates at: threadIndex ifAbsentPut: + [self initializeProcessor: processor forThreadIndex: threadIndex. - [self initializeProcessor: processor forThreadIndex: fakeThreadIndex. processor registerState]).! Item was added: + ----- Method: CogThreadManager>>returnToSchedulingLoopAndWakeThreadFor:source: (in category 'public api') ----- + returnToSchedulingLoopAndWakeThreadFor: threadAffinity source: sourceIndex + "Transfer the VM to a thread that is compatible with the given affinity. + Called from a thread that finds the highest priority runnable process is bound + to the given affinity." + <returnTypeC: #void> + "Make sure we do actually need to wake a thread" + self assert: (self vmOwnerIsCompatibleWith: threadAffinity) not. + self assert: threadAffinity ~= 0. + + ^ threadAffinity > 0 + ifTrue: [self assert: (threadAffinity between: 1 and: numThreads). + self returnToSchedulingLoopAndReleaseVMOrWakeThread: (threads at: threadAffinity) source: sourceIndex] + ifFalse: [|willingThread| + self assert: (self getVMOwner = threadAffinity negated). + "We know the thread affinity is 'any thread other then this one!!'." + willingThread := self ensureWillingThread. + willingThread ifNotNil: [self returnToSchedulingLoopAndReleaseVMOrWakeThread: willingThread source: sourceIndex]]! Item was changed: ----- Method: CogThreadManager>>saveRegisterStateForCurrentProcess (in category 'simulation') ----- saveRegisterStateForCurrentProcess "On switching osProcess we have to both - SAVE old process's register state - RESTORE new process's register state So what are the transitions? The transitions out (where processor state must be saved) are the opposites of tryLockVMOwnerTo:. Hence - waitOnOSSemaphore: - disownVM: - ioTransferTimeslice i.e. the continuations from here, disownVM:, and ioTransferTimeslice, will use tryLockVMOwnerTo: to regain control of the VM. So the register state to be restored at that point (if tryLockVMOwnerTo: succeeds) is the register state saved in one of the three places. But the processor is initialized in tryLockVMOwnerTo:, so only save the state if state is already present." <doNotGenerate> + self getVMOwner = CTMUnknownOwner ifTrue: [^ self]. cogit withProcessorHaltedDo: [| currentVMThread state | currentVMThread := self vmThreadForCurrentProcess. state := cogit processor registerState. self assertValidStackPointersInState: state forIndex: currentVMThread index. registerStates at: currentVMThread index put: state]! Item was changed: ----- Method: CogThreadManager>>setVMOwner: (in category 'public api') ----- setVMOwner: indexOrZero "An ugly accessor used in only three cases: 1. by ownVMFromUnidentifiedThread when the VM is first locked to the thread id of the unidentified thread, and then, once identified, to the thread's index. 2. by wakeVMThreadFor: used by the two-level scheduler to switch threads when a Smalltalk process switch occurs to a process affined to another thread. 3. to release the VM (set the owner to zero)" <inline: #always> + self assert: (self getVMOwner = self ioGetThreadLocalThreadIndex or: [self getVMOwner = -1]). + self assert: (self getVMOwner ~= indexOrZero). - self assert: (self getVMOwner ~= 0 and: [self getVMOwner ~= indexOrZero]). self cCode: '' inSmalltalk: [coInterpreter transcript ensureCr; f: 'setVMOwner: %d -> %d (%s)\n' printf: { self getVMOwner. indexOrZero. thisContext home sender selector }]. "TODO: We could make this a `release` ordering, which may perform better on ARM." self atomic_store: (self addressOf: vmOwner) _: indexOrZero! Item was changed: ----- Method: CogThreadManager>>simulateTryLockVMOwnerTo: (in category 'simulation') ----- simulateTryLockVMOwnerTo: threadIndex "In the real VM this is a direct call of #tryLockVMOwnerTo:. In the simulation this is where register state is restored, simulating a thread switch. State is stored in saveRegisterStateForCurrentProcess (sent by disownVM:, ioWaitOnOSSemaphore: and ioTransferTimeslice). The code here and in saveRegisterStateForCurrentProcess allow us to avoid the expensive and complex MultiProcessor hack. The idea here is to save the register state around the invocation of tryLockVMOwnerTo:, and set the register state to that for the owner, changing the state if ownership has changed, restoring it if ownership has not changed." <doNotGenerate> self deny: threadIndex = 0. ^cogit withProcessorHaltedDo: [| previousOwner currentOwner processor result | processor := cogit processor. "After switching, the 'current' owner will be the 'previous' owner. Though the value will be the same, let's still introduce a second variable that we can use after the switch to make it more clear what's going on." previousOwner := currentOwner := self getVMOwner. "If we currently have a VM owner, the register state should be + valid for that owner, otherwise it should be empty. + It may be CTMUnknownOwner (-1), in that case it should also be empty." + currentOwner > 0 - valid for that owner, otherwise it should be empty." - currentOwner ~= 0 ifTrue: [self assertValidStackPointersInState: processor registerState forIndex: currentOwner] ifFalse: [self assertEmptyRegisterStates: processor registerState]. result := self doTryLockVMOwnerTo: threadIndex. self assert: result = (threadIndex = self getVMOwner). result ifTrue: ["If we did actually change owners, assert that previously the processor was emtpy." previousOwner ~= self getVMOwner ifTrue: [self assertEmptyRegisterStates: processor registerState. self loadOrInitializeRegisterStateFor: threadIndex]]. coInterpreter transcript ensureCr; + f: 'tryLockVMOwner %d -> %d (%s) %s\n' + printf: { previousOwner. threadIndex. thisContext home sender selector. result ifTrue: ['ok'] ifFalse: ['FAILED'] }. - f: (result ifTrue: ['tryLockVMOwner %d -> %d (%s) ok\n'] ifFalse: ['tryLockVMOwner %d -> %d (%s) FAILED\n']) - printf: { previousOwner. threadIndex. thisContext home sender selector }. self assertValidProcessorStackPointersForIndex: self getVMOwner. result]! Item was changed: ----- Method: CogThreadManager>>threadIndex:isCompatibleWith: (in category 'public api-testing') ----- + threadIndex: aThreadIndex isCompatibleWith: threadAffinity + "Test if threadAffinity is ok to run on a thread with the given index." - threadIndex: aThreadIndex isCompatibleWith: processThreadId - "Test if processThreadId is ok to run on a thread with the given index." <inline: true> self assert: aThreadIndex > 0. + ^threadAffinity = 0 + or: [threadAffinity >= 0 + ifTrue: [aThreadIndex = threadAffinity] + ifFalse: [aThreadIndex ~= threadAffinity negated]]! - ^processThreadId = 0 - or: [processThreadId >= 0 - ifTrue: [aThreadIndex = processThreadId] - ifFalse: [aThreadIndex ~= processThreadId negated]]! Item was added: + ----- Method: CogThreadManager>>wakeVMThread: (in category 'scheduling') ----- + wakeVMThread: vmThread + <var: #vmThread type: #'CogVMThread *'> + <returnTypeC: #void> + | threadState | + self assert: (self vmIsOwned and: [(self vmOwnerIs: vmThread index) not]). + + "Instead of going through a #disownVM: call, directly set the new VM owner. + This has the advantage of avoiding a race for the different threads to become the new + VM owner. + In Simulation, this means we need to simulate a thread-switch." + self cCode: [] inSmalltalk: [ + self saveRegisterStateForCurrentProcess. + self loadOrInitializeRegisterStateFor: vmThread index]. + self setVMOwner: vmThread index. + + threadState := vmThread vmThreadState. + threadState = CTMUninitialized + ifTrue: [(self startThreadForThreadInfo: vmThread) ifFalse: [self releaseVM. "TODO: IS THIS SANE?"]] + ifFalse: + [self assert: ((threadState = CTMWantingOwnership + or: [threadState = CTMAssignableOrInVM]) + or: [threadState = CTMInitializing]). + self ioSignalOSSemaphore: (self addressOf: vmThread osSemaphore)]. + self ioTransferTimeslice! Item was removed: - ----- Method: CogThreadManager>>wakeVMThreadFor: (in category 'public api') ----- - wakeVMThreadFor: index - "Transfer the VM to the thread with index. Called from a thread that finds the - highest priority runnable process is bound to the thread with index index." - <returnTypeC: #void> - | vmThread threadState | - self assert: (self vmIsOwned and: [(self vmOwnerIs: index) not]). - self assert: (index between: 1 and: numThreads). - vmThread := threads at: index. - - "Instead of going through a #disownVM: call, directly set the new VM owner. - This has the advantage of avoiding a race for the different threads to become the new - VM owner. - In Simulation, this means we need to simulate a thread-switch." - self cCode: [] inSmalltalk: [ - self saveRegisterStateForCurrentProcess. - self loadOrInitializeRegisterStateFor: index]. - self setVMOwner: index. - - threadState := vmThread vmThreadState. - threadState = CTMUninitialized - ifTrue: [self startThreadForThreadInfo: vmThread] - ifFalse: - [self assert: ((threadState = CTMWantingOwnership - or: [threadState = CTMAssignableOrInVM]) - or: [threadState = CTMInitializing]). - self ioSignalOSSemaphore: (self addressOf: vmThread osSemaphore)]. - self ioTransferTimeslice! Item was removed: - ----- Method: CogVMSimulator>>ownerIndexOfProcess: (in category 'multi-threading simulation switch') ----- - ownerIndexOfProcess: aProcess - "This method includes or excludes CoInterpreterMT methods as required. - Auto-generated by CogVMSimulator>>ensureMultiThreadingOverridesAreUpToDate" - - ^self perform: #ownerIndexOfProcess: - withArguments: {aProcess} - inSuperclass: (cogThreadManager ifNil: [CoInterpreterPrimitives] ifNotNil: [CoInterpreterMT])! Item was removed: - ----- Method: CogVMSimulator>>setMaxWaitingPriorityTo: (in category 'multi-threading simulation switch') ----- - setMaxWaitingPriorityTo: minPriority - "This method includes or excludes CoInterpreterMT methods as required. - Auto-generated by CogVMSimulator>>ensureMultiThreadingOverridesAreUpToDate" - - ^self perform: #setMaxWaitingPriorityTo: - withArguments: {minPriority} - inSuperclass: (cogThreadManager ifNil: [CoInterpreterPrimitives] ifNotNil: [CoInterpreterMT])! Item was removed: - ----- Method: CogVMSimulator>>setOwnerIndexOfProcess:to:bind: (in category 'multi-threading simulation switch') ----- - setOwnerIndexOfProcess: aProcess to: anIndex bind: bind - "This method includes or excludes CoInterpreterMT methods as required. - Auto-generated by CogVMSimulator>>ensureMultiThreadingOverridesAreUpToDate" - - ^self perform: #setOwnerIndexOfProcess:to:bind: - withArguments: {aProcess. anIndex. bind} - inSuperclass: (cogThreadManager ifNil: [CoInterpreterPrimitives] ifNotNil: [CoInterpreterMT])! Item was changed: VMStructType subclass: #CogVMThread + instanceVariableNames: 'index state priority osSemaphore osThread disownFlags newMethodOrNull argumentCount inMachineCode cStackPointer cFramePointer awolProcIndex awolProcLength awolProcesses reenterThreadSchedulingLoop' - instanceVariableNames: 'index state priority osSemaphore osThread newMethodOrNull argumentCount inMachineCode cStackPointer cFramePointer awolProcIndex awolProcLength awolProcesses reenterThreadSchedulingLoop' classVariableNames: '' poolDictionaries: 'VMThreadingConstants' category: 'VMMaker-Multithreading'! !CogVMThread commentStamp: '<historical>' prior: 0! Instances of this class represent control blocks for native threads that cooperatively schedule the VM. See the class comment of CogThreadManager for full documentation. N.B. awolProcesses must be the last inst var.! Item was added: + ----- Method: CogVMThread>>awolProcesses: (in category 'accessing') ----- + awolProcesses: anObject + + ^ awolProcesses := anObject.! Item was added: + ----- Method: CogVMThread>>coerceTo:sim: (in category 'as yet unclassified') ----- + coerceTo: aSymbol sim: aSimulator + + self assert: aSymbol = #'CogVMThread *'. + ^ self! Item was added: + ----- Method: CogVMThread>>disownFlags (in category 'accessing') ----- + disownFlags + + ^ disownFlags! Item was added: + ----- Method: CogVMThread>>disownFlags: (in category 'accessing') ----- + disownFlags: anObject + + ^ disownFlags := anObject.! Item was changed: ----- Method: CogVMThread>>initializeThreadState (in category 'initialize-release') ----- initializeThreadState + "Unfortunately this cannot be inlined as Slang otherwise screws up the generation of the `atomic_store` call." + <inline: false> - <inline: #always> "In comparision to #initialize, this is also called in C code to initialize the VMThread, not just in the Smalltalk simulation." self cCode: [] inSmalltalk: [state := AtomicValue new]. + self atomic_store: (self addressOf: self state) _: CTMUninitialized.! - self atomic_store: (self addressOf: state) _: CTMUninitialized.! Item was changed: ----- Method: CogVMThread>>printOn: (in category 'printing') ----- printOn: aStream super printOn: aStream. aStream nextPutAll: ' index: '; print: index; nextPutAll: ' state: '; nextPutAll: (VMThreadingConstants keys + detect: [:k| k first == $C and: [(VMThreadingConstants classPool at: k) = state value]] + ifNone: [state value printString])! - detect: [:k| k first == $C and: [(VMThreadingConstants classPool at: k) = state]] - ifNone: [state printString])! Item was changed: ----- Method: CogVMThread>>setVmThreadState: (in category 'accessing') ----- setVmThreadState: anInteger + <inline: true> - "Unfortunately this cannot be inlined by Slang, as the generation otherwise screws - up the atomic_store operation." - <inline: false> |currentState| currentState := self vmThreadState. + currentState ~= anInteger + ifTrue: [currentState caseOf: { + [CTMUninitialized] -> [self assert: anInteger = CTMInitializing]. + [CTMInitializing] -> [self assert: anInteger = CTMAssignableOrInVM]. + [CTMAssignableOrInVM] -> [self assert: anInteger = CTMUnavailable]. + [CTMUnavailable] -> [self assert: (anInteger = CTMAssignableOrInVM + or: [anInteger = CTMWantingOwnership])]. + [CTMWantingOwnership] -> [self assert: anInteger = CTMAssignableOrInVM] + } otherwise: []]. - currentState caseOf: { - [CTMUninitialized] -> [self assert: anInteger = CTMInitializing]. - } otherwise: []. "The actual meat of the operation. The previous checks are only for debugging." + self atomic_store: (self addressOf: state) _: anInteger.! - self atomic_store: (self addressOf: self state) _: anInteger.! Item was changed: ----- Method: CogVMThread>>vmThreadState (in category 'accessing') ----- vmThreadState + <inline: true> + ^ self atomic_load: (self addressOf: state)! - "Unfortunately this cannot be inlined by Slang, as the generation otherwise screws - up the atomic_load operation." - <inline: false> - ^ self atomic_load: (self addressOf: self state)! Item was changed: ----- Method: FilePlugin>>primitiveFileReadPinningAndDisowning (in category 'file primitives') ----- primitiveFileReadPinningAndDisowning "This version of primitiveFileRead is for garbage collectors that support pinning and the multi-threaded VM. It actually does the own/disown dance." + | count startIndex array file slotSize elementSize bytesRead vmHandle wasPinned | - | count startIndex array file slotSize elementSize bytesRead threadIndexAndFlags wasPinned | <inline: true> <var: 'file' type: #'SQFile *'> <var: 'count' type: #'size_t'> <var: 'startIndex' type: #'size_t'> <var: 'slotSize' type: #'size_t'> <var: 'elementSize' type: #'size_t'> count := interpreterProxy positiveMachineIntegerValueOf: (interpreterProxy stackValue: 0). startIndex := interpreterProxy positiveMachineIntegerValueOf: (interpreterProxy stackValue: 1). array := interpreterProxy stackValue: 2. file := self fileValueOf: (interpreterProxy stackValue: 3). (interpreterProxy failed "buffer can be any indexable words or bytes object except CompiledMethod" or: [(interpreterProxy isWordsOrBytes: array) not]) ifTrue: [^interpreterProxy primitiveFailFor: PrimErrBadArgument]. slotSize := interpreterProxy slotSizeOf: array. (startIndex >= 1 and: [startIndex + count - 1 <= slotSize]) ifFalse: [^interpreterProxy primitiveFailFor: PrimErrBadIndex]. (wasPinned := interpreterProxy isPinned: array) ifFalse: [array := interpreterProxy pinObject: array]. + vmHandle := interpreterProxy disownVM: DisownVMForThreading. - threadIndexAndFlags := interpreterProxy disownVM: DisownVMForThreading. "Note: adjust startIndex for zero-origin byte indexing" elementSize := slotSize = 0 ifTrue: [1] ifFalse: [(interpreterProxy byteSizeOf: array) // slotSize]. bytesRead := self sqFile: file Read: count * elementSize Into: (interpreterProxy cCoerce: (interpreterProxy firstIndexableField: array) to: #'char *') At: startIndex - 1 * elementSize. + interpreterProxy ownVM: vmHandle. - interpreterProxy ownVM: threadIndexAndFlags. wasPinned ifFalse: [interpreterProxy unpinObject: array]. interpreterProxy failed ifFalse: [interpreterProxy methodReturnInteger: bytesRead // elementSize] "answer # of elements read"! Item was changed: ----- Method: SqueakSSLPlugin>>primitiveConnect (in category 'primitives') ----- primitiveConnect "Primitive. Starts or continues a client handshake using the provided data. Will eventually produce output to be sent to the server. Requires the host name to be set for the session. Returns: > 0 - Number of bytes to be sent to the server 0 - Success. The connection is established. -1 - More input is required. < -1 - Other errors. " + | start srcLen dstLen srcOop dstOop handle srcPtr dstPtr result wasSrcPinned wasDestPinned vmHandle | - | start srcLen dstLen srcOop dstOop handle srcPtr dstPtr result | <var: #srcPtr type: #'char *'> <var: #dstPtr type: #'char *'> <export: true> interpreterProxy methodArgumentCount = 5 ifFalse:[^interpreterProxy primitiveFail]. dstOop := interpreterProxy stackValue: 0. srcLen := interpreterProxy stackIntegerValue: 1. start := interpreterProxy stackIntegerValue: 2. srcOop := interpreterProxy stackValue: 3. handle := interpreterProxy stackIntegerValue: 4. interpreterProxy failed ifTrue:[^nil]. ((start > 0 and:[srcLen >= 0]) and:[(interpreterProxy isBytes: srcOop) and:[(interpreterProxy isBytes: dstOop) and:[(interpreterProxy byteSizeOf: srcOop) >= (start + srcLen - 1)]]]) ifFalse:[^interpreterProxy primitiveFail]. + + "Careful!! The object may move when being pinned!!" + (wasSrcPinned := interpreterProxy isPinned: srcOop) + ifFalse: [srcOop := interpreterProxy pinObject: srcOop]. + (wasDestPinned := interpreterProxy isPinned: dstOop) + ifFalse: [dstOop := interpreterProxy pinObject: dstOop]. + + "Pinning may fail (only if we're out of memory)" + (srcOop isNil or: [dstOop isNil]) + ifTrue: [^ interpreterProxy primitiveFail]. + srcPtr := interpreterProxy firstIndexableField: srcOop. dstPtr := interpreterProxy firstIndexableField: dstOop. srcPtr := srcPtr + start - 1. dstLen := interpreterProxy byteSizeOf: dstOop. + + vmHandle := interpreterProxy disownVM: DisownVMForThreading. + result := self cCode: 'sqConnectSSL(handle, srcPtr, srcLen, dstPtr, dstLen)' inSmalltalk:[handle. srcPtr. srcLen. dstPtr. dstLen. -2]. + + interpreterProxy ownVM: vmHandle. + wasSrcPinned ifFalse: [interpreterProxy unpinObject: srcOop]. + wasDestPinned ifFalse: [interpreterProxy unpinObject: dstOop]. + interpreterProxy failed ifTrue:[^nil]. interpreterProxy pop: interpreterProxy methodArgumentCount+1. interpreterProxy pushInteger: result.! Item was changed: SharedPool subclass: #VMThreadingConstants instanceVariableNames: '' + classVariableNames: 'AWOLProcessesIncrement CTMAssignableOrInVM CTMInitializing CTMUnavailable CTMUninitialized CTMUnknownOwner CTMWantingOwnership ThreadIdIndex ThreadIdShift' - classVariableNames: 'AWOLProcessesIncrement CTMAssignableOrInVM CTMInitializing CTMUnavailable CTMUninitialized CTMWantingOwnership ThreadIdIndex ThreadIdShift' poolDictionaries: '' category: 'VMMaker-Multithreading'! !VMThreadingConstants commentStamp: '<historical>' prior: 0! VMThreadingConstants ensureClassPool. CogThreadManager classPool keys do: [:k| VMThreadingConstants classPool declare: k from: CogThreadManager classPool]. CoInterpreterMT classPool keys do: [:k| VMThreadingConstants classPool declare: k from: CoInterpreterMT classPool].!

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev February 2024