Hi Jaromir,

I *hate* code like oldList class == LinkedList", "context methodClass == Semaphore", and "context methodClass == Mutex", which are all about implementation, not safely extensible, and difficult to understand. Instead we could introduce abstractions such as "isRunList" vs "isBlockingList", and "isSemaphore" and "isMutex", which would allow us to ast least extend the system if we ever introduced new forms of block (the "new" Mutex implementation was new 14 years ago). This: "context stackPtr

0 and: [context top == false]])]" could be a method on Context.

So let's try and rewrite to be more abstract and more intentional.

On Wed, Feb 1, 2023 at 7:22 AM Jaromir Matas mail@jaromir.net wrote:

Hi Eliot, all,

I'd like to expand a bit on releasing critical sections during termination presented in Kernel-jar.1498. I *think* the second part of the procedure can be further simplified (and generalized) as follows.

The point is we do not need to check the process is immediately beyond the wait/lock primitive (by checking selectorJustSentOrSelf); instead, just being still inside the #critical context should suffice to assume the process left the condition variable's primitive but hasn't made progress and an intervention is required.

suspendAndReleaseCriticalSection

            "Figure out if we are terminating a process that is in the

ensure: block of a critical section.

            If it hasn't made progress but is beyond the wait (which

we can tell by the oldList being

            one of the runnable lists, i.e. a LinkedList, not a

Semaphore or Mutex), then the ensure:

            block needs to be run."



            | oldList |

            "Suspend and unblock the receiver from a condition

variable using the suspend primitive #88.

            It answers the list the receiver was on before the

suspension."

            oldList := self suspendAndUnblock.

            suspendedContext ifNotNil: [:context |

                            (context method pragmaAt:

#criticalSection) ifNil: [^self].

                            (oldList isNil or: [oldList class ==

LinkedList]) ifFalse: [

                            "If still blocked at the condition

variable of a critical section, skip the rest of the current context."

                                            suspendedContext :=

context pc: context endPC.

                                            ^self].



                            "Now we know we are somewhere beyond the

wait/lock primitive in a critical section but

                            haven't made progress into the ensure

block; the only point of the following code is to identify

                            the type of condition variable we're in

and let the process enter the ensure block when applicable."

                            "If still haven't made progress into the

ensure block then it has not been activated, so step into it."

                            (context methodClass == Semaphore or: [



                            "If still haven't made progress into the

ensure block and the lock primitive just acquired ownership

                            (indicated by it answering false) then the

ensure block has not been activated, so step into it."

                            (context methodClass == Mutex and: [

                                            context stackPtr > 0 and:

[context top == false]])]) ifTrue: [

                            "In such cases we need to step into the

ensure block and let the unwind execute the ensure argument

                            block and, if the critical section itself

is already inside an unwind block, also the ensure receiver block."

                                            suspendedContext :=

context stepToCallee]]

The first part deals with the situation the process is blocked at the condition variable but has been just released from it by suspendAndUnblock (prim 88) to be able to proceed with termination; in this case we do need to skip the critical block because the critical block itself can be inside some outer ensure argument block.

I hope it's a bit clearer now. Any ideas would be greatly appreciated.

Best,

Jaromir

PS: here's Eliot's original explanation of #releaseCriticalSection: http://forum.world.st/Solving-termination-of-critical-sections-in-the-contex...

What's changed since then is we can now safely unwind critical sections and non-local returns inside ensure argument blocks.

*From: *Jaromir Matas mail@jaromir.net *Sent: *Monday, January 30, 2023 17:40 *To: *Squeak Dev squeak-dev@lists.squeakfoundation.org; Eliot Miranda eliot.miranda@gmail.com *Subject: *[squeak-dev] Improve termination behavior when in #critical section

Hi Eliot, Nicolas, or anyone interested in context juggling,

A few Process/Semaphore/Mutex tests have been left as expected failures in the current image. It's time to fix them. I'd very much appreciate if you could review/merge the improved versions of #terminate and *especially* #suspendAndReleaseCriticalSection that finally make all currently failing tests pass. Sent in Kernel-jar.1498 and KernelTests-jar.443.

In addition, I've added a few Mutex and Semaphore tests to further illustrate the behavior when terminating a process stuck in a #critical section, especially when the critical section itself is inside an unwind block (which has previously not been fully addressed). The steps in #suspendAndReleaseCriticalSection should, hopefully, be sufficiently commented.

In case you'd like to hear more detailed comments or explanations I'll be happy to elaborate.

The changes have been tested also in Cuis which gives me more confidence they're solid.

Thanks for any comments.

Best,

Jaromir

--

*Jaromír Matas*

mail@jaromir.net

Hi Eliot,

Here's a modified version using instance methods rather than class methods.

I'd very much appreciate your feedback whether such design is at all acceptable here.

I've been thinking about your suggestions:

Instead we could introduce abstractions such as "isRunList" vs "isBlockingList", and "isSemaphore" and "isMutex",

That'd be nicely extensible but we'd still have to do the "isRunList ifTrue: ... isSemaphore ifTrue: ..." thing. However, this is indeed faster and has fewer suspension points; would you hence consider preferable in this situation?

This: "context stackPtr > 0 and: [context top == false]])]" could be a method on Context.

I'm not sure how you mean it: I can't assign any reasonable meaning (name) to the above expression; it doesn't mean "I've just acquired ownership" which is what we're trying to find out because that meaning would require an additional condition: "I'm a #critical context with a condition variable receiver and I'm beyond the wait".

Many thanks for your help.

best, Jaromir

--

Jaromír Matas

mail@jaromir.net

From: Jaromir Matasmailto:mail@jaromir.net Sent: Friday, February 3, 2023 2:29 To: Eliot Mirandamailto:eliot.miranda@gmail.com Cc: The general-purpose Squeak developers listmailto:squeak-dev@lists.squeakfoundation.org Subject: RE: Improve termination behavior when in #critical section

Hi Eliot,

I *hate* code like oldList class == LinkedList...

Can't agree more :) How about this (also enclosed). If ok I will polish it a bit.

Process >> suspendAndReleaseCriticalSection "Figure out if we are terminating a process that is in the ensure: block of a critical section. If it hasn't made progress but is beyond the wait (which we can tell by the oldList being one of the runnable lists, i.e. a LinkedList, not a Semaphore or Mutex), then the ensure: block needs to be run."

"Suspend and unblock the receiver from a condition variable using suspend primitive #88. It answers the list the receiver was on before the suspension."

| oldList | oldList := self suspendAndUnblock ifNil: [LinkedList new]. ((suspendedContext ifNil: [^self]) method pragmaAt: #criticalSection) ifNil: [^self]. suspendedContext := oldList class releaseCriticalSection: suspendedContext

LinkedList class >> releaseCriticalSection: aContext

^aContext methodClass stepIntoCriticalSection: aContext

Mutex class >> releaseCriticalSection: aContext

^aContext pc: aContext endPC

Mutex class >> stepIntoCriticalSection: aContext

^(aContext stackPtr > 0 and: [aContext top == false]) ifTrue: [aContext stepToCallee] ifFalse: [aContext]

Semaphore class >> releaseCriticalSection: aContext

^aContext pc: aContext endPC

Semaphore class >> stepIntoCriticalSection: aContext

^aContext stepToCallee

--

Jaromír Matas

mail@jaromir.net

From: Eliot Mirandamailto:eliot.miranda@gmail.com Sent: Friday, February 3, 2023 0:09 To: Jaromir Matasmailto:mail@jaromir.net Cc: The general-purpose Squeak developers listmailto:squeak-dev@lists.squeakfoundation.org Subject: Re: Improve termination behavior when in #critical section

Hi Jaromir,

I *hate* code like oldList class == LinkedList", "context methodClass == Semaphore", and "context methodClass == Mutex", which are all about implementation, not safely extensible, and difficult to understand. Instead we could introduce abstractions such as "isRunList" vs "isBlockingList", and "isSemaphore" and "isMutex", which would allow us to ast least extend the system if we ever introduced new forms of block (the "new" Mutex implementation was new 14 years ago). This: "context stackPtr > 0 and: [context top == false]])]" could be a method on Context.

So let's try and rewrite to be more abstract and more intentional.

On Wed, Feb 1, 2023 at 7:22 AM Jaromir Matas <mail@jaromir.netmailto:mail@jaromir.net> wrote: Hi Eliot, all,

I'd like to expand a bit on releasing critical sections during termination presented in Kernel-jar.1498. I *think* the second part of the procedure can be further simplified (and generalized) as follows.

The point is we do not need to check the process is immediately beyond the wait/lock primitive (by checking selectorJustSentOrSelf); instead, just being still inside the #critical context should suffice to assume the process left the condition variable's primitive but hasn't made progress and an intervention is required.

suspendAndReleaseCriticalSection "Figure out if we are terminating a process that is in the ensure: block of a critical section. If it hasn't made progress but is beyond the wait (which we can tell by the oldList being one of the runnable lists, i.e. a LinkedList, not a Semaphore or Mutex), then the ensure: block needs to be run."

| oldList | "Suspend and unblock the receiver from a condition variable using the suspend primitive #88. It answers the list the receiver was on before the suspension." oldList := self suspendAndUnblock. suspendedContext ifNotNil: [:context | (context method pragmaAt: #criticalSection) ifNil: [^self].

(oldList isNil or: [oldList class == LinkedList]) ifFalse: [ "If still blocked at the condition variable of a critical section, skip the rest of the current context." suspendedContext := context pc: context endPC. ^self].

"Now we know we are somewhere beyond the wait/lock primitive in a critical section but haven't made progress into the ensure block; the only point of the following code is to identify the type of condition variable we're in and let the process enter the ensure block when applicable."

"If still haven't made progress into the ensure block then it has not been activated, so step into it." (context methodClass == Semaphore or: [

"If still haven't made progress into the ensure block and the lock primitive just acquired ownership (indicated by it answering false) then the ensure block has not been activated, so step into it." (context methodClass == Mutex and: [ context stackPtr > 0 and: [context top == false]])]) ifTrue: [

"In such cases we need to step into the ensure block and let the unwind execute the ensure argument block and, if the critical section itself is already inside an unwind block, also the ensure receiver block." suspendedContext := context stepToCallee]]

The first part deals with the situation the process is blocked at the condition variable but has been just released from it by suspendAndUnblock (prim 88) to be able to proceed with termination; in this case we do need to skip the critical block because the critical block itself can be inside some outer ensure argument block.

I hope it's a bit clearer now. Any ideas would be greatly appreciated.

Best, Jaromir

PS: here's Eliot's original explanation of #releaseCriticalSection: http://forum.world.st/Solving-termination-of-critical-sections-in-the-contex... What's changed since then is we can now safely unwind critical sections and non-local returns inside ensure argument blocks.

From: Jaromir Matasmailto:mail@jaromir.net Sent: Monday, January 30, 2023 17:40 To: Squeak Devmailto:squeak-dev@lists.squeakfoundation.org; Eliot Mirandamailto:eliot.miranda@gmail.com Subject: [squeak-dev] Improve termination behavior when in #critical section

Hi Eliot, Nicolas, or anyone interested in context juggling,

A few Process/Semaphore/Mutex tests have been left as expected failures in the current image. It's time to fix them. I'd very much appreciate if you could review/merge the improved versions of #terminate and *especially* #suspendAndReleaseCriticalSection that finally make all currently failing tests pass. Sent in Kernel-jar.1498 and KernelTests-jar.443.

In addition, I've added a few Mutex and Semaphore tests to further illustrate the behavior when terminating a process stuck in a #critical section, especially when the critical section itself is inside an unwind block (which has previously not been fully addressed). The steps in #suspendAndReleaseCriticalSection should, hopefully, be sufficiently commented.

In case you'd like to hear more detailed comments or explanations I'll be happy to elaborate.

The changes have been tested also in Cuis which gives me more confidence they're solid.

Thanks for any comments.

Best, Jaromir

--

Jaromír Matas

mail@jaromir.netmailto:mail@jaromir.net

-- _,,,^..^,,,_ best, Eliot