Hi Chris,

On Mon, Mar 25, 2024 at 4:30 PM Chris Muller <asqueaker@gmail.com> wrote:

Hi Marcel,

true means Processes running at the same priority are given fair time, round-robin style, as higher-priority processes return or go back to sleep. 

This violates the real-time semantics of the scheduler, which are that 

- processes are only preempted by higher-priority processes

- processes of the same priority are cooperative; another process of the same priority will only run if the currently running process at that priority yields or waits on a semaphore, etc.

This code forcing it to false appears to have been the root cause of the recent issues with source.squeak.org's responsiveness.

That makes sense, but the issue is that the server is relying on ill-defined behaviour, and should instead implement its own scheduler amongst the time hogs.

For example, the server could introduce a high-priority process that wakes up periodically (looping on a delay). On waking up it inspects the current set of running request serving processes, and if the current one has taken more than its time-slice the supervisor process can send yield to it, moving it to the back of the queue.

In this way one can implement time slicing for a subset of processes while allowing the rest of the system to provide sound concurrent code with a minimum of synchronisation.  See below.

  Clients were timing out because one big request would cause a backlog of 10 small requests to queue up behind it and, often, timeout.  Simply setting this true in a startup patch script seems to have fixed it.

yes, but it's a hack fix that breaks the rest of the system. See below.

There are no good explanations in the code for false. 

There are *excellent* reasons for false as the default.  And other Smalltalk implementations (especially VisualWorks) have been modified to work this way.  The old behaviour (Smalltalk processPreemptionYields == true) *IS A BUG*!! And I shout for good reason.  When the scheduler does not preempt (and note from above that *does not* prevent a subsystem from implementing preemption/time-slicing, etc, amongst a subset of processes), then a set of processes can safely maintain concurrent access to resources, and any other manner of cooperative behaviour, safely simply by using yield and wait. When Smalltalk processPreemptionYields == true, however, any code can be preempted at arbitrary times, and no safety guarantees can be provided. There is only a fig leaf of a real-time cooperative scheduler, not a reality.

Hence the solution here is *to fix the squeaksource server*, *not* to break the carefully thought out fix to the scheduling semantics that Smalltalk processPreemptionYields == false provides!!!!

This was discussed in some detail (obviously insufficient) years ago when the change was introduced to the Cog VM.

The only one I can remember from this mailing list I think was that you can get away easier not protecting your code with Semaphores, and instead depend on the known process switching behavior of the VM.  Perhaps for ultra high-performance applications?  It feels application-specific to me.  According to the comment, it also diverges from the Blue Book specification.

It's not necessarily to do with ultra-high-performance.  It's much more to do with comprehensibility and predictability.  A scheduler that provides preemption across priorities and round-robin cooperative scheduling within a priority has very desirable properties that make possible very simple implementation of robust concurrent code.

 

I don't mean to be disagreeable about the default, but true seems like the setting for smoother multitasking and more in the spirit of Squeak's liveness. 

Wrong, wrong, wrong, and wrong again. Smalltalk processPreemptionYields == true is a horrible bug.

What is the rationale for false?

See "*excellent* reasons" above.

Best,

  Chris

On Mon, Mar 25, 2024 at 12:00 PM Taeumel, Marcel via Squeak-dev <squeak-dev@lists.squeakfoundation.org> wrote:

Ah, this was "we have a new VM feature" migration code. We no longer need it, I suppose.

Can we make sure that the ReleaseBuilder sets this to "false" by default? :-)

Best,

Marcel

Am 25.03.2024 00:28:07 schrieb commits@source.squeak.org <commits@source.squeak.org>:

Chris Muller uploaded a new version of Kernel to project The Trunk:
http://source.squeak.org/trunk/Kernel-cmm.1562.mcz

==================== Summary ====================

Name: Kernel-cmm.1562
Author: cmm
Time: 24 March 2024, 6:27:34.572036 pm
UUID: 886bf6b9-a56b-4131-accf-3867a3836658
Ancestors: Kernel-ct.1561

Allow processPreemptionYields: to persist across image saves, as purported by its comment.

=============== Diff against Kernel-ct.1561 ===============

Item was changed:
----- Method: ProcessorScheduler class>>startUp: (in category 'background process') -----
startUp: resuming

- Smalltalk processPreemptionYields ifTrue: [
- Smalltalk processPreemptionYields: false].
-
resuming ifTrue: [
Smalltalk installLowSpaceWatcher.
self installIdleProcess].!

--

_,,,^..^,,,_

best, Eliot