Hi Eliot, hi all,

just in case: speaking of the LowSpaceWatcher, there's a bug in case of concurrent invocations of the LowSpaceWatcher which causes multiple LowSpaceWatchers running at the same time... if you try in the workspace:

p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork

My suggestion is in System-jar.1367 (please disregard my note in the text about a bug in terminate, it's no longer relevant).

best, Jaromir

On 24-Jan-24 8:06:45 PM, "Eliot Miranda" eliot.miranda@gmail.com wrote:

Hi Marcel, Hi All,

On Fri, Jan 19, 2024 at 6:33 AM Taeumel, Marcel via Squeak-dev squeak-dev@lists.squeakfoundation.org wrote:

...

Hi Eliot --

Hmm... our CI can successfully update a 22095 to HEAD without issues. Once per day. Hmm... I am not sure why the Debugger would be involved in a regular update. The debugger cue ist just for invocation. Even existing debugger windows should keep on working.

The reason my image locks up is that I'm playing with the low space notification and consequently the low space notifier is popping up frequently and I have to hit proceed. Some day I'll revise the low space code appropriately; it's on my list. But for the moment it shows that there's an update map missing.

When I load Tools-ct.1244 before updating, with the image starting at Tools-ct.1239, I am able to update.

...

What's the situation where the debugger wants to appear during an update?

Frequent low space notifications.

...

Best, Marcel

Cheers!

...

...

Am 18.01.2024 23:36:44 schrieb Eliot Miranda eliot.miranda@gmail.com:

Hi All,

I'm guessing there's some package ordering update missing to 

ensure the new debugger cue is defined. I just attempted to update a VMMaker image and got the following infinite recursion:

 0x1465199e8 s UndefinedObject>handleSignal: 0x1186898e0: a(n) 

UndefinedObject 0x146518030 s MessageNotUnderstood(Exception)>signal 0x146517c30: a(n) MessageNotUnderstood 0x146517818 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject 0x146517c78 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler 0x1465180e8 s ECToolSet class(StandardToolSet class)>handleError: 0x1195d6c88: a(n) ECToolSet 0x146518500 s ToolSet class>handleError: 0x118a26690: a(n) ToolSet 0x146518900 s UnhandledError>defaultAction 0x146518e50: a(n) UnhandledError 0x146518d98 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x1465191d0 s UnhandledError(Exception)>signal 0x146518e50: a(n) UnhandledError 0x146519668 s UnhandledError class>signalForException: 0x118a27df8: a(n) UnhandledError 0x146519aa0 s MessageNotUnderstood(Error)>defaultAction 0x1465185b8: a(n) MessageNotUnderstood 0x146519f38 s MessageNotUnderstood>defaultAction 0x1465185b8: a(n) MessageNotUnderstood 0x14651a370 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x1465189b8 s MessageNotUnderstood(Exception)>signal 0x1465185b8: a(n) MessageNotUnderstood 0x1465181a0 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject 0x146518600 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler 0x146518a70 s ECToolSet class(StandardToolSet class)>handleError: 0x1195d6c88: a(n) ECToolSet 0x146518e88 s ToolSet class>handleError: 0x118a26690: a(n) ToolSet 0x146519288 s UnhandledError>defaultAction 0x1465197d8: a(n) UnhandledError 0x146519720 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x146519b58 s UnhandledError(Exception)>signal 0x1465197d8: a(n) UnhandledError 0x146519ff0 s UnhandledError class>signalForException: 0x118a27df8: a(n) UnhandledError 0x14651a428 s MessageNotUnderstood(Error)>defaultAction 0x146518f40: a(n) MessageNotUnderstood 0x14651a8c0 s MessageNotUnderstood>defaultAction 0x146518f40: a(n) MessageNotUnderstood 0x14651acf8 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x146519340 s MessageNotUnderstood(Exception)>signal 0x146518f40: a(n) MessageNotUnderstood 0x146518b28 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject 0x146518f88 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler 0x1465193f8 s ECToolSet class(StandardToolSet class)>handleError: 0x1195d6c88: a(n) ECToolSet 0x146519810 s ToolSet class>handleError: 0x118a26690: a(n) ToolSet 0x146519c10 s UnhandledError>defaultAction 0x14651a160: a(n) UnhandledError 0x14651a0a8 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x14651a4e0 s UnhandledError(Exception)>signal 0x14651a160: a(n) UnhandledError 0x14651a978 s UnhandledError class>signalForException: 0x118a27df8: a(n) UnhandledError 0x14651adb0 s MessageNotUnderstood(Error)>defaultAction 0x1465198c8: a(n) MessageNotUnderstood 0x14651b248 s MessageNotUnderstood>defaultAction 0x1465198c8: a(n) MessageNotUnderstood 0x14651b680 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x146519cc8 s MessageNotUnderstood(Exception)>signal 0x1465198c8: a(n) MessageNotUnderstood 0x1465194b0 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject 0x146519910 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler _,,,^..^,,,_ best, Eliot

-- _,,,^..^,,,_ best, Eliot

Hi Marcel,

How can this happen?

good question... I remember it (doubling the low space watcher process) bit me when I was experimenting with termination, I thought it was caused by a bug in my code but it turned out it was there before too :) Unfortunately I can't recall/find the exact situations. So I just posted it to make you aware.

Would you elaborate on why this can happen at all?

If you ask about the mechanism of the doubling behavior, I attached the explanation to the changeset (System-jar.1367):

"Workspce example: p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork Here both proceses are scheduled in the run queue, then p1 wakes up, starts the LowSpaceProcess termination and waits on a semaphore until the termination is finished. Before the LowSpaceProcess can proceed, process p2 wakes up and starts the LowSpaceProcess termination just like p1 and waits on a semaphore until the termination is finished. Finally the LowSpaceProcess wakes up, unwinds, unblocks p1 a p2 and terminates. p1 now continues and installs a new LowSpaceProcess and then p2 does the same. The result will be two processes running the #lowSpaceWatcher method."

best, Jaromir

On 25-Jan-24 8:44:35 AM, "Taeumel, Marcel via Squeak-dev" squeak-dev@lists.squeakfoundation.org wrote:

Hi Jaromir --

How can this happen? The only invocations of #installLowSpaceWatcher are in

Debugger >> proceed Debugger >> windowIsClosing

Both are per definition not thread-safe (because GUI/Tool related) and must never be called from arbitrary processes.

Would you elaborate on why this can happen at all?

...

p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork

Best, Marcel

...

Am 25.01.2024 00:21:43 schrieb Jaromir Matas mail@jaromir.net:

Hi Eliot, hi all,

just in case: speaking of the LowSpaceWatcher, there's a bug in case of concurrent invocations of the LowSpaceWatcher which causes multiple LowSpaceWatchers running at the same time... if you try in the workspace:

p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork

My suggestion is in System-jar.1367 (please disregard my note in the text about a bug in terminate, it's no longer relevant).

best, Jaromir

On 24-Jan-24 8:06:45 PM, "Eliot Miranda" eliot.miranda@gmail.com wrote:

...

Hi Marcel, Hi All,

On Fri, Jan 19, 2024 at 6:33 AM Taeumel, Marcel via Squeak-dev squeak-dev@lists.squeakfoundation.org wrote:

...

Hi Eliot --

Hmm... our CI can successfully update a 22095 to HEAD without issues. Once per day. Hmm... I am not sure why the Debugger would be involved in a regular update. The debugger cue ist just for invocation. Even existing debugger windows should keep on working.

The reason my image locks up is that I'm playing with the low space notification and consequently the low space notifier is popping up frequently and I have to hit proceed. Some day I'll revise the low space code appropriately; it's on my list. But for the moment it shows that there's an update map missing.

When I load Tools-ct.1244 before updating, with the image starting at Tools-ct.1239, I am able to update.

...

What's the situation where the debugger wants to appear during an update?

Frequent low space notifications.

...

Best, Marcel

Cheers!

...

...

Am 18.01.2024 23:36:44 schrieb Eliot Miranda eliot.miranda@gmail.com:

Hi All,

I'm guessing there's some package ordering update missing to 

ensure the new debugger cue is defined. I just attempted to update a VMMaker image and got the following infinite recursion:

 0x1465199e8 s UndefinedObject>handleSignal: 0x1186898e0: a(n) 

UndefinedObject 0x146518030 s MessageNotUnderstood(Exception)>signal 0x146517c30: a(n) MessageNotUnderstood 0x146517818 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject 0x146517c78 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler 0x1465180e8 s ECToolSet class(StandardToolSet class)>handleError: 0x1195d6c88: a(n) ECToolSet 0x146518500 s ToolSet class>handleError: 0x118a26690: a(n) ToolSet 0x146518900 s UnhandledError>defaultAction 0x146518e50: a(n) UnhandledError 0x146518d98 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x1465191d0 s UnhandledError(Exception)>signal 0x146518e50: a(n) UnhandledError 0x146519668 s UnhandledError class>signalForException: 0x118a27df8: a(n) UnhandledError 0x146519aa0 s MessageNotUnderstood(Error)>defaultAction 0x1465185b8: a(n) MessageNotUnderstood 0x146519f38 s MessageNotUnderstood>defaultAction 0x1465185b8: a(n) MessageNotUnderstood 0x14651a370 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x1465189b8 s MessageNotUnderstood(Exception)>signal 0x1465185b8: a(n) MessageNotUnderstood 0x1465181a0 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject 0x146518600 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler 0x146518a70 s ECToolSet class(StandardToolSet class)>handleError: 0x1195d6c88: a(n) ECToolSet 0x146518e88 s ToolSet class>handleError: 0x118a26690: a(n) ToolSet 0x146519288 s UnhandledError>defaultAction 0x1465197d8: a(n) UnhandledError 0x146519720 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x146519b58 s UnhandledError(Exception)>signal 0x1465197d8: a(n) UnhandledError 0x146519ff0 s UnhandledError class>signalForException: 0x118a27df8: a(n) UnhandledError 0x14651a428 s MessageNotUnderstood(Error)>defaultAction 0x146518f40: a(n) MessageNotUnderstood 0x14651a8c0 s MessageNotUnderstood>defaultAction 0x146518f40: a(n) MessageNotUnderstood 0x14651acf8 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x146519340 s MessageNotUnderstood(Exception)>signal 0x146518f40: a(n) MessageNotUnderstood 0x146518b28 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject 0x146518f88 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler 0x1465193f8 s ECToolSet class(StandardToolSet class)>handleError: 0x1195d6c88: a(n) ECToolSet 0x146519810 s ToolSet class>handleError: 0x118a26690: a(n) ToolSet 0x146519c10 s UnhandledError>defaultAction 0x14651a160: a(n) UnhandledError 0x14651a0a8 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x14651a4e0 s UnhandledError(Exception)>signal 0x14651a160: a(n) UnhandledError 0x14651a978 s UnhandledError class>signalForException: 0x118a27df8: a(n) UnhandledError 0x14651adb0 s MessageNotUnderstood(Error)>defaultAction 0x1465198c8: a(n) MessageNotUnderstood 0x14651b248 s MessageNotUnderstood>defaultAction 0x1465198c8: a(n) MessageNotUnderstood 0x14651b680 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject 0x146519cc8 s MessageNotUnderstood(Exception)>signal 0x1465198c8: a(n) MessageNotUnderstood 0x1465194b0 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject 0x146519910 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler _,,,^..^,,,_ best, Eliot

-- _,,,^..^,,,_ best, Eliot

Hi Marcel,

On 25-Jan-24 10:28:07 AM, "Taeumel, Marcel via Squeak-dev" squeak-dev@lists.squeakfoundation.org wrote:

Hi Jaromir --

...

If you ask about the mechanism of the doubling behavior, I attached

the explanation to the changeset

Yes, you explained why a specific piece of code causes complications. Still, I want to understand, why this piece of code is relevant.

Hmm, I guess you're right my example is (most likely) not relevant in the current image. I can't replicate the behavior in a practical situation. May I ask you to remove System-jar.1367 from the Inbox then? Sorry for the noise :) Thanks! Jaromir

For example, I could present some Morphic code, executed in multiple processes, that would then cause issues only to argue that, for example, there should be more semaphores in Collections or Morphic routines. Yet, there are design choices and contracts we should adhere to. Also, to avoid unnecessary code. Here, Morphic itself is not thread-safe and that is okay. It's by design.

One should use #addDeferredUIMessage:, for example. :-)

Best, Marcel

...

Am 25.01.2024 10:16:26 schrieb Jaromir Matas mail@jaromir.net:

Hi Marcel,

...

How can this happen?

good question... I remember it (doubling the low space watcher process) bit me when I was experimenting with termination, I thought it was caused by a bug in my code but it turned out it was there before too :) Unfortunately I can't recall/find the exact situations. So I just posted it to make you aware.

...

Would you elaborate on why this can happen at all?

If you ask about the mechanism of the doubling behavior, I attached the explanation to the changeset (System-jar.1367):

"Workspce example: p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork Here both proceses are scheduled in the run queue, then p1 wakes up, starts the LowSpaceProcess termination and waits on a semaphore until the termination is finished. Before the LowSpaceProcess can proceed, process p2 wakes up and starts the LowSpaceProcess termination just like p1 and waits on a semaphore until the termination is finished. Finally the LowSpaceProcess wakes up, unwinds, unblocks p1 a p2 and terminates. p1 now continues and installs a new LowSpaceProcess and then p2 does the same. The result will be two processes running the #lowSpaceWatcher method."

best, Jaromir

On 25-Jan-24 8:44:35 AM, "Taeumel, Marcel via Squeak-dev" squeak-dev@lists.squeakfoundation.org wrote:

...

Hi Jaromir --

How can this happen? The only invocations of #installLowSpaceWatcher are in

Debugger >> proceed Debugger >> windowIsClosing

Both are per definition not thread-safe (because GUI/Tool related) and must never be called from arbitrary processes.

Would you elaborate on why this can happen at all?

...

p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork

Best, Marcel

...

Am 25.01.2024 00:21:43 schrieb Jaromir Matas mail@jaromir.net:

Hi Eliot, hi all,

just in case: speaking of the LowSpaceWatcher, there's a bug in case of concurrent invocations of the LowSpaceWatcher which causes multiple LowSpaceWatchers running at the same time... if you try in the workspace:

p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork

My suggestion is in System-jar.1367 (please disregard my note in the text about a bug in terminate, it's no longer relevant).

best, Jaromir

On 24-Jan-24 8:06:45 PM, "Eliot Miranda" eliot.miranda@gmail.com wrote:

...

Hi Marcel, Hi All,

On Fri, Jan 19, 2024 at 6:33 AM Taeumel, Marcel via Squeak-dev squeak-dev@lists.squeakfoundation.org wrote:

...

Hi Eliot --

Hmm... our CI can successfully update a 22095 to HEAD without issues. Once per day. Hmm... I am not sure why the Debugger would be involved in a regular update. The debugger cue ist just for invocation. Even existing debugger windows should keep on working.

The reason my image locks up is that I'm playing with the low space notification and consequently the low space notifier is popping up frequently and I have to hit proceed. Some day I'll revise the low space code appropriately; it's on my list. But for the moment it shows that there's an update map missing.

When I load Tools-ct.1244 before updating, with the image starting at Tools-ct.1239, I am able to update.

...

What's the situation where the debugger wants to appear during an update?

Frequent low space notifications.

...

Best, Marcel

Cheers!

...

>Am 18.01.2024 23:36:44 schrieb Eliot Miranda >eliot.miranda@gmail.com: > >Hi All, > > I'm guessing there's some package ordering update missing to >ensure the new debugger cue is defined. I just attempted to >update a VMMaker image and got the following infinite recursion: > > 0x1465199e8 s UndefinedObject>handleSignal: 0x1186898e0: >a(n) UndefinedObject > 0x146518030 s MessageNotUnderstood(Exception)>signal >0x146517c30: a(n) MessageNotUnderstood > 0x146517818 s UndefinedObject(Object)>doesNotUnderstand: new >0x1186898e0: a(n) UndefinedObject > 0x146517c78 s >ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: >a(n) ProcessorScheduler > 0x1465180e8 s ECToolSet class(StandardToolSet >class)>handleError: 0x1195d6c88: a(n) ECToolSet > 0x146518500 s ToolSet class>handleError: 0x118a26690: a(n) >ToolSet > 0x146518900 s UnhandledError>defaultAction 0x146518e50: a(n) >UnhandledError > 0x146518d98 s UndefinedObject>handleSignal: 0x1186898e0: >a(n) UndefinedObject > 0x1465191d0 s UnhandledError(Exception)>signal 0x146518e50: >a(n) UnhandledError > 0x146519668 s UnhandledError class>signalForException: >0x118a27df8: a(n) UnhandledError > 0x146519aa0 s MessageNotUnderstood(Error)>defaultAction >0x1465185b8: a(n) MessageNotUnderstood > 0x146519f38 s MessageNotUnderstood>defaultAction >0x1465185b8: a(n) MessageNotUnderstood > 0x14651a370 s UndefinedObject>handleSignal: 0x1186898e0: >a(n) UndefinedObject > 0x1465189b8 s MessageNotUnderstood(Exception)>signal >0x1465185b8: a(n) MessageNotUnderstood > 0x1465181a0 s UndefinedObject(Object)>doesNotUnderstand: new >0x1186898e0: a(n) UndefinedObject > 0x146518600 s >ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: >a(n) ProcessorScheduler > 0x146518a70 s ECToolSet class(StandardToolSet >class)>handleError: 0x1195d6c88: a(n) ECToolSet > 0x146518e88 s ToolSet class>handleError: 0x118a26690: a(n) >ToolSet > 0x146519288 s UnhandledError>defaultAction 0x1465197d8: a(n) >UnhandledError > 0x146519720 s UndefinedObject>handleSignal: 0x1186898e0: >a(n) UndefinedObject > 0x146519b58 s UnhandledError(Exception)>signal 0x1465197d8: >a(n) UnhandledError > 0x146519ff0 s UnhandledError class>signalForException: >0x118a27df8: a(n) UnhandledError > 0x14651a428 s MessageNotUnderstood(Error)>defaultAction >0x146518f40: a(n) MessageNotUnderstood > 0x14651a8c0 s MessageNotUnderstood>defaultAction >0x146518f40: a(n) MessageNotUnderstood > 0x14651acf8 s UndefinedObject>handleSignal: 0x1186898e0: >a(n) UndefinedObject > 0x146519340 s MessageNotUnderstood(Exception)>signal >0x146518f40: a(n) MessageNotUnderstood > 0x146518b28 s UndefinedObject(Object)>doesNotUnderstand: new >0x1186898e0: a(n) UndefinedObject > 0x146518f88 s >ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: >a(n) ProcessorScheduler > 0x1465193f8 s ECToolSet class(StandardToolSet >class)>handleError: 0x1195d6c88: a(n) ECToolSet > 0x146519810 s ToolSet class>handleError: 0x118a26690: a(n) >ToolSet > 0x146519c10 s UnhandledError>defaultAction 0x14651a160: a(n) >UnhandledError > 0x14651a0a8 s UndefinedObject>handleSignal: 0x1186898e0: >a(n) UndefinedObject > 0x14651a4e0 s UnhandledError(Exception)>signal 0x14651a160: >a(n) UnhandledError > 0x14651a978 s UnhandledError class>signalForException: >0x118a27df8: a(n) UnhandledError > 0x14651adb0 s MessageNotUnderstood(Error)>defaultAction >0x1465198c8: a(n) MessageNotUnderstood > 0x14651b248 s MessageNotUnderstood>defaultAction >0x1465198c8: a(n) MessageNotUnderstood > 0x14651b680 s UndefinedObject>handleSignal: 0x1186898e0: >a(n) UndefinedObject > 0x146519cc8 s MessageNotUnderstood(Exception)>signal >0x1465198c8: a(n) MessageNotUnderstood > 0x1465194b0 s UndefinedObject(Object)>doesNotUnderstand: new >0x1186898e0: a(n) UndefinedObject > 0x146519910 s >ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: >a(n) ProcessorScheduler >_,,,^..^,,,_ >best, Eliot

-- _,,,^..^,,,_ best, Eliot

Hi Marcel, all,

Documenting code - intentions, implementation details and tradeoffs etc. is a huge topic for me :)))

In mainstream languages is easy to find an answer to almost anything at stackoverflow etc. but in Squeak, given the lack of both documentation and millions of users, the in-image comments (in methods, classes) are, at least for me, crucial.

A special chapter would be documenting **changes** to the existing code: should we document reasons for each change? Was it a bugfix or a new feature, was it related to changes in other methods? Which tests address the change/bug? Are there some motivation examples?

A nice example would be `Context>>contextEnsure:`. A bug was found, `ctxt push: nil.` line was added to fix the bug plus a detailed note including a forum link. Perfect :) Now there's another bug in the method and I'd like to continue documenting the bug in a similar way - see Kernel-jar.1553 in the Inbox.

At the same time, in my opinion, the initial comment about the intentions and peculiarities of #contextEnsure: is insufficient (which is still an understatemnt). A good comment with examples would save hours of reverse engineering and guesswork (like the comment describing reasons for adding the `ctxt push: nil.` line) . (This is not to blame anyone, it's a 20+ years old code and I'm so grateful for all the effort that made Squeak alive and kicking! It's to say that for newcomers more detailed comments would be so much helpful.)

But back to the low space watcher. I didn't intend to study or "fix" the low space watcher but I accidentally observed some troubling behavior that seemed associated with the low space watcher and it was hard to learn what it does, how and why (without prior knowledge in this area). This is especially helpful during troubleshooting which, I'm sure, you all know too well :)

However, the big question is **where** to place such info. My opinion is **in the image**, as close to the code as possible, ideally inside the methods or class comments. In case of the low space watcher, however, in which method when the code is scattered across multiple methods? I can see some useful info in #lowSpaceThreshold but the first method you come into contact with is #installLowSpaceWatcher and #lowSpaceWatcher which don't say much.

How about placing more detailed or summarizing information in more complex cases like the low space watcher (with no class "LowSpaceWatcher" to hold the "central" description) to "documentation" class methods and just referencing them in the code comments (like "see more in SmalltalkImage>>docLowSpaceWatcher").

I also considered Squeak Help but writing Help requires extra effort; writing good comments is already very taxing so I wouldn't make it even more difficult :)

I'm afraid you didn't expect such a long text... sorry ;/

Thanks, Jaromir

On 26-Jan-24 9:32:52 AM, "Taeumel, Marcel via Squeak-dev" squeak-dev@lists.squeakfoundation.org wrote:

Hi Jaromir --

What would be a good place to document the current state and intentions about low-space watcher? I mean, what would have helped you in this case?

Best, Marcel

...

Am 25.01.2024 21:26:04 schrieb Jaromir Matas mail@jaromir.net:

Hi Marcel,

On 25-Jan-24 10:28:07 AM, "Taeumel, Marcel via Squeak-dev" squeak-dev@lists.squeakfoundation.org wrote:

...

Hi Jaromir --

...

If you ask about the mechanism of the doubling behavior, I attached

the explanation to the changeset

Yes, you explained why a specific piece of code causes complications. Still, I want to understand, why this piece of code is relevant.

Hmm, I guess you're right my example is (most likely) not relevant in the current image. I can't replicate the behavior in a practical situation. May I ask you to remove System-jar.1367 from the Inbox then? Sorry for the noise :) Thanks! Jaromir

...

For example, I could present some Morphic code, executed in multiple processes, that would then cause issues only to argue that, for example, there should be more semaphores in Collections or Morphic routines. Yet, there are design choices and contracts we should adhere to. Also, to avoid unnecessary code. Here, Morphic itself is not thread-safe and that is okay. It's by design.

One should use #addDeferredUIMessage:, for example. :-)

Best, Marcel

...

Am 25.01.2024 10:16:26 schrieb Jaromir Matas mail@jaromir.net:

Hi Marcel,

...

How can this happen?

good question... I remember it (doubling the low space watcher process) bit me when I was experimenting with termination, I thought it was caused by a bug in my code but it turned out it was there before too :) Unfortunately I can't recall/find the exact situations. So I just posted it to make you aware.

...

Would you elaborate on why this can happen at all?

If you ask about the mechanism of the doubling behavior, I attached the explanation to the changeset (System-jar.1367):

"Workspce example: p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork Here both proceses are scheduled in the run queue, then p1 wakes up, starts the LowSpaceProcess termination and waits on a semaphore until the termination is finished. Before the LowSpaceProcess can proceed, process p2 wakes up and starts the LowSpaceProcess termination just like p1 and waits on a semaphore until the termination is finished. Finally the LowSpaceProcess wakes up, unwinds, unblocks p1 a p2 and terminates. p1 now continues and installs a new LowSpaceProcess and then p2 does the same. The result will be two processes running the #lowSpaceWatcher method."

best, Jaromir

On 25-Jan-24 8:44:35 AM, "Taeumel, Marcel via Squeak-dev" squeak-dev@lists.squeakfoundation.org wrote:

...

Hi Jaromir --

How can this happen? The only invocations of #installLowSpaceWatcher are in

Debugger >> proceed Debugger >> windowIsClosing

Both are per definition not thread-safe (because GUI/Tool related) and must never be called from arbitrary processes.

Would you elaborate on why this can happen at all?

...

p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork

Best, Marcel

...

Am 25.01.2024 00:21:43 schrieb Jaromir Matas mail@jaromir.net:

Hi Eliot, hi all,

just in case: speaking of the LowSpaceWatcher, there's a bug in case of concurrent invocations of the LowSpaceWatcher which causes multiple LowSpaceWatchers running at the same time... if you try in the workspace:

p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork

My suggestion is in System-jar.1367 (please disregard my note in the text about a bug in terminate, it's no longer relevant).

best, Jaromir

On 24-Jan-24 8:06:45 PM, "Eliot Miranda" eliot.miranda@gmail.com wrote:

>Hi Marcel, Hi All, > >On Fri, Jan 19, 2024 at 6:33 AM Taeumel, Marcel via Squeak-dev >squeak-dev@lists.squeakfoundation.org wrote: >>Hi Eliot -- >> >>Hmm... our CI can successfully update a 22095 to HEAD without >>issues. Once per day. Hmm... I am not sure why the Debugger >>would be involved in a regular update. The debugger cue ist just >>for invocation. Even existing debugger windows should keep on >>working. > >The reason my image locks up is that I'm playing with the low >space notification and consequently the low space notifier is >popping up frequently and I have to hit proceed. Some day I'll >revise the low space code appropriately; it's on my list. But >for the moment it shows that there's an update map missing. > >When I load Tools-ct.1244 before updating, with the image >starting at Tools-ct.1239, I am able to update. > >> >>What's the situation where the debugger wants to appear during >>an update? > >Frequent low space notifications. > >> >>Best, >>Marcel > >Cheers! > >>>Am 18.01.2024 23:36:44 schrieb Eliot Miranda >>>eliot.miranda@gmail.com: >>> >>>Hi All, >>> >>> I'm guessing there's some package ordering update missing >>>to ensure the new debugger cue is defined. I just attempted to >>>update a VMMaker image and got the following infinite >>>recursion: >>> >>> 0x1465199e8 s UndefinedObject>handleSignal: 0x1186898e0: >>>a(n) UndefinedObject >>> 0x146518030 s MessageNotUnderstood(Exception)>signal >>>0x146517c30: a(n) MessageNotUnderstood >>> 0x146517818 s UndefinedObject(Object)>doesNotUnderstand: >>>new 0x1186898e0: a(n) UndefinedObject >>> 0x146517c78 s >>>ProcessorScheduler>debugContext:title:full:contents: >>>0x11869afe0: a(n) ProcessorScheduler >>> 0x1465180e8 s ECToolSet class(StandardToolSet >>>class)>handleError: 0x1195d6c88: a(n) ECToolSet >>> 0x146518500 s ToolSet class>handleError: 0x118a26690: a(n) >>>ToolSet >>> 0x146518900 s UnhandledError>defaultAction 0x146518e50: >>>a(n) UnhandledError >>> 0x146518d98 s UndefinedObject>handleSignal: 0x1186898e0: >>>a(n) UndefinedObject >>> 0x1465191d0 s UnhandledError(Exception)>signal >>>0x146518e50: a(n) UnhandledError >>> 0x146519668 s UnhandledError class>signalForException: >>>0x118a27df8: a(n) UnhandledError >>> 0x146519aa0 s MessageNotUnderstood(Error)>defaultAction >>>0x1465185b8: a(n) MessageNotUnderstood >>> 0x146519f38 s MessageNotUnderstood>defaultAction >>>0x1465185b8: a(n) MessageNotUnderstood >>> 0x14651a370 s UndefinedObject>handleSignal: 0x1186898e0: >>>a(n) UndefinedObject >>> 0x1465189b8 s MessageNotUnderstood(Exception)>signal >>>0x1465185b8: a(n) MessageNotUnderstood >>> 0x1465181a0 s UndefinedObject(Object)>doesNotUnderstand: >>>new 0x1186898e0: a(n) UndefinedObject >>> 0x146518600 s >>>ProcessorScheduler>debugContext:title:full:contents: >>>0x11869afe0: a(n) ProcessorScheduler >>> 0x146518a70 s ECToolSet class(StandardToolSet >>>class)>handleError: 0x1195d6c88: a(n) ECToolSet >>> 0x146518e88 s ToolSet class>handleError: 0x118a26690: a(n) >>>ToolSet >>> 0x146519288 s UnhandledError>defaultAction 0x1465197d8: >>>a(n) UnhandledError >>> 0x146519720 s UndefinedObject>handleSignal: 0x1186898e0: >>>a(n) UndefinedObject >>> 0x146519b58 s UnhandledError(Exception)>signal >>>0x1465197d8: a(n) UnhandledError >>> 0x146519ff0 s UnhandledError class>signalForException: >>>0x118a27df8: a(n) UnhandledError >>> 0x14651a428 s MessageNotUnderstood(Error)>defaultAction >>>0x146518f40: a(n) MessageNotUnderstood >>> 0x14651a8c0 s MessageNotUnderstood>defaultAction >>>0x146518f40: a(n) MessageNotUnderstood >>> 0x14651acf8 s UndefinedObject>handleSignal: 0x1186898e0: >>>a(n) UndefinedObject >>> 0x146519340 s MessageNotUnderstood(Exception)>signal >>>0x146518f40: a(n) MessageNotUnderstood >>> 0x146518b28 s UndefinedObject(Object)>doesNotUnderstand: >>>new 0x1186898e0: a(n) UndefinedObject >>> 0x146518f88 s >>>ProcessorScheduler>debugContext:title:full:contents: >>>0x11869afe0: a(n) ProcessorScheduler >>> 0x1465193f8 s ECToolSet class(StandardToolSet >>>class)>handleError: 0x1195d6c88: a(n) ECToolSet >>> 0x146519810 s ToolSet class>handleError: 0x118a26690: a(n) >>>ToolSet >>> 0x146519c10 s UnhandledError>defaultAction 0x14651a160: >>>a(n) UnhandledError >>> 0x14651a0a8 s UndefinedObject>handleSignal: 0x1186898e0: >>>a(n) UndefinedObject >>> 0x14651a4e0 s UnhandledError(Exception)>signal >>>0x14651a160: a(n) UnhandledError >>> 0x14651a978 s UnhandledError class>signalForException: >>>0x118a27df8: a(n) UnhandledError >>> 0x14651adb0 s MessageNotUnderstood(Error)>defaultAction >>>0x1465198c8: a(n) MessageNotUnderstood >>> 0x14651b248 s MessageNotUnderstood>defaultAction >>>0x1465198c8: a(n) MessageNotUnderstood >>> 0x14651b680 s UndefinedObject>handleSignal: 0x1186898e0: >>>a(n) UndefinedObject >>> 0x146519cc8 s MessageNotUnderstood(Exception)>signal >>>0x1465198c8: a(n) MessageNotUnderstood >>> 0x1465194b0 s UndefinedObject(Object)>doesNotUnderstand: >>>new 0x1186898e0: a(n) UndefinedObject >>> 0x146519910 s >>>ProcessorScheduler>debugContext:title:full:contents: >>>0x11869afe0: a(n) ProcessorScheduler >>>_,,,^..^,,,_ >>>best, Eliot >> > > >-- >_,,,^..^,,,_ >best, Eliot

+1 on this thread.I concur that HelpTopic creation by hand is fatally iniffecient and painful, however, it can be generated programmaticaly, which I do in Doc...a naive attempt to solve this problem.I agree that the documentation must be in image . Furthermore, if it is in-image it is automatically available on the web via Seaside. Too, the XTreams parsing can create other documentation formats and if it can do that, it can build the indexing database...I also agree that AI should do this.  Alas, I do not have the bandwidth to learn ai.However, what are the steps to building an indexing database? ( I am thinking Postgres here..)what does the schema look like?what gets put where?cordially. ---- On Fri, 26 Jan 2024 12:26:05 -0500 eliot.miranda@gmail.com wrote ----Hi Jaromir,On Jan 26, 2024, at 8:55 AM, Jaromir Matas mail@jaromir.net wrote:

Hi Marcel, all,Documenting code - intentions, implementation details and tradeoffs etc. is a huge topic for me :))) In mainstream languages is easy to find an answer to almost anything at stackoverflow etc. but in Squeak, given the lack of both documentation and millions of users, the in-image comments (in methods, classes) are, at least for me, crucial. A special chapter would be documenting **changes** to the existing code: should we document reasons for each change? Was it a bugfix or a new feature, was it related to changes in other methods? Which tests address the change/bug? Are there some motivation examples? A nice example would be `Context>>contextEnsure:`. A bug was found, `ctxt push: nil.` line was added to fix the bug plus a detailed note including a forum link. Perfect :) Now there's another bug in the method and I'd like to continue documenting the bug in a similar way - see Kernel-jar.1553 in the Inbox. At the same time, in my opinion, the initial comment about the intentions and peculiarities of #contextEnsure: is insufficient (which is still an understatemnt). A good comment with examples would save hours of reverse engineering and guesswork (like the comment describing reasons for adding the `ctxt push: nil.` line) . (This is not to blame anyone, it's a 20+ years old code and I'm so grateful for all the effort that made Squeak alive and kicking! It's to say that for newcomers more detailed comments would be so much helpful.) But back to the low space watcher. I didn't intend to study or "fix" the low space watcher but I accidentally observed some troubling behavior that seemed associated with the low space watcher and it was hard to learn what it does, how and why (without prior knowledge in this area). This is especially helpful during troubleshooting which, I'm sure, you all know too well :) However, the big question is **where** to place such info. My opinion is **in the image**, as close to the code as possible, ideally inside the methods or class comments. In case of the low space watcher, however, in which method when the code is scattered across multiple methods? I can see some useful info in #lowSpaceThreshold but the first method you come into contact with is #installLowSpaceWatcher and #lowSpaceWatcher which don't say much. How about placing more detailed or summarizing information in more complex cases like the low space watcher (with no class "LowSpaceWatcher" to hold the "central" description) to "documentation" class methods and just referencing them in the code comments (like "see more in SmalltalkImage>>docLowSpaceWatcher"). I also considered Squeak Help but writing Help requires extra effort; writing good comments is already very taxing so I wouldn't make it even more difficult :) I'm afraid you didn't expect such a long text... sorry ;/ But that’s to be expected given that you raise a critical issue. While being able to browse the images senders & implementors is powerful it is insufficient. Things that would be powerful to have in-image would be:- being able to find the Monticello package commits corresponding to the current and previous versions of a method- being able to find message threads on squeak-dev et al referring to specific methods, packages, and commits- being able to find the help topic(s) that mention a specific method, selector, etcNow Chris Muller has already done part of this with the commits database indexing commits by method version, but it’s not in every package and is an add-on, not in the base system.This is fundamentally about indexing, which is fundamentally about comprehension and search. Sounds like an ideal project for someone to use ChatGPT-like capabilities to build the indexing database.. Thanks, JaromirEliot_,,,^..^,,,_ (phone)

On 26-Jan-24 9:32:52 AM, "Taeumel, Marcel via Squeak-dev" squeak-dev@lists.squeakfoundation.org wrote:

Hi Jaromir --

What would be a good place to document the current state and intentions about low-space watcher? I mean, what would have helped you in this case?

Best, Marcel

Am 25.01.2024 21:26:04 schrieb Jaromir Matas mail@jaromir.net:

Hi Marcel,

On 25-Jan-24 10:28:07 AM, "Taeumel, Marcel via Squeak-dev" squeak-dev@lists.squeakfoundation.org wrote:

Hi Jaromir --

If you ask about the mechanism of the doubling behavior, I attached the explanation to the changeset

Yes, you explained why a specific piece of code causes complications. Still, I want to understand, why this piece of code is relevant.

Hmm, I guess you're right my example is (most likely) not relevant in the current image. I can't replicate the behavior in a practical situation. May I ask you to remove System-jar.1367 from the Inbox then? Sorry for the noise :)

Thanks!

Jaromir

For example, I could present some Morphic code, executed in multiple processes, that would then cause issues only to argue that, for example, there should be more semaphores in Collections or Morphic routines. Yet, there are design choices and contracts we should adhere to. Also, to avoid unnecessary code. Here, Morphic itself is not thread-safe and that is okay. It's by design.

One should use #addDeferredUIMessage:, for example. :-)

Best, Marcel

Am 25.01.2024 10:16:26 schrieb Jaromir Matas mail@jaromir.net:

Hi Marcel,

How can this happen?

good question... I remember it (doubling the low space watcher process) bit me when I was experimenting with termination, I thought it was caused by a bug in my code but it turned out it was there before too :) Unfortunately I can't recall/find the exact situations. So I just posted it to make you aware.

Would you elaborate on why this can happen at all?

If you ask about the mechanism of the doubling behavior, I attached the explanation to the changeset (System-jar.1367):

"Workspce example:     p1 := [Smalltalk installLowSpaceWatcher] fork.      p2 := [Smalltalk installLowSpaceWatcher] fork Here both proceses are scheduled in the run queue, then p1 wakes up, starts the LowSpaceProcess   termination and waits on a semaphore until the termination is finished. Before the LowSpaceProcess

can proceed, process p2 wakes up and starts the LowSpaceProcess termination just like p1 and waits

on a semaphore until the termination is finished. Finally the LowSpaceProcess wakes up, unwinds,

unblocks p1 a p2 and terminates. p1 now continues and installs a new LowSpaceProcess and then

p2 does the same. The result will be two processes running the #lowSpaceWatcher method."

best, Jaromir

On 25-Jan-24 8:44:35 AM, "Taeumel, Marcel via Squeak-dev" squeak-dev@lists.squeakfoundation.org wrote:

Hi Jaromir --

How can this happen? The only invocations of #installLowSpaceWatcher are in

Debugger >> proceed Debugger >> windowIsClosing

Both are per definition not thread-safe (because GUI/Tool related) and must never be called from arbitrary processes.

Would you elaborate on why this can happen at all?

p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork

Best, Marcel

Am 25.01.2024 00:21:43 schrieb Jaromir Matas mail@jaromir.net:

Hi Eliot, hi all,

just in case: speaking of the LowSpaceWatcher, there's a bug in case of concurrent invocations of the LowSpaceWatcher which causes multiple LowSpaceWatchers running at the same time... if you try in the workspace:

p1 := [Smalltalk installLowSpaceWatcher] fork. p2 := [Smalltalk installLowSpaceWatcher] fork

My suggestion is in System-jar.1367 (please disregard my note in the text about a bug in terminate, it's no longer relevant).

best, Jaromir

On 24-Jan-24 8:06:45 PM, "Eliot Miranda" eliot.miranda@gmail.com wrote:

Hi Marcel, Hi All,

On Fri, Jan 19, 2024 at 6:33 AM Taeumel, Marcel via Squeak-dev squeak-dev@lists.squeakfoundation.org wrote:

Hi Eliot --

Hmm... our CI can successfully update a 22095 to HEAD without issues. Once per day. Hmm... I am not sure why the Debugger would be involved in a regular update. The debugger cue ist just for invocation. Even existing debugger windows should keep on working.

The reason my image locks up is that I'm playing with the low space notification and consequently the low space notifier is popping up frequently and I have to hit proceed.  Some day I'll revise the low space code appropriately; it's on my list.  But for the moment it shows that there's an update map missing.

When I load Tools-ct.1244 before updating, with the image starting at Tools-ct.1239, I am able to update.

What's the situation where the debugger wants to appear during an update?

Frequent low space notifications.

Best, Marcel

Cheers!

Am 18.01.2024 23:36:44 schrieb Eliot Miranda eliot.miranda@gmail.com:

Hi All,

I'm guessing there's some package ordering update missing to ensure the new debugger cue is defined.  I just attempted to update a VMMaker image and got the following infinite recursion:

0x1465199e8 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject      0x146518030 s MessageNotUnderstood(Exception)>signal 0x146517c30: a(n) MessageNotUnderstood      0x146517818 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject      0x146517c78 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler      0x1465180e8 s ECToolSet class(StandardToolSet class)>handleError: 0x1195d6c88: a(n) ECToolSet      0x146518500 s ToolSet class>handleError: 0x118a26690: a(n) ToolSet      0x146518900 s UnhandledError>defaultAction 0x146518e50: a(n) UnhandledError      0x146518d98 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject      0x1465191d0 s UnhandledError(Exception)>signal 0x146518e50: a(n) UnhandledError      0x146519668 s UnhandledError class>signalForException: 0x118a27df8: a(n) UnhandledError      0x146519aa0 s MessageNotUnderstood(Error)>defaultAction 0x1465185b8: a(n) MessageNotUnderstood      0x146519f38 s MessageNotUnderstood>defaultAction 0x1465185b8: a(n) MessageNotUnderstood      0x14651a370 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject      0x1465189b8 s MessageNotUnderstood(Exception)>signal 0x1465185b8: a(n) MessageNotUnderstood      0x1465181a0 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject      0x146518600 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler      0x146518a70 s ECToolSet class(StandardToolSet class)>handleError: 0x1195d6c88: a(n) ECToolSet      0x146518e88 s ToolSet class>handleError: 0x118a26690: a(n) ToolSet      0x146519288 s UnhandledError>defaultAction 0x1465197d8: a(n) UnhandledError      0x146519720 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject      0x146519b58 s UnhandledError(Exception)>signal 0x1465197d8: a(n) UnhandledError      0x146519ff0 s UnhandledError class>signalForException: 0x118a27df8: a(n) UnhandledError      0x14651a428 s MessageNotUnderstood(Error)>defaultAction 0x146518f40: a(n) MessageNotUnderstood      0x14651a8c0 s MessageNotUnderstood>defaultAction 0x146518f40: a(n) MessageNotUnderstood      0x14651acf8 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject      0x146519340 s MessageNotUnderstood(Exception)>signal 0x146518f40: a(n) MessageNotUnderstood      0x146518b28 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject      0x146518f88 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler      0x1465193f8 s ECToolSet class(StandardToolSet class)>handleError: 0x1195d6c88: a(n) ECToolSet      0x146519810 s ToolSet class>handleError: 0x118a26690: a(n) ToolSet      0x146519c10 s UnhandledError>defaultAction 0x14651a160: a(n) UnhandledError      0x14651a0a8 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject      0x14651a4e0 s UnhandledError(Exception)>signal 0x14651a160: a(n) UnhandledError      0x14651a978 s UnhandledError class>signalForException: 0x118a27df8: a(n) UnhandledError      0x14651adb0 s MessageNotUnderstood(Error)>defaultAction 0x1465198c8: a(n) MessageNotUnderstood      0x14651b248 s MessageNotUnderstood>defaultAction 0x1465198c8: a(n) MessageNotUnderstood      0x14651b680 s UndefinedObject>handleSignal: 0x1186898e0: a(n) UndefinedObject      0x146519cc8 s MessageNotUnderstood(Exception)>signal 0x1465198c8: a(n) MessageNotUnderstood      0x1465194b0 s UndefinedObject(Object)>doesNotUnderstand: new 0x1186898e0: a(n) UndefinedObject      0x146519910 s ProcessorScheduler>debugContext:title:full:contents: 0x11869afe0: a(n) ProcessorScheduler

_,,,^..^,,,_

best, Eliot

"The hard part of creating documentation is not the tools "this is false.there are mega- tons of Squeak documentation out there and the prtcentage created by the Help tools approaches zero."You cannot auto-generate sensible documentation."false.  the AI  out there is eminently sensible and will only get better. "Simply gathering up class and/or method comments is virtually useless, unless by chance those comments are very well written. Which in general is not the case."correctseveral  things here...Gathering up in-image documention does border on useless, but bringing out-image information in image is doable...squeakers do this everyday.Associating/ Inserting out-image documentation with/ Into  the relevant in-image stuff , given the wonders we are seeing with AI, seems self evidently doable to me ( Doc was/is a human attempt at this task)"Which in general is not the case."....there is a reason for this....coders in flow have to break flow to document. i,e coding and writing about coding are fundamentally different mental activities...one might as well ask coders to paint portraiture as documentation...you will get the same result ...no paintings.Regarding indexing in-image,   from my very limited reading on LLM , the datasets are huge....furthermore, to say that this capability should be in-image is to say that all of monticello and git repos must be in image...which is empircly false.t ---- On Fri, 26 Jan 2024 18:29:21 -0500 tim@rowledge.org wrote ----

I concur that HelpTopic creation by hand is fatally iniffecient and painful, however, it can be generated programmaticaly, which I do in Doc...a naive attempt to solve this problem.

You cannot auto-generate sensible documentation. Simply gathering up class and/or method comments is virtually useless, unless by chance those comments are very well written. Which in general is not the case.

And even then it would be quite important to have the method comments arranged in some order that helps make narrative sense.

And even then, class comments should be about the class (duh!) and that leaves out the whole question of *system* documentation. Which is much more important because that is where the functioning of the group(s) of classes gets explained, and without that explanation you are in real trouble.

A method comment should explain what the method is for, and ought to mention any known limitations or problems, should reference any standards or papers relating to it, etc. Comments within the code should help elucidate what the code is *meant* to be doing. Yes, code tells you what it does; it does not tell you what was intended. I promise you, the two often diverge significantly. Consider the careful comments in LargePositiveInteger>>#digitMul33: etc.

A class comment should give some outline of what the class is about so yo can quickly get a higher level idea. It should mention any known algorithms introduced or papers the ideas are based on so that debugging and improving can be done more effectively. People might need to work out what is going on 25 years later!

Somewhere, there has to be documentation about bigger things; how to connect to & use a database, how to use the morphs for XX, how to handle code management, dealing with bugs, etc etc. That is a very good thing it have in the HelpBrowser world, and probably *also* the swiki. We do have ways to read swiki pages into the Help browsers but I suspect it could be improved a lot, as indeed could the swiki content.

The HelpBrowser seems to be a decent tool these days; you can edit pages, add new ones and so forth. The hard part of creating documentation is not the tools - even using ancient junk like vi can be survived - but working out what to say, making it make sense, making is decently complete, keeping it up to date. Good tutorial type content is important too, and that can be a real pain to keep up to date.

I also agree that AI should do this. Alas, I do not have the bandwidth to learn ai.

I really wouldn't consider any current AI to be any good for this. You'll get the Boris Johnson version, blather and gibberish that looks good for a few moments until you spot something you do already know about and that it has made an utter hash of.

However, what are the steps to building an indexing database? ( I am thinking Postgres here..)

I doubt postgreSQL is appropriate here, fine DB though it is. For a start, it's not in the image if it's in a DB. One might consider Magma if you really need to keep info outside the image. One might consider sets of imagesegments as away of keeping help pages or books; they load very fast.

Strings in-image would surely be the fastest thing to search, especially with clever indexing algorithms. Yes, it would increase the size of the image, maybe by a lot - but for a development image where you want tools and support, why would that be a problem? Does it really matter if a full-fat Squeak developer image is several gigabytes in size? Take a look at how big the packages for some very minor tools are these days. For comparison, some work related Squeak images with seaside, refactoring browser, postgresql, sixx, a whole raft of specific code & data, and still with EToys & the games etc, is 130Mb. When running it has never gone past 200Mb working set. I could run 30 or more simultaneously on a Pi without using any virtual memory. Pfui!

tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Never trust a computer you can't lift.

HiI use READMEfoo methods as a means of personal documentation all the time within specific method categories.I strongly doubt that a README method category would be of use because of the bandwith needed to place the current README in context.In Emacs, I can get help on the current thing I am looking at .WTF? ....now that! is a great method to send to a class to have  to answer all our questions in real time.[ ]  wtf?"A block can be thought of as an anonymous function, ....see terse information <here> detailed infirmation <here> or browse the wiki <here>{1 .  2 . 3}  wtf?Transcript wtf?1 + 2 * 3 wtf?etc...'wtf? ' the message centric help system.should prove popular on y combinator... ---- On Sat, 27 Jan 2024 13:46:39 -0500 mail@jaromir.net wrote ----Hi all,

At the moment I'm more worried about the contents than the form. Without contents even a perfect form is empty.

I keep notes about every difficult method I come across (I'd forget otherwise). Placing detailed notes directly in methods doesn't feel right - it may (1) clutter the methods (2) create long chains of versions when updating the comments frequently. My idea - at least before a better form is available - would be to agree on using "documentation" class methods for each method to keep such extended comments, e.g. #docContextEnsure: under the Context class, which would allow (potentially) frequent updates without affecting the methods themselves. It would also not discourage one from writing the comments because deleting comments wouldn't leave a trace in the methods history.

Also, if in the future someone created an intelligent Help system based on whichever technology (AI, DB, Help, Swiki) it would be easy to access the contents of the documentation methods or simply delete them all to cleanup the system.

I can see one potential problem though - access rights to save to Trunk. Anytime I create or update a comment, who's going to save it to the Trunk...

About AI: I personally very much like consulting with ChatGPT about mainstream topics like OOP, FP etc because it has a vast knowledge base available. Of course the information provided is often inaccurate and controversial, sometimes I even make the chat bot reverse its "opinion" by 180 degrees but that shouldn't be surprising, I made the same experience with human teachers too :)) Unlike humans, however, the bot is always open for discussion and "willing" to consider my arguments, clever or stupid, and correct its "opinion" if necessary ;)

(To avoid confusion: By no means I suggest using AI to replace a teacher; but it's good as a sparring partner with a vast knowledge base; IMO it's like talking Wikipedia++ with some inference capabilities - can you trust it? Not really. Is it good to have? For sure.)

However, I have no opinion whether it's even feasible to feed ChatGPT with the existing (probably limited) Smalltalk data to be able to discuss at least on a similar level as e.g. in case of Python or Scala.

But back to Earth, before someone volunteers to spend hundreds (if not thousands) of hours on an intelligent help system, I'd like to start with simple steps like writing more detailed comments with examples to methods. Do you think "documentation" class methods is a way forward? At least as an interim solution?

(Tim) A method comment should explain what the method is for, and

ought to mention any known limitations or problems, should reference any standards or papers relating to it, etc. Comments within the code should help elucidate what the code is *meant* to be doing. Yes, code tells you what it does; it does not tell you what was intended. I promise you, the two often diverge significantly.

(Eliot) Things that would be powerful to have in-image would be:

- being able to find the Monticello package commits corresponding to the current and previous versions of a method - being able to find message threads on squeak-dev et al referring to specific methods, packages, and commits - being able to find the help topic(s) that mention a specific method, selector, etc

.... can't agree more!

Thanks, Jaromir

On 27-Jan-24 12:29:21 AM, "Tim Rowledge" tim@rowledge.org wrote:

...

I concur that HelpTopic creation by hand is fatally iniffecient and painful, however, it can be generated programmaticaly, which I do in Doc...a naive attempt to solve this problem.

You cannot auto-generate sensible documentation. Simply gathering up class and/or method comments is virtually useless, unless by chance those comments are very well written. Which in general is not the case.

And even then it would be quite important to have the method comments arranged in some order that helps make narrative sense.

And even then, class comments should be about the class (duh!) and that leaves out the whole question of *system* documentation. Which is much more important because that is where the functioning of the group(s) of classes gets explained, and without that explanation you are in real trouble.

A method comment should explain what the method is for, and ought to mention any known limitations or problems, should reference any standards or papers relating to it, etc. Comments within the code should help elucidate what the code is *meant* to be doing. Yes, code tells you what it does; it does not tell you what was intended. I promise you, the two often diverge significantly. Consider the careful comments in LargePositiveInteger>>#digitMul33: etc.

A class comment should give some outline of what the class is about so yo can quickly get a higher level idea. It should mention any known algorithms introduced or papers the ideas are based on so that debugging and improving can be done more effectively. People might need to work out what is going on 25 years later!

Somewhere, there has to be documentation about bigger things; how to connect to & use a database, how to use the morphs for XX, how to handle code management, dealing with bugs, etc etc. That is a very good thing it have in the HelpBrowser world, and probably *also* the swiki. We do have ways to read swiki pages into the Help browsers but I suspect it could be improved a lot, as indeed could the swiki content.

The HelpBrowser seems to be a decent tool these days; you can edit pages, add new ones and so forth. The hard part of creating documentation is not the tools - even using ancient junk like vi can be survived - but working out what to say, making it make sense, making is decently complete, keeping it up to date. Good tutorial type content is important too, and that can be a real pain to keep up to date.

...

I also agree that AI should do this. Alas, I do not have the bandwidth to learn ai.

I really wouldn't consider any current AI to be any good for this. You'll get the Boris Johnson version, blather and gibberish that looks good for a few moments until you spot something you do already know about and that it has made an utter hash of.

...

However, what are the steps to building an indexing database? ( I am thinking Postgres here..)

I doubt postgreSQL is appropriate here, fine DB though it is. For a start, it's not in the image if it's in a DB. One might consider Magma if you really need to keep info outside the image. One might consider sets of imagesegments as away of keeping help pages or books; they load very fast.

Strings in-image would surely be the fastest thing to search, especially with clever indexing algorithms. Yes, it would increase the size of the image, maybe by a lot - but for a development image where you want tools and support, why would that be a problem? Does it really matter if a full-fat Squeak developer image is several gigabytes in size? Take a look at how big the packages for some very minor tools are these days. For comparison, some work related Squeak images with seaside, refactoring browser, postgresql, sixx, a whole raft of specific code & data, and still with EToys & the games etc, is 130Mb. When running it has never gone past 200Mb working set. I could run 30 or more simultaneously on a Pi without using any virtual memory. Pfui!

tim

tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Never trust a computer you can't lift.

On 2024-01-26 08:32, Taeumel, Marcel via Squeak-dev wrote:

Hi Jaromir --

What would be a good place to document the current state and intentions about low-space watcher? I mean, what would have helped you in this case?

Best, Marcel

One way to document the low space watcher might be to turn it into a proper object with a good class comment.

The low space watcher process is currently attached to a class variable in class SmalltalkImage. This is reasonable and appropriate, but it would be just as reasonable to have a LowSpaceWatcher object attached to the class variable, and let the LowSpaceWatcher own the process and implement methods related for that process. The semaphore and primitive calls could also move from SmalltalkImage to LowSpaceWatcher.

Class SmalltalkImage has a lot of responsibilities already, so moving the responsibility for managing low space handling to a new class called LowSpaceWatcher might be a reasonable thing to do. A happy side effect would be to let this new class serve as documentation for the intended behavior of the low-space watcher process.

Dave

I decided to give this idea a try, and I think that it does help with the documentation. Please try loading System-dtl.1445 from the inbox, then evaluate "HelpBrowser openOn: LowSpaceWatcher". Let me know what you think of it :-)

Dave

On 2024-01-27 18:51, Jaromir Matas wrote:

Hi Dave,

Interesting, I wondered why the low space watcher doesn't have its class :)

best, Jaromir

On 27-Jan-24 4:53:59 AM, lewis@mail.msen.com wrote:

On 2024-01-26 08:32, Taeumel, Marcel via Squeak-dev wrote: Hi Jaromir

What would be a good place to document the current state and intentions about low-space watcher? I mean, what would have helped you in this case?

Best, Marcel

One way to document the low space watcher might be to turn it into a proper object with a good class comment.

The low space watcher process is currently attached to a class variable in class SmalltalkImage. This is reasonable and appropriate, but it would be just as reasonable to have a LowSpaceWatcher object attached to the class variable, and let the LowSpaceWatcher own the process and implement methods related for that process. The semaphore and primitive calls could also move from SmalltalkImage to LowSpaceWatcher.

Class SmalltalkImage has a lot of responsibilities already, so moving the responsibility for managing low space handling to a new class called LowSpaceWatcher might be a reasonable thing to do. A happy side effect would be to let this new class serve as documentation for the intended behavior of the low-space watcher process.

Dave

Hi Dave, one more note: I tried `LowSpaceWatcher default signalLowSpace` and it looks like a few more `self`s need to be replaced with `Smalltalk` in #lowSpaceWatcher - (changeset attached). Thanks, Jaromir

On 29-Jan-24 1:43:15 PM, "Jaromir Matas" mail@jaromir.net wrote:

Hi Dave,

I like this very much. It's so much clearer what's going on!

A small suggestion - how about to run LowSpaceWatcher>>#install critically to avoid the (theoretical) possibility to start two low space watcher processes simultaneously?

(Like if you run this in the workspace: [LowSpaceWatcher install] fork. [LowSpaceWatcher install] fork )

It's nothing critical but I'd like to know your opinion - please see System-jar.1446.

Thanks, Jaromir

On 29-Jan-24 2:18:40 AM, lewis@mail.msen.com wrote:

...

I decided to give this idea a try, and I think that it does help with the documentation. Please try loading System-dtl.1445 from the inbox, then evaluate "HelpBrowser openOn: LowSpaceWatcher". Let me know what you think of it :-)

Dave

On 2024-01-27 18:51, Jaromir Matas wrote:

...

Hi Dave,

Interesting, I wondered why the low space watcher doesn't have its class :)

best, Jaromir

On 27-Jan-24 4:53:59 AM, lewis@mail.msen.com wrote:

...

On 2024-01-26 08:32, Taeumel, Marcel via Squeak-dev wrote:

...

Hi Jaromir --

What would be a good place to document the current state and intentions about low-space watcher? I mean, what would have helped you in this case?

Best, Marcel

One way to document the low space watcher might be to turn it into a proper object with a good class comment.

The low space watcher process is currently attached to a class variable in class SmalltalkImage. This is reasonable and appropriate, but it would be just as reasonable to have a LowSpaceWatcher object attached to the class variable, and let the LowSpaceWatcher own the process and implement methods related for that process. The semaphore and primitive calls could also move from SmalltalkImage to LowSpaceWatcher.

Class SmalltalkImage has a lot of responsibilities already, so moving the responsibility for managing low space handling to a new class called LowSpaceWatcher might be a reasonable thing to do. A happy side effect would be to let this new class serve as documentation for the intended behavior of the low-space watcher process.

Dave

Hi Dave,

On 30-Jan-24 12:10:15 AM, lewis@mail.msen.com wrote:

Hi Jaromir,

You are right that I missed those updates in the lowSpaceWatcher method, thank you! I think I see at least one more similar slip

You probably mean `^ LowSpaceWatcher installLowSpaceWatcher`... I missed that one too :)

Q: You left #lowSpaceThreshold in SmalltalkImage... intentionally? (I'm just curious, still learning good OOP practice)

, I'll review it more carefully before posting anything further.

Regarding protecting the #install method, I can't really think of a case where this would be needed,

I can't either; the only reason I keep mentioning this "issue" is I had spent hours trying to find a bug in my code before realizing it was the low space watcher's unfortunate behavior. So I'm trying to (1) get it documented or (2) possibly "fixed" to save anyone from encountering this again.

Of course you're right the behavior is harmless from a functional point of view.

but you are much closer to the issues related to process scheduling so you may be better able than me to judge. The actual failure modes associated with having two instances of LowSpaceWatcher would not be serious. Only one of the instances would be functional (if you are lucky) and the other would be harmlessly broken.

My main goal in posting this was to see if it makes the low space mechanism easier to understand and to document. I think it does help in that regard,

Indeed!

so maybe after a bit of cleanup we can move it to trunk.

Dave

On 2024-01-29 22:04, Jaromir Matas wrote:

...

Hi Dave, one more note: I tried `LowSpaceWatcher default signalLowSpace` and it looks like a few more `self`s need to be replaced with `Smalltalk` in #lowSpaceWatcher - (changeset attached). Thanks, Jaromir

On 29-Jan-24 1:43:15 PM, "Jaromir Matas" mail@jaromir.net wrote:

...

Hi Dave,

I like this very much. It's so much clearer what's going on!

A small suggestion - how about to run LowSpaceWatcher>>#install critically to avoid the (theoretical) possibility to start two low space watcher processes simultaneously?

(Like if you run this in the workspace: [LowSpaceWatcher install] fork. [LowSpaceWatcher install] fork )

It's nothing critical but I'd like to know your opinion - please see System-jar.1446.

Thanks, Jaromir

On 29-Jan-24 2:18:40 AM, lewis@mail.msen.com wrote:

...

I decided to give this idea a try, and I think that it does help with the documentation. Please try loading System-dtl.1445 from the inbox, then evaluate "HelpBrowser openOn: LowSpaceWatcher". Let me know what you think of it :-)

Dave

On 2024-01-27 18:51, Jaromir Matas wrote:

...

Hi Dave,

Interesting, I wondered why the low space watcher doesn't have its class :)

best, Jaromir

On 27-Jan-24 4:53:59 AM, lewis@mail.msen.com wrote:

...

On 2024-01-26 08:32, Taeumel, Marcel via Squeak-dev wrote: >Hi Jaromir -- > >What would be a good place to document the current state and >intentions about low-space watcher? I mean, what would have >helped you in this case? > >Best, >Marcel >

One way to document the low space watcher might be to turn it into a proper object with a good class comment.

The low space watcher process is currently attached to a class variable in class SmalltalkImage. This is reasonable and appropriate, but it would be just as reasonable to have a LowSpaceWatcher object attached to the class variable, and let the LowSpaceWatcher own the process and implement methods related for that process. The semaphore and primitive calls could also move from SmalltalkImage to LowSpaceWatcher.

Class SmalltalkImage has a lot of responsibilities already, so moving the responsibility for managing low space handling to a new class called LowSpaceWatcher might be a reasonable thing to do. A happy side effect would be to let this new class serve as documentation for the intended behavior of the low-space watcher process.

Dave

Hi Dave,

On 30-Jan-24 5:10:55 PM, lewis@mail.msen.com wrote:

On 2024-01-30 09:58, Jaromir Matas wrote:

...

On 30-Jan-24 12:10:15 AM, lewis@mail.msen.com wrote:

...

You are right that I missed those updates in the lowSpaceWatcher method, thank you! I think I see at least one more similar slip

You probably mean `^ LowSpaceWatcher installLowSpaceWatcher`... I missed that one too :)

Yes. Also, looking at the #lowSpaceWatcher method overall, it is rather long. I think it might easier to read (and debug) if it was split into some smaller methods that reflect the separate steps in the low space handling.

I've noticed the initial condition in #lowSpaceWatcher ``` Smalltalk garbageCollectMost <= Smalltalk lowSpaceThreshold ifTrue: [self garbageCollect <= Smalltalk lowSpaceThreshold ifTrue: [^ Beeper beep]]. ``` could perhaps be replaced with ``` Smalltalk okayToProceedEvenIfSpaceIsLow ifFalse: [^ self]. ``` (not sure about the beep vs confirm though)

...

Q: You left #lowSpaceThreshold in SmalltalkImage... intentionally? (I'm just curious, still learning good OOP practice)

For this exercise, I left lowSpaceThreshold in class SmalltalkImage because it seems to be naturally associated with the image itself, as opposed to the LowSpaceWatcher mechanism. It is also used by DisplayScreen and Debugger (indirectly) so it seemed most natural to let SmalltalkImage be responsible for knowing its low space threshold, and let the low space watcher ask the image about it.

That makes sense, I've overlooked the senders, thanks for helping me understand! Does the same reasoning apply to leaving #okayToProceedEvenIfSpaceIsLow under SmalltalkImage? Thanks again :)

...

...

, I'll review it more carefully before posting anything further.

Regarding protecting the #install method, I can't really think of a case where this would be needed,

I can't either; the only reason I keep mentioning this "issue" is I had spent hours trying to find a bug in my code before realizing it was the low space watcher's unfortunate behavior. So I'm trying to (1) get it documented or (2) possibly "fixed" to save anyone from encountering this again.

Of course you're right the behavior is harmless from a functional point of view.

I would be inclined not to add a mutex unless there is a real need for it. I worry that someone reading the code might assume that the mutex was required for function reasons, which might make the low space watcher harder to understand.

Ok, readability wins :) I admit unless I can provide a convincing example, it's just clouding the code.

But I don't think that I really understand the concurrency issue that you were seeing, so I might be wrong.

...

...

but you are much closer to the issues related to process scheduling so you may be better able than me to judge. The actual failure modes associated with having two instances of LowSpaceWatcher would not be serious. Only one of the instances would be functional (if you are lucky) and the other would be harmlessly broken.

My main goal in posting this was to see if it makes the low space mechanism easier to understand and to document. I think it does help in that regard,

Indeed!

Good :-)

Dave