[OpenSmalltalk/opensmalltalk-vm] macOS --metal backwards compatibility regarding primitives 126, 127, 231 (Issue #627)
As of https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/2d7105db755928fd908..., there can be flickering or invisible Morphic dragging if no image-side composition buffer is used.
In Squeak, that composition buffer is either already used or can be activated for older images by hand via:
```Smalltalk PasteUpMorph disableDeferredUpdates: true. Project allMorphicProjects do: [:ea | ea world canvas: nil]. ```
Also see: - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037907.html - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037888.html - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037891.html - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037897.html
For the macOS `--metal` backend, one might actually implement that extra video buffer here: https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/2d7105db755928fd90823...
And then read from that buffer and **not** the `displayBits` here: https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/2d7105db755928fd90823...
...but **only if** the `deferDisplayUpdates` flag is `false` again.
**NOTE THAT** the old approach to always commit content from `displayBits` to the graphics backend entailed a *serious performance regression* as it slows down the interpreter loop to a maximum of 60 FPS, regardless of other stuff waiting in the image side. That is, primitives 127 and 231 were awfully slow and blocking on macOS. At least that got better in https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/2d7105db755928fd908...
-------------
Another, more flexible, approach would be to fix **primitive 126** to actually *fail* if a certain graphics backend does not support deferred updates. This might work if platform-specific code would be able to let that primitive fail. See: - `StackInterpreterPrimitives >> primitiveDeferDisplayUpdates` in VMMaker
This would work for recent versions of Cuis. However, the existing code in older Squeak images did not correctly invalidate the canvas and thus was not even able to flip between image-side and vm-side composition buffer for different results of primitive 126. Thus, such a fix would also only work from here on for future versions of Squeak and Cuis.
http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037922.html
As I said before, this is also a good solution for Cuis. So, my suggestion, is to simply make primitive 126 fail on MacOs.
This wouldn't affect older images checking for the platform (as you say, for a performance improvement), but would fix older "untweaked" images, solving any concern about strict back compatibility for the VM.
Well, this would be trivial to make backwards compatible were the necessary imager fix only PasteUpMorph disableDeferredUpdates: true. Here's how. 1. introduce a new flag bit in the image flags (alongside full screen, preemption yields, send wheel events, do mixed arithmetic, et al). This bit is unset in old images 2. have the beDIsplay primitive #102 interrogate this bit and if unset invoke primitiveDeferDisplayUpdates's innards with false as the effective argument 3. set the flag bit in new images
Now if there's a way of side effecting the canvas in old images we have a solution, no?
I don't really agree that having a flag bit because of _Morphic_ is not overkill. Letting 126 fail on OS X makes more sense to me…
So, if I see that correctly, this is the table of affected user bases
| | Win | Mac | Linux | |---| :---: | :---: | :---: | | Squeak <=5.3, disableDeferredUpdates: true | ✅ | ✅ | ✅ | | Squeak <=5.3, disableDeferredUpdates: false | ✅ | ❌ | ✅ | | Squeak <=5.3, updated via update map | ✅ | ✅ | ✅ | | Squeak 6 alpha, not updated | do not use | do not use | do not use | | Squeak 6 alpha, updated | ✅ | ✅ | ✅ |
Please note that the Mac user base is by far the smallest nowadays.
Well, this would be trivial to make backwards compatible were the necessary old image fix only
PasteUpMorph disableDeferredUpdates: true.
I think it might already be since the morphic canvas is set to `nil` when shutting down the image.
introduce a new flag bit in the image flags
Wouldn't this decrease compatibility of newer images with older VMs?
On Mon, May 2, 2022 at 6:26 AM Marcel Taeumel notifications@github.com wrote:
Well, this would be trivial to make backwards compatible were the necessary old image fix only PasteUpMorph disableDeferredUpdates: true.
I think it might already be since the morphic canvas is set to nil when shutting down the image.
introduce a new flag bit in the image flags
Wouldn't this decrease compatibility of newer images with older VMs?
But that's not a relevant concern. A new image can always be run on an updated VM. The issue is to *not break older images*. A VM on its own is not an application; it is analogous to a real piece of hardware. An image, on the other hand, *is* an application. New VMs should *never* change the semantics of older images with which they're compatible. That's akin to a newer processor running older binaries incorrectly; entirely unacceptable.
Here's what the Go designers have to say about the issue in the latest CACM:
"A final kind of consistency is that of the language and libraries over time. For the first few years of Go's existence, we tinkered with and adjusted it in each weekly release. Users often had to change their programs when updating to a new Go version. Automated tools reduced the burden, but manual adjustments were also necessary. Starting with Go version 1, released in 2012, *we publicly committed to making only backwards-compatible changes to the language and standard library*, so that programs would continue running unchanged when compiled with newer Go versions. That commitment attracted industry and has encouraged not just long-lived engineering projects but also other efforts, such as books, training courses, and a thriving ecosystem of third-party packages." (my emphasis added)
This is a rational approach to backwards compatibility, and is one most Smalltalk VMs, and certainly all the ones I've been associated with, have maintained. It is OK to change the VM to no longer support older images; it is *not* ok to break older images in a VM upon which those images still run.
— Reply to this email directly, view it on GitHub https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/627#issuecomment-1114878968, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIJPEWYWNAVJSGIQGQEKFWTVH7JXTANCNFSM5URRNFFQ . You are receiving this because you are subscribed to this thread.Message ID: OpenSmalltalk/opensmalltalk-vm/issues/627/1114878968@github.com
Note that the very mechanism of "deferring display updates" and thus relying on a VM-side composition buffer could actually be a feature with a larger impact. However, I am not aware of a single project that relies on this other than Morphic window dragging. Many graphics-related Smalltalk projects are rather fiddling around with hardware acceleration via FFI or custom VM plugins. That is, the primitives 126, 127, and 231 are not that useful outside Morphic and MVC. And for those projects, BitBLT (i.e., #copyBits onto Form) might only be a fail-safe if anything.
Looking backwards at older images, macOS users can still (a) enable fast/ST80 window dragging or (b) manually enable the image-side composition buffer as described above. That is, if the users are even able to open their older at all, which is tricky for 32-bit images on a modern macOS version.
If the VM is hardware, then Squeak is not an application but an operating system. And not all operating systems run on new hardware. They might need patches. And that's okay. They need patches to make the actual applications in them work.
I think that you current perspective on compatibility between VM and Image is unnecessarily restrictive. The image is not an application but an operating system.
The issue is to *not break older images*.
We do not do that. They just open fine. They run just fine. You can do 99.9999% of what you could do before. Please argue at the level of Morphic window dragging. Otherwise we cannot make any progress on this matter. You are way too generic for this matter here.
---------------
On a different matter, the way somebody introduced OS event pumping during regular method invocation is a much more serious deal breaker. (I think that only the image should do that.) It was pure luck that that has worked until recently. Just take a look at the fix in `DisplayScreen >> #restore` and then tell me again that we have to go back to the seriously slow `--metal` backend from before just to make the calls to `ioProcessEvents` in `checkForEventsMayContextSwitch` work.
---------------
You should but you cannot always change the VM in isolation. It will work 99.999% of the time. But not always.
Well, this would be trivial to make backwards compatible were the necessary old image fix only `PasteUpMorph disableDeferredUpdates: true.`
I have no idea how you mean by that. Are you okay with running some code in an image to make it compatible or are you not okay with that?
For the sake of piece, I will invest some more time and look into how difficult it would be to let primitive 126 fail for platform-specific graphics backends.
On Fri, May 6, 2022 at 2:23 AM Marcel Taeumel ***@***.***> wrote:
If the VM is hardware, then Squeak is not an application but an operating system. And not all operating systems run on new hardware. They might need patches. And that's okay. They need patches to make the actual applications in them work.
I think that you current perspective on compatibility between VM and Image is unnecessarily restrictive. The image is not an application but an operating system.
An image may be include operating system (e.g. a trunk dev image), but is more. An image may be a deployed application (e.g. stripped of the IDE, or including a restricted IDE, e.g. Scratch). Vuewing an image as "an operating system" is vague. Do OS's include all the apps that can run on them? Not for me. So an image is potentially much more, and potentially much less, than an OS. In any case, it is something that expects certain semantics of the VM upon which it runs, and it is unreasonable and unacceptabvle for that VM to arbitrarily change its semantics, unless it is fixing bugs.
—
Reply to this email directly, view it on GitHub https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/627#issuecomment-1119423119, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADY5VUG3QJHIES2EPP7HX4DVITQKRANCNFSM5URRNFFQ . You are receiving this because you commented.Message ID: ***@***.***>
HI,
How about as a solution that we add a VM flag to enable the backwards compatibility for MacOS Metal? Make the default what works quickly on the current systems but, if someone has a problem it is possible to return to the old behaviour.
Then we can see how often it seems needed and possibly remove this in the future.
It is true that causing problems on old images is a problem and we have had a somewhat recent discussion with Ted? about this where some of his old images no longer run.
But... since we are talking about MacOS the bigger problem with MacOS is Apple's decision to disallow running 32 bit code, and, our 32 bit images requiring a 32 bit VM. And that was Teds problem with his old images.
We are a small group and there are limits as to what we can do. I think that we should try to work well on current systems even if it means that older systems stop working. It is sad, I have some of those older systems, but that is life..
cheers
bruce
On 2022-05-06T21:49:52.000+02:00, Eliot Miranda ***@***.***> wrote:
On Fri, May 6, 2022 at 2:23 AM Marcel Taeumel ***@***.***> wrote:
If the VM is hardware, then Squeak is not an application but an
operating
system. And not all operating systems run on new hardware. They
might need
patches. And that's okay. They need patches to make the actual
applications
in them work. I think that you current perspective on compatibility between VM
and Image
is unnecessarily restrictive. The image is not an application but
an
operating system.
An image may be include operating system (e.g. a trunk dev image), but is more. An image may be a deployed application (e.g. stripped of the IDE, or including a restricted IDE, e.g. Scratch). Vuewing an image as "an operating system" is vague. Do OS's include all the apps that can run on them? Not for me. So an image is potentially much more, and potentially much less, than an OS. In any case, it is something that expects certain semantics of the VM upon which it runs, and it is unreasonable and unacceptabvle for that VM to arbitrarily change its semantics, unless it is fixing bugs. —
Reply to this email directly, view it on GitHub
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/627#issuecomment-1119423119;,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADY5VUG3QJHIES2EPP7HX4DVITQKRANCNFSM5URRNFFQ;
. You are receiving this because you commented.Message ID: ***@***.***>
-- _,,,^..^,,,_ best, Eliot — Reply to this email directly, view it on GitHub [https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/627#issuecomment-11...], or unsubscribe [https://github.com/notifications/unsubscribe-auth/ACEXJIWH2IR3B6KWXFQ6Y5LVIV...]. You are receiving this because you are subscribed to this thread.Message ID:
***@***.***>
Done. See PR https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/631. If there are no objections, I will merge it and finish the OSVM release.
This along with VMMaker.oscog-mt.3184 looks like a straighforward solution to the problem.
On Fri, May 6, 2022 at 2:38 AM Marcel Taeumel ***@***.***> wrote:
The issue is to *not break older images*.
We do not do that. They just open fine. They run just fine. You can do 99.9999% of what you could do before. Please argue at the level of Morphic window dragging. Otherwise we cannot make any progress on this matter. You are way too generic for this matter here.
Ah, that's a good point. SO if it only affects interactive behaviour in an image which has an IDE I could live with the "you have to execute code to fix things" position. I'll not live happily, but I'll live :-)
On a different matter, the way somebody introduced OS event pumping during regular method invocation is a much more serious deal breaker. (I think that only the image should do that.) It was pure luck that that has worked until recently. Just take a look at the fix in DisplayScreen >> #restore and then tell me again that we have to go back to the seriously slow --metal backend from before just to make the calls to ioProcessEvents in checkForEventsMayContextSwitch work.
You should but you cannot always change the VM in isolation. It will work 99.999% of the time. But not always.
— Reply to this email directly, view it on GitHub https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/627#issuecomment-1119435768, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADY5VUCOOSAYYCNHQO57DYTVITSC3ANCNFSM5URRNFFQ . You are receiving this because you commented.Message ID: ***@***.***>
vm-dev@lists.squeakfoundation.org
-
David T Lewis -
edoneel
-
Eliot Miranda -
Eliot Miranda
-
Marcel Taeumel
-
Tobias Pape