Vm-dev September 2022

vm-dev@lists.squeakfoundation.org

11 participants
21 discussions

[OpenSmalltalk/opensmalltalk-vm] BitBlt rgbMul is broken for depth 1 and not correctly rounded above depth 4 (Issue #651)
by Nicolas Cellier 24 Nov '23

24 Nov '23

In depth 1, the resulting bits will always be 0. It's not a big problem because rgbMul is just a bitAnd operation at this depth. So a quick workaround would be to detect the case in BitBltSimulation destDepth = 1 ifTrue: [^self bitAnd: sourceWord with: destinationWord]. That would also accelerate the Bit BLock Transfer operation, so it's a good hack. But there is more. What we want is multiply ratios in interval [0,1]. dstRatio * srcRatio Our implementation is scaled ratio (scaled by `1 << nBits - 1`): src := (srcRatio * scale) rounded. dst := (dstRatio * scale) rounded. So what we want is: ((dst/scale) * (src/scale) * scale) rounded that is: (dst*src / (1<<nBits-1)) rounded Unfortunately, that's the other grief with the current implementation used for rounding: (dst+1)*(src+1) - 1 >> nBits It only equals correctly rounded operation for depths 2 and 4. For rounding we might use: (((dst/scale) * (src/scale) + 0.5) * scale) truncated. that is expressed with truncated division: dst*src + (scale+1//2) // scale So here is a nicer formulation for doing the job at any depth (including 5bits rgb channels for 16 bits depth) with correctly rounded division: aux := src * dst + (1 << (nBits - 1)). "add mid-scale for rounding" result := aux << (nBits - 1) + aux << (nBits -1). "divide by scale" This is because instead of dividing by scale, we can multiply by shifted inverse (sort of double precision), then shift right. (2 to: 32) allSatisfy: [:nBits | (1 << (nBits * 2) / (1 << nBits - 1)) rounded = (1 << nBits + 1)]. Multiplying by this inverse is easy and cheap: x * (1 << nBits + 1) = (x << nBits + x). And then applying the right shift `>> (2 * nBits)` is equivalent to: x >> nBits + x >> nBits. We must first add 0.5 (scaled), that is `src * dst + (1 << (nBits -1))` - our formulation of aux, and we're done. We verify: { (0 to: 1<<20-1) allSatisfy: [:i | (1<<9+i)>>10+ (1<<9+i)>>10 = (i/1023) rounded]. (0 to: 1<<18-1) allSatisfy: [:i | (1<<8+i)>>9+ (1<<8+i)>>9 = (i/511) rounded]. (0 to: 1<<16-1) allSatisfy: [:i | (1<<7+i)>>8+ (1<<7+i)>>8 = (i/255) rounded]. (0 to: 1<<14-1) allSatisfy: [:i | (1<<6+i)>>7+ (1<<6+i)>>7 = (i/127) rounded]. (0 to: 1<<12-1) allSatisfy: [:i | (1<<5+i)>>6+ (1<<5+i)>>6 = (i/63) rounded]. (0 to: 1<<10-1) allSatisfy: [:i | (1<<4+i)>>5+ (1<<4+i)>>5 = (i/31) rounded]. (0 to: 1<<8-1) allSatisfy: [:i | (1<<3+i)>>4+ (1<<3+i)>>4 = (i/15) rounded]. (0 to: 1<<6-1) allSatisfy: [:i | (1<<2+i)>>3+ (1<<2+i)>>3 = (i/7) rounded]. (0 to: 1<<4-1) allSatisfy: [:i | (1<<1+i)>>2+ (1<<1+i)>>2 = (i/3) rounded]. } allSatisfy: #yourself. The nice thing is that above down-scaling operation can be multiplexed. Suppose that we have p groups of 2*nBits `M` holding square-scale multiplication of each channel concatenated in a double-Word-Mul. doubleWordMul = Mp .... M5 M3 M1 Note we arrange to have odd channels in low word, and even channels in high word. We first form a `groupMask` on a word with (p+1)/2 groups of nBits alternating all one `i` and all zero `o`, `oioi...ioi`. channelMask := 1 << nBits - 1. groupMask := 0. 0 to: wordBits // (2 * nBits) do: [:i | groupMask = groupMask << (2 * nBits) + channelMask]. Where wordBits is the number of bits in a word (usually we want to operate on 32 bits words in BitBlt). We form the `doubleGroupMask` on a double-word with p groups of 2*nBits `oi`: doubleGroupMask := groupMask >> nBits. doubleGroupMask := doubleGroupMask << wordBits + groupMask. And we perform the division by scale: doubleWordMul := (doubleWordMul >> nBits bitAnd: doubleGroupMask) + doubleWord >> nBits bitAnd: doubleGroupMask. At this stage we obtain a double word containing scaled multiplicands interleaved with groups of nBits zeros: o mp ... o m3 o m1 Now the final result can be obtained by shifting back: doubleWordMul >> (wordBits - nBits) + (doubleWordMul bitAnd: groupMask) The only problem remaining is how to obtain the squared-scale multiplicands. It would be easy to form the alternate even-odd channels for each src and dst operands: doubleWordSrc := src >> nBits bitAnd: groupMask. doubleWordSrc := doubleWordSrc << wordBits + (src bitAnd: groupMask). doubleWordDst := dst >> nBits bitAnd: groupMask. doubleWordDst := doubleWordDst << wordBits + (dst bitAnd: groupMask). we now get `o sp ... o s3 o s1` and `0 dp ... o d3 o d1`, but we would now need a SIMD integer multiplication operating on groups of 2*nBits in parallel... We don't have that, at least in portable C code. So we still have to emulate it with a loop. half := 1 << (nBits - 1). shift := 0. doubleWordMul := 0 0 to: nChannels - 1 do: [:i | doubleWordMul := doubleWordMul + (((doubleWordSrc >> shift bitAnd: channelMask) * (doubleWordSrc >> shift bitAnd: channelMask) + half) << shift). shift := shift + nBits + nBits]. We know that each operation cannot overflow on upper neighbour group of 2*nBits, because the maximum value is: (1<<nBits-1) squared + (1 << (nBits-1)) = 1 << (2*nBits) - (2*(1<<nBits)) + (1 << (nBits-1)) - 1 < (1 << (2*nBits) - 1) It remains the odd case of 16 bits depth, which has 3 groups of 5 bits and a leading zero. I believe that above algorithm works without splitting in two half-words... To be tested. We have gathered the pieces for a correctly rounded almost-multiplexed rgbMul. Somehow have our cake and eat it too. -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/651 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/651(a)github.com>

2 4

[OpenSmalltalk/opensmalltalk-vm] Allow choosing an image via zenity when no arguments are provided (#408)
by Leon Matthes 05 Jun '23

05 Jun '23

You can view, comment on, or merge this pull request online at: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408 -- Commit Summary -- * Add file chooser dialog to squeak.sh.in * Add file chooser dialog to bin.squeak.sh.in -- File Changes -- M platforms/unix/config/bin.squeak.sh.in (13) M platforms/unix/config/squeak.sh.in (13) -- Patch Links -- https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408.patch https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408

4 8

[OpenSmalltalk/opensmalltalk-vm] Proposed Document Addition: riscv64 porting notes (PR #650)
by Ken Dickey 23 Jan '23

23 Jan '23

A slightly cleaned up version of what I poster earlier. Just a schematic description which may be of use down the line. [Doc only; no code changes] You can view, comment on, or merge this pull request online at: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/650 -- Commit Summary -- * proposed doc * nits * added ref -- File Changes -- A PortingNotesRISCV64.md (322) -- Patch Links -- https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/650.patch https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/650.diff -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/650 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/pull/650(a)github.com>

1 3

[OpenSmalltalk/opensmalltalk-vm] Document VM release process (Issue #637)
by Marcel Taeumel 05 Nov '22

05 Nov '22

Here is a draft. ----------- How to make a new RC of the OSVM: 1. Tag specific commit with its UTC timestamp, e.g., 202205110711 for May 11, 2022 at 7:11 2. Run all "Build for*" workflows manually for that tag 3. Verify that a pre-release was created for that tag, containing all build artifacts How to make a new release of the OSVM: 1. Do a new RC (see above) 2. Edit that GitHub pre-release to be an actual release 3. Change the description to have some release notes on the changes How to make use of OSVM-RC in squeak-app bundles: 1. Maybe do a new RC (see above) 2. In any branch's bundle.yml, set VM_RC_TAG in prepare-bundles job to the RC tag How to make use of a new OSVM release in squeak-app bundles: 1. Maybe make a new VM release (see above) 2. Update the files in http://files.squeak.org/base/ for the Squeak versions of your choice 3. :warning: Watch out for minor differences between OSVM packaging and what vm-*.zip expects. For example, Linux has an extra indirection that needs to be removed. And those macOS .dmg packages have to be converted to .zip for squeak-app. -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/637 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/637(a)github.com>

2 1

[OpenSmalltalk/opensmalltalk-vm] Memory leak (Issue #642)
by smalltalking 11 Oct '22

11 Oct '22

I noticed that recent VMs leak a few hundred kilobytes of memory every 3-6 seconds. It happens even if you just start VM and do nothing with the image. I tested it with various 64-bit Linux VMs. There seems to be no upper limit on the leaked memory. The leak happens even with the -memory option set. The size of the object memory is not affected by the leak, saved images remain small and the image does not see the increase in memory usage. Affected VMs include: 5.0-202206021410, 5.0-202205110711, 5.0-202204190959 Unaffected VMs include: 5.0-202112022203 and 5.0-202112201228 The 5.0-202206021410 assert and debug VMs leak the memory as well. -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/642 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/642(a)github.com>

7 18

[OpenSmalltalk/opensmalltalk-vm] macOS --metal backwards compatibility regarding primitives 126, 127, 231 (Issue #627)
by Marcel Taeumel 11 Oct '22

11 Oct '22

As of https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/2d7105db755928fd90…, there can be flickering or invisible Morphic dragging if no image-side composition buffer is used. In Squeak, that composition buffer is either already used or can be activated for older images by hand via: ```Smalltalk PasteUpMorph disableDeferredUpdates: true. Project allMorphicProjects do: [:ea | ea world canvas: nil]. ``` Also see: - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037907.html - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037888.html - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037891.html - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037897.html For the macOS `--metal` backend, one might actually implement that extra video buffer here: https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/2d7105db755928fd9082… And then read from that buffer and **not** the `displayBits` here: https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/2d7105db755928fd9082… ...but **only if** the `deferDisplayUpdates` flag is `false` again. **NOTE THAT** the old approach to always commit content from `displayBits` to the graphics backend entailed a *serious performance regression* as it slows down the interpreter loop to a maximum of 60 FPS, regardless of other stuff waiting in the image side. That is, primitives 127 and 231 were awfully slow and blocking on macOS. At least that got better in https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/2d7105db755928fd90… ------------- Another, more flexible, approach would be to fix **primitive 126** to actually *fail* if a certain graphics backend does not support deferred updates. This might work if platform-specific code would be able to let that primitive fail. See: - `StackInterpreterPrimitives >> primitiveDeferDisplayUpdates` in VMMaker This would work for recent versions of Cuis. However, the existing code in older Squeak images did not correctly invalidate the canvas and thus was not even able to flip between image-side and vm-side composition buffer for different results of primitive 126. Thus, such a fix would also only work from here on for future versions of Squeak and Cuis. -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/627 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/627(a)github.com>

6 16

Test
by christoph.thiede＠student.hpi.uni-potsdam.de 21 Sep '22

21 Sep '22

This is a test. Can you receive this message? Can I? Sorry for the noise ... Best, Christoph --- Sent from Squeak Inbox Talk

2 2

[OpenSmalltalk/opensmalltalk-vm] pushRcvr fails for context instances whose sender has been set to an Integer instance (Issue #654)
by Christoph Thiede 19 Sep '22

19 Sep '22

Example to reproduce (in Squeak): ```smalltalk (Context basicNew: 16) privSender: 1; pc ``` For me, this reproducibly crashes the VM. <details><summary>Stack backtrace</summary><pre><code> [00007ff7460a73f7] ??? + 0x173f7 in SqueakConsole.exe [00007ff7465372cc] Cog method with nil selector + 0xbc in CogCode [00007ff746401520] ceReturnToInterpreterTrampoline + 0x0 in CogCode [00007ff7478f04f6] ??? + 0x0 in (null) [00007ff7478f09e8] ??? + 0x0 in (null) [00007ff746498543] Cog method with nil selector + 0x213 in CogCode [00007ff746402906] on:do: + 0xa6 in CogCode [00007ff746401520] ceReturnToInterpreterTrampoline + 0x0 in CogCode [00007ff746401550] ceBaseFrameReturnTrampoline + 0x0 in CogCode</code></pre></details> Other examples: ```smalltalk (Context basicNew: 16) privSender: 1; method. "nil -- doesn't crash" (Context basicNew: 16) privSender: 1; receiver. "nil -- doesn't crash" (Context basicNew: 16) privSender: 1; sender. "crashes!" (Context basicNew: 16) privSender: 1; isMorph. "false - doesn't crash" (Context basicNew: 16) privSender: 1; yourself. "aContext - beware! crashes one or two seconds later without a backtrace" ``` Unless the context instance is executed by the VM, this should not happen. This is an annoying limitation for "heap fuzzing", i.e., randomly creating and assigning object instances, as done in SimulationStudio, for instance. -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/654 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/654(a)github.com>

2 4

Re: [Vm-dev] [list] Cannot receive plain-text messages
by Thiede, Christoph 18 Sep '22

18 Sep '22

Hi Dave, Hi Tobias, Hi all, I took another look at the multipart sources of the vm-lists which I cannot read in my client. The difference seems to be that on squeak-dev, the order of MIME parts is: multipart/mixed - multipart/alternative - - text/plain - - text/html - text/plain (Content-Disposition: inline, the empty stub) Whereas on vm-dev, the outer MIME parts are ordered the other way around: multipart/mixed - text/plain (Content-Disposition: inline, the empty stub) - multipart/alternative - - text/plain - - text/html I confirmed by experiment that my mail client (Outlook Web App) only displays the first multipart element, even if it is empty, so the order matters indeed. I also sent a test message to vm-dev (please forgive me the noise!) with just a single multipart/alternative, and the mailing list automatically wrapped it into a multipart/mixed. So Tobias seems to be wrong and mailman does indeed manipulate the contents of each message. I think this behavior violates rfc1341<https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html>: > Mail gateways, relays, and other mail handling agents are commonly known to alter the top-level header of an RFC 822 message. In particular, they frequently add, remove, or reorder header fields. Such alterations are explicitly forbidden for the body part headers embedded in the bodies of messages of type "multipart." And later: > In general, user agents that compose multipart/alternative entities should place the body parts in increasing order of preference, that is, with the preferred format last. For fancy text, the sending user agent should put the plainest format first and the richest format last. Receiving user agents should pick and display the last format they are capable of displaying. So according to the RFC, mailman/the list should not add its empty inline field at all (it just adds no value to the reader, right?). But if it needs to stick with that extra field (why?), according to the RFC, the current order is already correct. Also, my client clearly violates this point of the RFC as it displays the first instead of the last mixed element. Dave, mailadmins, whoever is responsible for this list, do you have any option to reconfigure mailman to have it work just like it works for squeak-dev, or just remove the superfluous mime element at all? I will also try to report this bug to Microsoft, but my gut tells me that I won't be able to convince them to follow the standards ... they never did. :-) Best, Christoph ---------------------------------------------------------------------- Message: 1 Date: Wed, 4 May 2022 18:51:12 +0200 From: Tobias Pape <Das.Linux(a)gmx.de> To: Open Smalltalk Virtual Machine Development Discussion <vm-dev(a)lists.squeakfoundation.org> Subject: Re: [Vm-dev] [list] Cannot receive plain-text messages Message-ID: <4D28BEB3-A6D6-41EE-83FE-0961B758F257(a)gmx.de> Content-Type: text/plain; charset=utf-8 > On 4. May 2022, at 18:36, Thiede, Christoph <Christoph.Thiede(a)student.hpi.uni-potsdam.de> wrote: > > Hi Marcel, > > (currently I have switched to the digest subscription option so that I can receive your answers :D) > > Thanks for the details! > > > ... and Tobias just told me that mailman does not change that stuff but forwards the content it gets. So the issue is how mail clients or SMTP servers push out that stuff. And we cannot control that. > > This is strange ... > > Object pinning vs. garbage collection > > > We are sending messages for commits to VMMaker and Trunk to vm-dev and squeak-dev. Do those look okay on vm -dev for you? > > I have not subscribed enough time ago to receive any VMMaker message. I will report back. :-) > > > What about my messages to vm-dev from April 2022? Were you subscribed then? > > For instance in the thread "Object pinning vs. garbage collection", I cannot read any of your messages: > Well, OWA is dumb as heck. It has no idea of multipart mixed, apparently and freely reëncodes emails you sent over it. Avoid it like plague. -t > <pastedImage.png> > > > Best, > Christoph > > Von: Vm-dev <vm-dev-bounces(a)lists.squeakfoundation.org> im Auftrag von vm-dev-request(a)lists.squeakfoundation.org <vm-dev-request(a)lists.squeakfoundation.org> > Gesendet: Mittwoch, 4. Mai 2022 16:41 Uhr > An: vm-dev(a)lists.squeakfoundation.org > Betreff: Vm-dev Digest, Vol 191, Issue 2 > > Send Vm-dev mailing list submissions to > vm-dev(a)lists.squeakfoundation.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.squeakfoundation.org/mailman/listinfo/vm-dev > or, via email, send a message with subject or body 'help' to > vm-dev-request(a)lists.squeakfoundation.org > > You can reach the person managing the list at > vm-dev-owner(a)lists.squeakfoundation.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Vm-dev digest..." > > > Today's Topics: > > 1. [list] Cannot receive plain-text messages (Thiede, Christoph) > 2. Re: [list] Cannot receive plain-text messages (Marcel Taeumel) > 3. Re: [list] Cannot receive plain-text messages (Marcel Taeumel) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 4 May 2022 13:58:03 +0000 > From: "Thiede, Christoph" > <Christoph.Thiede(a)student.hpi.uni-potsdam.de> > To: "vm-dev(a)lists.squeakfoundation.org" > <vm-dev(a)lists.squeakfoundation.org> > Subject: [Vm-dev] [list] Cannot receive plain-text messages > Message-ID: > <3483f711b6594b8bbf9a313a2af667c3(a)student.hpi.uni-potsdam.de> > Content-Type: text/plain; charset="iso-8859-1" > > Hi all, > > > while I have subscribed to squeak-dev for a longer time, I only subscribed recently to vm-dev. Unfortunately, I cannot view most messages in my e-mail client - I only see an attachment of the HTML file. It appears as if the plaintext version of the message is missing and the HTML version is classified incorrectly. This does not regard messages from the OSVM bot (GitHub) only but also normal messages sent by Eliot, Marcel, et al. - but at the same time, the messages sent by the same person to squeak-dev look fine on my end. > > > In other mailing clients (and in Squeak Inbox Talk), everything looks fine, but I do not want to switch away from OWA. In the message headers, I could not find any interesting difference. My subscription settings for both lists are identical. I also noted that the pipermail archives do not look identical, but this is probably unrelated. > > > Any ideas on what might cause this, and any chance to fix this in the mailing list configuration/servers? > > > Thanks in advance, > > Christoph >

2 1

[ANN] OpenSmalltalk VM 2022 released (tag 202205110711)
by Marcel Taeumel 15 Sep '22

15 Sep '22

Hi all! We just released the next version of the OpenSmalltalk VM. Please find the binaries here: https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/202205110711 [https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/202205110711] (see VMMaker.oscog-mt.3184 and update.oscog-mt.6.mcm) That version will be used in the upcoming Squeak 6.0 and also updated bundles for Squeak 5.3. And probably in upcoming Cuis releases. :-) Here is an attempt of a change log (since 2020): - Adds ARMv8/Aarch64/ARM64 JIT incl. support for Apple M1 - Adds "fast C primitives" via #FastCPrimitiveFlag - Adds support for catching exceptions in FFI callouts - Adds #primitiveScreenScaleFactor (for DPI-aware images) - Adds primitives 568 and 578 complementing 88 (primitiveSuspend) - Adds #primitiveMultipleBytecodeSetsActive to update image format for SistaV1 - Adds VectorEnginePlugin - Fixes regressions in ARMv6 support - Fixes performance regressions of -metal and -opengl backends on macOS - Fixes -core-graphics backend on macOS - Fixes Retina scaling on macOS, i.e., support "backing scale factor" - Fixes primitive 126 to fail on graphics backends w/o composition buffer - Fixes regressions in vm-display-fbdev on Linux - Fixes time sync (e.g., for DST) on Windows - Fixes UDP binding on Windows I am sure that I forgot something especially in plugin code. Please expand on this. BIG THANKS to everybody who has worked on this release! Personally, I would like to thank Eliot, who is a great software architect who keeps on making the OSVM faster with every commit. Thank you! Best, Marcel (on behalf of the OSVM core dev team)

5 6

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev September 2022