Vm-dev July 2022

vm-dev@lists.squeakfoundation.org

15 participants
78 discussions

[OpenSmalltalk/opensmalltalk-vm] Allow choosing an image via zenity when no arguments are provided (#408)
by Leon Matthes 05 Jun '23

05 Jun '23

You can view, comment on, or merge this pull request online at: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408 -- Commit Summary -- * Add file chooser dialog to squeak.sh.in * Add file chooser dialog to bin.squeak.sh.in -- File Changes -- M platforms/unix/config/bin.squeak.sh.in (13) M platforms/unix/config/squeak.sh.in (13) -- Patch Links -- https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408.patch https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408.diff -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/408

4 8

[OpenSmalltalk/opensmalltalk-vm] Document VM release process (Issue #637)
by Marcel Taeumel 05 Nov '22

05 Nov '22

Here is a draft. ----------- How to make a new RC of the OSVM: 1. Tag specific commit with its UTC timestamp, e.g., 202205110711 for May 11, 2022 at 7:11 2. Run all "Build for*" workflows manually for that tag 3. Verify that a pre-release was created for that tag, containing all build artifacts How to make a new release of the OSVM: 1. Do a new RC (see above) 2. Edit that GitHub pre-release to be an actual release 3. Change the description to have some release notes on the changes How to make use of OSVM-RC in squeak-app bundles: 1. Maybe do a new RC (see above) 2. In any branch's bundle.yml, set VM_RC_TAG in prepare-bundles job to the RC tag How to make use of a new OSVM release in squeak-app bundles: 1. Maybe make a new VM release (see above) 2. Update the files in http://files.squeak.org/base/ for the Squeak versions of your choice 3. :warning: Watch out for minor differences between OSVM packaging and what vm-*.zip expects. For example, Linux has an extra indirection that needs to be removed. And those macOS .dmg packages have to be converted to .zip for squeak-app. -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/637 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/637(a)github.com>

2 1

[OpenSmalltalk/opensmalltalk-vm] Memory leak (Issue #642)
by smalltalking 11 Oct '22

11 Oct '22

I noticed that recent VMs leak a few hundred kilobytes of memory every 3-6 seconds. It happens even if you just start VM and do nothing with the image. I tested it with various 64-bit Linux VMs. There seems to be no upper limit on the leaked memory. The leak happens even with the -memory option set. The size of the object memory is not affected by the leak, saved images remain small and the image does not see the increase in memory usage. Affected VMs include: 5.0-202206021410, 5.0-202205110711, 5.0-202204190959 Unaffected VMs include: 5.0-202112022203 and 5.0-202112201228 The 5.0-202206021410 assert and debug VMs leak the memory as well. -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/642 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/642(a)github.com>

7 18

[OpenSmalltalk/opensmalltalk-vm] macOS --metal backwards compatibility regarding primitives 126, 127, 231 (Issue #627)
by Marcel Taeumel 11 Oct '22

11 Oct '22

As of https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/2d7105db755928fd90…, there can be flickering or invisible Morphic dragging if no image-side composition buffer is used. In Squeak, that composition buffer is either already used or can be activated for older images by hand via: ```Smalltalk PasteUpMorph disableDeferredUpdates: true. Project allMorphicProjects do: [:ea | ea world canvas: nil]. ``` Also see: - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037907.html - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037888.html - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037891.html - http://lists.squeakfoundation.org/pipermail/vm-dev/2022-April/037897.html For the macOS `--metal` backend, one might actually implement that extra video buffer here: https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/2d7105db755928fd9082… And then read from that buffer and **not** the `displayBits` here: https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/2d7105db755928fd9082… ...but **only if** the `deferDisplayUpdates` flag is `false` again. **NOTE THAT** the old approach to always commit content from `displayBits` to the graphics backend entailed a *serious performance regression* as it slows down the interpreter loop to a maximum of 60 FPS, regardless of other stuff waiting in the image side. That is, primitives 127 and 231 were awfully slow and blocking on macOS. At least that got better in https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/2d7105db755928fd90… ------------- Another, more flexible, approach would be to fix **primitive 126** to actually *fail* if a certain graphics backend does not support deferred updates. This might work if platform-specific code would be able to let that primitive fail. See: - `StackInterpreterPrimitives >> primitiveDeferDisplayUpdates` in VMMaker This would work for recent versions of Cuis. However, the existing code in older Squeak images did not correctly invalidate the canvas and thus was not even able to flip between image-side and vm-side composition buffer for different results of primitive 126. Thus, such a fix would also only work from here on for future versions of Squeak and Cuis. -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/627 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/627(a)github.com>

6 16

[ANN] OpenSmalltalk VM 2022 released (tag 202205110711)
by Marcel Taeumel 15 Sep '22

15 Sep '22

Hi all! We just released the next version of the OpenSmalltalk VM. Please find the binaries here: https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/202205110711 [https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/tag/202205110711] (see VMMaker.oscog-mt.3184 and update.oscog-mt.6.mcm) That version will be used in the upcoming Squeak 6.0 and also updated bundles for Squeak 5.3. And probably in upcoming Cuis releases. :-) Here is an attempt of a change log (since 2020): - Adds ARMv8/Aarch64/ARM64 JIT incl. support for Apple M1 - Adds "fast C primitives" via #FastCPrimitiveFlag - Adds support for catching exceptions in FFI callouts - Adds #primitiveScreenScaleFactor (for DPI-aware images) - Adds primitives 568 and 578 complementing 88 (primitiveSuspend) - Adds #primitiveMultipleBytecodeSetsActive to update image format for SistaV1 - Adds VectorEnginePlugin - Fixes regressions in ARMv6 support - Fixes performance regressions of -metal and -opengl backends on macOS - Fixes -core-graphics backend on macOS - Fixes Retina scaling on macOS, i.e., support "backing scale factor" - Fixes primitive 126 to fail on graphics backends w/o composition buffer - Fixes regressions in vm-display-fbdev on Linux - Fixes time sync (e.g., for DST) on Windows - Fixes UDP binding on Windows I am sure that I forgot something especially in plugin code. Please expand on this. BIG THANKS to everybody who has worked on this release! Personally, I would like to thank Eliot, who is a great software architect who keeps on making the OSVM faster with every commit. Thank you! Best, Marcel (on behalf of the OSVM core dev team)

5 6

[OpenSmalltalk/opensmalltalk-vm] VM crash (probably wrong forward pointer calculation) (Issue #614)
by unique75m 07 Sep '22

07 Sep '22

This I what I get in the VS2022 debugger. Of course, at the moment I do not understand the code there, but maybe hopefully Eliot can see something interesting in it. I leave VS open and look tomorrow at the code, maybe I understand a bit of it :-) Seems there is something wrong with the forward pointers. I assume the longAt(referent) fails? I guess it is a macro, but VS could not find the definition. ########## Code area - the --> line is failing with access violation ########### /* begin literalCountOfMethodHeader: */ assert((((header) & 7) == 1)); numLiterals = ((header >> 3)) & AlternateHeaderNumLiteralsMask; numSlots = numLiterals + LiteralStart; l9: /* end numStrongSlotsOf:format:ephemeronInactiveIf: */; for (i = 0; i < numSlots; i += 1) { referent = longAt((referrer + BaseHeaderSize) + (((sqInt)((usqInt)(i) << (shiftForWord()))))); if ((!(referent & (tagMask())))) { /* a forwarding pointer could be because of become: or scavenging. */ --> if ((!((longAt(referent)) & ((classIndexMask()) - (isForwardedObjectClassIndexPun()))))) { /* begin followForwarded: */ assert(isUnambiguouslyForwarder(referent)); /* begin fetchPointer:ofMaybeForwardedObject: */ referent1 = longAt((referent + BaseHeaderSize) + (0U << (shiftForWord()))); while (((!(referent1 & (tagMask())))) && ((!((longAt(referent1)) & ((classIndexMask()) - (isForwardedObjectClassIndexPun())))))) { /* begin fetchPointer:ofMaybeForwardedObject: */ referent1 = longAt((referent1 + BaseHeaderSize) + (0U << (shiftForWord()))); } referent = referent1; ########## Call stack ############ Squeak.exe!scavengeReferentsOf(__int64 referrer) Zeile 42680 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\src\spur64.cog\cointerp.c (42680) Squeak.exe!scavengeRememberedSetStartingAt(__int64 n) Zeile 42760 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\src\spur64.cog\cointerp.c (42760) Squeak.exe!scavengeLoop() Zeile 42542 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\src\spur64.cog\cointerp.c (42542) Squeak.exe!doScavenge(__int64 tenuringCriterion) Zeile 47599 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\src\spur64.cog\cointerp.c (47599) Squeak.exe!scavengingGCTenuringIf(__int64 tenuringCriterion) Zeile 57959 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\src\spur64.cog\cointerp.c (57959) Squeak.exe!sufficientSpaceAfterGC(__int64 numBytes) Zeile 58691 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\src\spur64.cog\cointerp.c (58691) Squeak.exe!checkForEventsMayContextSwitch(__int64 mayContextSwitch) Zeile 62762 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\src\spur64.cog\cointerp.c (62762) Squeak.exe!handleStackOverflowOrEventAllowContextSwitch(__int64 mayContextSwitch) Zeile 66037 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\src\spur64.cog\cointerp.c (66037) Squeak.exe!ceStackOverflow(__int64 contextSwitchIfNotNil) Zeile 15593 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\src\spur64.cog\cointerp.c (15593) [Externer Code] Squeak.exe!ioInitHeartbeat() Zeile 420 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\platforms\win32\vm\sqWin32Heartbeat.c (420) Squeak.exe!interpret() Zeile 2875 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\src\spur64.cog\cointerp.c (2875) Squeak.exe!sqMain(int argc, char * * argv) Zeile 1761 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\platforms\win32\vm\sqWin32Main.c (1761) Squeak.exe!WinMain(HINSTANCE__ * hInst, HINSTANCE__ * hPrevInstance, char * lpCmdLine, int nCmdShow) Zeile 1851 unter C:\Users\joerg\Persoenlich\Entwicklung\Squeak\vmssource\trunk\platforms\win32\vm\sqWin32Main.c (1851) [Externer Code] ########## Locals ############ classFormat 662660168776 __int64 contextSize 72198296718755736 __int64 fmt 2 __int64 foundNewReferentOrIsWeakling 0 __int64 header 140698790658008 __int64 header1 140698758633160 __int64 i 37029628 __int64 newLocation 811789632 __int64 numLiterals 64 __int64 numSlots 167772160 __int64 numSlots1 255 unsigned __int64 numSlots2 167772160 unsigned __int64 objOop1 72057594139401638 __int64 referent 139599658561184 __int64 referent1 0 __int64 referrer 140700938141704 __int64 sp 72058702004626295 __int64 ############## Second try ################## I needed to restart it again, here are my new local values in debugger classFormat 973468582328 __int64 contextSize 72198310804720808 __int64 fmt 2 __int64 foundNewReferentOrIsWeakling 0 __int64 header 140699864399832 __int64 header1 140698758633160 __int64 i 81749860 __int64 newLocation 2070121416 __int64 numLiterals 64 __int64 numSlots 335544320 __int64 numSlots1 255 unsigned __int64 numSlots2 335544320 unsigned __int64 objOop1 72057594296693111 __int64 referent 139620721982368 __int64 referent1 0 __int64 referrer 140711869546504 __int64 sp 72058702004626295 __int64 numSlots seems to me very wrong. If I do some calculations I get also not the same value that the debugger says me. Here is the code numLiterals = ((header >> 3)) & AlternateHeaderNumLiteralsMask; numSlots = numLiterals + LiteralStart; AlternateHeaderNumLiteralsMask seems to be 0x7fff LiteralStart seems to be 1 For me is: numLiterals = ((140699864399832 >> 3)) & 0x7fff = 32763 —> does not match the debugger local, where is the 64 coming from Seems to me somebody has overridden already the „header“ variable, which seems to be wrong. Could it be that some other thread is writing in the wrong memory area and override my values? Jörg ########## System Information ############## Gerätename timemachine Prozessor AMD Ryzen 9 3900X 12-Core Processor 3.80 GHz Installierter RAM 32,0 GB Geräte-ID F6FA897B-DDB1-44D6-9BF3-8BD1110AA754 Produkt-ID 00326-10048-08575-AA867 Systemtyp 64-Bit-Betriebssystem, x64-basierter Prozessor Stift- und Toucheingabe Für diese Anzeige ist keine Stift- oder Toucheingabe verfügbar. Edition Windows 10 Home Version 20H2 Installiert am ‎23.‎03.‎2021 Betriebssystembuild 19042.1526 Leistung Windows Feature Experience Pack 120.2212.4170.0 Visual Studio 2022 win64x64\squeak.cog.spur VM version ??? I ask Eliot how i can see that on my current local github clone and post it later ############# Code for reproducation ############## Simply execute the following Smalltalk code. Maybe you need to run it multiple times, for me the crash happens sometimes at the 3rd or 4th try. | oc | oc := OrderedCollection new. 400000000 timesRepeat: [oc add: Object new] -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/614 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/614(a)github.com>

1 4

OSVM on WebAssembly
by Manuel Leuenberger 31 Aug '22

31 Aug '22

Hi, Ever since WebAssembly became a thing, I was wondering if this could become a target for VMs. People are already compiling FFMPEG and other complex tools. So I thought I would try as well. So here I am to report to whom it may concern: OSVM compiles to WebAssembly, starts up (nearly), then looping infinitely Meaning: The VM mmaps the image file, loads plugins (SecurityPlugin made EXTERNAL), starts interpreter loop, but then loops the same bytecode sequence forever Code lives at https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma… <https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma…> if you want to try it out. Below is the current Readme, including a short list of issues. Maybe some of you could give me a hint? Cheers, Manuel pharo.stack.spur.wasm Compiles OSVM Stack interpreter to WebAssembly using the Emscripten compiler. Emscripten can be used as a drop-in replacement for gcc/clang and cmake. Based on MinHeadless Linux 32bit sources, as Emscripten provides Linux-like environment (pthreads, nanosleep, dlopen, file system). Check the latest few commits of maenu to see changed files. <https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma…>Current issues Most adjustments are just putting EMSCRIPTEN in a macro or script. Should be fine, but should be tested to not interfere with other builds. Compiles and runs, but seems to be stuck in initial GC and Heartbeat. Those could be related to incorrect get/set64() implementation. Removed mmap address hint, as it caused errors. Using argv eval '1 + 3' to do a simple eval does not terminate. Interpreter repeats these bytecodes forever (what is this?): 332 384 385 348 501 339 320 401 272 380 <https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma…>Build & Run <https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma…>1. Install Emscripten I installed Emscripten SDK <https://emscripten.org/docs/getting_started/downloads.html> to get an all-in-one package. <https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma…>2. Grab an image Grab a 32bit Smalltalk image and but it in the image folder. I used Pharo 9. cd building/minheadless.cmake/x86/pharo.stack.spur.wasm mkdir image cd image curl https://get.pharo.org/32/90 | bash <https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma…>3. Build VM ./mvm_configure_variant debug Debug && make -C debug install <https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma…>4. Run a web server emrun --port 9090 --serve_root ../../../../ --no_browser . <https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma…>5. Launch VM http://localhost:9090/building/minheadless.cmake/x86/pharo.stack.spur.wasm/… <http://localhost:9090/building/minheadless.cmake/x86/pharo.stack.spur.wasm/…> <https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma…>6. Inspect running VM The VM is compiled with DWARF debug information, which is understood by the Chrome debugger. So we can step through the C sources of the WebAssembly, pretty nifty. <https://github.com/maenu/opensmalltalk-vm/tree/Cog/building/minheadless.cma…>Resources Inspect WebAssembly (at the bottom) <https://webassembly.org/getting-started/developers-guide/> Emscripten doc (Porting) <https://emscripten.org/docs/porting/index.html> Emscripten settings <https://emsettings.surma.technology/>

2 6

[OpenSmalltalk/opensmalltalk-vm] BitBlt rule blendAlphaScaled may not be correct (Issue #643)
by MariusDoe 22 Aug '22

22 Aug '22

The `BitBlt` blending rule `Form blendAlphaScaled` (`34`) doesn't blend the colors `0x00000000` (source; fully transparent) and `0xFFFFFFFF` (destination; white) correctly. The output is `0xFEFEFEFE` (slightly transparent white), whereas it should be `0xFFFFFFFF` (fully opaque white). I think the problem lies in [the implementation](https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/a45e… of `BitBltSimulation>>#alphaBlendScaled:with:`. When calculating the summand containing the `destinationWord`, a right bit shift by `8` is performed to normalize the result after multiplying with `unAlpha` (semantically a division by `256`). However, `unAlpha` is in the range `0x00` - `0xFF` and thus a division by `0xFF = 255` should be used instead. I think that the other bit shifts by `8` in the function are ok, as they are only used to extract or compose certain bytes and not to (semantically) divide a value by `256`. This problem causes the described behavior, because the following computation is performed in each channel: ```((0xFF * 0xFF) >> 8) + 0x00 = 0xFE01 >> 8 = 0xFE``` A division by `0xFF` produces the expected result: ```((0xFF * 0xFF) / 0xFF) + 0x00 = 0xFE01 / 0xFF = 0xFF``` Code to reproduce: ``` | src dst | src := (Form extent: 1 @ 1 depth: 1) colorAt: 0 @ 0 put: Color black; "transparency is applied with the fillColor:" yourself. dst := (Form extent: 1 @ 1 depth: 32) colorAt: 0 @ 0 put: Color white; "16rFFFFFFFF" yourself. dst copyBits: (0 @ 0 corner: 1 @ 1) from: src at: 0 @ 0 clippingBox: (0 @ 0 corner: 1 @ 1) rule: Form blendAlphaScaled fillColor: Color transparent. "16r00000000" (dst pixelValueAt: 0 @ 0) hex "16rFEFEFEFE" ``` Notes: - I did not thoroughly test the proposed change, just calculated the result for the described case. - I am no expert in writing high performance code, but a potential improved implementation may not use a division but rather a more optimized combination of `+`, `*` and `>>` to increase performance. A quick search on the internet shows some possible alternatives. - I read in the contribution guidelines, that the VM is developed in Smalltalk and the C code is only generated from the Smalltalk code. I did not have the time to set up a VM image and just looked into the C code to find the problem. I hope the problem description still helps. - The described behavior doesn't occur when blending two 32 bit `Form`s. I think this is due to `alphaSourceBlendBits32` being called in this case, which is optimized for edge cases with full or zero transparency and handles those correctly (it doesn't call `alphaBlendScaledwith`). -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/643 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/issues/643(a)github.com>

3 7

[OpenSmalltalk/opensmalltalk-vm] Refixed slip in mvm (PR #646)
by Ken Dickey 02 Aug '22

02 Aug '22

Elided 1 line (TARGET_ARCH) You can view, comment on, or merge this pull request online at: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/646 -- Commit Summary -- * Refixed slip in mvm -- File Changes -- M building/linux64riscv/squeak.stack.spur/build/mvm (1) -- Patch Links -- https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/646.patch https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/646.diff -- Reply to this email directly or view it on GitHub: https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/646 You are receiving this because you are subscribed to this thread. Message ID: <OpenSmalltalk/opensmalltalk-vm/pull/646(a)github.com>

2 10

Re: [Vm-dev] Help Request: gdb usage et al [RV64]
by ken.dickey＠whidbey.com 01 Aug '22

01 Aug '22

Better code sequence got to calling entryThunk(), which hit gdb breakpoint. Now on to "could not locate Callback instance corresponding to thunk address" -- but in Smalltalk! ;^) Thanks again to all, -KenD

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev July 2022