Vm-dev March 2018

vm-dev@lists.squeakfoundation.org

46 participants
218 discussions

VM build questions from last board meeting
by tim Rowledge 30 Mar '18

30 Mar '18

At the last SOB meeting we had a couple of minutes discussion relating to some recent VM issues; this is just to wave the flag and remind us that they are out there and could do with answering by anyone that Knows The Truth. a) Squeak3D plugin is only built on ARM config. Simple oversight in config files? b) B3DAcceleratorPlugin on Pi; it needs assorted gl.h files and there is no easy to spot doc of what packages have been installed on the autobuild Pi that provide them. In general that lead us to wonder if we can get some better documentation about the whole build/test/bintray setup on squeak.org so that we spread the knowledge a bit better. c) Also Pi related - the Pi vms appear to be itimer versions, which appear to give somewhat variable performance. I have no problems building thread timer vms, which seem much more consistent. Why is the thread version a problem in the autobuilder world? Or is it a simple oversight? And why is the package naming of the form squeak.cog.spur_linux32ARMv6_itimer_201803052012.tar.gz but the VM directory still named as sqcogspurlinuxhtRPi and thus looking like the threaded timer? tim -- tim Rowledge; tim(a)rowledge.org; http://www.rowledge.org/tim BASIC is to computer programming as QWERTY is to typing.

11 36

Generate Sources
by Guido Chari 30 Mar '18

30 Mar '18

Hi. I am not able to generate the sources for the VM. I am following these steps: 1. Download a squeak image (MAC 32 bits) from the official website. I have to do so because if I run "buildspurtrunkvmmakerimage.sh" from the latest Cog branch of opensmalltalk I got a dmg and when the script (or me) tries to open it, it appears a warning (the following disk image could not be opened) and nothing more happens. 2. When I file in the UpdateSqueakTrunkImage it hangs in: 'intalling kernel-nice.1127' so I am not able to update the image. 3. If I try to generate sources without updating the image running "VMMaker generateSqueakSpurCog64VM" I got a PrintFormatString>>transformVMMaker mnu. Can anybody point me out what am I doing wrong? Best Regards, Guido.

4 5

SiliconSqueak and RISC-V J Extension
by Jecel Assumpcao Jr. 29 Mar '18

29 Mar '18

Some of you know that I have been working on the design of the SiliconSqueak processor which is optimized for the OpenSmalltalk VM. I am currently redesigning it to be a proper extension to the RISC-V standard instruction set (https://riscv.org/). While the result will not be as technically elegant as previous versions, it should be much more interesting commercially. RISC-V was started at Berkeley in 2010 and has a growing community which I expect will make it the third most important instruction set after the x86 and ARM very soon. It was created to be expandable, so you start with one of the four integer instruction subsets (RV32I, RV64I, RV128I or RV32E with only 16 registers for embedded systems) and then you can optionally have some of the standard extensions: http://linuxgizmos.com/files/riscv_ref_card1.jpg http://linuxgizmos.com/files/riscv_ref_card2.jpg Integer Multiplication and Division (M) Atomics (A) Single-Precision Floating-Point (F) Double-Precision Floating-Point (D) The IMAFD combination is considered to be popular enough that you can use G (for General) instead. So RV32G is the same thing as RV32IMAFD. Some more standard extensions are in the process of being defined: Quad-Precision Floating-Point (Q) Decimal Floating-Point (L) 16-bit Compressed Instructions (C) Bit Manipulation (B) Dynamic Languages (J) Transactional Memory (T) Packed-SIMD Extensions (P) Vector Extensions (V) User-Level Interrupts (N) Non standard extensions are also allowed. For example: Xhwacha means the processor has the Hwacha vector extension that is different from the V vector extension that is being defined. So a processor named RV32GXsisq would be a general 32 bit RISC-V with SIliconSQueak extensions (which I will describe below). I am in the process of becoming a RISC-V Foundation member so I have join the J Extension work group since I feel I can help though the "J" is meant to imply Java and Javascript. It would also be a good idea if any needless incompatiblity between Xsisq and J can be avoided. My goal is to both improve the performance and efficiency of the bytecode interpreter and make the processor a better target for adaptive compilation. The ARM folks also did this in Jazalle 1 (later renamed DBX - direct bytecode execution) and Jazelle 2 (renamed RCT - runtime compilation target and later Thumb EE). I want to modify Cog to both work with RV32G and with my extensions and have the simulator interface with a cycle accurate processor simulator that can let us measure the effects of the extensions for both simple implementations and fancier out-of-order execution engines. It would be nice to have data to guide the design of these extensions before all this work, but the fact that critical operations are done via macros in the OpenSmalltalk VM (as they should!) instead of subroutines make it hard to know how much time is spent on PICs or allocating new objects, for example. == Xsisqbytecodes This is a special execution mode which uses two extra regiters: IP pointing to the next bytecodes and BCTableBase which points to an aligned 1024 byte table in memory. The instruction at BCTableBase + (* (char *) IP++) << 4 is executed. If the instruction is some kind of branch then bytecode mode is exited, otherwise the next bytecode is fetched. Combined with the next extension, the most common bytecodes can be interpreted by a single RISC-V instruction. == Xsisqstack Registers x16 to x31 are remapped to a moving window (not quite like RISC I, RISC II, RISC III (SOAR - Smalltalk On A RISC) or RISC IV (Spur)) but in the same spirit. In addition, register X15 becomes an alias for one of the others so you can push and pop to the stack implicitly instead of using extra instructions. When combined with the next extension, each register and word on the stack gets a 33rd (or 65th or 129th) bit that can't be changed by the software and which distinguishes between raw values and tagged values. == Xsisqobjectmemory Normal RISC-V load and store instructions generate a virtual address by adding a 12 bit immediate value to a register value. This virtual address is translated to a physical address by the MMU and that is used to access the cache. In the object addressing mode the value in the register is considered a virtual object ID and the immediate value is an offset. They are not added but used separately by the cache hash to access the desired word. On a cache miss an object table is used to find the physical address. The object formats are known so loads and stores can tell raw values from tagged words. Both V3 and Spur images use direct pointers and so can't take advantage of this mode, though the RoarVM could probably be adapted since it uses object tables. == Xsisqpic A special PIC execution mode is entered with an instruction that reads an object's class. The instruction cache loads a line hashed by both the class and the current PC and the instructions there are executed until some branch happens. So it is like a combination of call and switch. This means that PICs take the same time no matter how many entries they have. If all levels of caches miss then this means that the compiler has to be called to create a new PIC entry for this PC/class combination. Besides these four extensions, there is a Xteam extension which allows a group of cores to work together on a single piece of code. Using this will require adding a new compiler to Cog, so I won't go into it here. There are interesting possible extensions which I have not worked on: - support for GC (though Xsisqobjectmemory does help) like read or write barriers - support for execution counters - support of JIT like the hardware accelerators in this RISC-V to VLIW runtime compiler: > https://riscv.org/wp-content/uploads/2017/05/Wed1545-HybridDBT-Rokicki.pdf Any other ideas? -- Jecel

3 6

Re: [Vm-dev] [Pharo-dev] Pharo application packager for OS X
by Christophe Demarey 29 Mar '18

29 Mar '18

Hi, In addition to the previous email, I found the source code of the javapackager wrapper: http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/6df3a2267733/src/macosx… Python does almost the same thing: https://bitbucket.org/ronaldoussoren/py2app/src/b69e0ee6546ab157969524dd128… They use a shared lib I would like to have a small utility to easily create an OS X application out of a Pharo image. Other languages hav such tools like javapackager[1] for Java or py2app for Python. Basically, these tools provide a small executable that manages arguments, is able to find the language runtime in the app bundle and call a function from a dylib that creates and run the VM (see [3] and [4]). Could we do the same with OpenSmalltalk VM (embed it as a lib and run it from another exe)? Regards, Christophe [1] https://docs.oracle.com/javase/8/docs/technotes/tools/unix/javapackager.html [2] https://py2app.readthedocs.io/en/latest/ [3] http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/file/6df3a2267733/src/macosx… [4] https://bitbucket.org/ronaldoussoren/py2app/src/b69e0ee6546ab157969524dd128…

1 1

Re: [Vm-dev] [Pharo-dev] Some segfault crashes and freezes when trying to re-run saved pharo 7 images
by Eliot Miranda 28 Mar '18

28 Mar '18

Hi Holger, On Mon, Mar 26, 2018 at 8:32 AM, Holger Freyther <holger(a)freyther.de> wrote: > > > > On 26. Mar 2018, at 00:00, Eliot Miranda <eliot.miranda(a)gmail.com> > wrote: > > > > Hi Holger, > > Hey! > > > > Is the intent of CCallOut with: aBlock to collect and defer > deallocations until the block completes? I think it's nice but complex and > wonder how general it is. But it seems like it would impact a lot of code > and require a lot of effort changing existing code bases. My handle scheme > is only intended to fix the issue of images crashing on startup. The > problems with crashing on startup being a) one loses one's work and b) the > issue is hard to debug. That, for me, motivates something like the simple > fix I p[roposed. I'm not standing in the way of something more beautiful, > but I do believe that one shouldn't make the perfect the enemy of the good. > > By all means, let's have a smart pointer! It will be beneficial for > freetype and many other places. Nobody likes crashing images (for a mistake > made in a previous run). > > > My intent with the CCallOut is inhibit image saving until the code is > outside a sequence of (interruptible, hence the backward jump int the > example) C calls working with one or more pieces of manually managed > memory. But maybe it is best to call it by what it does instead of finding > a name of where it is used. > > I can't come up with a better example right now. But I think a > Smartpointer wouldn't be of much help with a sequence of strtok calls. > > word := CStringAPI strtok: cStringPtr safeBytes separator: '\r'. > [word isNil] whileFalse: [ > word = 'Foo'... > word = 'Bla'... > word := CStringAPI strtok: nil separator: '\r' > ------------- Image Save happening right here and word is not nil ----- > ]. > > I hope this is more clear. > Right. So the VM could have a flag that, when set, causes the snapshot primitive to fail, and that flag would be cleared only when all FFI cals had unwound. Would that be enough? What would you have the image do if snapshot fails because FFI calls are in progress? > holger > _,,,^..^,,,_ best, Eliot

1 0

Proposed mod to FilePlugin>>primitiveFileAtEnd for non-regular files
by Alistair Grant 27 Mar '18

27 Mar '18

FilePlugin currently splits files in to two groups: 1) Stdio streams and 2) everything else. To test for the end of file, FilePlugin>>primitiveFileAtEnd: 1) Uses feof() for stdio streams. 2) Compares the current position to the file size for everything else. This returns the expected results for regular files, but fails for non-regular files, e.g.: (FileSystem / 'dev' / 'urandom') binaryReadStream next: 8. returns an empty array. On Unix, the proper way to check is to read past the end of the file and then call feof() to confirm that it is in fact at the end. Pharo (and I assume, Squeak) has plenty of code that assumes that a file position >= the file size means that the file is at the end, and as stated above this generally works for regular files. This patch modifies FilePlugin>>primitiveFileAtEnd to: a) Keep the current behaviour of using the file position test for regular files. b) Keep the current behaviour of using feof() for stdio streams. c) Use feof() for non-regular files, e.g. /dev/urandom. This allows existing code to continue to function, and allows non-regular files to be tested correctly. After applying the patch, the example code above answers the expected result, e.g.: (FileSystem / 'dev' / 'urandom') binaryReadStream next: 8. " #[179 136 227 226 28 147 197 125]" On Windows, as far as I can tell, all files are regular, and the position test is used regularly, so no change is requried. https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/232 contains the changes. The CI builds failed, but for different reasons to do with sista. If I don't hear any dissenting opinion in a few days I'll integrate this in. Cheers, Alistair

3 4

Re: [Vm-dev] [Pharo-dev] Image crashing on startup, apparently during GC
by Eliot Miranda 26 Mar '18

26 Mar '18

Hi Cyril, On Sat, Mar 24, 2018 at 1:50 AM, Cyril Ferlicot D. <cyril.ferlicot(a)gmail.com > wrote: > Le 23/03/2018 à 21:52, Eliot Miranda a écrit : > > Hi Damien, > > > > Indeed the image is corrupt at start-up. See below. > > > > > > Right. This VM is prior to the bug fixes in VMMaker.oscog-eem.2320: > > > > Spur: > > Fix a bad bug in SpurPlnningCompactor. > > unmarkObjectsFromFirstFreeObject, used when the compactor requires more > > than one pass due to insufficient savedFirstFieldsSpace, expects the > > corpse of a moved object to be unmarked, but > > copyAndUnmarkObject:to:bytes:firstField: only unmarked the target. > > Unmarking the corpse before the copy unmarks both. This fixes a crash > > with ReleaseBuilder class>>saveAsNewRelease when non-use of cacheDuring: > > creates lots of files, enough to push the system into the multi-pass > regime. > > > > > > Pharo urgently needs to upgrade the VM to one more up to date than 2017 > > 08 27 (in fact more up-to-date than opensmalltalk/vm commit > > 0fe1e1ea108e53501a0e728736048062c83a66ce, Fri Jan 19 13:17:57 2018 > > -0800). The bug that VMMaker.oscog-eem.2320 fixes can result in image > > corruption in large images, and can occur (as it has here) at start-up, > > causing one's work to be irretrievably lost. > > > > Hi Eliot, > > I think that there is a lot of people who would like to get a newer > stable vm for Pharo 6.1 and 7. The problem is that it is hard to know > which VM are stable enough to be promoted as stable. > > Some weeks ago Esteban tried to promote a VM as stable and he had to > revert it the same day because a regression occurred in the VM. > > If you're able to tell us which vms are stable in those present at > http://files.pharo.org/vm/pharo-spur32/ and > http://files.pharo.org/vm/pharo-spur64/ it would be a great help. > > Even better would be for the pharo community to have a way to know which > vms are stable or not without having to ask you. > The way to do that is to run tests on the CI infrastructure and mark VMs that pass the tests as stable. That;s what happens on travis with the Cog VMs built there. > > Have a nice day. > > > > > -- > > _,,,^..^,,,_ > > best, Eliot > -- > Cyril Ferlicot > https://ferlicot.fr > > -- _,,,^..^,,,_ best, Eliot

1 0

TMethod>>outputConditionalDefineFor:on: proposed modification
by Alistair Grant 23 Mar '18

23 Mar '18

Hi Everyone, VMMaker plugin C generation allows the option: pragma to be used to conditionally compile methods in specific VMs, e.g.: primitiveForPharoOnWin32 <option: #_WIN32> <option: #PharoVM> ^self doStuff will generate C code along the lines of: #if _WIN32 && PharoVM sqInt primitiveForPharoOnWin32() { ... } #endif The problem is that on non Win32 platforms this will cause a compiler error as _WIN32 isn't defined, and the resulting directive is invalid. Modifying TMethod>>outputConditionalDefineFor:on: to generate: #if defined(_WIN32) && defined(PharoVM) sqInt primitiveForPharoOnWin32() { ... } #endif avoids the problem. Any comment or opposition to me submitting this change? Thanks, Alistair

2 2

cute, we crash gcc...
by Tobias Pape 23 Mar '18

23 Mar '18

https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/355802605#L2458

4 12

Re: [Vm-dev] [squeak-dev] VM crash upgrading via trunk from 5.1-17325 (420->421)
by Tm Jhnsn 21 Mar '18

21 Mar '18

Hi vm-dev, Continuing my thread from squeak-dev. On Mar 20, 2018, at 1:03 PM, Tim Johnson wrote: >>> Good. I'm off the hook :-) Your subject line is misleading; this is a lock up, not a VM crash. On Mac OS X I see the lock up but see no VM crash. Does it help that you can start from the 6.0alpha download (6.0alpha) instead of starting from scratch? > > Whoops! I'm sorry Eliot. The one time the Win32 VM actually popped up a dialog window (instead of an in-image offer to drop to evaluator), it suggested there had been a VM problem. I could never repeat the pop-up of that dialog window, though. Thanks... I will be more careful about my terminology. Attached is a crash.dmp file. This crash happened on the Win32 VM, squeak.cog.spur_win32x86_201803080952. IIRC, I was trying to upgrade from #420 to #421. The VM popped up a dialog box indicating some VM statistics and last opcode executed. I failed to capture a screenshot. I was never able to make the same situation crash the VM again after maybe 5 attempts — it only brought me an offer to enter the emergency evaluator. Thanks, Tim

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev March 2018