Vm-dev July 2011

vm-dev@lists.squeakfoundation.org

31 participants
164 discussions

ffi-test-main.c fails on FreeBSD
by Gary Dunn 12 Aug '16

12 Aug '16

In conjunction with getting Open Cobalt running on FreeBSD I have been looking into the problems with SqueakFFIPrims on FreeBSD and have reached a point that is beyond my understanding. I am not sure if the fix should come from the VM side or the FreeBSD port maintainer. I needed to get source code back into the ports tree so I did a make in /usr/ports/lang/squeak. I already had the 3.9 tarball, so that just repopulated the source and did the initial build. In the "files" folder I found patch files typical for a FreeBSD port. These patch source files from a generic tarball to work with FreeBSD. The one of interest was patch-platforms__unix__plugins__SqueakFFIPrims__ffi-config which contains ----x-----x----- --- platforms/unix/plugins/SqueakFFIPrims/ffi-config.org Wed Apr 26 20:27:53 2006 +++ platforms/unix/plugins/SqueakFFIPrims/ffi-config Wed Apr 26 20:29:00 2006 @@ -39,6 +39,7 @@ case ${abi} in linux) abi=sysv;; + freebsd*) abi=sysv;; darwin*) abi=darwin;; *) abi=libffi; lib="-lffi";; esac -----x-----x----- I then had the folder work/Squeak-3.9-7/platforms/unix/plugins/SqueakFFIPrims In it is a file 00README which explains how to test FFI. It says to run ffi-test-config but I have no such file. It says to type "make" to build a test suite but the build failed. It says to try with make CPU=any ABI=libffi LIB=-lffi which also failed. I traced the problem to the gcc search paths for includes and libraries -- turns out FreeBSD only searches /usr/include and /usr/lib when the required files are in /usr/local/include and /usr/local/lib. After some fiddling I finally had a "main" to test. It failed: ffi-test-main.c failed at line 361. Here is a bit from that file, ending with line 361: ffiInitialize(); for (ul= 0; ul < 15; ++ul) ffiPushSingleFloat(fa[ul]); GO(FFITypeSingleFloat, many); f= ffiReturnFloatValue(); ffiCleanup(); CHECK(f - ff < FLT_EPSILON); I worked out the CHECK macro but could not find what FLT_EPSILON is. At that point I decided to stop and come to you for help. Where do I go from here? -- Gary Dunn, Honolulu osp(a)aloha.com http://openslate.net/ http://e9erust.blogspot.com/ Sent from Slate001

5 8

Squeak VM port to Google Native Client
by Yoshiki Ohshima 19 Feb '12

19 Feb '12

Hello, I'm playing with Google Native Client a bit and made a port of Squeak VM to it. It is far from complete (even the state where backspace or Cmd-p etc. does not work, sound does not work, etc.), It now reached to the state where I can say the basic part of it is working. The source code is available at: http://github.com/yoshikiohshima/NaClSqueak . Also, f you have Google Chrome 11 beta available, try to turn on the Native Client on (by typing "about:flags" into the title bar) and access: http://tinlizzie.org/~ohshima/squeak/squeak.html (The first try may take some time.) - It is gnuify-ed but we're still learning the JIT situation of NaCl. So, it' need some work to port Cog. - The image data is embedded as a C array in this executable. You can remove EMBEDDED_IMAGE_FILE flag off, and comment/uncomment the corresponding part of squeak.html, it can load the image separately via XMLHttpRequest of the browser. - There is some strange issue when you do fast drag the system window. - As you can see, the source code is derived from the Unix VM. It was hard for me to write some CMake stuff to set it up for the cross compilation. A help to merge it back to the Unix code would be greatly appreciated. - If Google takes over the world and all apps become a Native Client (may be even other browsers^^;), this port may be useful^^; -- Yoshiki

9 25

Cross compile Window from Unix/Mac os
by Jean Baptiste Arnaud 14 Dec '11

14 Dec '11

Since the last Igor ( thank ) update it 's possible to cross compile windows VM(last version) from MacOs and Unix. Using Mingw and cmake. Now, i do for Cog, and for ARM9 processor. My process imply to fix some window file cause some include are wrong, then it the cross compiler don't find some include beginning by Uppercase(cause the file isn't beginning by uppercase). In /Platform/win32/vm/sqWin32Intel.c #include <float.h> instead #include <Float.h> in /Platform/win32/plugin/SocketPlugin/sqWin32NewNet.c #include <ws2tcpip.h> instead #include <Ws2tcpip.h> Regard Jean Baptiste Arnaud jbaptiste.arnaud(a)gmail.com

4 8

Direct object pointers vs indirect ones pros and cons
by Igor Stasenko 26 Oct '11

26 Oct '11

Hi , all RoarVM using object tables, while Squeak VM using direct pointers for object. This is a basic element of VM design, and i wonder, how much impact it does on overall VM speed. Both variants having own advantages/disadvantages, while i think, that with good JIT an extra indirection could be almost insignificant. But having indirect pointers to objects opens quite sexy perspectives. Being able to freely choose object(s) location means that: - its quite easy to implement object memory paging (swapping between memory and disc), - place particular object(s) into special memory location (good for FFI, object pinning etc) - #become: is O(1), instead of O(heap size) The downside of indirect pointers is, of course, a higher memory traffic, which directly impacts all operations everywhere. What else? I'd like to know, what you thinking about it, and why Squeak VM, in particular, using direct object pointers? What are this choice based on? I'd like to know. Maybe i missing something. -- Best regards, Igor Stasenko AKA sig.

10 23

About sqSCCVersion.h
by Igor Stasenko 13 Sep '11

13 Sep '11

Hello, i was looking how to make same trick for git, to embed the revision information into the source and found no easy solution. Using gitattributes is powerful, but too tedious for me to set it up fast and too much info for doing simple things. (You may look http://progit.org/book/ch7-2.html) So, to make it fast, i will use shell script to generate output. Unfortunately, it makes little sense to use #define GIT 1 (in sqSCCVersion.h) because its easier to just include generated file: #include "vmVersionInfo.h" // defines REVISION_STRING static char *sourceVersionString() { return REVISION_STRING; } Where vmVersionInfo.h contents is generated dynamically by shell script. To extract an URL a following command does it well: git config --get remote.origin.url Which shows: git@gitorious.org:cogvm/blessed.git To extract commit info, i using: git show HEAD --pretty="Commit: %H Date: %cd By: %cn <%cE>" Which shows: b0df6dc6902a9eb05de02782a77712d855d0422e Date: Wed Jul 20 16:22:16 2011 +0200 By: Igor Stasenko <siguctua(a)gmail.com> So, i will create a script which will generate a header file with these two lines concatenated, and then just include it into modified sqSCCVersion.h The problem is that i cannot put invocation of this script it into cmake config files, because then it will work only if you check out source from git , but not for tarballs. So, to generate the version info file this command will run separately on server. The output of sourceVersionString() function will be something like that: git@gitorious.org:cogvm/blessed.git Commit: b0df6dc6902a9eb05de02782a77712d855d0422e Date: Wed Jul 20 16:22:16 2011 +0200 By: Igor Stasenko <siguctua(a)gmail.com> If you have better suggestions how this string should look like (format, fields) i am here to listen. :) -- Best regards, Igor Stasenko AKA sig.

3 4

Re: [Pharo-project] Troubles with #flushCache and #run:with:in:
by Mariano Martinez Peck 11 Aug '11

11 Aug '11

Maybe cc'ing VM mailing list can help. On Fri, Jul 1, 2011 at 9:32 AM, Lukas Renggli <renggli(a)gmail.com> wrote: > Hi Eliot, > > I am using one of the latest VMs from your site (VM.r2434) and I > continue to have subtle problems with objects as methods (#flushCache, > #run:with:in:). > > The issue is that the test coverage in Pharo is kind of broken on Cog > for a long time already. It reports methods as not covered that are > clearly covered, and tests seem to randomly fail. > > I suspected that there is something wrong with the coverage code > itself. So I started to experiment with TestCoverage>>flushCache and > noticed that the current implementation > > TestCoverage>>flushCache > self reference methodSymbol flushCache > > performs not that well: The set of not covered methods is wrong and > many tests suddenly fail. If I replace it with > > TestCoverage>>flushCache > self reference actualClass flushCache > > I actually get accurate coverage information, but there are still a > few tests constantly failing. I tried to use all possible combinations > of #flushCache (also calling it on the compiled method), but only > flushing the cache on the class seems to work properly. So far so > good, but I really wonder what the correct way is to flush the cache? > :-) > > For my experiments I was using the package 'AST-Tests-Semantics'. This > is a small package with lots of test methods that cover each method > but one (RBSemanticAnnotationMisssing>>#isResumable). Now the "real" > problem is that when running these tests in coverage mode, the same 4 > tests always fail: > > RBSemanticTest>>testBlockScope > RBSemanticTest>>testCascadeReceiver > RBSemanticTest>>testClassVariableBinding > RBSemanticTest>>testGlobalVariableBinding > > Not sure of how to debug that? Do you have an idea why these otherwise > passing tests suddenly fail? If you want to try to reproduce you can > use any Pharo image with the tests loaded, or use those that I used: > > > http://jenkins.lukas-renggli.ch/job/Development/lastSuccessfulBuild/artifac… > > http://jenkins.lukas-renggli.ch/job/Development/lastSuccessfulBuild/artifac… > > Open the Test Runner, select 'AST-Tests-Semantics' and 'Run Coverage'. > > Any help or clarification would be appreciated :-) > > Lukas > > -- > Lukas Renggli > www.lukas-renggli.ch > > -- Mariano http://marianopeck.wordpress.com

3 6

Profiling and delay accuracy
by Andreas Raab 06 Aug '11

06 Aug '11

Hi Folks - For a long time now I had this feeling that MessageTally reports pretty bogus results in many situations without being able to precisely track down why this would be the case. Today I was looking at primitiveResponse() and, much to my surprise, noticed that the checkForInterrupts() that used to be there for years is gone (see Squeak3.6 for example which still has it). The sole reason for this check was to get both high delay accuracy and good profiling behavior. Consider this example: fc := FormCanvas on: Display. MessageTally spyOn:[ 1 to: 100 do:[:i| fc fillColor: Color white. fc printString. ]. ]. Using 3.6 this will be measured correctly as: **Tree** 87.5% {243ms} FormCanvas>>fillColor: |87.5% {243ms} GrafPort>>fillRect:offset: | 87.5% {243ms} GrafPort>>copyBits 12.5% {35ms} FormCanvas(Object)>>printString 12.5% {35ms} FormCanvas(Object)>>printStringLimitedTo: whereas the 3.10 reports it as **Tree** 99.6% {670ms} FormCanvas(Object)>>printString 94.9% {639ms} primitives 4.6% {31ms} FormCanvas(Object)>>printStringLimitedTo: I think you can see the problem and just imagine what this does to any cumulative results. I also wonder what that change does to platforms that don't have high-resolution asynchronous timers. This used to be the one place where we would catch timer changes at high resolution and unless the platform forces high resolution timer interrupts asynchronously (by resetting the interrupt counter) we must be loosing a lot in terms of delay accuracy. (it just so happens that the Windows VM *does* do asynchronous timer interrupts so I never noticed this but I'm very curious about the difference on Linux which typically still has pretty bad timer granularity). Any opinions? My feeling is that the timer check should go back in for all of the above reasons. At the very least a quickCheckForInterrupt() is required but I actually think putting the full check back in is worth it. Cheers, - Andreas

3 3

Object format(s) as a contract with VM?
by Igor Stasenko 02 Aug '11

02 Aug '11

Hello, i thought about object formats and thought how to make them more flexible and came to following idea. Today's squeak and cog vms using a 4 bits in object header which denoting an object format: So, there is 16 possible object formats supported by VM: formatOf: oop " 0 no fields 1 fixed fields only (all containing pointers) 2 indexable fields only (all containing pointers) 3 both fixed and indexable fields (all containing pointers) 4 both fixed and indexable weak fields (all containing pointers). 5 unused 6 indexable word fields only (no pointers) 7 indexable long (64-bit) fields (only in 64-bit images) 8-11 indexable byte fields only (no pointers) (low 2 bits are low 2 bits of size) 12-15 compiled methods: # of literal oops specified in method header, followed by indexable bytes (same interpretation of low 2 bits as above) " And virtual machine (mostly an ObjectMemory class) knows difference between those numbers and how to deal with objects depending what format they having. The bad side of it, that most semantics around formats is hardcoded and spreaded across various methods, like: lastPointerOf: oop "Return the byte offset of the last pointer field of the given object. Works with CompiledMethods, as well as ordinary objects. Can be used even when the type bits are not correct." | fmt sz methodHeader header contextSize | <inline: true> <asmLabel: false> header := self baseHeader: oop. fmt := self formatOfHeader: header. fmt <= 4 ifTrue: [(fmt = 3 and: [self isContextHeader: header]) ifTrue: ["contexts end at the stack pointer" contextSize := self fetchStackPointerOf: oop. ^ CtxtTempFrameStart + contextSize * BytesPerWord]. sz := self sizeBitsOfSafe: oop. ^ sz - BaseHeaderSize "all pointers"]. fmt < 12 ifTrue: [^ 0]. "no pointers" "CompiledMethod: contains both pointers and bytes:" methodHeader := self longAt: oop + BaseHeaderSize. ^ (methodHeader >> 10 bitAnd: 255) * BytesPerWord + BaseHeaderSize See how smelly this code is? What i was thinking about is , that what if number which denotes an object format instead of just being a number, point to a table of functions, which implement a behavior, specific for that format. Then the above method could be turned into something as short and nice as following: lastPointerOf: oop <inline: true> <asmLabel: false> | table | table := (self formatTableOf: oop). table performSomeOperation: oop param: param which , when translated to C, will look like: sqInt formatNumber; formatNumber = fetchFormatOf(oop); return ObjectFormats[formatNumber].performSomeOperation(oop, param). So, the table is simple list of function pointers (which very closely resembling a method dictionary in smalltalk), to provide an implementation of certain behavior for given object format: struct ObjectFormatFunctions { int (*)(sqInt oop); // firstFixedSlot int (*)(sqInt oop); // numFixedSlots: void* (*)(sqInt oop); // lastPointerOf: .. firstPointerOf: .. numIndexableSlots: .. indexableSlotSize: ... etc ) so, then instead of hardcoding and spreading the smelly code around the VM, we can write something like: (self formatTableOf: oop) performSomeOperation: oop params: params and code no longer needs to know if format = 0 or 4 or whatever, but just dispatching to function using formats table. Then all methods which dealing with object format could be placed in nicely organized class structure and can be translated to C in form of tables. The downside of it, of course, that additional level of indirection will cause a serious slowdown to interpreter. Because C compiler don't sees directly the implementation of operation for certain format and must call a function, pointer to which held in a corresponding format table. Moreover, you cannot inline those functions, since you don't know which one will be used at concrete place because of dynamic dispatch. How we can deal with it? Well, we still can hardcode the stuff (granting that we are not modifying object format table(s) at runtime), so we can write: fmt = 5 ifTrue: [ ^ self inlineFunctionOfFormat: 5 oop: oop params: params ] this requires some trickery in C code generator, but it is still doable. So, in overall i think that drawbacks could be cleverly mitigated if not completely avoided. So, lets talk what opportinities it could give us: VM could support a dynamically changeable object format(s), if we extend the number of possible formats to more than 16. So, for first 16 numbers things will be hardcoded as today, but for higher numbers, VM will always dispatch using format table. The cost of indirection can be amortized quite well especially in presence of JIT (it can simply inline the code for corresponding semantics to access various bits in object when compiling the methods of class with such format). Then we could dynamically change (or create new formats) at run time. But currently we have only 4 bits for object format. Where we can get some more? A solution is extremely simple! We have a 5 bits in object header for compact class index. If we merge them with object format bits 4+5 = 9 -- 512 possible object formats! The trick is that we still can pretty easily identify special objects by reserving a concrete format number for them (hey we have plenty of numbers) So, then format numbers ranged from 0..15 will identify hardcoded formats , known by VM from beginning. And formats from 16..48 is for special objects. and rest 512-48 is dynamically defined. Of course, we could change the object header to look differently for new VM, then there may be more (or less) than 9 bits for object format. Not really matters, because concept remains the same: VM could know that the only field which it can safely access by itself is an object header. For accessing other fields (if any) it should use functions provided in object format table. And the last thing: why i titled topic format as a contract? Because the next logical step of it is to imprint formats in a kind of form of manifest, and store them in image file. Then VM, when booting an image will read manifest and translate it into machine code before first attempt to access any object(s) in object memory. So, what you thinking about it? It could be an overkill , but i like the idea that with such approach we could change the object format(s) dynamically at run time, without the need of changing VM. It also structuring the code in VM quite nicely which will serve clarity. It also opens a wide field for experiments with different object formats. -- Best regards, Igor Stasenko AKA sig.

4 5

SqueakVMPathAnswersResources preference
by Eliot Miranda 01 Aug '11

01 Aug '11

Hi All, I just added code to the Cog branch for the Mac VM so that whether primVmPath/primitiveVMPath answers the directory containing the VM.app or answers the Resources directory within the .app is controlled by the SqueakVMPathAnswersResources preference in the Info.plist file. primVmPath is used to determine where to derive the sources file from. By default primVmPath answers the directory containing the VM.app and therefore the sources file is at the same level, e.g. /Applications/Squeak.app & /Applications/SqueakV41.sources. But some installations (e.g. Newspeak and the EToys all-in-one) may find it more convenient to put the sources in VM.app/Contents/Resources. This means one file to install, not 2, etc. I think it useful that this behavior be in the standard VM. Esteban, if you agree, perhaps you could integrate into the trunk platforms/Mac OS? -- best, Eliot

3 7

A trick to speedup #become: (but not becomeForward:)
by Igor Stasenko 01 Aug '11

01 Aug '11

An interesting observation: if two oops which we need to swap: oop1 and oop2 taking exactly same space in memory, then all we need to do is to swap their memory contents, instead of scanning heap and updating pointers! This makes a become operation extremely cheap for such pairs! As for objects of different sizes, we could still use slow algorithm. But knowing that if two objects with same size will be swapped much faster, a developers could adapt their algorithms to exploit this feature (use fixed-size objects instead of variable-sized ones etc). Magma using proxies, which then #becomeForward: to real objects when reified. This is very costly operation and main reason why it so slow on loading objects from server. To speed thing up, a special trick can be used: - each time server sends an object ID (instead of real object) it could also send its size in bytes (or even more clever - you can encode object size in its ID ;) ). Then a client will create a proxy for given object and will try to match the size, if its possible. (nobody said that proxy can't be variable-sized, right?) Then once client requesting to reify given proxy with real object from database, VM will just overwrite a proxy's memory contents with real object. -- Best regards, Igor Stasenko AKA sig.

4 11

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev July 2011