Vm-dev November 2013

vm-dev@lists.squeakfoundation.org

30 participants
103 discussions

ffi-test-main.c fails on FreeBSD
by Gary Dunn 12 Aug '16

12 Aug '16

In conjunction with getting Open Cobalt running on FreeBSD I have been looking into the problems with SqueakFFIPrims on FreeBSD and have reached a point that is beyond my understanding. I am not sure if the fix should come from the VM side or the FreeBSD port maintainer. I needed to get source code back into the ports tree so I did a make in /usr/ports/lang/squeak. I already had the 3.9 tarball, so that just repopulated the source and did the initial build. In the "files" folder I found patch files typical for a FreeBSD port. These patch source files from a generic tarball to work with FreeBSD. The one of interest was patch-platforms__unix__plugins__SqueakFFIPrims__ffi-config which contains ----x-----x----- --- platforms/unix/plugins/SqueakFFIPrims/ffi-config.org Wed Apr 26 20:27:53 2006 +++ platforms/unix/plugins/SqueakFFIPrims/ffi-config Wed Apr 26 20:29:00 2006 @@ -39,6 +39,7 @@ case ${abi} in linux) abi=sysv;; + freebsd*) abi=sysv;; darwin*) abi=darwin;; *) abi=libffi; lib="-lffi";; esac -----x-----x----- I then had the folder work/Squeak-3.9-7/platforms/unix/plugins/SqueakFFIPrims In it is a file 00README which explains how to test FFI. It says to run ffi-test-config but I have no such file. It says to type "make" to build a test suite but the build failed. It says to try with make CPU=any ABI=libffi LIB=-lffi which also failed. I traced the problem to the gcc search paths for includes and libraries -- turns out FreeBSD only searches /usr/include and /usr/lib when the required files are in /usr/local/include and /usr/local/lib. After some fiddling I finally had a "main" to test. It failed: ffi-test-main.c failed at line 361. Here is a bit from that file, ending with line 361: ffiInitialize(); for (ul= 0; ul < 15; ++ul) ffiPushSingleFloat(fa[ul]); GO(FFITypeSingleFloat, many); f= ffiReturnFloatValue(); ffiCleanup(); CHECK(f - ff < FLT_EPSILON); I worked out the CHECK macro but could not find what FLT_EPSILON is. At that point I decided to stop and come to you for help. Where do I go from here? -- Gary Dunn, Honolulu osp(a)aloha.com http://openslate.net/ http://e9erust.blogspot.com/ Sent from Slate001

5 8

Do we have the new primitive?? [WAS] Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?
by Mariano Martinez Peck 16 Dec '15

16 Dec '15

> All I can say is that I am impressed by the numbers it is really much >> faster. >> I still don't understand why I send this email with a subject say >> IdentitySet because what I really need is a fast/large IdentityDictionary >> :( Anyway, there's a place where we can use this LargeIdentitySet in Fuel >> I think). >> >> So Levente, you say this is not possible to adapt this for dictionary? >> can >> we contact Eliot to provide such a primitive? >> > > As promised, I uploaded my LargeIdentityDictionary implementation to > http://leves.web.elte.hu/**squeak/**LargeIdentityDictionary.st<http://leves.web.elte.hu/squeak/LargeIdentityDictionary.st>. > The numbers will be a bit worse compared to LargeIdentitySet, because of > the lack of the primitive, but it's still 2-3x faster than other solutions > (IdentityDictionary, PluggableIdentityDictionary, subclassing, etc). I'm > about to propose this primitive with other improvements on the vm-dev list. > > Hi Eliot/Levente. What is the status of this? Do we have already the new primitive? If true, how can we adapt LargeIdentitySet to use such new primitive? Thanks! > > Levente > > >> thanks >> >> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >> >> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>> >>> On 16.12.2011 03:26, Levente Uzonyi wrote: >>> >>>> >>>> >>>>> How about my numbers? :) >>>>> >>>>> "Preallocate objects, so we won't count gc time." >>>>> n := 1000000. >>>>> objects := Array new: n streamContents: [ :stream | >>>>> n timesRepeat: [ stream nextPut: Object new ] ]. >>>>> >>>>> set := IdentitySet new: n. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>>> >>>>> set := LargeIdentitySet new. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>>> >>>>> set := (PluggableSet new: n) >>>>> hashBlock: [ :object | object identityHash * 4096 + object class >>>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>>> equalBlock: [ :a :b | a == b ]; >>>>> yourself. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>>> >>>>> >>>>> I also have a LargeIdentityDictionary, which is relatively fast, but >>>>> not >>>>> as fast as LargeIdentitySet, because (for some unknown reason) we don't >>>>> have a primitive that could support it. If we had a primitive like >>>>> primitive 132 which would return the index of the element if found or >>>>> 0 if >>>>> not, then we could have a really fast LargeIdentityDictionary. >>>>> >>>>> >>>>> Levente >>>>> >>>>> Hehe yes, if writing a version fully exploiting the limited range, >>>> that's >>>> probably the approach I would go for as well. >>>> (IAssuming it's the version at http://leves.web.elte.hu/** >>>> squeak/LargeIdentitySet.st<htt**p://leves.web.elte.hu/squeak/** >>>> LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>>> > >>>> ) >>>> >>>> Mariano commented in the version at http://www.squeaksource.com/** >>>> FuelExperiments <http://www.squeaksource.com/**FuelExperiments<http://www.squeaksource.com/FuelExperiments>> >>>> that it's >>>> >>>> slow for them, which I guess is due to not adopting #identityHash calls >>>> to >>>> #basicIdentityHash calls for Pharo: >>>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>>> >>>> Uploaded a new version to the MC repository which is adapted for Pharo, >>>> on the same machine my numbers were taken from, it does the same test >>>> as I >>>> used above in 871 ms. (181 with preallocation). >>>> >>>> >>> Cool. One more thing: in Squeak the method using primitive 132 directly >>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> this was also added to Pharo, then the #pointsTo: sends should be changed >>> to #instVarsInclude:, otherwise Array can be reported as included even if >>> it wasn't added. >>> I'll upload my LargeIdentityDictionary implementation to the same place >>> this evening, since it's still 2-3 factor faster than other solutionts >>> and >>> there seem to be demand for it. >>> >>> >>> Levente >>> >>> >>> Cheers, >>>> Henry >>>> >>>> >>>> >>>> >>> >> >> -- >> Mariano >> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >> >> > -- Mariano http://marianopeck.wordpress.com

4 12

Nicolas Cellier added to VMMaker developers (was: VM Maker: VMMaker-dtl.328.mcz)
by David T. Lewis 22 Jan '14

22 Jan '14

Done. Thanks Nicolas! Note, our source.squeak.org server seems to be having problems at the moment, so there is some risk that this update may be lost if the image gets restarted. I will double check tomorrow and make sure that the addition is permanent. Dave On Fri, Nov 01, 2013 at 12:44:29PM -0700, Eliot Miranda wrote: > Hi David, Nicolas, > > I'd like Nicolas to become a trusted committer. Nicolas's > contributions recently are first-class. Do you agree? If so, can you make > it so? > Nicolas, thanks. I think you're only the fourth person after myself, Lars > Wassermann and Igor Stasenko to hack the Cogit. I really appreciate it. > > > On Thu, Oct 31, 2013 at 4:55 PM, <commits(a)source.squeak.org> wrote: > > > > > David T. Lewis uploaded a new version of VMMaker to project VM Maker: > > http://source.squeak.org/VMMaker/VMMaker-dtl.328.mcz > > > > ==================== Summary ==================== > > > > Name: VMMaker-dtl.328 > > Author: dtl > > Time: 31 October 2013, 7:48:11.404 pm > > UUID: 952f8e54-a106-406f-8f0e-3c588fb14748 > > Ancestors: VMMaker-dtl.327 > > > > VMMaker 4.12.8 > > Mantis 7794: BitBltSimulation primitiveDisplayString fails to advance destX > > Fix by Nicolas Cellier: > >

3 3

StackSimulator somewhat running in Pharo2.
by phil＠highoctane.be 28 Nov '13

28 Nov '13

Here is a picture of it. ObjectMemory and the StackVM are there. Lots of cleaning to do, but we'll make it. Next step: find out why the Display isn't rendered properly. StartUp problem? Phil

1 0

Simulator & Pharo [was: Second coming of Java...]
by phil＠highoctane.be 27 Nov '13

27 Nov '13

Hello, I gave it a shot based on the generator.image that PharoVM uses to generate the VM. But none of : (| vm | vm := StackInterpreterSimulator newWithOptions: #(). vm openOn: '/Users/eliot/Cog/startreader.image'. vm openAsMorph; run) and (| vm | vm := CogVMSimulator newWithOptions: #(ObjectMemory Spur32BitCoMemoryManager Cogit StackToRegisterMappingCogit). vm desiredNumStackPages: 8. vm objectMemory setCheckForLeaks: 1. vm openOn: '/Users/eliot/Cog/spurreader.image'. vm openAsMorph; toggleTranscript; halt; run) is going to work right away as newWithOptions: isn't there in that image. So, there is a initializeWithOptions: #(). When trying out that one, I do get another error. In fact, the initializeWithOptions: method is wrong: initializeWithOptions: optionsDictionary "ObjectMemory initializeWithOptions: Dictionary new" self initBytesPerWord: (optionsDictionary at: #BytesPerWord ifAbsent: [4]). ... the thing is doing #BytesPerWork ifAbsent: ... :-( Of course, as #() is an Array an not a Dictionary. So, I tried: StackInterpreterSimulator initializeWithOptions: Dictionary new. vm := StackInterpreterSimulator new. Which got the ball rolling. But then, of course, same story as here: http://forum.world.st/Got-quot-Error-basicNew-failed-quot-when-running-Inte… Well, based on Clement's comments, I fiddled a bit with the TranscriptStream, which I turned into a simple Transcript and let go of the local image name for the display window. Also, did transcript := Transcript. displayForm := (ImageMorph fromString: 'Display has not yet been installed') form. in StackInterpreterSimulator initialize. So, here is the current stance: | vm | StackInterpreterSimulator initializeWithOptions: Dictionary new. vm := StackInterpreterSimulator new. vm openOn: 'C:\MinGW\msys\1.0\home\User\pharo\pharo.image'. vm openAsMorph. vm run. the morph opens. Now, when I do run, I do get a problem in: InterpreterStackPage>>headFP: pointer There is an self assert: (pointer = 0 or: [pointer < baseAddress and: [realStackLimit - (LargeContextBytes / 2) <= pointer]]). And LargeContextBytes is nil. I looked around for initialization for that one. It is only in the InterpreterStackPage class >> initialize So I did; | vm | StackInterpreterSimulator initializeWithOptions: Dictionary new. vm := StackInterpreterSimulator new. InterpreterStackPage initialize. vm openOn: 'C:\MinGW\msys\1.0\home\User\pharo\pharo.image'. vm openAsMorph. vm run. Then I got a blinking square running in the top corner. No amount of stopping processes would interrupt it. So the simulator is running the run loop: self fetchNextBytecode. [true] whileTrue: [self assertValidExecutionPointers. atEachStepBlock value. "N.B. may be nil" self dispatchOn: currentBytecode in: BytecodeTable. self incrementByteCount]. and relinquishing processor every once in a while and this square is shown (Display reverse below] ... (which is where I end up when interrupting) ioRelinquishProcessorForMicroseconds: microseconds "In the simulator give an indication that we're idling and check for input." Display reverse: (0@0 extent: 16@16). Sensor peekEvent ifNotNil: [self forceInterruptCheck]. Processor activeProcess == UIManager default uiProcess ifTrue: [World doOneCycle]. microseconds >= 1000 ifTrue: [(Delay forMilliseconds: microseconds + 999 // 1000) wait] ifFalse: [Processor yield] If one could tell me how to get the display to show, it would be welcome. The display instVar contains stuff in the debugger but isn't ok to open with something like: In the transcript, I do see things related to initialization: (56) Looking for primitiveSetGCBiasToGrowGCLimit in vm Looking for module ... not found (3751) Looking for primitiveFileWrite in FilePlugin Looking for module FilePlugin (3751) Looking for secCanCreatePathOfSize in SecurityPlugin Looking for module SecurityPlugin ... loaded (3751) Looking for secCanDeletePathOfSize in SecurityPlugin (3751) Looking for secCanGetFileTypeOfSize in SecurityPlugin (3751) Looking for secCanListPathOfSize in SecurityPlugin (3751) Looking for secCanSetFileTypeOfSize in SecurityPlugin (3751) Looking for secDisableFileAccess in SecurityPlugin (3751) Looking for secCanDeleteFileOfSize in SecurityPlugin (3751) Looking for secCanOpenFileOfSizeWritable in SecurityPlugin (3751) Looking for secCanRenameFileOfSize in SecurityPlugin (3751) Looking for secHasFileAccess in SecurityPlugin ... loaded (3774) Looking for primitiveFileSize in FilePlugin (4390) Looking for primitiveCompareString in MiscPrimitivePlugin Looking for module MiscPrimitivePlugin ... loaded (4623) Looking for primitiveDirectoryDelimitor in FilePlugin (7806) Looking for primitiveIndexOfAsciiInString in MiscPrimitivePlugin (57216) Looking for primitiveStringHash in MiscPrimitivePlugin (177977) Looking for primitiveFindFirstInString in MiscPrimitivePlugin (179055) Looking for primitiveFileOpen in FilePlugin (179815) Looking for primitiveFileGetPosition in FilePlugin (179832) Looking for primitiveFileAtEnd in FilePlugin (179873) Looking for primitiveFileRead in FilePlugin (182837) Looking for primitiveFileSize in FilePlugin (182868) Looking for primitiveFileSetPosition in FilePlugin (186627) Looking for primDigitCompare in LargeIntegers Looking for module LargeIntegers ... loaded (186688) Looking for primDigitMultiplyNegative in LargeIntegers (186884) Looking for primNormalizeNegative in LargeIntegers (186985) Looking for primDigitAdd in LargeIntegers (186986) Looking for primNormalizePositive in LargeIntegers (187038) Looking for primDigitSubtract in LargeIntegers Is there a way to stop the interpreter in order to debug things? I saw: runAtEachStep: aBlock self initStackPages. self loadInitialContext. self internalizeIPandSP. self fetchNextBytecode. [true] whileTrue: [self assertValidExecutionPointers. aBlock value: currentBytecode. self dispatchOn: currentBytecode in: BytecodeTable. self incrementByteCount]. localIP := localIP - 1. "undo the pre-increment of IP before returning" self externalizeIPandSP How to use that? Why do I not see the display in the sim? I saw a "quitBlock". What is this about? I am not far from getting this working... Thanks for some clues! Phil --- Philippe Back Dramatic Performance Improvements Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 Mail:phil@highoctane.be | Web: http://philippeback.eu Blog: http://philippeback.be | Twitter: @philippeback Youtube: http://www.youtube.com/user/philippeback/videos High Octane SPRL rue cour Boisacq 101 | 1301 Bierges | Belgium Pharo Consortium Member - http://consortium.pharo.org/ Featured on the Software Process and Measurement Cast - http://spamcast.libsyn.com Sparx Systems Enterprise Architect and Ability Engineering EADocX Value Added Reseller

3 4

Re: [squeak-dev] Stack VM = Squeak VM?
by gettimothy 27 Nov '13

27 Nov '13

Cool. One more thing to clarify: does running the simulator require having Bochs installed? (From what I can see, yes, but its not clear from your instructions here: http://www.mirandabanda.org/cogblog/build-image/ ) Finally, here is my game-plan. 1. I am going to finish my overview of the Blue Book Object Memory etc and NOT go into a study of the details in the last chapters of part 4--I will refer back to it when needed to understand a specific situation. 2. Get Bochs running on a 32 bit capable slackware linux (My bochs install on my 32 bit capable Slackware dies because of a bug in binutils-22. Slackware has a new 14.1 release out with the updated binutils, so hopefully I can get squeak/bochs running together on linux. it installs fine on my pure 64 bit boot. chicken,egg...) 3 Verify that me not having installed Bochs is why I could install your Cog dev environment but not be able to launch the simulations. 4. I will then study the Stack VM and try to replicate what you did with Stack VM in 64 bits on Spur using the steps you posted a few weeks ago: Read the latest two articles on my blog (A Spur gear for Cog & Lazy Become). Make a VMMaker image (see e.g. Cog Blog :: Building a Cog Development Image). Then read SpurMemoryManager, /especially/ its class comment. Then read its subclasses Spur32BitMemoryManager (functional) & Spur64BitMemoryManager (completely untested). Then read SpurBootstrap32. Try and create a 32-bit Spur image and run it. Then try and implement SpurBootstrap64. Try and create a 64-bit Spur image and run it. Sound like a plan? thx. tty ---- On Wed, 27 Nov 2013 11:21:19 -0800 Eliot Miranda <eliot.miranda(a)gmail.com> wrote ---- Hi tty, On Wed, Nov 27, 2013 at 9:19 AM, gettimothy <gettimothy(a)zoho.com> wrote: Oops, I posted to squeak-dev instead of vm-dev. I have added a CC so we can continue there if you wish. Ahhhh, I see what's going on now, its an incremental approach..makes sense now. Yes. Qwaq got something faster sooner. The phone folks got a faster VM that doesn't require a JIT (see StackVM on Android, iPhone etc). I got a simpler simulator for Spur. Lots of advantages. Smalltalk-80 VM (Blue Book) -> Squeak VM (OE-Tour.pdf)->Stack VM (http://www.mirandabanda.org/cogblog/2009/01/14/under-cover-contexts-and-the… (JIT, Stack To Register Mapping, ..http://www.mirandabanda.org/cogblog/about-cog/) ->Spur (changing the garbage collector and the object representation) ->Lazy Become (http://www.mirandabanda.org/cogblog/2013/09/13/lazy-become-and-a-partial-re…) >From your post here: http://www.mirandabanda.org/cogblog/2008/12/12/simulate-out-of-the-bochs/ I gather you used the StackVM with Alien and Bochs as your base to build the JIT for Cog . That's right. So then, in an earlier response to my inquiry on the ~$1,000,000 bounty for cmakeifying the process thread, you wrote ...then you could instead have a go at getting the Stack VM working in 64-bits, and that would involve getting the 64-bit VM simulator working in VMMaker. Which makes sense. Basically, I would be doing a 64 bit version of follow the leader on your existing 32 bit work where it should all come together at Spur. Does that sound about right? Exactly right. With the StackVM simulator you can get a 64-bit Spur memory manager working. Then you can get an x86-64 code generator going with the Cog VM simulator. tty ---- On Wed, 27 Nov 2013 08:47:03 -0800 Eliot Miranda <eliot.miranda(a)gmail.com> wrote ---- On Wed, Nov 27, 2013 at 8:05 AM, Levente Uzonyi <leves(a)elte.hu> wrote: On Wed, 27 Nov 2013, gettimothy wrote: I am confused as to what the term 'Stack VM' refers to. "The StackInterpreter is an intermediate step after closures and before the JIT to ensure steady progress and on-time delivery of a substantially faster VM. The essential point, of course, is that a stack organization suits the use of native call and return instructions whose use, along with in-line cacheing techniques, substantially improve send and return performance." - http://www.mirandabanda.org/cogblog/2009/01/14/under-cover-contexts-and-the… I would say it's a CogVM without a JIT. Exactly. Another way of saying it is that the StackVM is an interpreter that doesn't use contexts. Instead it uses stack frames, much like a conventional language implementation. And remember "doesn't use contexts" doesn't mean "doesn't have contexts"; instead context objects are created when the program asks for them instead of on every send. The StackVM's performance is about 1.5x the Context Interpreter VM. The Cog VM is about 5x the Context Interpreter VM. Levente > Here is the outline I have. > > Smalltalk VM = Blue Book > Squeak VM = Tim Rowledge's OE-Tour.pdf Stack VM = ?? Cog = Squeak VM redesigned per http://www.mirandabanda.org/cogblog/about-cog/ thx, tty -- best,Eliot -- best,Eliot

2 1

Re: [squeak-dev] Stack VM = Squeak VM?
by gettimothy 27 Nov '13

27 Nov '13

Oops, I posted to squeak-dev instead of vm-dev. I have added a CC so we can continue there if you wish. Ahhhh, I see what's going on now, its an incremental approach..makes sense now. Smalltalk-80 VM (Blue Book) -> Squeak VM (OE-Tour.pdf)->Stack VM (http://www.mirandabanda.org/cogblog/2009/01/14/under-cover-contexts-and-the… (JIT, Stack To Register Mapping, ..http://www.mirandabanda.org/cogblog/about-cog/) ->Spur (changing the garbage collector and the object representation) ->Lazy Become (http://www.mirandabanda.org/cogblog/2013/09/13/lazy-become-and-a-partial-re…) >From your post here: http://www.mirandabanda.org/cogblog/2008/12/12/simulate-out-of-the-bochs/ I gather you used the StackVM with Alien and Bochs as your base to build the JIT for Cog . So then, in an earlier response to my inquiry on the ~$1,000,000 bounty for cmakeifying the process thread, you wrote ...then you could instead have a go at getting the Stack VM working in 64-bits, and that would involve getting the 64-bit VM simulator working in VMMaker. Which makes sense. Basically, I would be doing a 64 bit version of follow the leader on your existing 32 bit work where it should all come together at Spur. Does that sound about right? tty ---- On Wed, 27 Nov 2013 08:47:03 -0800 Eliot Miranda <eliot.miranda(a)gmail.com> wrote ---- On Wed, Nov 27, 2013 at 8:05 AM, Levente Uzonyi <leves(a)elte.hu> wrote: On Wed, 27 Nov 2013, gettimothy wrote: I am confused as to what the term 'Stack VM' refers to. "The StackInterpreter is an intermediate step after closures and before the JIT to ensure steady progress and on-time delivery of a substantially faster VM. The essential point, of course, is that a stack organization suits the use of native call and return instructions whose use, along with in-line cacheing techniques, substantially improve send and return performance." - http://www.mirandabanda.org/cogblog/2009/01/14/under-cover-contexts-and-the… I would say it's a CogVM without a JIT. Exactly. Another way of saying it is that the StackVM is an interpreter that doesn't use contexts. Instead it uses stack frames, much like a conventional language implementation. And remember "doesn't use contexts" doesn't mean "doesn't have contexts"; instead context objects are created when the program asks for them instead of on every send. The StackVM's performance is about 1.5x the Context Interpreter VM. The Cog VM is about 5x the Context Interpreter VM. Levente > Here is the outline I have. > > Smalltalk VM = Blue Book > Squeak VM = Tim Rowledge's OE-Tour.pdf Stack VM = ?? Cog = Squeak VM redesigned per http://www.mirandabanda.org/cogblog/about-cog/ thx, tty -- best,Eliot

1 0

Re: [Pharo-dev] The Second Coming of Java article
by Eliot Miranda 27 Nov '13

27 Nov '13

On Tue, Nov 26, 2013 at 12:49 PM, phil(a)highoctane.be <phil(a)highoctane.be>wrote: > FWIW, I'd love to have a working Pharo bytecode interpreter that works. > VMMaker currently doesn't have one it seems (earlier experiments didn't > worked for me). > > I am very interested with the VM, read the blue book, understand the > primitives, can somewhat read bytecode but what is needed now is the > ability to run/debug a VM inside Pharo itself. GDB'ing is okay but a pain > in the ass to understand what's going on. > > Also read the Tour of the OE of Tim Rowledge and Porting the VM etc. > > Also looked at the VMMaker package (Interpreter and Object Memory) + Slang. > > Now, getting an working interpreter would help me reach the next step. I > am not talking about the Stack interpreter, but the plain Interpreter. > > Any plans? > David Lewis and I want to see the Cog branch and the VMMaker proper merged and I definitely want the standard Interpreter to be married to Spur. But I have no cycles to do this, and I don't think David has many either. Volunteers welcome. > > Phil > > --- > Philippe Back > Dramatic Performance Improvements > Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 > Mail:phil@highoctane.be | Web: http://philippeback.eu > Blog: http://philippeback.be | Twitter: @philippeback > Youtube: http://www.youtube.com/user/philippeback/videos > > High Octane SPRL > rue cour Boisacq 101 | 1301 Bierges | Belgium > > Pharo Consortium Member - http://consortium.pharo.org/ > Featured on the Software Process and Measurement Cast - > http://spamcast.libsyn.com > Sparx Systems Enterprise Architect and Ability Engineering EADocX Value > Added Reseller > > > > > On Tue, Nov 26, 2013 at 9:33 PM, Stéphane Ducasse < > stephane.ducasse(a)inria.fr> wrote: > >> >> On Nov 26, 2013, at 1:21 PM, phil(a)highoctane.be wrote: >> >> I downloaded ST/X this morning and looked around for a couple hours. >> >> Impressive system. And impressive clients list/activities etc. >> <rant> >> And wow, Claus is yet another individual with top notch computer chops... >> Seems that this community has the highest density of high perf brains. I >> feel humbled indeed. >> </rant> >> >> What struck me was the speed. >> >> >> 15 years of working improving the VM pays off. >> This is why the work of Eliot on Spur are also important. What is also >> important is that more people can work on the VM >> for cleaning it so that we get a documented and good vehicule. >> And this will take time. We are working on a prototype to get forced to >> think about VM and learn. But this prototype will >> not replace Cog in the near future. So we should be happy that cog is >> there and improving. >> >> Stef >> >> Pharo really needs a huge speedup on the UI front. I am using a top of >> the line desktop system and still Pharo sometimes feels slow (some is due >> to algorithms - like the finder tool taking ages for a lot of things, but >> some is due to the UI system). >> >> This is not linked to the VM, as the MVC projects in Squeak are uber fast. >> >> Is the Text rewrite going to have an impact on this? >> >> On the front of interoperability, I personally have chosen to go the >> RabbitMQ route, it allows me to wire all kinds of things together with some >> room for scale. >> >> On the Java front itself, in the TCL community, we do have tclbend and >> also a package to do the same thing as STX:LIBJAVA. Truth be told, there >> hasn't been much traction in there. It works fine but that's it. >> http://www2.tcl.tk/1313 - usage sample http://www2.tcl.tk/14919 >> It also has fell behind in terms of keeping up with the latest versions >> of the environment. >> >> In PHP, there is the Quercus and Caucho Resin Server that allows to run >> PHP on top of the JVM and invoke Java from there. Some people run very fast >> stuff on that, and benefiting from all JEE abilities (JMS, JTA, Clustering, >> JAAS...) is really nice. http://www.caucho.com/resin-3.1/doc/quercus.xtp >> >> >> Phil >> >> >> >> >> On Tue, Nov 26, 2013 at 12:26 PM, Jan Vrany <jan.vrany(a)fit.cvut.cz>wrote: >> >>> On 26/11/13 10:40, Serge Stinckwich wrote: >>> >>>> On Tue, Nov 26, 2013 at 10:54 AM, Jan Vrany <jan.vrany(a)fit.cvut.cz> >>>> wrote: >>>> >>>>> On 26/11/13 03:28, askoh wrote: >>>>> >>>>>> >>>>>> Bravo Jan and your collaborators. You have done it. >>>>>> >>>>> >>>>> >>>>> Thanks. >>>>> >>>>> >>>>>> Anything preventing STX:LIBJAVA from being used in production >>>>>> environments? >>>>>> >>>>> >>>>> >>>>> Good question. STX:LIBJAVA is still a research phase. However, if >>>>> everything >>>>> goes fine, we might have first project using it for real >>>>> in couple months. >>>>> >>>>> >>>> What is important is also the licence. Do you use an open-source >>>> licence ? >>>> >>> >>> Strictly speaking - no. For various, historical reasons. >>> >>> The Smalltalk part STX:LIBJAVA support code is available under >>> the same terms as Smalltalk/X itself [1]. >>> >>> The code of the VM is not publicly available for various reasons, >>> though it is possible to get an access. Ask Claus Gittinger if >>> you're interested in details. >>> >>> >>>> Anything preventing Pharo and VisualWorks for using the technology >>>>>> also? >>>>>> >>>>>> Short answer: Time and money. >>>>> The way we did it requires significant changes to the virtual machine >>>>> (as we believe this is the only way to get a decent performance). >>>>> Indeed, if >>>>> somebody going to pay for it, everything's possible ;-) >>>>> >>>> >>>> You should try to ask ESUG about financial support. >>>> >>>> >>> Maybe I should. But frankly - how many people in this community are like >>> "That would be great, I need this feature!" Raise hands. :-) >>> >>> Cheers, Jan >>> >>> [1] http://www.exept.de/cgi-bin/viewvc.cgi/stx/README?view=markup >>> >>> >>> >> >> > -- best, Eliot

4 3

VM Maker: VMMaker-dtl.330.mcz
by commits＠source.squeak.org 27 Nov '13

27 Nov '13

David T. Lewis uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker-dtl.330.mcz ==================== Summary ==================== Name: VMMaker-dtl.330 Author: dtl Time: 26 November 2013, 9:33:55.06 pm UUID: 4dabd4f4-3bed-4c10-9c4e-4bfa69037719 Ancestors: VMMaker-dtl.329 Fix loading of format 6505 (Cog) images in InterpreterSimulator in float word order fixup. An interpreter must send #lengthOf:baseHeader:format: to its object memory, not to self. No change to generated code, so versionString is unchanged. =============== Diff against VMMaker-dtl.329 =============== Item was changed: ----- Method: Interpreter>>install:inAtCache:at:string: (in category 'indexing primitives') ----- install: rcvr inAtCache: cache at: atIx string: stringy "Install the oop of this object in the given cache (at or atPut), along with its size, format and fixedSize" | hdr fmt totalLength fixedFields | <var: #cache type: 'sqInt *'> hdr := objectMemory baseHeader: rcvr. fmt := (hdr >> 8) bitAnd: 16rF. (fmt = 3 and: [self isContextHeader: hdr]) ifTrue: ["Contexts must not be put in the atCache, since their size is not constant" ^ self primitiveFail]. + totalLength := objectMemory lengthOf: rcvr baseHeader: hdr format: fmt. - totalLength := self lengthOf: rcvr baseHeader: hdr format: fmt. fixedFields := objectMemory fixedFieldsOf: rcvr format: fmt length: totalLength. cache at: atIx+AtCacheOop put: rcvr. stringy ifTrue: [cache at: atIx+AtCacheFmt put: fmt + 16] "special flag for strings" ifFalse: [cache at: atIx+AtCacheFmt put: fmt]. cache at: atIx+AtCacheFixedFields put: fixedFields. cache at: atIx+AtCacheSize put: totalLength - fixedFields. ! Item was changed: ----- Method: Interpreter>>stObject:at: (in category 'array primitive support') ----- stObject: array at: index "Return what ST would return for <obj> at: index." | hdr fmt totalLength fixedFields stSize | <inline: false> hdr := objectMemory baseHeader: array. fmt := (hdr >> 8) bitAnd: 16rF. + totalLength := objectMemory lengthOf: array baseHeader: hdr format: fmt. - totalLength := self lengthOf: array baseHeader: hdr format: fmt. fixedFields := objectMemory fixedFieldsOf: array format: fmt length: totalLength. (fmt = 3 and: [self isContextHeader: hdr]) ifTrue: [stSize := self fetchStackPointerOf: array] ifFalse: [stSize := totalLength - fixedFields]. ((objectMemory oop: index isGreaterThanOrEqualTo: 1) and: [objectMemory oop: index isLessThanOrEqualTo: stSize]) ifTrue: [^ self subscript: array with: (index + fixedFields) format: fmt] ifFalse: [self primitiveFail. ^ 0].! Item was changed: ----- Method: Interpreter>>stObject:at:put: (in category 'array primitive support') ----- stObject: array at: index put: value "Do what ST would return for <obj> at: index put: value." | hdr fmt totalLength fixedFields stSize | <inline: false> hdr := objectMemory baseHeader: array. fmt := (hdr >> 8) bitAnd: 16rF. + totalLength := objectMemory lengthOf: array baseHeader: hdr format: fmt. - totalLength := self lengthOf: array baseHeader: hdr format: fmt. fixedFields := objectMemory fixedFieldsOf: array format: fmt length: totalLength. (fmt = 3 and: [self isContextHeader: hdr]) ifTrue: [stSize := self fetchStackPointerOf: array] ifFalse: [stSize := totalLength - fixedFields]. ((objectMemory oop: index isGreaterThanOrEqualTo: 1) and: [objectMemory oop: index isLessThanOrEqualTo: stSize]) ifTrue: [self subscript: array with: (index + fixedFields) storing: value format: fmt] ifFalse: [self primitiveFail]! Item was changed: ----- Method: Interpreter>>stSizeOf: (in category 'array primitive support') ----- stSizeOf: oop "Return the number of indexable fields in the given object. (i.e., what Smalltalk would return for <obj> size)." "Note: Assume oop is not a SmallInteger!!" | hdr fmt totalLength fixedFields | <inline: false> hdr := objectMemory baseHeader: oop. fmt := (hdr >> 8) bitAnd: 16rF. + totalLength := objectMemory lengthOf: oop baseHeader: hdr format: fmt. - totalLength := self lengthOf: oop baseHeader: hdr format: fmt. fixedFields := objectMemory fixedFieldsOf: oop format: fmt length: totalLength. (fmt = 3 and: [self isContextHeader: hdr]) ifTrue: [^ self fetchStackPointerOf: oop] ifFalse: [^ totalLength - fixedFields]! Item was changed: ----- Method: InterpreterPrimitives>>primitiveInstVarAt (in category 'object access primitives') ----- primitiveInstVarAt | index rcvr hdr fmt totalLength fixedFields value | index := self stackIntegerValue: 0. rcvr := self stackValue: 1. self successful ifTrue: [hdr := objectMemory baseHeader: rcvr. fmt := hdr >> 8 bitAnd: 15. + totalLength := objectMemory lengthOf: rcvr baseHeader: hdr format: fmt. - totalLength := self lengthOf: rcvr baseHeader: hdr format: fmt. fixedFields := objectMemory fixedFieldsOf: rcvr format: fmt length: totalLength. (index >= 1 and: [index <= fixedFields]) ifFalse: [self primitiveFail]]. self successful ifTrue: [value := self subscript: rcvr with: index format: fmt]. self successful ifTrue: [self pop: argumentCount + 1 thenPush: value]! Item was changed: ----- Method: InterpreterPrimitives>>primitiveInstVarAtPut (in category 'object access primitives') ----- primitiveInstVarAtPut | newValue index rcvr hdr fmt totalLength fixedFields | newValue := self stackTop. index := self stackIntegerValue: 1. rcvr := self stackValue: 2. self successful ifTrue: [hdr := objectMemory baseHeader: rcvr. fmt := hdr >> 8 bitAnd: 15. + totalLength := objectMemory lengthOf: rcvr baseHeader: hdr format: fmt. - totalLength := self lengthOf: rcvr baseHeader: hdr format: fmt. fixedFields := objectMemory fixedFieldsOf: rcvr format: fmt length: totalLength. (index >= 1 and: [index <= fixedFields]) ifFalse: [self primitiveFail]]. self successful ifTrue: [self subscript: rcvr with: index storing: newValue format: fmt]. self successful ifTrue: [self pop: argumentCount + 1 thenPush: newValue]! Item was changed: ----- Method: InterpreterPrimitives>>primitiveStringReplace (in category 'indexing primitives') ----- primitiveStringReplace " <array> primReplaceFrom: start to: stop with: replacement startingAt: repStart <primitive: 105> " | array start stop repl replStart hdr arrayFmt totalLength arrayInstSize replFmt replInstSize srcIndex | array := self stackValue: 4. start := self stackIntegerValue: 3. stop := self stackIntegerValue: 2. repl := self stackValue: 1. replStart := self stackIntegerValue: 0. self successful ifFalse: [^ self primitiveFail]. (objectMemory isIntegerObject: repl) ifTrue: ["can happen in LgInt copy" ^ self primitiveFail]. hdr := objectMemory baseHeader: array. arrayFmt := hdr >> 8 bitAnd: 15. + totalLength := objectMemory lengthOf: array baseHeader: hdr format: arrayFmt. - totalLength := self lengthOf: array baseHeader: hdr format: arrayFmt. arrayInstSize := objectMemory fixedFieldsOf: array format: arrayFmt length: totalLength. (start >= 1 and: [start - 1 <= stop and: [stop + arrayInstSize <= totalLength]]) ifFalse: [^ self primitiveFail]. hdr := objectMemory baseHeader: repl. replFmt := hdr >> 8 bitAnd: 15. + totalLength := objectMemory lengthOf: repl baseHeader: hdr format: replFmt. - totalLength := self lengthOf: repl baseHeader: hdr format: replFmt. replInstSize := objectMemory fixedFieldsOf: repl format: replFmt length: totalLength. (replStart >= 1 and: [stop - start + replStart + replInstSize <= totalLength]) ifFalse: [^ self primitiveFail]. "Array formats (without byteSize bits, if bytes array) must be same " arrayFmt < 8 ifTrue: [arrayFmt = replFmt ifFalse: [^ self primitiveFail]] ifFalse: [(arrayFmt bitAnd: 12) = (replFmt bitAnd: 12) ifFalse: [^ self primitiveFail]]. srcIndex := replStart + replInstSize - 1. "- 1 for 0-based access" arrayFmt <= 4 ifTrue: ["pointer type objects" start + arrayInstSize - 1 to: stop + arrayInstSize - 1 do: [:i | objectMemory storePointer: i ofObject: array withValue: (objectMemory fetchPointer: srcIndex ofObject: repl). srcIndex := srcIndex + 1]] ifFalse: [arrayFmt < 8 ifTrue: ["32-bit-word type objects" start + arrayInstSize - 1 to: stop + arrayInstSize - 1 do: [:i | objectMemory storeLong32: i ofObject: array withValue: (objectMemory fetchLong32: srcIndex ofObject: repl). srcIndex := srcIndex + 1]] ifFalse: ["byte-type objects" start + arrayInstSize - 1 to: stop + arrayInstSize - 1 do: [:i | objectMemory storeByte: i ofObject: array withValue: (objectMemory fetchByte: srcIndex ofObject: repl). srcIndex := srcIndex + 1]]]. "We might consider comparing stop - start to some value here and using forceInterruptCheck" self pop: argumentCount "leave rcvr on stack"! Item was changed: ----- Method: ObjectMemorySimulator>>firstIndexableField: (in category 'memory access') ----- firstIndexableField: oop "NOTE: overridden from Interpreter to add coercion to CArray" | hdr fmt totalLength fixedFields | self returnTypeC: 'void *'. hdr := self baseHeader: oop. fmt := (hdr >> 8) bitAnd: 16rF. + totalLength := self lengthOf: oop baseHeader: hdr format: fmt. - totalLength := interpreter lengthOf: oop baseHeader: hdr format: fmt. fixedFields := self fixedFieldsOf: oop format: fmt length: totalLength. fmt < 8 ifTrue: [fmt = 6 ifTrue: ["32 bit field objects" ^ self cCoerce: (self pointerForOop: oop + self baseHeaderSize + (fixedFields << 2)) to: 'int *']. "full word objects (pointer or bits)" ^ self cCoerce: (self pointerForOop: oop + self baseHeaderSize + (fixedFields << self shiftForWord)) to: 'oop *'] ifFalse: ["Byte objects" ^ self cCoerce: (self pointerForOop: oop + self baseHeaderSize + fixedFields) to: 'char *']!

1 0

VM Maker: VMMaker.oscog-eem.526.mcz
by commits＠source.squeak.org 27 Nov '13

27 Nov '13

Eliot Miranda uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-eem.526.mcz ==================== Summary ==================== Name: VMMaker.oscog-eem.526 Author: eem Time: 26 November 2013, 6:03:25.857 pm UUID: f9b89c93-d691-49b9-a39c-2de1201b301a Ancestors: VMMaker.oscog-eem.525 Implement following of potentially forwarded literal variables in the Cogit. =============== Diff against VMMaker.oscog-eem.525 =============== Item was added: + ----- Method: CogObjectRepresentation>>genEnsureRegNotForwarded:scratchReg: (in category 'compile abstract instructions') ----- + genEnsureRegNotForwarded: reg scratchReg: scratch + "Make sure that the obejct in reg is not forwarded. By default there is + nothing to do. Subclasses for memory managers that forward will override." + ^0! Item was added: + ----- Method: CogObjectRepresentationForSpur>>genEnsureRegNotForwarded:scratchReg: (in category 'compile abstract instructions') ----- + genEnsureRegNotForwarded: reg scratchReg: scratch + "Make sure that the obejct in reg is not forwarded." + | loop ok | + <var: #ok type: #'AbstractInstruction *'> + <var: #loop type: #'AbstractInstruction *'> + self assert: reg ~= scratch. + loop := cogit Label. + self genGetClassIndexOfNonImm: reg into: scratch. + cogit CmpCq: objectMemory isForwardedObjectClassIndexPun + R: TempReg. + ok := cogit JumpNonZero: 0. + self genLoadSlot: 0 sourceReg: reg destReg: reg. + cogit Jump: loop. + ok jmpTarget: cogit Label. + ^0! Item was changed: ----- Method: SimpleStackBasedCogit>>genPushLiteralVariable: (in category 'bytecode generators') ----- genPushLiteralVariable: literalIndex <inline: false> | association | association := self getLiteral: literalIndex. "N.B. Do _not_ use ReceiverResultReg to avoid overwriting receiver in assignment in frameless methods." self annotate: (self MoveCw: association R: ClassReg) objRef: association. + objectRepresentation genEnsureRegNotForwarded: ClassReg scratchReg: TempReg. objectRepresentation genLoadSlot: ValueIndex sourceReg: ClassReg destReg: TempReg. self PushR: TempReg. ^0! Item was changed: ----- Method: SimpleStackBasedCogit>>genStorePop:LiteralVariable: (in category 'bytecode generators') ----- genStorePop: popBoolean LiteralVariable: litVarIndex <inline: false> | association | self assert: needsFrame. association := self getLiteral: litVarIndex. self annotate: (self MoveCw: association R: ReceiverResultReg) objRef: association. + objectRepresentation genEnsureRegNotForwarded: ReceiverResultReg scratchReg: TempReg. popBoolean ifTrue: [self PopR: ClassReg] ifFalse: [self MoveMw: 0 r: SPReg R: ClassReg]. traceStores > 0 ifTrue: [self CallRT: ceTraceStoreTrampoline]. ^objectRepresentation genStoreSourceReg: ClassReg slotIndex: ValueIndex destReg: ReceiverResultReg scratchReg: TempReg! Item was changed: ----- Method: StackToRegisterMappingCogit>>genPushLiteralVariable: (in category 'bytecode generators') ----- genPushLiteralVariable: literalIndex <inline: false> | association freeReg | freeReg := self ssAllocatePreferredReg: ClassReg. association := self getLiteral: literalIndex. "N.B. Do _not_ use ReceiverResultReg to avoid overwriting receiver in assignment in frameless methods." "So far descriptors are not rich enough to describe the entire dereference so generate the register load but don't push the result. There is an order-of-evaluation issue if we defer the dereference." self annotate: (self MoveCw: association R: TempReg) objRef: association. + objectRepresentation genEnsureRegNotForwarded: TempReg scratchReg: freeReg. objectRepresentation genLoadSlot: ValueIndex sourceReg: TempReg destReg: freeReg. self ssPushRegister: freeReg. ^0! Item was changed: ----- Method: StackToRegisterMappingCogit>>genStorePop:LiteralVariable: (in category 'bytecode generators') ----- genStorePop: popBoolean LiteralVariable: litVarIndex <inline: false> | topReg valueReg association constVal | self flag: 'with better register allocation this wouldn''t need a frame. e.g. use SendNumArgs instead of ReceiverResultReg'. self assert: needsFrame. optStatus isReceiverResultRegLive: false. "N.B. No need to check the stack for references because we generate code for literal variable loads that stores the result in a register, deferring only the register push." association := self getLiteral: litVarIndex. constVal := self ssTop maybeConstant. "Avoid store check for immediate values" (self ssTop type = SSConstant and: [(objectRepresentation shouldAnnotateObjectReference: constVal) not]) ifTrue: [self ssAllocateRequiredReg: ReceiverResultReg. self annotate: (self MoveCw: association R: ReceiverResultReg) objRef: association. + objectRepresentation genEnsureRegNotForwarded: ReceiverResultReg scratchReg: TempReg. self ssStorePop: popBoolean toPreferredReg: TempReg. traceStores > 0 ifTrue: [self CallRT: ceTraceStoreTrampoline]. ^objectRepresentation genStoreImmediateInSourceReg: TempReg slotIndex: ValueIndex destReg: ReceiverResultReg]. ((topReg := self ssTop registerOrNil) isNil or: [topReg = ReceiverResultReg]) ifTrue: [topReg := ClassReg]. self ssPop: 1. self ssAllocateCallReg: topReg. "for the ceStoreCheck call in genStoreSourceReg:... below" self ssPush: 1. valueReg := self ssStorePop: popBoolean toPreferredReg: topReg. valueReg = ReceiverResultReg ifTrue: [self MoveR: valueReg R: topReg]. self ssAllocateCallReg: ReceiverResultReg. self annotate: (self MoveCw: association R: ReceiverResultReg) objRef: association. + objectRepresentation genEnsureRegNotForwarded: ReceiverResultReg scratchReg: TempReg. + traceStores > 0 ifTrue: - traceStores > 0 ifTrue: [self MoveR: topReg R: TempReg. self CallRT: ceTraceStoreTrampoline]. ^objectRepresentation genStoreSourceReg: topReg slotIndex: ValueIndex destReg: ReceiverResultReg scratchReg: TempReg!

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev November 2013