Vm-dev May 2012

vm-dev@lists.squeakfoundation.org

37 participants
74 discussions

Bochs simulator plug-in structure
by Chris Cunnington 12 May '12

12 May '12

http://osrcon.ca/screen4.png A slight correction. The host/vm image is only frozen, when it's scrolling output in that Morphic window. When it's not, the vm image can accept queries and produce details about the running vm. Very cool. Chris

1 0

Bochs simulator plug-in structure
by Lars 12 May '12

12 May '12

Hi all, I am Lars, one of this years GSoC students, and my project is implementing an ARM Simulator plugin and subsequently an ARM JIT for the Cog VM[1]. Last week, I had a meeting with my mentors, Eliot and Stefan, and I will try to summarize my understanding of the Cog VM structure here. The main topic of our chat was the overall structure of the project and a timeline, and the functions of the Bochs simulator for the CogVM x86 JIT. This summary is based on our protocols and my understanding and might not be correct. The Bochs x86 simulator[2] is a CogVM plugin writen in C to interface with the Bochs x86-64 C++ simulator[3]. The Bochs simulator can emulate an Intel x86 CPU and i.a. common I/O devices, a BIOS, and different instruction extensions like MMX. Of all that, only disassembling, single and multiple instruction execution and error decoding is used. The Bochs plugin keeps track of the ByteArray into which code has been compiled. It has to do so, because the Smalltalk VM garbage collector may move it around in memory. The plugin implements several (5?) primitives, which are: single instruction execution on given memory in form of a ByteArray; using the Bochs' disassembler, in order to compare the supposed meaning of some bytes from a ByteArray with its interpretation by an emulated CPU; execution of code from the byte array until an interrupt or error. Note that the CogVM uses a heartbeat (~once a second?) (for GC?) which is an interrupt and thus the simulator will not run longer than between two heartbeats; accessing the error codes which are provided by the Bochs; accessing the register state of the simulated CPU? The ByteArray to provide for the simulator contains an image: Objects, a Stack, and Instructions. But whenever the Bochs code comes to instructions where it would have to interpret Smalltalk code, the instructions are generated in such a way, that it accesses an illegal memory address and thus stops, back to the Smalltalk VM, which then interprets the Smalltalk Code and afterwards continues the simulation. So the plugin also maintains these false addresses. The simulator is used to have stepwise debugging and automated tests before coping with a real CPU, where debugging would be much more tedious. There has not been a definitive decision on which ARM simulator to use, but according to a previous try[mail was directly to Eliot], most open source ARM simulators are not worth using. I.a., QEMU is not viable[4]. So far, a good choice seems to be the GNU ARMulator[5]. Another one mentioned in our chat was SkyEye[6]. In order to use/test the simulator, an ARM compiler is needed. The CogVM includes an abstract instruction set as one of two basic intermediate languages. These instruction are for an abstract 2 register machine (because x86 is a 2 register machine) and are implemented in subclasses of CogAbstractInstruction for actual IAs. Instances of CogAbstractInstruction are single instructions for that abstract machine. The x86 subclass is CogIA32Instructions. ARM actually is a 3 register machine. Those abstract instruction objects are created by an instance of the Cogit and should provide assembler which then populates a ByteArray. This was an abstract overview of some two key parts which are needed for JITting. We meet again next week and Eliot plans to show me around the code, which implements the above described functionalities. At the moment, my semester is still running, so there might not be much progress. But in the beginning of June, most of the lectures will be done. All the best, Lars [1] http://gsoc2012.esug.org/projects/arm-jitter [2] http://www.mirandabanda.org/cogblog/2008/12/12/simulate-out-of-the-bochs/ [3] http://bochs.sourceforge.net/ [4] http://lists.squeakfoundation.org/pipermail/vm-dev/2012-March/010210.html [5] http://en.wikipedia.org/wiki/ARMulator [6] http://sourceforge.net/projects/skyeye/?source=directory

5 5

Bochs simulator plug-in structure
by Chris Cunnington 12 May '12

12 May '12

http://www.mirandabanda.org/files/Cog/VM/VM.r2550/ http://www.osrcon.ca/screen.png http://www.osrcon.ca/screen2.png http://www.osrcon.ca/screen3.png "you need a Boch plugin to run this code. This plugin is not shipped with VMs built by Jenkins server. Not sure about Eliot's builds." Running this image on the latest Cog build, which has Bochs, which you can see from the screen shots, is an advance. The simulator appears, but the whole image is frozen. Until it throws an error, as seen in screen2.png, the only living thing is the scrolling output in the bottom left pane. Of course, none of this may be a problem. I use the 2.7 InterpreterSimulatorLSB, where the simulated image runs, while the host image, the one with the vm is frozen. My solution is to plant a 'self halt' in Interpreter>>primtiveSomeInstance. Then I evaluate SystemTracer allInstances. Bang. The simulated image goes up in a puff of smoke leaving a debugger in the host image, which is pure gold. I can explore the values in the ObjectMemory and such. I imagine I could do something similar here. Chris

1 0

Bochs simulator plug-in structure
by Chris Cunnington 12 May '12

12 May '12

http://www.osrcon.ca/Eliot_Simulator.zip FWIW, here's the 4.1 I got from Deep Into Smalltalk in 2011 that runs the Bochs simulator (on a Mac). I saw Eliot make it work. When I ran the code at the time in the Workspace on the right, I think it only got as far as 'redirecting object pointers'. Now it throws a primitiveNewCPU failed error, while trying to instantiate BochsIA32Alien class>>new. That aside, it may be illustrative for somebody. Chris

2 1

VM Maker: VMMaker-oscog-golubovsky.138.mcz
by commits＠source.squeak.org 12 May '12

12 May '12

Dmitry Golubovsky uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker-oscog-golubovsky.138.mcz ==================== Summary ==================== Name: VMMaker-oscog-golubovsky.138 Author: golubovsky Time: 12 May 2012, 9:03:51 am UUID: 74cbe75b-0ecc-46f1-b933-ca088a8b69a1 Ancestors: VMMaker-oscog-golubovsky.137 Added primOnScreenKbd to show/hide onscreen Android keyboard upon Smalltalk code request. =============== Diff against VMMaker-oscog-golubovsky.137 =============== Item was added: + ----- Method: AndroidPlugin>>primOnScreenKbd (in category 'system primitives') ----- + primOnScreenKbd + | what vmcls vmobj bmmeth jmsg | + <var: 'what' type: 'int'> + <var: 'vmcls' type: 'jclass'> + <var: 'vmobj' type: 'jobject'> + <var: 'bmmeth' type: 'jmethodID'> + <export: true> + what := interpreterProxy stackIntegerValue: 0. + vmcls := self getVMClass. + vmcls ifNil: [^ interpreterProxy primitiveFail]. + vmobj := self getVMObject. + vmobj ifNil: [^ interpreterProxy primitiveFail]. + bmmeth := self getMethod: 'showHideKbd' withSig: '(I)V' inClass: vmcls. + bmmeth ifNil: [^ interpreterProxy primitiveFail]. + self callVoidMethod: bmmeth On: vmobj with: what. + interpreterProxy pop: 1. + !

1 0

VM Maker: Android-Base-golubovsky.24.mcz
by commits＠source.squeak.org 12 May '12

12 May '12

Dmitry Golubovsky uploaded a new version of Android-Base to project VM Maker: http://source.squeak.org/VMMaker/Android-Base-golubovsky.24.mcz ==================== Summary ==================== Name: Android-Base-golubovsky.24 Author: golubovsky Time: 12 May 2012, 8:51:20 am UUID: 23a9a082-6343-46b0-ba4a-6573a4377464 Ancestors: Android-Base-golubovsky.23 Added primitive onScreenKbd: anInteger. When called with 1, onscreen keyboard appears on screen, with 0 it disappears. =============== Diff against Android-Base-golubovsky.23 =============== Item was added: + ----- Method: Android class>>onScreenKbd: (in category 'system primitives') ----- + onScreenKbd: anInteger + "show/hide the onscreen keyboard" + <primitive: 'primOnScreenKbd' module: 'AndroidPlugin'> + self primitiveFailed.!

1 0

Re: [Vm-dev] Help/advice tracking down a squeak-vm regression
by Amos Blanton 11 May '12

11 May '12

> Amos, can you please clarify if further development of the Scratch > image is planned? No further development is planned by us. However, John and I spent a little time looking into this today to see about an easy fix. I'll pass on my impressions, and let him correct / make additional suggestions as necessary. There seem to be (at least) two issues preventing the current version of Scratch from working with later VMs. 1. The sound primitive changes David mentioned above (thanks for tracking this down, David!) Mantis 0007454: Removal of obsolete prim support for closures is a problem for Scratch images http://bugs.squeak.org/view.php?id=7454 John felt he could manage fixing this, but not soon. It's probably a couple hours of work. 2. However, even using an extremely simple image that just writes to a file, we get errors with later versions of the VM. Using this image: http://dl.dropbox.com/u/1979035/msqueak.image on 32 bit Lucid, running Squeak 4.4.7.2357 from Squeakvm.org, we get the following error: lightnin@386-lucid:~/Dropbox/MicroSqueak for Elliot$ squeak msqueak.image Exit to debugger at user request 2004272328 Object>handleExceptionName:context: 2004272236 Object>error: 2004272084 Object>primitiveFailed 2004271940 File>primOpen:writable: 2004271848 File>openReadWrite: 2004271308 >append:toFile: 2004271216 >log: 2004271124 >start 2004260548 [] in ProcessorScheduler>installStartProcess So there's seems to be an issue with file io that could be causing problems. It's difficult to know how hard this would be to fix in the Scratch image, and if there are other issues that would need to be addressed. If there is a squeak dev with time and interest who could look around and share their impressions, that would really help get a better grip on the scale of the problem(s). So it seems like there are a few ways forward: 1. Make a Scratch compatible VM, as David has suggested. 2. Ask heroic, generous, kind Squeak developers to help change the Scratch.image to make it compatible with later VMs. They can then re-release it under the GPL, or give it to us to release as an update to the source package. One problem with 1.) is that we won't be able to benefit from further development of a true 64 bit VM. (Still a possibility in the future I hope?) If we fork an earlier VM just for Scratch, we can have a working, easily installable 32 bit package, but all 64 bit users will need to go to the command line to force the installation of the 32 bit package on their 64 machines. This seems like a significant hurdle for many end users - especially the kind who would be interested in Scratch. Ideally, we want them to be able to install Scratch from their normal package manager on whatever distro they use. But perhaps 1.) is still the best / only solution, if 2.) isn't feasible. Best, -Amos On Thu, May 10, 2012 at 3:49 AM, Miriam Ruiz <miriam(a)debian.org> wrote: > 2012/5/10 David T. Lewis <lewis(a)mail.msen.com>: > > > Amos, can you please clarify if further development of the Scratch > > image is planned? If yes, then updating the Scratch image may be > > the right thing to do. If no, then we can take a harder look at > > coming up with a way to produce a backward compatible VM. If neither > > of these is feasible, then it will probably be necessary to arrange > > a separate source distribution for a Scratch VM. > > Hi! > > Sorry for the delay in answering to this thread. I think that it is a > very sensible suggestion you're making. There is one thing I'm > concerned about, though: if it is finally necessary to include an > aditional older version of the VM in Debian to be able to run Scratch, > as other people have suggested earliuer,¿who will maintain this VM? > > I mean, I can manage the packaging and some shallow patches if it is > necessary, as I do with all my packages, but as I'm not especially > proficient with the VM's code not I ever worked with it at a serious > level, so there's limits in what I can do. I suppose that Squeak > developers won't like to have to maintain two versions of the VM, and > the will probably just keep supporting, evolving and solving bugs from > the latest. I don't know if Scratch developers are willing or have > resources to solve big problems in the VM in case they appear, so that > leaves me probably close to being alone in fixing stuff in the VM if > something serious appear. I must confess that this perspective kind of > scares me. > > To sum up, if we finally have to keep a separate version of the VM for > running Scratch (and BYOB and other derivatives), we have to also find > out how to handle bugs and improvements in the VM in case that they're > needed. > > Greetings, > Miry >

3 2

Re: [Pharo-project] FFI in Fedora
by Casimiro de Almeida Barreto 10 May '12

10 May '12

On 09-05-2012 22:15, Schwab,Wilhelm K wrote: > my understanding is that 64 bits are not well supported at this time. > It's probably not what you want to hear, but have you thought of > making a 32 vm on your 64 bit machine? There are several possible combinations: 1) 32bit VM and 32bit image. 2) 64bit VM and 32bit image. 3) 64bit VM and 64bit image. As far as I know, there are no suitable (meaning stable, up to date and maintained) 64bit images. Thus option #3 is quite non-existent unless for image development purposes. It is posssible to have the traditional SqueakVM (stack vm) compiled in 64bit mode. It will work all right with 32bit images. It will interface ok with OS as far as you don't want to make your programs link to external libraries. This bar includes not only FFI but plugins as well. As far as I know, currently CogVM must suffer changes in order to be compiled in 64bit mode. So, option #2 is workable but very limited. It is possible to have 32bit VM and 32bit image provided you have a complete 32bit environment around you. That includes lots of things. For instance, some recent hardware comes with "optimus" technology for scaling GPU functionality depending on load. In Linux there is no support for "optimus" but a "wrap around" solution called bumblebee. Bumblebee allows activating and deactivating NVIDIA GPU by hand by means of a kernel module, a daemon and an application (optirun) but those depend on hardware architecture (i686 or x86_64). Thus squeak/pharo and anything depending on SqueakVM and PharoVM will fail in this environment. But there are lots of other things that depend on hardware architecture that won't work well if you have 32bit VM & image. By the way, currently bumblebee does not support statically linked OpenGL libraries (perhaps because it uses a VirtualGL server that masks OpenGL inside non OpenGL X-Window sessions but I have to further investigate that). Thing is: 32bit architecture is quite dead. There won't be new computers with this architecture (even ARM is 64bit and A15 has an interesting address structure allowing 1TByte memory space). Also, dynamic evolving OSes like Fedora (and even OS X) won't be stuck in the old 32bit model for long. Until 15 days ago the hardware platforms available to me were a 4 years old desktop computer and a 3 years old laptop. Both 32bits. Most of my developments involved interfaces with CUDA stuff (libraries I developed) and Fann (artificial neural network libraries). Then I got "upgraded" (both computers) and nothing is working and I have to choose between trying to fix things and keep working with squeak/pharo or to port everything to java/python/C++. Besides, I think it is opportune to move this discussion to VMdev, pharo and squeak lists. CdAB > > > > > ------------------------------------------------------------------------ > (...)

1 0

Re: [Pharo-project] StackVM on iOS: the return
by Esteban Lorenzano 10 May '12

10 May '12

Hi, yes... cmake is still not making a good job on creating xcode projects. For now, I touch that manually after generation, I still didn't find the time to correct it... if you find a way, please let me know. btw... I think this vm related stuff belongs to vm-dev list, I'm redirecting there. best, Esteban On May 10, 2012, at 9:46 AM, phil(a)highoctane.be wrote: > Hello > > I've been updating the blessed/ thing from git (pull, merge master), taken a fresh image, loaded the 4.7 and generated sources. > > Cmake and fix works. > > Project opens in XCode 4.3 > > But... > > The linking step fails because the .a files for the plugins aren't found. > > I can't figure out why since all this Cmake machinery makes a ton of scripts all over the place. > > Any clue to get me out of that? > > Thanks! > > Philippe Back

2 1

Re: [Vm-dev] Help/advice tracking down a squeak-vm regression
by David T. Lewis 10 May '12

10 May '12

Well, I for one would not put "John Maloney on the team" in the same sentence with "unfortunately" ;-) In any case, to bracket the problem a bit I tried running a Scratch image on the 3.7-7 Unix VM from http://squeakvm.org/unix/ and it seems to work fine. This VM was built from source in March 2005. Source code for that VM is at http://squeakvm.org/unix/release/Squeak-3.7-7.src.tar.gz (which may not include the the Scratch plugins, but is otherwise a workable source code base). Current Squeak VMs do not support a Scratch image, so the differences are due to changes in the VM / image interface that took place since 2005. Time permitting I will try next week to identify the major changes that that have taken place since 2005 that would affect Scratch support. In principle it should be possible to produce a VM that will run Scratch along with current Squeak/Pharo/etc although I'm afraid that this will be a difficult task in practice, so a separate source distribution for a VM to run Scratch may be a more pragmatic approach. With respect to patching the image, this is an extremely easy thing to do, but a difficult thing to manage. Unless there is active development of the Scratch image itself, it is probable better to focus effort on providing a VM that runs the Scratch image as currently distributed. But that is an idea that you might want to run by John for a second opinion if you can. Dave On Wed, May 09, 2012 at 01:48:19PM -0400, Amos Blanton wrote: > Thanks to everyone who is looking into this issue. > > Unfortunately, the only current member of the Scratch Team with Squeak > experience is John Maloney, and he is tied up working on Scratch 2.0 at the > moment. I'm told it will not be possible to divert his time towards > looking deeply into these issues - although he probably can respond to > general questions. > > So - if bundling the VM seems like the best solution, that would be fine. > We do bundle the VM in the Mac / Windows version of Scratch. The latest > official Debian package also bundles an older VM (not included in the > current GPL'd Scratch source tarball, but still available here: > http://www.assembla.com/spaces/scratchonlinux/trac_subversion_tool ). > However, we were once told that this may run afoul of packaging guidelines, > and that we should try to avoid it if possible. Perhaps those on thread > with more knowledge of packaging guidelines for Debian can say more? > > In addition - if someone would like to look into making changes to the > Scratch.image to improve compatibility with later VMs, that would be great. > I'm not sure how patching works with Squeak images, and if it would be > possible for packagers to patch the current source. But if that's > difficult, we could update the source package based on recommendations / > changes from the OSS / Squeak community. > > > On Tue, May 8, 2012 at 10:52 AM, David T. Lewis <lewis(a)mail.msen.com> wrote: > > > On Tue, May 08, 2012 at 02:39:45PM +0200, Bert Freudenberg wrote: > > > > > > > > > On 08.05.2012, at 14:21, David T. Lewis wrote: > > > > > > > > > > > On Tue, May 08, 2012 at 10:44:26AM +0100, Alex Bradbury wrote: > > > >> > > > >> David and Igor: thanks for your responses. Comments below. > > > >> > > > >> On 7 May 2012 00:28, David T. Lewis <lewis(a)mail.msen.com> wrote: > > > >>> If I remember correctly, Scratch derives from a Squeak 3.6 image and > > uses > > > >>> a VM of similar vintage. Since the days of Squeak 3.6, there have > > been > > > >>> a number of fundamental changes in the Squeak VM. I recall some > > discussion > > > >>> of this perhaps a couple of years ago, and I think that the > > prevailing > > > >>> view was that it made more sense to bring the Scratch image up to > > date > > > >>> with respect to the VM changes, as opposed to trying to make a single > > > >>> VM that could be backward compatible all the way to Squeak 3.6. > > > >> > > > >> Though the current squeak image did work flawlessly in squeak-vm 4.0? > > > >> So the upstream view is that this is not officially supported and the > > > >> correct solution if it no longer works on 4.4 is to just use an older > > > >> vm? > > > >> > > > >>> From a license point of view, is there any reason that the Scratch VM > > > >>> and source code could not be distributed on Debian? If this is > > acceptable, > > > >>> then I expect that the fastest way to achieve a working distribution > > would > > > >>> involve arranging for Scratch to run using a VM at e.g. > > /usr/bin/scratch, > > > >>> and Squeak to run with /usr/bin/squeak or /usr/bin/cog. > > > >> > > > >> It does seem that the Ubuntu packages bundles its own > > > >> scratch_squeak_vm. Either that, or supporting the installation of > > > >> squeak-vm 4.0 alongside 4.4 might be good options. For my purposes > > > >> with the Raspberry Pi, I'm probably just going to install and pin the > > > >> squeeze squeak-vm (4.0) until a better solution comes along. > > > >> > > > >> Alex > > > > > > > > On 8 May 2012 10:44, Alex Bradbury <asb(a)asbradbury.org> wrote: > > > >>> Though the current squeak image did work flawlessly in squeak-vm 4.0? > > > >>> So the upstream view is that this is not officially supported and the > > > >>> correct solution if it no longer works on 4.4 is to just use an older > > > >>> vm? > > > >> > > > >> squeak image -> scratch image > > > >> > > > >> Alex > > > > > > > > As far as I know, Scratch images have not worked on standard Squeak VMs > > > > for a number of years, and Scratch has always relied on shipping its > > > > own VM. In addition to fundamental changes in the VM itself that have > > > > taken place in recent years, I believe that the Scratch version of the > > > > VM also includes a couple of plugins (add-in functionality) that are > > > > required for Scratch but not included in Squeak. > > > > > > > > I think I see one source of confusion here. I just read the link at > > > > http://info.scratch.mit.edu/Source_Code and find the following > > statement: > > > > > > > > "Since Scratch simply uses a stock Squeak virtual machine, we are > > > > not distributing the Squeak virtual machine source code. Both the > > > > source code and pre-compiled binaries for the Squeak virtual machine > > > > are available at www.squeakvm.org." > > > > > > > > Unfortunately, I don't think that this is true, and hopefully someone > > > > can make a correction to that web page. > > > > > > > > The squeakvm.org code repository along with VMMaker at > > source.squeak.org > > > > does contain fully versioned copies of the Squeak VM, presumably > > including > > > > all or most of the code in the current Scratch VM (except for the > > Scratch > > > > specific plugins). So it would probably be possible to build a working > > > > Scratch VM using code from the standard Squeak VM repositories. But > > > > I fear this would take some work on the part of someone who knows > > > > Scratch well enough to find and build the right versions and put the > > > > pieces together. > > > > > > > > Dave > > > > > > Well, actually, Scratch (based on Squeak 2.8 IIRC) worked fine up to the > > 3.10 VM. E.g. OLPC's Scratch package does not bundle a VM but uses the > > stock Fedora Squeak VM (3.10). > > > > > > For a very long time we managed to keep VMs being able to run all Squeak > > images back to 1.x. The promise was new VM + old image = fine, but new > > images may require new VMs. Upgrading the VM should not unnecessarily break > > old images. > > > > > > So what did we change in 4.x VMs that it does not work anymore? > > > > I am not certain, but we will need to figure it if it is causing problems > > now for Scratch and Debian. I probably will not be able to look into > > this myself until next week though. > > > > I know that I can run a Squeak 3.6 image on the most up-to-date > > SqueakVM, but going back further in time gets problematic, and it > > definitely is not possible to run a 1.x image on current VMs. > > > > My recollection (not necessarily correct) is that the mechanism for > > delivering events from the VM to the image was changed in a fundamental > > way. There would probably also be some issues related to the special > > objects array and primitive number assignments. > > > > I think there was some discussion of this on the vm-dev list a > > couple of years ago, and some related emails with John Maloney at > > the time. I don't have a link right now, but I'll see if I can > > find it. > > > > Dave > > > > > > > > If these changes are really unavoidable and incompatible, then we should > > indeed advise distros to package both 3.x and 4.x VMs. > > > > > > - Bert - > > > > > > >

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev May 2012