Vm-dev January 2010

vm-dev@lists.squeakfoundation.org

28 participants
47 discussions

Re: [Vm-dev] [Fwd: Image file loading]
by Igor Stasenko 23 Jan '10

23 Jan '10

Let me restate my proposal, because during private talk with John, we figured out that there is some misunderstanding. Platform or 'support' code is C code written by platform maintainer. Interpreter code is generated by VMMaker , and written by us :) What i proposing: Interpreter provides a single API function: Interpreter>>readImageFrom: handle, where handle is just a (void*) pointer, which only role is to serve as an argument , which interpreter should use to identify this resource, when calling a service functions, provided by platform code. In our specific case, the only service function, which we need is to ask platform code to read a next N bytes from given resource and store result at specified location: Platform>>readFrom: object into: memoryLocation size: bytesToRead here, the memoryLocation is a pointer to buffer, which Interpreter can obtain by allocating the memory or just a temporary buffer on stack. Now, it should be more clear, what Interpreter>>readImageFrom: handle, does: Interpreter>>readImageFrom: handle | header | self var: 'header' declareC: 'char[64]'. "first, read first 64 bytes - an image header' (Platform readFrom: handle into: header size: 64) ~~ 64 ifTrue: [ ^ nil ]. "parse the header, and check if it is valid etc" ((self badHeaderIn: header ) ifTrue: [ ^ nil ]. "Fail sooner" memsize := header longAt: xxx. (self isNotEnoughMemoryFor: memsize ) ifTrue: [ ^ nil ]. .... "now, we're ready to allocate the memory for image" memory := Platform allocate: memsize + self breathingRoom. "now read the image into allocated area" Platform readFrom: handle into: memory size: memsize. ---------------- Since allocation and reading is both controlled by platform code, one could do any tricks with it, like reserve an address space, but not commit it into physical memory, but instead lazily read only pages which is not yet in memory using a page-fault mechanism etc. -- Best regards, Igor Stasenko AKA sig.

2 1

[Fwd: Image file loading]
by Andreas Raab 22 Jan '10

22 Jan '10

Apologies, accidentally posted to squeak-dev instead of vm-dev. - A. -------- Original Message -------- Subject: Image file loading Date: Tue, 19 Jan 2010 14:06:00 -0800 From: Andreas Raab <andreas.raab(a)gmx.de> Reply-To: The general-purpose Squeak developers list <squeak-dev(a)lists.squeakfoundation.org> To: The general-purpose Squeak developers list <squeak-dev(a)lists.squeakfoundation.org> Newsgroups: gmane.comp.lang.smalltalk.squeak.general Folks - I think that one of the more clever hacks I did for the Android VM deserves proper generalization in the Interpreter. Since on Android the image is part of the (compressed) application package it's not easily possible to pass a file handle into the VM and have the VM read the image as an external file. Instead, I'm preloading the image file into memory and then hacked an implementation of the sqImageFile* and sqAlloc* functions that would basically operate on the image in memory. I think we should formalize this idea for two reasons: a) It allows preloading the image into memory on platforms where the image may not be in an external file. b) It trivially allows mmap()ing the entire image into memory (I think John does that on the iPhone). What I'm proposing is an additional VM entry point, called Interpreter>>readImageFromHeap: heapStart size: heapSize which takes to arguments: The pointer to the beginning of the (pre-allocated) memory portion where the image file has been loaded or mapped and a size for the allocated memory portion. The VM then assumes that beginning at heapStart there is an image provided that it needs to properly prepare (i.e., perform byte-reversal, pointer adjustment etc). The VMs would generally use this along the lines of: memStart = self malloc(heapSize); fread(imageFile, memStart, fsize(imageFile)); readImageFromHeapSize(memStart, heapSize); and of course a backwards compatible version of readImageFromFile:HeapSize:StartingAt: could do the same (although I'd rather drop that function altogether since it removes the need for supporting various sqImageFile* functions). I'd also say that the same pattern should be used for *writing* image files, that is we assemble the entire image in memory then call sqImageFileWrite(fileName, heapStart, heapSize) and sqImageFileWrite is really just a combo of fopen(); fwrite(); fclose(); I think this scheme is much nicer, less complicated and offers more options for implementing image loading than what we currently have. Comments? Cheers, - Andreas

5 24

issue with cmake
by John McIntosh 22 Jan '10

22 Jan '10

I noticed if you mkdir bld cd bld ../platforms/unix/cmake/configure --CFLAGS="-foo" -- Setting CFLAGS=-foo -- Using CFLAGS -g -fomit-frame-pointer -O2 make -n confirms that the CFLAGS used for gcc is the "-g -fomit-frame-pointer -O2" not the -foo you asked for. Now if you toss chicken entrails about you might get -- Setting CFLAGS=-fum -- Setting CFLAGS=-foo ---Setting -g -fomit-frame-pointer -O2 --Using CFLAGS -fum Or you repeat the ../platforms/unix/cmake/configure --CFLAGS="-foo" a couple of times, then it takes well maybe... Or various hand waving of CFLAGS= rm CMakeCache.txt maybe some can explain the magic, or fix the script? -- =========================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================

3 5

microsecond timing for GC work
by John M McIntosh 21 Jan '10

21 Jan '10

The attached change set and sq.h enable microsecond timing for GC work. Please review for consideration for VMMaker. I've added a named primitive primitiveMicrosecondClock "This is a named (not numbered) primitive in the null module (ie the VM)" self export: true. self pop: 1 thenPush: (self positive64BitIntegerFor: (self ioMicroSeconds)). I used the 'sqLong' data type, but we don't seem to have a usqLong type to get unsigned long long ? Since in this case I'd like the 64bit value to be positive only value... Where on OS-X Cocoa VM V5 it is sqLong ioMicroSeconds(void) { //API Documented struct timeval now; sqLong theTimeIs; gettimeofday(&now, 0); if ((now.tv_usec-= startUpTime.tv_usec) < 0) { now.tv_usec+= 1000000; now.tv_sec-= 1; } now.tv_sec-= startUpTime.tv_sec; theTimeIs = now.tv_usec; theTimeIs = theTimeIs + now.tv_sec * 1000000; return theTimeIs; } This follows the pattern for ioMicroMSecs which has it roots in the Unix code, which is: sqInt ioMSecs(void) { struct timeval now; gettimeofday(&now, 0); if ((now.tv_usec-= startUpTime.tv_usec) < 0) { now.tv_usec+= 1000000; now.tv_sec-= 1; } now.tv_sec-= startUpTime.tv_sec; return lowResMSecs= (now.tv_usec / 1000 + now.tv_sec * 1000); } An interesting thing here btw is that the unix variation uses "startUpTime" which gets set to a clock time at VM startup time, so the value zero is the start time of VM, versus a time from the squeak epoch time start value. Also I use theTimeIs to ensure we don't have a 64bit overflow in the math ** Assuming the C compiler actually realizes the math has to be long long for the now.tv_sec * 1000000 (interesting question perhaps).... *** and then I had to alter aCCodeGenerator var: #statGCTime type: #'sqLong'. aCCodeGenerator var: #statFullGCMSecs type: #'sqLong'. aCCodeGenerator var: #statIGCDeltaTime type: #'sqLong'. aCCodeGenerator var: #statIncrGCMSecs type: #'sqLong'. and startTime in fullGC and incrementalGC so that they are 64bit values. -- =========================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================

5 18

getLongFromFileswap bug?
by John M McIntosh 21 Jan '10

21 Jan '10

I'm sure David passed a note by my email a week back about the logic to read the header wasn't quite right and some problem with extraVMMemory. So I've noticed that foo->fullScreenFlag = getLongFromFileswap(f, swapBytes); returns 1 on a 64bit image. Is this mmm true or am I mistaken in what's in that slot? -- =========================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================

2 1

Mac VM 5.4b1 ships
by John M McIntosh 21 Jan '10

21 Jan '10

I've push the latest cocoa based 32/64bit & 64/64bit macintel/powerpc VM 5.4b1 This includes the logic for the microsecond clock, to call the clock the change set is attached. I've also built the 64/64bit VM for Intel that works with 64bit squeak images. Look for the folder experimental/64bits in the regular places found via http://smalltalkconsulting.com/squeak.html or ftp://ftp.smalltalkconsulting.com/experimental/64bit/ or http://homepage.mac.com/johnmci/FileSharing.html click to experimental, 64bits In a sub folder the file names are: Squeak 64-32 5.4b1.app.zip or Squeak 64-64 5.4b1.app.zip This *beta* VM runs in 32 or 64 bit mode on macIntel, and 32bit mode on powerpc to support Squeak 32bit images. Includes the freetype plugin, and requires os-x 10.5 or higher. The *beta* 64/64 VM runs only in 64 bit mode on macIntel to support Squeak 64bit images. It does not include FreeType at this time, and it requires os-x 10.5 or higher. -- =========================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================

1 0

[ANN] Squeak on Android
by Andreas Raab 21 Jan '10

21 Jan '10

Folks - Sorry for being quiet over the weekend but I was busy with my new toy - a shiny new unlocked Nexus One which arrived last week fresh from the Google store. As a result of my weekend distraction you can now download and run Squeak on any Android based phone. If you have an Android phone, simply search for "squeak" in the Android market place and it should find it right there (sorry, Google doesn't seem to give web access to the apps in the store so I can't send a link). Please be aware that this is not a full port yet. It's a weekend effort to show the basic feasibility. Lots (and I mean *lots*) of things are still missing from a full port (among those is text input and network support to name just two of the more glaring ones). However, I would be *very* interested to hear if (and how well) it works for other Android-based cell phones. So if you have a Motorola Droid or or a T-Mobile G1 give it a shot and post some benchmark results. Performance on the Nexus One is about what one would expect: With roughly 1M sends/sec and 30M bytecodes/sec it's not exactly rocking but it's quite usable for most tasks on a mobile device. (Input is *terrible* though; Squeak's UI is not made for fat-fingered clicks like mine :-) Since the code itself is quite messy at this point, it isn't published anywhere yet. If people are interested in contributing, please let me know and I'll share what I have even before it's cleaned up. Be warned though, unless you know how to deal with both the Android SDK and NDK, Java, JNI, and the Squeak VM it will be a very steep learning curve. And of course, attached a little screenshot (taken on the emulator; I haven't found out how to use my phone to make pictures of my phone :) Cheers, - Andreas PS. I don't recall micro benchmarks on the iPhone - John can you repost those to see how well the hardware stacks up?

2 5

Re: Numbered primitives in images
by Bert Freudenberg 18 Jan '10

18 Jan '10

Fwd from John Maloney ... - Bert - Begin forwarded message: > > From: John Maloney <jmaloney(a)media.mit.edu> > Date: 16. Januar 2010 16:21:59 MEZ > To: Bert Freudenberg <bert(a)freudenbergs.de> > Cc: Ian Piumarta <ian(a)vpri.org> > Subject: Re: [Vm-dev] Re: [squeak-dev] Numbered primitives in images > > Hi, Bert. > > Thanks for the alert. > > Yep, this would be a problem for Scratch. It would prevent sharing projects on the website. Normally, I would not mind sticking with an older VM, but that there is also a change in the sound support on Linux from ALSA to Pulse Audio that we need, and work on that is still ongoing. So we may need to use a current VM, at least on Linux. > > Any chance there are some other numbered primitives that could be co-opted for the closure prims? As I recall there were lots of unused numbered primitives. > > -- John > > On Jan 16, 2010, at 9:57 AM, Bert Freudenberg wrote: >> That may be a problem ... >> >> - Bert - >> >> Begin forwarded message: >>> >>> From: "David T. Lewis" <lewis(a)mail.msen.com> >>> Date: 16. Januar 2010 15:55:02 MEZ >>> To: "K. K. Subramaniam" <subbukk(a)gmail.com> >>> Cc: vm-dev(a)lists.squeakfoundation.org >>> Subject: [Vm-dev] Re: [squeak-dev] Numbered primitives in images >>> Reply-To: Squeak Virtual Machine Development Discussion <vm-dev(a)lists.squeakfoundation.org> >>> >>> >>> On Sat, Jan 16, 2010 at 08:16:06AM +0530, K. K. Subramaniam wrote: >>>> On Friday 15 January 2010 10:46:45 pm Bert Freudenberg wrote: >>>>> On 15.01.2010, at 16:58, K. K. Subramaniam wrote: >>>>>> Scratch 1.4 (Linux) of 30-Nov-2009 on 3.11-3 VM. >>>>>> Break into Scratch and do "Socket initializeNetwork". >>>>> >>>>> So it does not occur in normal operation? >>>> Only if you define "normal" as "offline" ;-). Any operation which uses networking >>>> (url block) goes through this method and it will fail when run under the >>>> 3.11.3 VM built from unix-3.11.3-vmm or svn trunk. >>>> >>>> BTW, Scratch image is just an example which has numbered primitives. >>>> >>>> The 3.10-4 VM has primitiveObsoleteIndexedPrimitive in slot 207 in >>>> platforms/unix/src/vm/interp.c while the 3.11.3 tar.gz and the trunk contain >>>> primitiveFail. The change crept in r1935 through vmm-dtl-126. >>>> >>>> Is this intentional or an oversight? >>> >>> Sorry I was not able to reply earlier. The change was intentional, >>> but the effect on Scratch was not. >>> >>> Some of the obsolete primitives had to be dropped when we added the >>> block closure support. This happened in February 2009 (VMMaker-eem.114 >>> in the SqS/VMMaker). Specifically, we changed from this, which redirects >>> the numbered primitives: >>> >>> "Networking Primitives (200-225) - NO LONGER INDEXED" >>> (200 225 primitiveObsoleteIndexedPrimitive) >>> >>> to this, which reallocates the numbers for closure support and for >>> future COG integration: >>> >>> "new closure primitives (were Networking primitives)" >>> (200 primitiveClosureCopyWithCopiedValues) >>> (201 primitiveClosureValue) "value" >>> (202 primitiveClosureValue) "value:" >>> (203 primitiveClosureValue) "value:value:" >>> (204 primitiveClosureValue) "value:value:value:" >>> (205 primitiveClosureValue) "value:value:value:value:" >>> (206 primitiveClosureValueWithArgs) "valueWithArguments:" >>> >>> (207 209 primitiveFail) "reserved for Cog primitives" >>> >>> The obsolete primitive redirection had been in place for at least five >>> years, so backward compatibility had been preserved for some time. >>> >>> Question for Subbu: Is this an important issue may affect a large number >>> of Scratch users? In other words, are there kids trying to run Scratch, >>> and their Scratch image stops working properly after Squeak gets updated >>> to use a new VM? >>> >>> And a thought for the vm-dev list: I don't know if there might be a >>> technical solution to this, but it might be possible to use a strategy >>> in which the some of the old primitive lookups are preserved (at least >>> for the range 207 through 225) if and only if the image format is >>> pre-closures. Follow ups to this on vm-only. >>> >>> Dave >

6 11

Re: [squeak-dev] Numbered primitives in images
by David T. Lewis 17 Jan '10

17 Jan '10

On Sat, Jan 16, 2010 at 08:16:06AM +0530, K. K. Subramaniam wrote: > On Friday 15 January 2010 10:46:45 pm Bert Freudenberg wrote: > > On 15.01.2010, at 16:58, K. K. Subramaniam wrote: > > > Scratch 1.4 (Linux) of 30-Nov-2009 on 3.11-3 VM. > > > Break into Scratch and do "Socket initializeNetwork". > > > > So it does not occur in normal operation? > Only if you define "normal" as "offline" ;-). Any operation which uses networking > (url block) goes through this method and it will fail when run under the > 3.11.3 VM built from unix-3.11.3-vmm or svn trunk. > > BTW, Scratch image is just an example which has numbered primitives. > > The 3.10-4 VM has primitiveObsoleteIndexedPrimitive in slot 207 in > platforms/unix/src/vm/interp.c while the 3.11.3 tar.gz and the trunk contain > primitiveFail. The change crept in r1935 through vmm-dtl-126. > > Is this intentional or an oversight? Sorry I was not able to reply earlier. The change was intentional, but the effect on Scratch was not. Some of the obsolete primitives had to be dropped when we added the block closure support. This happened in February 2009 (VMMaker-eem.114 in the SqS/VMMaker). Specifically, we changed from this, which redirects the numbered primitives: "Networking Primitives (200-225) - NO LONGER INDEXED" (200 225 primitiveObsoleteIndexedPrimitive) to this, which reallocates the numbers for closure support and for future COG integration: "new closure primitives (were Networking primitives)" (200 primitiveClosureCopyWithCopiedValues) (201 primitiveClosureValue) "value" (202 primitiveClosureValue) "value:" (203 primitiveClosureValue) "value:value:" (204 primitiveClosureValue) "value:value:value:" (205 primitiveClosureValue) "value:value:value:value:" (206 primitiveClosureValueWithArgs) "valueWithArguments:" (207 209 primitiveFail) "reserved for Cog primitives" The obsolete primitive redirection had been in place for at least five years, so backward compatibility had been preserved for some time. Question for Subbu: Is this an important issue may affect a large number of Scratch users? In other words, are there kids trying to run Scratch, and their Scratch image stops working properly after Squeak gets updated to use a new VM? And a thought for the vm-dev list: I don't know if there might be a technical solution to this, but it might be possible to use a strategy in which the some of the old primitive lookups are preserved (at least for the range 207 through 225) if and only if the image format is pre-closures. Follow ups to this on vm-only. Dave

5 4

Ubuntu Linux 9 and unix install
by John M McIntosh 16 Jan '10

16 Jan '10

Ok, I'm just trying to install squeak on Ubuntu linux 9. when I do the make it seems to want to install sqUnixSoundPulseAudio although the readme says something about OSS being the unix deault and pulse audio an optional install? However on a base Ubuntu Linux 9 system when it attempts to compile sqUnixSoundPulseAudio.c it freaks and gives error: pulse/simple.h: No such file or directory and we are dead... So how do I configure it with either no sound? Or pick some other sound Or figure out how to get Sound pulse on Ubuntu to make the squeak make logic happy? -- =========================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================

8 18

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev January 2010