Vm-dev April 2009

vm-dev@lists.squeakfoundation.org

22 participants
37 discussions

An idea about better finalization support
by Igor Stasenko 23 Apr '09

23 Apr '09

In recent topic on Seaside list, someone has asked, why Seasider's abandon the idea of using weak references (and use finalization technique when they die). The main issue was the slowness of weak finalization process implemented in Squeak. The problem: - sometimes developer needs to get a notification from system when particular object become garbage, so an additional processing could be activated, called finalization. How things currently working: Squeak provides a default finalization scheme implemented by WeakRegistry class, where one could register a pair of weak reference and executor. Its connected with finalization process , which governed by WeakArray class. Upon each GC, the finalization process awakes and sending #finalizeValues to all objects, registered in FinalizationDependents collection. When a WeakRegistry receives this message, it iterates over own WeakIdentityKeyDictionary instance to find a pairs, for which key (weak ref) became nil, and sends #finalize to the value (executor). The problem with such approach, is a linear time consumption spent for scanning all registered key>value pairs. If you have thousands of objects registered in that way, while only few of them could become garbage, the finalization process still will spend a fair amount of time, trying to determine which ones became garbage. To eliminate this waste, we need to implement a scheme in VM, which can help us to reach only those objects which is died during current GC cycle, leaving rest untouched. Here is what i'd like to propose: - add a special class Object weakSubclass: #WeakReferenceWithNotification instanceVariableNames: 'list next' classVariableNames: '' poolDictionaries: '' category: 'Collections-Weak' so, that VM can check this special kind of weak reference in ObjectMemory>>finalizeReference: method. A 'list' ivar can point to any object, we only need to be sure it having at least one slot to hold a list head: Object subclass: #FinalizationList instanceVariableNames: 'head' classVariableNames: '' poolDictionaries: '' category: 'Collections-Weak' now, all we need is to modify #finalizeReference: method, to do following at the moment of storing nil pointer: " this is not actually the slang code, just to illustrate what it does " (oop class isKindOf: WeakReferenceWithNotification) ifTrue: [ | list | list := oop at: ListSlotIndex. oop at: NextSlotIndex put: (list at: HeadSlotIndex). list at: HeadSlotIndex put: oop. ]. What it does, it simply links given oop to the list. And as you may suppose, if we have initial state: - list with head == nil - multiple instances with WeakReferenceWithNotification , all pointing to our list, and having a weak refs, then after GC, if some of the refs become nilled, in a list's head we will have a list of weak references we need to finalize (or do anything else we want to). Then a finalization process in case of WeakRegistry, don't needs to go through all weak references which are registered in itself, instead it could simply do: "suppose we use a subclass " WeakReferenceWithNotification weakSubclass: #WeakReferenceWithExecutor instanceVariableNames: 'executor' classVariableNames: '' poolDictionaries: '' category: 'Collections-Weak' ... ref := list head. ref notNil whileTrue: [ ref executor finalize. ref := ref next. ]. list head: nil. P.S. Me hopes ;) , that i successfully shown that the actual changes to VM is quite simple and minimal. But effects of such change could have a major impact on a finalization scheme performance. -- Best regards, Igor Stasenko AKA sig.

4 8

help with autoconf...
by Eliot Miranda 22 Apr '09

22 Apr '09

Hi All, I need to modify the autoconf scripts in the unix build to arrange to omit the -fomit-frame-pointer flag form on e of the files in Cog. I've asked Ian for help but he's not responded for whatever reason and I would like to solve the issue up front. Is anyone familiar enough with autoconf and has the free time to hold my hand through the process. I haven't got the patience to learn autoconf myself (fundamentally because I think its a heap of crap, waaaay too complicated to be worth the effort) but I would like to have the rest of the VM compiled with -fomit-frame-pointer, essentially for plugin performance, so I don't want to take the easy route and simply omit -fomit-frame-pointer throguhout, and I don't want to spend the time e.g. porting the Newspeak make framework, which would be a good thing in the long run. TIA Eliot

3 9

(fwd) Re: 6502
by Jecel Assumpcao Jr 22 Apr '09

22 Apr '09

Here is a discussion that I started on another list, but which we agreed that it would be better to have it here. I should mention that I do see that version numbers are too linear to properly represent development that often branches in complex ways. I would personally prefer to use image version numbers as we do now (to indicate radical changes) and have some special in-image object to represent the features that are needed. It could be a well known array of symbols (or strings) like #(exceptions weakPointers exupery utf8input) so the VM could check that it can fully work with that image. But I think something like this is even less likely to be accepted than incrementing 6502... ====forwarded==== Hi Jecel - I think this discussion is probably better done on either Squeak-dev or VM-dev, so I'll only add one short comment. The current VMs do support both the pre- and the post-closure format in precisely the way you are describing. Eliot made it so that an image is being marked as closure version if the closure creation bytecode is ever executed so that one may load a pre-closure image, run the conversion script, and save it as a post-closure version. In the future (i.e., when using the Stack VM) the pre-closure format will no longer be supported but for the interim period both are supported. Cheers, - Andreas Jecel Assumpcao Jr wrote: > I'm posting this here since it is a policy issue, but I would be equally > happy to discuss this in vm-dev or squeak-dev if anyone thinks those > would be better options. > > One interesting tool that we have in Squeak is the image version number. > This is supposed to allow a VM to check that it is compatible with some > image before trying to use it. Most images still use the original 6502 > version, with the 64 bit images having a different number (68000, so it > seems we have a theme of "important microprocessors in Apple history"). > > I think that the policy so far has been to save this tool for something > really big, so changes to the VM in the past have not resulted in new > image version numbers. An example would be the introduction of > exceptions. Certainly the new VMs could fully handle old images - these > would just never trigger the newly added code. Unfortunately, old VMs > would not be able to deal with new images. They might seem to for quite > a while and only if an exception were actually triggered would things > break down. So casual tests might indicate bidirectional compatibility > even if specific tests would prove that not to be the case. So keeping > the old number would lead some people (but not most, who tend to use the > very latest VMs) into trouble while bumping up the number would make the > new VMs refuse to load images even though it is perfectly fine for them > to do so. > > A simple solution would be for VMs to look for a list of compatible > image numbers rather than a single one. The new exception VMs would > accept either 6502 or 6503 images, for example. This would allow all > valid combinations to continue to work while preventing new images from > running on the old VMs where they could break. Of course, one worry is > that this list might grow to be a significant overhead for future VMs, > but that is unlikely to happen (specially if numbers are assigned wisely > so simple ranges could cover most cases). > > It seems that the fruits of Eliot's Cog work is being adopted in pieces. > There are several VMs and images out there using his closure changes, > and I hope we will next see the Stack Pages, the Jitter, new header and > the cleaner bytecodes get widely used. This last upgrade will no doubt > force us to select a new image number, but will all the previous ones > stick with 6502? > > An interesting thread on squeak-dev and vm-dev is about non ASCII input > from keyboard and clipboard. My own experience from this came late last > week when I was forced to reinstall all my applications after a hardware > replacement. So I went to download.squeak.org (nice to have this new > URL) and got the latest Linux VM for Squeak. This image (I use Celeste > for my email) seemed to work just fine as long as I was replying in > English, but pressing the key for ~ got me a debugger. Testing showed > that any deadkey character caused the same problem. A quick look at the > release history part of the Unix VM page showed that 3.10-1 had changed > the keyboard input. So I thought it was silly for me to try to use a 3.9 > image with a 3.10-4 VM and tested a 3.10.2 image instead. But it has the > same error - I suppose only the Etoys image has the needed code to deal > with the new VM events. My opinion is that proper use of the image > version number could at least have generated a nice warning (no big deal > for me - I just got a 3.7-7 VM instead and am using that, but what about > "normal" users?). > > I was hoping we could hold off all image changes for one big leap in > Squeak 5.0 since I have been comparing such a transition to the changes > Apple has gone through with the Mac and how disruptive this can be. But > now I feel this battle has been already lost and the current incremental > path will be the norm. So could we at least agree to use the only tool > we have, the image version number, to let people know what is supposed > to work and what isn't? > > Many cheers, > -- Jecel

1 0

Re: Pasting unicode (was Re: Pharo:: Re: Cuis update)
by Andreas Raab 21 Apr '09

21 Apr '09

Bert Freudenberg wrote: > Below that the paste from ru.wikipedia. These are actual question marks > (ASCII 63), the Mac VM does not paste Unicode even if setting > SqueakEncodingType to UTF-8 (only the extended clipboard plugin does this). Fortunately, at least that part is very easy to fix. I asked our CEO if we could release our clipboard support code back to the community and he agreed, so find attched an sqMacUIClipBoard.c version which does support Unicode copy/paste properly via UTF-8 (this should work as a drop-in replacement to the existing sqMacUIClipBoard.c). Brought to you by the friendly folks at Qwaq. Cheers, - Andreas

3 5

float word order
by Eliot Miranda 20 Apr '09

20 Apr '09

Hi All, I see that Float 32-bit word order is big-endian (PowerPC) on all platforms. This is a pain for performance and a pain for code generation in Cog. For example using SSE2 instructions it is trivial to swizzle a PowerPC-layout Float into an xmm register using the PSHUFD SSE2 instruction but tediously verbose to swizzle on write, because one has to swizzle to an xmm register which is hence destructive, which means three instructions (shuffle, write, unshuffle) just to write a Float result. Yes, ok 2 extra instructions is small potatoes, but they're still starch. So I wonder what would the impact be of maintaining Floats in platform order? There are a number of possible solutions. 1. Floats are always in platform order and swizzled on image load when moving from little-endian to big-endian or vice verce. Image code must be rewritten to take the platform's endianness into account. (requires an image rewrite) 2. As for 1 but the image is isolated from the change by providing two primitives, primitiveFloatAt and primitiveFloatAtPut which are implemented with selectors at: basicAt: at:put: and basicAt:put: on Float. These primitives map index 1 onto the most significant word and index 2 onto the least significant word. (requires no image rewrite, but does require a file-in of the four implementations) 3. as for 1 but the image is isolated from the change by providing four primitives primitiveFloatLowWord, primitiveFloatLowWordPut primitiveFloatHighWord & primitiveFloatHighWordPut (requires as much of a rewrite of image code as 1) 4. as per 1 but provide two primitives primitiveFloatBits prmitiveFloatBitsPut which answer or store 64-bit non-negative integers. (requires as much of a rewrite of image code as 1 but is cleaner and scales to 128 bit floats) 5. modify the existing at:[put:] primitives to check for Float receivers, e.g. (and in our Qwaq images Float has a compact class index of 6) from commonVariable:at:cacheIndex: fmt < 8 ifTrue: "Bitmap (& Float!!)" [(self compactClassIndexOf: oop) == ClassFloatCompactClassIndex ifTrue: [result := self fetchLong32: 2 - index ofObject: rcvr] ifFalse: [result := self fetchLong32: index - 1 ofObject: rcvr]. ^self positive32BitIntegerFor: result]. This slows down at: access for Bitmap and complicates an already overcomplicated, and performance-critical, primitive 6. eat it. do the swizzling on every float access 6. is apparently painless but actually absurd because we're unnecessarily throwing away performance for no good reason. 5. ditto, not for Float but for Bitmap access (and Bitmap is used in the vm simulator ;) ) 2. is my recommendation because it has least effort for adopters of solutions that provide maximum performance Opinions & alternatives? Especially, what are the likely issues of moving to platform Float order? Best Eliot

5 8

Linux package maintainers need help (was Re: Can you launch a squeak image on an etoys vm in Ubuntu?)
by Bert Freudenberg 16 Apr '09

16 Apr '09

(Sorry, this is going to be a longer post, covering a lot of basics, but I feel even most Linux Squeakers are unaware of how open-source software actually gets to users in Linux) On 15.04.2009, at 09:05, Jerome Peace wrote: > Chris Kassopulo ckasso at sprynet.com Tue Apr 14 02:00:01 UTC 2009 > wrote: >> I was told via bug report that Ubuntu doesn't support OSS at all. >> >> It is the opposite of what you are trying to do but on my Ubuntu box, >> I removed the Ubuntu squeak related packages and installed the DEB >> squeak-vm package. I can open squeak and etoys images with the >> command: squeak /path_to_image/image_name.image and sound works. >> > Atually this sounds like it is on the path to a solution. > > Can we get a DEB package that we can offer from the website in such > a way > that when Ubuntu goes searching for a squeak to install it installs > that vm? > > Bert: That that sound like the way to go? Not really, that would just be a workaround. In the closed-source world (Mac, Win) typically the software authors provide binary packages for end users. This is even true for open- source software on these platforms, the authors provide ready-to- install packages, separate from with the source code. That's why we have Windows and Mac downloads on our website. It's a one-size-fits- all approach, and all work is done by the authors. Not so in Linux. Here, building the binary packages that fit into a specific Linux distribution is typically done by users of that Linux version. These volunteers ("package maintainers") take the source code from an "upstream" author and make a binary package from that, typically by writing a few scripts that compile and install the software in the right place and with the right supporting files (which may be different for each Linux distribution). Because of these volunteers it is that you can simply install about *any* free software from your distro's "software installer". Every single one of these thousands of packages was added and is maintained by someone. As you can guess, package maintainers are usually no experts in the software they package. They will try the canonical "./configure; make; make install" command, and if that appears to work they make a package from that, do a few tweaks maybe and release it. People can install that package easily now, and everything appears to be fine. Except it's not. E.g., unless the package builder declared a build- dependency on the "alsa-lib-devel" package (that's what it is called in Fedora anyway) the Squeak VM will be built without ALSA sound support. The "configure" script is smart enough to not build the "vm- sound-ALSA" plugin if it does not find the necessary files. It also writes out a message that the plugin was disabled. But that is easy to overlook, and also not fatal - there are good reasons for not wanting to build that plugin. Now what needs to happen is that a user who discovers that sound is not working must file a bug report for the package *maintainer*. Not with the author (us), because that would bypass the maintainer. So use your distro's bug tracker, not bugs.squeak.org. The maintainer either fixes the problem, or opens an "upstream" bug with the authors. Sometimes it's even a chain of "upstreams", in Ubuntu's case the package may come from Debian, so a user would file a report at Ubuntu's tracker, the Ubuntu maintainer at Debian's tracker, and the Debian maintainer with us. If instead we bypass the maintainers, we get precisely the chicken-and- egg problem we are in now. The distro packages do not work well, so the experienced developers do not use them but compile their own. Which leaves the packages in their bad state because nobody knowledgeable cares about them. To be fair, in the "sound on Ubuntu" case in fact the proper procedure was followed: https://bugs.launchpad.net/ubuntu/+source/squeak-vm/+bug/235146 but apparently did not get fully resolved yet. Not sure what needs to happen next in this specific case. But in general, we should try to make it as easy for package maintainers as possible. Even if it is just by letting them know that their package is used. So Squeak developers on Linux should try to use the distro's packages, and if they are insufficient, give feedback. Also, the source code should come with clear instructions on how to build it, what dependencies there are etc. Maybe having a mailing list specifically for maintainers would be useful? But even just tracking down what packages are already out there, who is maintaining them etc. would be very valuable. And monitoring the bug trackers, participating on the distro-maintainers lists etc. Here is just a short list for squeak packages in various distros: Debian: http://packages.debian.org/search?keywords=squeak-vm Fedora: https://admin.fedoraproject.org/pkgdb/packages/name/squeak-vm Ubuntu: https://launchpad.net/ubuntu/+source/squeak-vm OpenSuSE: http://download.opensuse.org/repositories/devel:/languages:/smalltalk/openS… Gentoo: http://gentoo-portage.com/dev-lang/squeak Arch: http://aur.archlinux.org/packages.php?ID=13021 The good news is that once there are "good" packages in the major Linux distributions, the smaller ones can follow suite much more easily. The big issues of e.g. what build dependencies exist are solved by then, and everyone can learn from each other. Squeakers have been pretty much ignorant of the larger open-source community in the past. But I think there is a lot to be gained by becoming a proper part of it. I have been working towards that goal for a while now, in particular as part of the OLPC / Sugar development efforts. Etoys is distributed as part of Sugar and depends on an up-to- date Squeak-VM package, so we do have allies in the Sugar maintainers. And everyone can help - as I wrote above, no deep VM knowledge is necessary to monitor bug trackers or keep contact with the maintainers. So there. Who else wants to get involved? - Bert - PS: Note that I am primarily concerned with the VM packages for now. Packaging images is a somewhat separate issue, simpler because no dependencies are involved, and at the same time harder because there is no established practice of where to put them in the file system hierarchy. There used to be the license issue with images, too, but we should shoot for proper packaging with the license-clean upcoming 4.0 release. If you want to discuss this, please start a different thread.

3 3

Can Squeak VM be build with Visual Studio?
by Torsten Bergmann 16 Apr '09

16 Apr '09

If it wont work with VS the Eclipse CDT (C/C++ Development Tooling) may be another option if you look for an IDE instead of the command line. It works well with makefiles of any kind and is easy to use with GCC/MingW on Windows. http://www.eclipse.org/downloads (69MB download + local installed Java) Havent tried to compile/debug Squeak with it but should work out of the box. Bye Torsten -- Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger01

1 0

Can Squeak VM be build with Visual Studio?
by askoh 16 Apr '09

16 Apr '09

What problems would be encountered? How to solve them, if possible? Thanks, Aik-Siong Koh -- View this message in context: http://www.nabble.com/Can-Squeak-VM-be-build-with-Visual-Studio--tp23071492… Sent from the Squeak vm-dev mailing list archive at Nabble.com.

2 1

NEVER MIND: Can FFI be used with an Alien-aware VM?
by Jay Hardesty 16 Apr '09

16 Apr '09

Hello again - MY MISTAKE - turns out that the plugins at ftp.squeak.org/3.8/mac/carbonplugins/ are for PowerPC, so of course they didn't work on a Mac Intel. I instead copied the plugins (SqueakFFIPrims.bundle and TestOSAPlugin.bundle)from the Resources folder inside my 3.8VM, and both FFI and Applescript now work fine. SORRY for the false alarm - Thanks as always Jay

2 1

Can FFI be used with an Alien-aware VM?
by Jay Hardesty 16 Apr '09

16 Apr '09

Hi Elliot - Thanks for the reply - you're right - as explained my other message I dumbly had the wrong bundle versions installed. Thanks again Jay

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev April 2009