Squeak-dev June 2010

squeak-dev@lists.squeakfoundation.org

96 participants
269 discussions

Start a nNew thread

stream>>#nextPutAll: changed to return collection instead of the stream?
by Rob Withers 11 Jul '10

11 Jul '10

This has completely broken SSL. :-(

5 19

OSCog updated
by Eliot Miranda 08 Jul '10

08 Jul '10

Hi All, I've updated http://ftp.squeak.org/Cog with new source, OpenSourceCog-oscog.16.tar.gz/VMMaker-oscog.16.mcz that fixes the three issues people have found: Nicholas' crash on loading SUnit.dtl.?? which was a very nasty bug in the GC'ing of some trace logs the VM maintains. It was down to a difference between Smalltalk's \\ and C's %, specifically that the latter can answer something of different sign to the modulus. The failure of GeneratorTest, superficially due to cannotReturn: crashes, but fundamentally to do with inter-page returns to frames within the page not preserving the frames above, which can be in use due to co-routining (swapSender: usage). I hope this will fix Seaside on Cog. Please let me know. Crashes due to temps occurring in very simple blocks (that Cog compiles frameless) e.g. [ | b | 5 ] value. HTH Eliot

5 7

Teleplace Cog VMs are now available
by Eliot Miranda 08 Jul '10

08 Jul '10

Hi All, it gives me great pleasure to announce that the Teleplace Cog VMs are now available. Huge thanks to all at Teleplace who have given me the opportunity to build Cog and release it as open source, been willing guinea pigs braving its bugs, and providing indispensable participation in getting Cog to its current state. Huge thanks are also due to the original Back To The Future team whose VMMaker Cog extends to write the VM, and to Peter Deutsch from whom I've taken many ideas. This release contains two VMs. The Stack VM, is a cross-platform interpreter that uses context-to-stack mapping to achieve modest performance gains. The Cog VM is a just-in-time compiler that currently supports only x86 that builds upon the Stack VM to achieve substantial performance improvements. The release is in the form of a Monticello package containing the VMMaker source and a tarball containing the platform sources, the generated sources and a Squeak 4.1 image containing the VMMaker sources. Download both at http://ftp.squeak.org/Cog/VMMaker-oscog.11.mcz http://ftp.squeak.org/Cog/OpenSourceCog.tar.gz Cog VMs: The Cog VMs are Squeak/Croquet VMs that run closure Squeak/Croquet/Pharo/Cuis images. The VMs support existing plugin source but will require plugins to be recompiled as the VM_PROXY_MAJOR plugin api has been extended. This release contains two distinct VMs, the StackInterpreter and the Cogit. The StackInterpreter is a fully-portable plug-in replacement for the current closure Squeak VMs and images. The Stack VM uses context-to-stack mapping and a somewhat improved garbage collector to achieve modest but useful performance gains in the 10% to 15% range. The StackInterpreter is intended to supersede the Squeak VM on platforms where the Cogit cannot be used. The Cogit extends the StackInterpreter with a just-in-time compiler that uses aggressive inline caching techniques to deliver substantial performance gains in the 3x to 15x range, depending on benchmark. The Cogit currently supports only x86 and the floating-point primitives and parts of the platform support code depend on SSE2. I hope members of the community will attempt to port it, e.g. to ARM, PowerPC and x86-64. The Cogit (excuse the pun) is so named because it is both an interpreter and a JIT, choosing not to generate machine code for large methods, interpreting them instead, the default policy being not to JIT methods with more than 60 literals. The Cog VM requires a few minor image changes all in image/NecessaryImageChangesForCogToWork.1.cs. The JIT's machine-code SmallInteger primitives insist on a SmallInteger receiver so the primitives in LargePositiveInteger = ~= bitAnd: bitOr: butShift: and bitXor: cannot be used and these methods must be deleted. The Cogit inlines the address of the Character instance table, Smalltalk specialObjectsArray at: 25, into the machine-code at: primitive for faster ByteString>>at: and so the table cannot be rebuilt in SmalltalkImage>>recreateSpecialObjectsArray. The new version preserves the existing table. Both VMs maintain floats in platform order to ease implementation of machine code floating-point primitives, and hence internally are in little-endian order instead of big-endian in current Squeak images. While the VMs convert float order automatically on load they do require special accessing primitives Float>>basicAt: & Float>>basicAt:put: that undo the reversal and answer Float contents in big-endian order so that e.g. Float>>hash is unchanged. The methods assume these primitives can fail, allowing the code to be used on current Squeak VMs. The image/VMMaker-Squeak4.1.image is a Squeak 4.1 image, runnable with the current Squeak VMs, that contains these changes, and can hence also be run with a Cog VM. But beware, once an image has been saved on Cog it cannot be run by an existing Squeak VM, because existing VMs cannot undo the Float order change. Platform Subsystem: Most of the platform subsystem is unchanged but there are some important changes that need description. The biggest change is the heartbeat and the clock in platforms/unix/vm/sqUnixHeartbeat.c and platforms/win32/vm/sqWin32Heartbeat.c. The Cog VMs avoid the slow and variable interruptCheckCounter, folding the event check into the stack overflow check on frame build. The heartbeat, typically 500Hz or 1KHz, changes the stackLimit to a value that will always fail. On the next frame building send the VM will enter stack overflow handling that, as a side effect, will also check for events. This is more efficient than the update of interruptCheckCounter and much more regular. If one is running code that executes long-running primitives (e.g. large integer arithmetic) the counter approach will result in too low an interrupt check frequency, and conversely if one is running normal code the interrupt check frequency can be very high. The heartbeat also maintains a 64-bit microsecond clock, UTC microseconds from 1901, from which the backward-compatible millisecond and second clocks are derived. Primitives exist to answer UTC microseconds and local microseconds. Updating the clock in the heartbeat results in a 1 or 2 millisecond resolution but avoids the cost of accessing the OS time on every prim tie which we've found important for performance at Teleplace. The 64-bit microsecond clocks provide a unified time basis and eliminate wrapping (for the next 54,000 years at least). I hope community images will move to these clocks. It's worked well in VisualWorks. Another significant change is in the external semaphore table support code. This is now lock-free at the cost of having to specify a maximum number of external semaphores at start-up (default 256). The support code for the lock-free data structures are processor-specific and is currently implemented only for x86 and gcc-compatible compilers; see platforms/Cross/vm/{sqAtomicOps.h,sqMemoryFence.h}. There is also improved crash reporting code that prints a primitive log and a C backtrace in addition to the Smalltalk backtrace. See platforms/Mac OS/vm/sqMacMain.c, platforms/unix/vm/sqUnixMain.c, platforms/win32/vm/sqWin32Intel.c & platforms/win32/vm/sqWin32Backtrace.c. Finally there is support for the QVMProfiler, a pc-sampling profiler for profiling at the VM level. See platforms/unix/vm/sqUnixVMProfile.c and platforms/win32/vm/sqWin32VMProfile.c. The profiler itself is in the VMMaker image described below in Qwaq-VMProfiling. There are also changes to do with Teleplace-specific extensions to the HostWindowPlugin but these are not essential to Cog. VMMaker and Slang: The image/VMMaker-Squeak4.1.image Squeak 4.1 image contains the complete Cog VMMaker with necessary support code for simulation. This image was used to generate the sources in the src and stacksrc directories. Cog's VMMaker is substantially revised and extended from the current VMMaker. It supports multiple classes, not just Interpreter and superclasses, because both context-to-stack mapping and the Cogit are too complex to write monolithically. Classes can specify ancilliaryClasses and ancilliaryStructClasses, such as CoInterpreterStackPage, CogMethod and CogAbstractInstruction. The Monticello package version is included in the header of all generated files and constitutes the version stamp for generated code. Code is generated in sorted order so that minor changes in the Smalltalk source produce correspondingly minor changes in the generated code. The gnuification step is built-in to VMMaker. No effort has been made to maintain 64-bit compatibility. Apologies, this was unaffordable. The VMMaker generates a single source tree used by all platforms. Instead of deciding at generation time whether to use the Interpreter struct the generated code depends on the SQ_USE_GLOBAL_STRUCT define which can be overridden in platform makefiles. All plugins live in src/plugins and platform makefiles along with plugins.int and plugins.ext files in the build subdirectories decide which plugins are built as external or internal. The VM Generation Workspace from Workspace.text workspace contains dots to generate the sources. We no longer use the VMMakerTool since there should be nothing platform-specific in the generated sources (if we add ports to other ISAs all their source can be included and selected as required by the platform makefiles). Since the Cogit generates x86 machine code simulation is much more complex. There is a support plugin, platforms/Cross/plugins/BochsIA32Plugin that depends on a large simulation of the x86 family implemented in C++ (see processors/IA32/bochs) and on Alien. I use the simulator frequently (but note that I haven't had time to build a working version for Squeak 4.1). I have tested Cog simulation in this image, running on the image/VMMaker-Squeak4.1.image itself. The VM Simulation Workspace in the VMMaker image contains an example doit that starts the simulator. Be patient, even on a fast machine unhibernating the Squeak display background image takes nearly a minute. Native fonts do not (yet) simulate correctly, but the system runs. But note that I have only attempted to build and run the simulator on Mac OS X. I expect Bochs can be built on linux and win32 but I have not tried. By the way, I've not described how to run the Bochs simulator on the current Squeak VM. That's because the plugin depends on the heartbeat to break out of simulation occasionally via a new interpreterProxy entry point setInterruptCheckChain. As this isn't supported by the current Squeak VMs the plugin would require modification. So to simulate first build either of the Cog VMs and then run the simulation with it. There are a number of unpublished changes to the base other than those in NecessaryImageChangesForCogToWork.1.cs. This is partly laziness on my part, partly avoiding publishing things in advance of Cog. These changes are better motivated once Cog is in use. There are changes to the "translated primitives" (see implementors of translatedPrimitives) which replace messages with method tags for generation directives. The Cog VMMaker uses Object>>perform:with:with:with:with: & Object>>perform:with:with:with:with:with: during simulation, and Collection>>#fold: & SquenceableCollection>>#copyUpThrough: during generation. Object>>inline: and Object var:declareC:, which are mispackaged in Kernel in Squeak 4.1 are obsolete (method tags being used instead) and have been removed. I have changed Integer>>hex and Integer>>hex8 back to their original semantics as of 3.8. Backward compatibility is important and one can easily add new selectors if one wants different functionality. VMMaker was here first ;) Tarball: The top-level directories in the tarball are src the tree for the Cog generated sources including all plugins stacksrc/vm the directory containing the Stack VM source (plugins can be taken from above) platforms the usual svn platform tree but including Cog specific changes such as the heartbeat processors the tree containing simulation support code, i.e. the bochs C++ x86 simulation library, along with a potential ARM, PowerPC & MIPS simulator, Skeye. image the Cog-prepared Squeak 4.1 VMMaker image scripts some svn scripts to revert unchanged plugins that haven't really changed cygwinbuild the win32 build directory winbuild the old win32 build directory for minnow gcc 2.95. Not entirely obsolete as the cygwin build as yet fails to generate a functional FFIPlugin macbuild the CoreVM.xcodeproj and support build projects for Mac OS X 10.5 or better unixbuild the build directory for linux Building Cog: Each build directory above contains a HowToBuild file that describes building in more detail. The build directories only contain Cogit makefiles. f you want to build a Stack VM you're on your own but this is very close to the existing Squeak VM build. Status: The Cogit VM has been our sole VM at Teleplace for nearly a year. We do occasionally find bugs and there are almost certainly areas of functionality that we have not touched (for example I know that co-routining does not yet work). If you find a bug please try and create a reproducible test case and let me know. I can't promise to take a look or fix it but I am motivated to do so and will try my best as time allows. Better still if you find and fix bugs be sure to let me know. License (MIT): All contributions from Teleplace in this release are Copyright (c) 2010 Teleplace, Inc. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. Eliot Miranda June 2010

26 40

Cog performance
by Levente Uzonyi 06 Jul '10

06 Jul '10

Hi, I was curious how much speedup Cog gives when the code has only a few message sends, so I ran the following "benchmark": | s1 s2 | Smalltalk garbageCollect. s1 := String streamContents: [ :stream | 1000 timesRepeat: [ 'aab' do: [ :e | stream nextPut: e; cr ] ] ]. s2 := String streamContents: [ :stream | 1000 timesRepeat: [ 'abb' do: [ :e | stream nextPut: e; cr ] ] ]. [ TextDiffBuilder from: s1 to: s2 ] timeToRun. The above pattern makes TextDiffBuilder >> #lcsFor:and: run for a while. My results are a bit surprising: CogVM: 2914 SqueakVM: 1900 MessageTally shows that (I wonder if it's accurate with Cog at all) CogVM's garbage collector is a bit better, but it runs the code slower than SqueakVM: CogVM: **Leaves** 60.6% {1886ms} TextDiffBuilder>>lcsFor:and: 36.2% {1127ms} DiffElement>>= 1.8% {56ms} ByteString(String)>>= **GCs** full 1 totalling 153ms (5.0% uptime), avg 153.0ms incr 21 totalling 76ms (2.0% uptime), avg 4.0ms tenures 13 (avg 1 GCs/tenure) root table 0 overflows SqueakVM: **Leaves** 46.8% {888ms} TextDiffBuilder>>lcsFor:and: 35.3% {670ms} DiffElement>>= 9.8% {186ms} ByteString(String)>>compare:with:collated: 6.9% {131ms} ByteString(String)>>= **GCs** full 3 totalling 254ms (13.0% uptime), avg 85.0ms incr 301 totalling 110ms (6.0% uptime), avg 0.0ms tenures 272 (avg 1 GCs/tenure) root table 0 overflows Is Cog slower because #to:do: loops are not optimized, or is there some other reason for the slowdown? Levente

4 8

4.1 GUI Changes - Buttons not working in my code
by Jon Hylands 04 Jul '10

04 Jul '10

Hi Everyone, So, the new Cog VM has finally given me a reason to jump onto 4.1. I downloaded Andreas' 4.1 Windows zip file, installed it on my robot, and it starts up and runs with no issues. I loaded my Brainbot server code from my Monticello repository, but whenever I try and open any of my GUIs, the buttons are non-clickable. In fact, I can't even bring up their halo - when I right click, it switches between the system window and the alignment morph, but never the buttons. If I explore the alignment morph, I can of course see all the submorphs where they should be. This code works in a 3.10.2 image, which is what I've been using up until now. I'm adding PluggableButtonMorph objects to an AlignmentMorph, which is then added to a SystemWindowWithButton. Has something about these things changed between 3.10.2 and 4.1? Thanks, Jon

3 4

Re: Cryptography
by Rob Withers 03 Jul '10

03 Jul '10

Hi David, I am CCing the squeak-dev list. I had tried to contact Ron over the weekend, since he is the admin for the SqueakSource Cryptography package. I needed to add you as a member of the Crypto team. That's right, you are a member now. I didn't hear from him so the email address may be bad. We'll see if we hear from him in the future. I hope so. Do you have the link to the Crypto mailing list? I agree we should do this in public. I would suggest the squeak-dev list. We may have several "forks" and/or subsequent work done from the root package. The root package for SSL is in the Monticello repository http://www.squeaksource.com/Cryptography, but it may not be the latest. I think that the SSL package there is the latest...Whoa! I just checked again and new SSL packages were inserted 2 days ago. The latest there used to be SSL-rww.4.mcz and now there is a SSL-jrd.12.mcz. The root package for Cryptography is in the same repository, but it is less clear which is the latest and greatest. There is a mixture of version numbers. I am using Cryptography-cmm.13.mcz. All tests pass except for a Rindael test and an X509 test (due to bad validity dates). I know there is packages in http://croquet-src-01.oit.duke.edu:8886/Contributions. SSL look like the ones now in the Cryptography repository above. The Cryptography packages in Contributions look like they are also in the Cryptography repository, but mixed up with other work. In Contributions, there is the sequence of packages: Cryptography-RJT.10.mcz (the root package) Cryptography-jrd.11.mcz Cryptography-jrd.12.mcz Cryptography-mtf.13.mcz These are interwoven with other packages in the Cryptography repository, with the same version numbers. Lastly, David, it sounded like you have developments that you wanted to integrate. Cheers, Rob -------------------------------------------------- From: "C. David Shaffer" <cdshaffer(a)acm.org> Sent: Tuesday, June 29, 2010 2:23 PM To: "Rob Withers" <reefedjib(a)yahoo.com>; "Ron Teitelbaum" <Ron(a)USMedRec.com> Subject: Cryptography > Ron and Rob, > > I see that there is a Cryptography mailing list but that there hasn't > been a post since 2009. I would like updates to Cryptography to happen > "in public" since I think a lot of people are interested in these > packages. I can post to either forum. > > Based on what I've seen, integrating work that has been done may take a > few 'back-and-forths' so I just need to know in what forum to raise > questions. > > Finally, I know I'm not part of the Crypto team and I'm no expert in > cryptography or even the Squeak frameworks. What I see, however, is the > there is some code harvesting to be done and I think I can do a little > of it without causing any harm. I'm using SSL and MD5+HMAC heavily > under Squeak3.9 and Squeak4.1 (soon) and so I can exercise parts of the > framework past what is already covered by unit tests. Any other bugs > that I introduce can be fixed by harsh words from those that use those > bits of functionality. > > David >

4 6

BlockClosure asContext losing bound variable?
by Rob Withers 02 Jul '10

02 Jul '10

I am getting a walkback on this code, with a valid vat: EventualProcess class>>#newOnVat: vat ^ super new suspendedContext: [ vat processSends. Processor terminateActive] asContext; priority: Processor userSchedulingPriority; eventualName: 'vat thread'; vat: vat; yourself once it has been resumed. The exception is UndefinedObject(Object)>>doesNotUnderstand: #processSends. Somehow the vat variable inside the Closure/MethodContext has become nil. Anyone know about this stuff? Many thanks, Rob

3 2

Daily Commit Log
by commits＠source.squeak.org 01 Jul '10

01 Jul '10

Changes to Trunk (http://source.squeak.org/trunk.html) in the last 24 hours: http://lists.squeakfoundation.org/pipermail/packages/2010-June/003491.html Name: System-eem.345 Ancestors: System-tbn.344 Changes to ChangeRecord to support remove unchanged definitions and file-in reorganizations in the ChangeList. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2010-June/003492.html Name: Monticello-eem.396 Ancestors: Monticello-ar.395 Make sure MCWorkingCopy receives system change notifications by reregistering for them on startup. File-out-all for PatchBrowser (package changes). ============================================= http://lists.squeakfoundation.org/pipermail/packages/2010-June/003493.html Name: Collections-eem.366 Ancestors: Collections-ul.365 Much faster Dictionary>>unreferencedKeys (single pass over class hierarchy) speeds up removeUnreferencedKeys. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2010-June/003494.html Name: Collections-eem.367 Ancestors: Collections-eem.366 Fix bug in Symbol class>>selectorsContaining: which won''t find selectors containing digits. Add Symbol class>>selectorsMatching:. Both the above used in improved MessageNames ============================================= http://lists.squeakfoundation.org/pipermail/packages/2010-June/003495.html Name: Monticello-eem.396 Ancestors: Monticello-ar.395 Make sure MCWorkingCopy receives system change notifications by reregistering for them on startup. File-out-all for PatchBrowser (package changes). ============================================= http://lists.squeakfoundation.org/pipermail/packages/2010-June/003496.html Name: Tools-eem.248 Ancestors: Tools-jcg.247 Make MessageNames accept semicolon-separated patterns, so "at:put*;*remove*" answers all selectors beginning with "at:put" and all including remove. Needs Collections-eem.367. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2010-June/003497.html Name: Tools-eem.249 Ancestors: Tools-eem.248 ChangeList improvements - recognize class definitions and reorganization doits and class comments so they can be diffed - include class defs and comments in "remove unchanged definitions" so they are removed as well as unchanged methods, and in select methods for this class - provide a "select changes containing pattern" operation - provide a "select itemps for existig classes" operation - use class's parser class (if it exists) when parsing selectors, defaulting the selector to unparsableSelector if it can't be parsed. ============================================= http://lists.squeakfoundation.org/pipermail/packages/2010-June/003498.html Name: ToolBuilder-SUnit-nice.14 Ancestors: ToolBuilder-SUnit-ul.13 _ -> := =============================================

1 0

Very strange bug with Cog VM
by Stéphane Rollandin 30 Jun '10

30 Jun '10

hello, I stumbled upon this: 1) open an image with Cog 2) open a Worspace, evaluate 0.0 closeTo: 0 -> true (so far, so good) 3) now evaluate (1.0 closeTo: 1) and: [0.0 closeTo: 0] -> false ! 4) evaluate again 0.0 closeTo: 0 -> false ! 5) save the image and quit. 6) reopen the image, start again at 2... ??? Stef

3 6

The Trunk: ToolBuilder-SUnit-nice.14.mcz
by commits＠source.squeak.org 30 Jun '10

30 Jun '10

Nicolas Cellier uploaded a new version of ToolBuilder-SUnit to project The Trunk: http://source.squeak.org/trunk/ToolBuilder-SUnit-nice.14.mcz ==================== Summary ==================== Name: ToolBuilder-SUnit-nice.14 Author: nice Time: 30 June 2010, 10:27:57.162 pm UUID: 727c9741-25e8-422e-a205-6a8bd371dc0c Ancestors: ToolBuilder-SUnit-ul.13 _ -> := =============== Diff against ToolBuilder-SUnit-ul.13 =============== Item was changed: ----- Method: ButtonStub>>isEnabled (in category 'simulating') ----- isEnabled + enabled ifNil: [enabled := spec model perform: spec enabled]. - enabled ifNil: [enabled _ spec model perform: spec enabled]. ^ enabled! Item was changed: ----- Method: MenuStub>>click: (in category 'as yet unclassified') ----- click: aString | item | + item := self items detect: [:ea | ea label = aString] ifNone: [^ self]. - item _ self items detect: [:ea | ea label = aString] ifNone: [^ self]. item action isSymbol ifTrue: [self model perform: item action] ifFalse: [item action value]!

1 0

Jump to page: