Hi Tom,

On Fri, Jun 23, 2017 at 1:35 AM, Tom Beckmann tomjonabc@gmail.com wrote:

Dear VM devs,

I'm trying to allocate a Bitmap of around 16MB size in a plugin via `interpreterProxy instantiateClass: interpreterProxy classBitmap indexableSize: size`. However, I always get NULL back. I already tried wrapping the primitive call in `retryWithGC:until:`, with the same effect. Running a normal `Bitmap new: 16000000` from a workspace works flawlessly.

I'm on a Ubuntu 16.04 64bit, running a squeak 32bit built from trunk.

Is there anything to watch out for? More precisely, I'm trying to copy the contents of an image buffer generated from a pdf via libpoppler at 144dpi to a new bitmap in squeak. The primitive and how I invoke it can be seen here: https://pastebin.com/9NCdnx2J (line 50 is the call in question)

Thanks in advance for any pointers!

I want to endorse what Bert said; allocating in the image is a much better approach. But I did want to explain what you're seeing, assuming you're using Spur (Squeak 5.x, Pharo 6).

The Spur old space heap is segmented. It grows in increments defined by vm parameter 25 (Smalltalk vmParameterAt:[put:]), and shrinks whenever free memory is above parameter 24 and GC empties one or more segments. It will not grow memory beyond parameter 67, if that parameter is non-zero:

24 memory threshold above which to shrink object memory (read-write) 25 memory headroom when growing object memory (read-write) 67 the maximum allowed size of old space in bytes, 0 implies no internal limit (Spur only).

Parmeter 25 defaults to 16Mb (grow in increments of 16Mb) and 24 to 32 Mb (only shrink while 32Mb or more of empty segments exist).

Currently memory will /only/ grow in these circumstances - under image control via growMemoryByAtLeast: - when scavenging tenures sufficient objects to old space that old space must grow - when a fullGC fails to ensure there is at least vm parameter 25's worth of free space - when an allObjects or allInstances primitive needs more room to answer its result

The current allocator does /not/ try and grow memory in the allocation routines (which are accessed in plugins via instantiateClass:indexableSize: and in the image via new: basicNew: et al), failing instead. This gives the image the chance to mediate GC and heap growth (see e.g. Behavior>>handleFailing[Failing]BasicNew:).

Perhaps instantiateClass:indexableSize: /should/ allow growth but I prefer it remaining the same. You would then rewrite your primitive to - when it fails to allocate memory via instantiateClass:indexableSize: fail with interpreterProxy primitiveFailFor: PrimErrNoMemory - have the Smalltalk primitive failure code check the primitive error code (see senders of primitive:module:error:) and if the failure code is #'insufficient object memory' grow memory by the necessary amount and retry the primitive (a la handleFailingBasicNew:)

To the general audience, I think information like the above is key to being able to understand and exploit the system effectively, but where should it reside? Clearly "in my head" is not satisfactory. It belongs somewhere in the image, but it needs to be somewhere where people can find it and/or will look. Suggestions for an "architectural information" documentation section gratefully received. Object class>>whatIsAPrimitive might perhaps work or perhaps be overloaded.

_,,,^..^,,,_ best, Eliot