On Wed, Jan 12, 2005 at 11:09:17PM -0800, Dan Ingalls wrote:
Ian and I were talking today about simple storage management schemes and realized that Squeak's current garbage collector is "so wonderful" that it has shielded us completely from a useful piece of information.
We were specifically curious how a non-moving garbage collector would perform. For such a beast, when an object is freed it is maintained in a structure from which it can later be reallocated. Most such approaches use some form of "buddy" system to coalesce adjacent free blocks but, inevitably there is some fragmentation of memory.
So here's the instrumentation challenge: Write a log of all the allocation and release operations from running Squeak through, say, the macro benchmarks.
There's a pretty simple place to find all the allocations in the VM. Then at each GC, each unmarked object that the compactor steps over should cause an effective release operation to be logged.
Such a log could then be fed to any number of block management algorithms to get a sense of what kind of fragmentation to expect. This, in turn, would tell us what kind of trade-off exists between the memory footprint used and the time between catastrophic reorganization (ie when you have to do a full compaction).
Partial credit: If you have real experience with a Squeak-like system that worked this way, give some report on the fragmentation behavior experienced.
References to papers with practical results in the area would also be of interest.
Thanks
- Dan
For my entry in the hotly contested better-late-than-never category, I am attaching AllocationLoggingPlugin-dtl.cs, a memory allocation logging goodie. Along with the change set, I've attached the modified sqVirtualMachine.c and sqVirtualMachine.h files and corresponding diff files (I'm not sure which is most convenient). These required a new entry to permit the allocation logging plugin to register its logging function for the interpreter to use.
From the preamble:
Log memory allocation activity to a file. Caution - may create very large log files.
AllocationLogger new log: 'memory.log' while: [Array new: 2000000] AllocationLogger new log: 'memory.log' while: [Smalltalk garbageCollect] AllocationLogger new log: 'memory.log' while: [MacroBenchmarks run]
The AllocationLoggingPlugin arranges for the interpreter to log interesting memory allocation and deallocation events to an external log file. The plugin registers a logging function (#log:with:with) that is called by the object memory to record events.
Primitives are provided to open and close a log file, and to control what events are to be included in the log. Records in the log file have comma separated fields with time stamp (ioMSecs), a memory function code such as 'Alloc' or 'Free', and two integer values that pertain to the memory function. See #initializeMemoryLoggingConstants for the memory functions that can be logged.
This plugin requires an update to the sqVirtualMachine.c and sqVirtualMachine.h to define the logging function.
In sqVirtualMachine.c:
#if VM_PROXY_MINOR > 6 VM->memoryLoggingFunction= NULL; /* set by a plugin */ #endif
In sqVirtualMachine.h:
#define VM_PROXY_MINOR 7
#if VM_PROXY_MINOR > 6 /* function to be provided by a plugin */ int (*memoryLoggingFunction)(int code, int param1, int param2); #endif
Note: #allocateChunk: also contains a change for a low space interrupt fix, commented out for this AllocationLoggingPlugin distribution. The two change sets will need to be merged.
Dave