I'm wondering what 'shrink' means in the following VM parameter comments:
"24 memory threshold above which shrinking object memory (rw)"
and
"32 number of shrink memory requests (read-only)"
My first thought is that a shrink represents Squeak release memory back to the operating system once it is no longer needed.
As I've mentioned before we are trying to reduce the RAM footprint of the www.squeak.org process and are watching various related figures.
We can see the value in VM parameter 32 increase, but while the value for parameter 3 (end of memory) goes down as you would expect, the operating system's view of the memory usage does not change.
Ken
It *would* give the memory back to the operating system. But we don't anymore again read the following
/* Note: * * The code allows memory to be overallocated; i.e., the initial * block is reserved via mmap() and then the unused portion * munmap()ped from the top end. This is INHERENTLY DANGEROUS since * malloc() may randomly map new memory in the block we "reserved" * and subsequently unmap()ped. Enabling this causes crashes in * Croquet, which makes heavy use of the FFI and thus calls malloc() * all over the place. * * For this reason, overallocateMemory is DISABLED by default. * * The upshot of all this is that Squeak will claim (and hold on to) * ALL of the available virtual memory (or at least 75% of it) when * it starts up. If you can't live with that, use the -memory * option to allocate a fixed size heap. */
On 2010-10-22, at 6:24 AM, Ken Causey wrote:
I'm wondering what 'shrink' means in the following VM parameter comments:
"24 memory threshold above which shrinking object memory (rw)"
and
"32 number of shrink memory requests (read-only)"
My first thought is that a shrink represents Squeak release memory back to the operating system once it is no longer needed.
As I've mentioned before we are trying to reduce the RAM footprint of the www.squeak.org process and are watching various related figures.
We can see the value in VM parameter 32 increase, but while the value for parameter 3 (end of memory) goes down as you would expect, the operating system's view of the memory usage does not change.
Ken
-- =========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
On Fri, 2010-10-22 at 09:23 -0700, John M McIntosh wrote:
It *would* give the memory back to the operating system. But we don't anymore again read the following
/* Note:
- The code allows memory to be overallocated; i.e., the initial
- block is reserved via mmap() and then the unused portion
- munmap()ped from the top end. This is INHERENTLY DANGEROUS since
- malloc() may randomly map new memory in the block we "reserved"
- and subsequently unmap()ped. Enabling this causes crashes in
- Croquet, which makes heavy use of the FFI and thus calls malloc()
- all over the place.
- For this reason, overallocateMemory is DISABLED by default.
- The upshot of all this is that Squeak will claim (and hold on to)
- ALL of the available virtual memory (or at least 75% of it) when
- it starts up. If you can't live with that, use the -memory
- option to allocate a fixed size heap.
*/
John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
OK, I should have replied to this aspect before, sorry about that.
I'm sure I'm not telling you anything here I just want to be clear we are using the same terminology or that you at least understand what I'm saying:
Note that under Linux there are two (really more than two, but two major) memory figures. One is Virtual Set Size (VSZ) and the other is Resident Set Size (RSS). It's nothing new that with Squeak on Linux if you don't specify a -memory option it mmaps a gigabyte and this is reflected in the VSZ, in my experience this value never changes, I'm not sure what would happen if more than a gigabyte was actually addressed. I'm not at all concerned about the VSZ value because this does not count against true allocated RAM. So I may be wrong but I believe what you are referring to in VSZ and is irrelevant to us.
My problem instead is with RSS which represents an actual cost in used RAM. What I am expecting to see is that the RSS value I see is in some way representative of the end of memory (vm parameter 3) figure. This is true initially, but as I've said before, when the end of memory drops due to a GC, the RSS does not.
Setting the value of -memory is not going to help here. That's simply going to change the VSZ and set a lower ceiling, a limit on the RSS. The reality is that the way the website is emulated there are short periods of time in which it needs one or two hundred megabytes of memory, but these times are short lived and this is evidence by the end of memory figure dropping back to the normal base level.
Ken
Ok, well I think you are into forensic here. So let me make some suggestions.
Have a timer based task that wakes up and looks for the allocated memory > what 200MB? When that happens do as a one time shot a count of all the instance allocation sizes, then look at dumping all the process stacks so you understand what everything is doing at the time that it wanted & was using 200MB.
I note the dreadful 1994 defaults for stack dumps are too low, you might need to look at that code that creates the stack dump to print more data. For Sophie we changed things from dumping 4000 bytes to providing upwards of 400Kbytes to help diagnose things.
If you find that you've a ByteArray that's 150MB then maybe you could override new: for ByteArray and place some diagnostics for dumping the caller's stack so you can determine why it's wanting 150MB for a few 100 ms.
On 2010-10-22, at 12:37 PM, Ken Causey wrote:
Note that under Linux there are two (really more than two, but two major) memory figures. One is Virtual Set Size (VSZ) and the other is Resident Set Size (RSS). It's nothing new that with Squeak on Linux if you don't specify a -memory option it mmaps a gigabyte and this is reflected in the VSZ, in my experience this value never changes, I'm not sure what would happen if more than a gigabyte was actually addressed. I'm not at all concerned about the VSZ value because this does not count against true allocated RAM. So I may be wrong but I believe what you are referring to in VSZ and is irrelevant to us.
Yes the VSZ would include the GB mmap, this value is a promise(lie) about what the app could at anytime want.
My problem instead is with RSS which represents an actual cost in used RAM. What I am expecting to see is that the RSS value I see is in some way representative of the end of memory (vm parameter 3) figure. This is true initially, but as I've said before, when the end of memory drops due to a GC, the RSS does not.
If you take a 50MB image and expand it for 100ms to 200MB then RSS will show 200MB, If the image vm parameter 3 goes back to 60MB this will NOT be reflected in the RSS value because the operating system memory manager gave you the 200MB, this value will decay back to 60MB+ over time but that decay is function of some very smart algorithms in the operating system which depend on how much RAM other process are using, how much CPU timing Squeak is using. As you know Squeak chews CPU so the operating system might pull idle pages from Squeak at a slower rate.
Frankly here you need to do a "vm_stat 1" to understand if the operating system is under any paging issues. If the RSS is high and the vm parm 3 low, then I'd fully expect the operating system has zero paging going on so there is NO effort on it's part to steal pages from the Squeak process because that's expensive and Squeak has shown it wanted the pages so why pull them back?
It's simplistic to consider a high RSS as bad, I'd rather be concerned about seeing a page_in/out rate in vm_stat showing 30+ then I'd bitch... Note that at 30 pages/second that's like 1000 ms for hard disk rotation/access/seek, bad bad, but I guess in new school thought it's only 300ms.
I'll bet page_in/out rate is zero..
Setting the value of -memory is not going to help here. That's simply going to change the VSZ and set a lower ceiling, a limit on the RSS. The reality is that the way the website is emulated there are short periods of time in which it needs one or two hundred megabytes of memory, but these times are short lived and this is evidence by the end of memory figure dropping back to the normal base level.
Ken
-- =========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
Thanks for the info John. We'll take this and watch things for a while.
You are quite probably right about my reading too much into the RSS value. I've felt like on the occasions when the RSS does go down that it is because it is moving into swap. What has triggered my concern is when free memory is down to single digit megabytes and swap is half used. Looking at the process list the biggest offender that seems to stick out is the www.squeak.org process. And so I'm working on it perhaps worrying about issues of no real concern. I still need a bit of convincing though but you have provided information that will help me decide.
Thank you,
Ken
On Fri, 2010-10-22 at 13:11 -0700, John M McIntosh wrote:
Ok, well I think you are into forensic here. So let me make some suggestions.
Have a timer based task that wakes up and looks for the allocated memory > what 200MB? When that happens do as a one time shot a count of all the instance allocation sizes, then look at dumping all the process stacks so you understand what everything is doing at the time that it wanted & was using 200MB.
I note the dreadful 1994 defaults for stack dumps are too low, you might need to look at that code that creates the stack dump to print more data. For Sophie we changed things from dumping 4000 bytes to providing upwards of 400Kbytes to help diagnose things.
If you find that you've a ByteArray that's 150MB then maybe you could override new: for ByteArray and place some diagnostics for dumping the caller's stack so you can determine why it's wanting 150MB for a few 100 ms.
On 2010-10-22, at 12:37 PM, Ken Causey wrote:
Note that under Linux there are two (really more than two, but two major) memory figures. One is Virtual Set Size (VSZ) and the other is Resident Set Size (RSS). It's nothing new that with Squeak on Linux if you don't specify a -memory option it mmaps a gigabyte and this is reflected in the VSZ, in my experience this value never changes, I'm not sure what would happen if more than a gigabyte was actually addressed. I'm not at all concerned about the VSZ value because this does not count against true allocated RAM. So I may be wrong but I believe what you are referring to in VSZ and is irrelevant to us.
Yes the VSZ would include the GB mmap, this value is a promise(lie) about what the app could at anytime want.
My problem instead is with RSS which represents an actual cost in used RAM. What I am expecting to see is that the RSS value I see is in some way representative of the end of memory (vm parameter 3) figure. This is true initially, but as I've said before, when the end of memory drops due to a GC, the RSS does not.
If you take a 50MB image and expand it for 100ms to 200MB then RSS will show 200MB, If the image vm parameter 3 goes back to 60MB this will NOT be reflected in the RSS value because the operating system memory manager gave you the 200MB, this value will decay back to 60MB+ over time but that decay is function of some very smart algorithms in the operating system which depend on how much RAM other process are using, how much CPU timing Squeak is using. As you know Squeak chews CPU so the operating system might pull idle pages from Squeak at a slower rate.
Frankly here you need to do a "vm_stat 1" to understand if the operating system is under any paging issues. If the RSS is high and the vm parm 3 low, then I'd fully expect the operating system has zero paging going on so there is NO effort on it's part to steal pages from the Squeak process because that's expensive and Squeak has shown it wanted the pages so why pull them back?
It's simplistic to consider a high RSS as bad, I'd rather be concerned about seeing a page_in/out rate in vm_stat showing 30+ then I'd bitch... Note that at 30 pages/second that's like 1000 ms for hard disk rotation/access/seek, bad bad, but I guess in new school thought it's only 300ms.
I'll bet page_in/out rate is zero..
Setting the value of -memory is not going to help here. That's simply going to change the VSZ and set a lower ceiling, a limit on the RSS. The reality is that the way the website is emulated there are short periods of time in which it needs one or two hundred megabytes of memory, but these times are short lived and this is evidence by the end of memory figure dropping back to the normal base level.
Ken
--
John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
Further to this you should consider the setting for setGCBiasToGrow and what are the numbers out of
3 end of memory (read-only) 31 number of grow memory requests (read-only) 32 number of shrink memory requests (read-only)
I'll suggest you chart them by time to understand what the spikes are? does it go from 60MB to 180MB then back to 60MB in a second? and pound #31 up by 20?
On 2010-10-22, at 6:24 AM, Ken Causey wrote:
I'm wondering what 'shrink' means in the following VM parameter comments:
"24 memory threshold above which shrinking object memory (rw)"
and
"32 number of shrink memory requests (read-only)"
My first thought is that a shrink represents Squeak release memory back to the operating system once it is no longer needed.
As I've mentioned before we are trying to reduce the RAM footprint of the www.squeak.org process and are watching various related figures.
We can see the value in VM parameter 32 increase, but while the value for parameter 3 (end of memory) goes down as you would expect, the operating system's view of the memory usage does not change.
Ken
-- =========================================================================== John M. McIntosh johnmci@smalltalkconsulting.com Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
vm-dev@lists.squeakfoundation.org
-
John M McIntosh -
Ken Causey