On Thu, 2015-12-31 at 13:14 -0800, KenD wrote:
On Thu, 31 Dec 2015 15:29:02 -0500 "Phil (list)" pbpublist@gmail.com wrote:
At this stage I'm expecting to find that it's my code doing the leaking into the image, and not a base image or VM bug. My original thought was that I had some orphaned reference loops floating around but ran across a post that indicated that unrooted reference loops should not be a problem for the VM to gc.
Making sure such object nets are unrooted as viewed by the gc is usually the problem.
Are you making use of finalization?
Not currently. I seem to recall that one issue I ran into was that there were no guarantees as to when finalize would get called and I also remember something about one or more situations where finalize might not get called at all (though I could be misremembering that / thinking of some random Java scenario I ran into.) As a result, when I need finalizers I tend to do it explicitly/manually which is obviously error prone but didn't consider that there might be other less obvious side-effects involved. Aside from situations involving FFI where you might have resources external to the VM involved, are there common cases when failing to finalize a resource (streams perhaps? That's definitely one thing I'm seeing a lot of lying around) might result in objects failing to be gc'd? (I don't doubt it's possible, just trying to understand the scenarios)
Ephemerons in the newer Spur image are useful here -- but it will take some work to get an updated Cuis image in the Spur memory format. Cuis does have a variety of weak collections which can be very helpful.
I use weak collections extensively and am quite curious to play around with ephemerons as soon as Cuis is ready :-)
I've done all of the obvious things I could think of to minimize rooted references including closing workspaces, browsers, inspectors, terminated Smalltalk processes etc. Any ideas as to what other possible ways these objects could be rooted?
I tend to think of expected object lifetimes. Having done real-time systems, one thinks of pre-allocating objects and explicitly managing object lifetimes.
If you have expected lifetimes and a "duty cycle" you should be able to nil out ivars/globals when you reach the expected "end-of-life" to orphan object nets. The run the full collection a couple of times and check that the memory is being reclaimed.
You may be able to gc to get the memory usage, nil out a bunch of references, gc twice again, then check if the particular bunch of references made a difference. Pick a new bunch of references and retry..
Also, you might be able to collect noted references when assigned to get candidates for testing.
Sometimes thinking contrariwise works. You might take a look at the objects saved in a SmartRefStream to see if things are joined in ways you don't expect. You might write a filter which elides expected object references but notes outliers.
All good food for thought. In this particular instance, I literally have what should be an inactive image (i.e. I've shut down and de- referenced everything I can think to and am just trying to understand where/why so many objects are still being held onto)
What tools could you build to help yourself be successful?
I've already got one in mind: looking for object graphs with similar structures. I just need to understand all of the ways in which they can be rooted to go looking for them.
Also you could send me an image and we could Skype/GoogleHangout. I sometimes find describing the problem to someone else in itself causes "aha" to happen.
I appreciate the offer and will take you up on it. I think I have tracked down at least one scenario that you should be able to replicate from a base Cuis image. I'll email the details.
Good luck, -KenD
Thanks, Phil