Thx Eliot for that detailed description. What might be the impact of a
default-pin every object passed to FFI?
Cheers -ben
On 30 Aug 2017 11:46 PM, "Eliot Miranda" <eliot.miranda(a)gmail.com> wrote:
Hi Torsten, Hi All,
re pinDuring:
> On Aug 28, 2017, at 9:17 AM, Torsten Bergmann <astares(a)gmx.de> wrote:
>
> As some of you are aware of Spur added memory pinning and in Pharo 6.1
(#60510) as well as Pharo 7
> we currently have
>
> Object>>pin
> Object>>unpin
> Object>>isPinned
>
> and also Object>>setPinned:.
>
> Object pinning is explained by Eliot here [1].
>
> I would propose two changes (at least for Pharo but ideally also for
Squeak):
>
> 1. IMHO #setPinned: should be moved to a private protocol
> 2. Provide a #pinDuring: aBlock method like:
>
> pinDuring: aBlock
> [ self pin.
> aBlock value ] ensure: [ self unpin ]
>
> so we can ensure that a pinned object is unpinned after an operation.
>From an implementation perspective, I suspect this is unnecessary. There
are two reasons, one to do with collecting pinned objects and the other to
do with compaction.
Pinning an object forces it into oldSpace if it was not there already, but
does not prevent it being garbage collected. So any un references pinned
object in oldSpace will be collected at the next full GC.
It is possible (but in implemented) to move not large objects back to
newSpace but the forwarder left in oldSpace will prevent its collection
until the next full GC so trying to move an object back into newSpace is
pointless. Conclusion, there is no point unpinning for garbage
collectability.
The compactor does a good job of moving unpinned objects around pinned
objects, typically leaving very small gaps before pinned objects. [One can
observe how much free space exists in an image file to get an idea of how
much free space this causes. It's low]. The number of uncollectable pinned
objects in an application, which will be objects that are long-lived that
have been passed through the FFI, is likely to be small. Conclusion there
is little to be gained in avoiding fragmentation by unpinning long-lived
objects pinned on an FFI call.
In summary, collectible ones objects will be collected anyway, and the few
long-lived pinned objects should not cause significant fragmentation. So
pinDuring: is unlikely to provide any benefit. Simply pin and leave it to
the GC and its compactor to clean up afterwards.
> What do you think?
>
> Also I wonder why FFICallbackTests>>#testCqsortWithByteArray is sending
#pin but not
> #unpin afterwards. Maybe that is the reason why there is a flag in this
method that it
> is crashing on Windows.
>
> Bye
> T.
>
> [1] http://lists.squeakfoundation.org/pipermail/squeak-dev/2017-
August/195129.html
Torsten Bergmann uploaded a new version of Kernel to project The Trunk:
http://source.squeak.org/trunk/Kernel-tbn.1112.mcz
==================== Summary ====================
Name: Kernel-tbn.1112
Author: tbn
Time: 30 August 2017, 11:21:46.282311 pm
UUID: a4399bb0-13e7-7a4f-a0a1-d78bfbd3012e
Ancestors: Kernel-nice.1111
Cleanup pinning protocols
- isPinned moves from "system primitive" to "pinning"
- pin moves from "system primitive" to "pinning"
- unpin moves from "system primitive" to "pinning"
- setPinned: moves from "system primitive" to "private" (as this should not be part of the public API, people should use pin/unpin methods)
(similar to https://github.com/pharo-project/pharo/pull/224 for Pharo)
=============== Diff against Kernel-nice.1111 ===============
Item was changed:
+ ----- Method: Object>>isPinned (in category 'pinning') -----
- ----- Method: Object>>isPinned (in category 'system primitives') -----
isPinned
"Answer if the receiver is pinned. The VM's garbage collector routinely moves
objects as it reclaims and compacts memory. But it can also pin an object so
that it will not be moved, which can make it easier to pass objects out through
the FFI."
<primitive: 183 error: ec>
^self primitiveFailed!
Item was changed:
+ ----- Method: Object>>pin (in category 'pinning') -----
- ----- Method: Object>>pin (in category 'system primitives') -----
pin
"The VM's garbage collector routinely moves objects as it reclaims and compacts
memory. But it can also pin an object so that it will not be moved, which can make
it easier to pass objects out through the FFI. Objects are unpinnned when created.
This method ensures an object is pinned, and answers whether it was already pinned."
^self setPinned: true!
Item was changed:
+ ----- Method: Object>>setPinned: (in category 'private') -----
- ----- Method: Object>>setPinned: (in category 'system primitives') -----
setPinned: aBoolean
"The VM's garbage collector routinely moves objects as it reclaims and compacts
memory. But it can also pin an object so that it will not be moved, which can make
it easier to pass objects out through the FFI. Objects are unpinnned when created.
This primitive either pins or unpins an object, and answers if it was already pinned."
<primitive: 184 error: ec>
^self primitiveFailed!
Item was changed:
+ ----- Method: Object>>unpin (in category 'pinning') -----
- ----- Method: Object>>unpin (in category 'system primitives') -----
unpin
"The VM's garbage collector routinely moves objects as it reclaims and compacts
memory. But it can also pin an object so that it will not be moved, which can make
it easier to pass objects out through the FFI. Objects are unpinnned when created.
This method ensures an object is unpinned, and answers whether it was pinned."
^self setPinned: false!
Hi Torsten, Hi All,
re pinDuring:
> On Aug 28, 2017, at 9:17 AM, Torsten Bergmann <astares(a)gmx.de> wrote:
>
> As some of you are aware of Spur added memory pinning and in Pharo 6.1 (#60510) as well as Pharo 7
> we currently have
>
> Object>>pin
> Object>>unpin
> Object>>isPinned
>
> and also Object>>setPinned:.
>
> Object pinning is explained by Eliot here [1].
>
> I would propose two changes (at least for Pharo but ideally also for Squeak):
>
> 1. IMHO #setPinned: should be moved to a private protocol
> 2. Provide a #pinDuring: aBlock method like:
>
> pinDuring: aBlock
> [ self pin.
> aBlock value ] ensure: [ self unpin ]
>
> so we can ensure that a pinned object is unpinned after an operation.
From an implementation perspective, I suspect this is unnecessary. There are two reasons, one to do with collecting pinned objects and the other to do with compaction.
Pinning an object forces it into oldSpace if it was not there already, but does not prevent it being garbage collected. So any un references pinned object in oldSpace will be collected at the next full GC.
It is possible (but in implemented) to move not large objects back to newSpace but the forwarder left in oldSpace will prevent its collection until the next full GC so trying to move an object back into newSpace is pointless. Conclusion, there is no point unpinning for garbage collectability.
The compactor does a good job of moving unpinned objects around pinned objects, typically leaving very small gaps before pinned objects. [One can observe how much free space exists in an image file to get an idea of how much free space this causes. It's low]. The number of uncollectable pinned objects in an application, which will be objects that are long-lived that have been passed through the FFI, is likely to be small. Conclusion there is little to be gained in avoiding fragmentation by unpinning long-lived objects pinned on an FFI call.
In summary, collectible ones objects will be collected anyway, and the few long-lived pinned objects should not cause significant fragmentation. So pinDuring: is unlikely to provide any benefit. Simply pin and leave it to the GC and its compactor to clean up afterwards.
> What do you think?
>
> Also I wonder why FFICallbackTests>>#testCqsortWithByteArray is sending #pin but not
> #unpin afterwards. Maybe that is the reason why there is a flag in this method that it
> is crashing on Windows.
>
> Bye
> T.
>
> [1] http://lists.squeakfoundation.org/pipermail/squeak-dev/2017-August/195129.h…
I have not only several Squeak images , also Cuis and Cuis forks.
Here a quick resume
Squeak from 1.x until 4.1 could be open by old stack machines like John
McIntosh for Mac and old Unix stack VM which still instal in Ubuntu and
Ubuntu forks this days.
4.6 is the release which change to Cog,
In fact I have 4.6 derivated from before the change and I could run this old
images in G4 Power PC Macs and Pentium IV Ubuntu boxes
For 4.6 from http://ftp.squeak.org/4.6/ you see Squeak4.6-15102 but this
could be updated from trunk and still be Cog.
>From 5.1 until today the VM must be Spur.
Spur do not open older images.
Cog could open some 4.x , but you was forced to continue using Cog, .image
can’t be opened again by stack
VM’s
My two cents.
I currently learning webassembly , so if people here like learning I’m
ready to exchange.
I have in the cooking phase of a Cuis derivate image which have KomHV for
the back end and Amber , Morphic.js and sevearl Git .js useful libraries on
the front .
Edgar
@morplenauta
Since I upgraded to MacOS 10.12, I've had a devil of a time finding a VM
that will reliably run various images I have lying around. Can someone
point me to a good one?