Hi Avi,
You might think me crazy for trying to store morphs in GOODS, but that exactly what I want to do. Currently, storing them works (e.g. I've tried out Morph new) as long as I don't have them displaying in the world prior to committing.
With respect to ReferenceStreams, you can store a Morph or set of morphs in a stream and it appropriately does not chase the WorldMorph and try to stick that in the stream.
Is there anyway you'd consider adding a special case (or showing me where I could put on) that would allow me to store Morphs in and out of GOODS so that I don't have to remove them from the world first before committing?
Just let me know if this is not something you want to spend time on. I just find it would be very handy for sharing morphs that are already pre-configured on the screen in a certain way.
Regards,
John
On Sep 22, 2004, at 5:20 PM, John Pierce wrote:
Is there anyway you'd consider adding a special case (or showing me where I could put on) that would allow me to store Morphs in and out of GOODS so that I don't have to remove them from the world first before committing?
Hi John,
If the issue is not trying to serialize WorldMorph, have a look at the specialObjects dictionary in KKDatabase. This has as its keys objects that shouldn't themselves be committed to the database, but may be referenced from objects that are. Its values are unique negative numbers - these need to be the same for the same conceptual object across all your images/connections. The KKDatabase instance itself gets stored as the first such special object, with an ID of -1. You could store the WorldMorph in there as -2.
Avi
The KKDatabase instance itself gets stored as the first such special object, with an ID of -1. You could store the WorldMorph in there as -2.
Avi
Yes! That works. If you append the following line:
specialObjects at: ActiveWorld put: -2.
to KKDatabase>>initializeWithConnection: then you can take an on-screen morph and store in the database and re-fetch it later. Now, one caveat I have found is that color instances of the morph don't get reconstituted from GOODS correctly. I am not quite sure how to fix it.
All I know is that if you put a "Morph new" into GOODS, come back later (it is important to try this on another connection so as to force getting a new instance) and show the stored morph in the world you will get a BitBlt (Fraction?) walkback. You might think that the morph just doesn't work from GOODS (and it doesn't). But if you inspect the bad morph and say to its color instance "flushCache" then you CAN show the morph again -- all is well.
So taking this rabit trail one more step I noticed that the two cache variables of Color are a SmallInteger and a Bitmap. The small integer (bit depth) seems fine on the reconstituted object, but the Bitmap (cachedBitPattern), which looks okay upon inspection in the inspector, is just somehow foul when it came out of GOODS.
Bitmap is a variableWordSubclass and I know GOODs recently received some attention in this area. I can't suggest a test case other than this long roundabout dialog that I provided. I don't visually see any difference in this bad bitmap vs. a good one so I can't tell you what is wrong with it other than BitBlt doesn't like it the bad one that looks fine, but something under the covers is going on.
I wish I could provide a suggestion or possible fix, but does anyone have any ideas?
Regards,
John
Hi all,
I wish I could provide a suggestion or possible fix, but does anyone have any ideas?
I was able to work up some test code. Again, you must be running Squeak 3.7 release plus have GOODS.73-avi.mcz loaded with my local patch to see this problem. See my former email for my patch to store live morphs in a GOODS db.
Run the following code to see what I am talking about -- I don't get what is going on with the busted bitmap that appears indistinguisable from a good one.
*TRY THE FOLLOWING CODE BELOW TO SEE WHAT I MEAN*
"Assumes you have a new GOODs server launched already with the following parameters" client _ KKDatabase onHost: 'localhost' port: 9999. client commitWithRetry: [client root: Dictionary new].
joe _ Morph new. joe openInWorld. joe position: joe position + 100.
client commitWithRetry: [client root at: 'joe' put: joe]. joe abandon.
client _ KKDatabase onHost: 'localhost' port: 9999. (client root at: 'joe') openInWorld. "should see BitBlt error here!" ActiveWorld restoreMorphicDisplay. "do this to see broken joe -- notice, he's in the right position!"
"remember bad bitmap" badBitmap _ (client root at: 'joe') color instVarNamed: #cachedBitPattern.
"fix the cachedBitPattern of joe using a new one from Color blue" (client root at: 'joe') color instVarNamed: #cachedBitPattern put: (Color blue instVarNamed: #cachedBitPattern).
"let's try to draw joe again -- all is well!" (client root at: 'joe') resumeAfterDrawError. ActiveWorld restoreMorphicDisplay. "might need to do this to see fixed joe"
"really weird that the object that is broken is no different than the object that fixes it??" badBitmap = ((client root at: 'joe') color instVarNamed: #cachedBitPattern)
- Any thoughts!
John
I wish I could provide a suggestion or possible fix, but does anyone have any ideas?
Ok, so I think I am on to something here. I actually am discovering that the BitBlt machinery gets really angry when an instance of Color coming from GOODS has, as its cachedBitPattern instVar, a KKObjectProxy rather than the unproxied bitmap it was expecting.
So -- is this a BitBlt problem that it won't take a ProtoObject (in this case, an instance of KKObjectProxy)? Or is there a way to ensure that an object coming out of GOODs always gets totally unproxied (yuck)? Too bad that BitBlt doesn't handle this better, but I suspect it is looking for an instance of a variable subclass and, instead, it finds an instance of ProtoObject and gives me the fraction? error.
The really hard thing to solve is that it is not the Color instance of my Morph that is problematic. It is rather the instance variable "cachedBitPattern" of the Color instance that is causing BitBlt so much grief. I am looking for a direction to go here if anyone can provide advice.
Of course, I want a general solution and not just "call flushCache" on the Color instance of the morph I am storing in GOODS.
Regards,
John
Ok, so I think I am on to something here. I actually am discovering that the BitBlt machinery gets really angry when an instance of Color coming from GOODS has, as its cachedBitPattern instVar, a KKObjectProxy rather than the unproxied bitmap it was expecting.
That is to be expected. Mind you BitBlt runs as a primitive which means it does NOT send any messages but rather does hard type checks on the iVars and fails if it sees anything that doesn't look correct.
So -- is this a BitBlt problem that it won't take a ProtoObject (in this case, an instance of KKObjectProxy)? Or is there a way to ensure that an object coming out of GOODs always gets totally unproxied (yuck)? Too bad that BitBlt doesn't handle this better, but I suspect it is looking for an instance of a variable subclass and, instead, it finds an instance of ProtoObject and gives me the fraction? error.
That is correct. One way to deal with this would be to have a method like (for example) #unproxy which you can send to the iVars of BitBlt and which answers true if any of them have been unproxied. Then you could fix the failure code to check any of the (relevant) iVars and whether they have been unproxied and retry (a very similar mechanism is used for Form>>unhibernate which is a nice example for that mechanism).
The really hard thing to solve is that it is not the Color instance of my Morph that is problematic. It is rather the instance variable "cachedBitPattern" of the Color instance that is causing BitBlt so much grief. I am looking for a direction to go here if anyone can provide advice.
Simple: Don't store the cachedBitPattern. As the name says it's a cache and therefore there is no reason to store it with the rest. Alternatively, dump it when the color comes back in (this is less efficient in space terms but might be easier to implement).
Cheers, - Andreas
On Sep 27, 2004, at 5:02 AM, John Pierce wrote:
I wish I could provide a suggestion or possible fix, but does anyone have any ideas?
Ok, so I think I am on to something here. I actually am discovering that the BitBlt machinery gets really angry when an instance of Color coming from GOODS has, as its cachedBitPattern instVar, a KKObjectProxy rather than the unproxied bitmap it was expecting.
Yes, any time you end up passing a proxy into a prim you're gonna have trouble.
I can't think of a great way of dealing with this in the general case. In this case, you could override Color class>>goodsFieldDescriptors to not include a descriptor for the cachedBitPattern (so that it won't get written to or read from GOODS). Slightly more general would be to add a field type something like KKAutoloadReferenceField that would always bring the full object in right away, rather than using a proxy, and mark cachedBitPattern as having this field type - this would probably require adding an #allowProxy flag to KKRequestObject that would be set to false by KKAutoloadReferenceField>>referenceAt:, and then checked by KKDatabase>>objectForKey:.
Another really specific solution would be to add Color as a new immediate value type. We did this for Character when we were having very similar problems a while back. Since object references in GOODS are 6 bytes, we have plenty of tag bits to play with...
Avi
Yes, any time you end up passing a proxy into a prim you're gonna have trouble.
Yeah, I see that now, but gosh, this is a big implementation detail (I call them pipes) jutting up through the floor. The real trouble is that I don't know where these things will come out of the woodwork. Yeah, I can avoid doing this for BitBlt in the future, but what other prim will be my downfall with respect to proxied objects in the future?
I can't think of a great way of dealing with this in the general case. In this case, you could override Color class>>goodsFieldDescriptors to not include a descriptor for the cachedBitPattern (so that it won't get written to or read from GOODS).
Of course, it isn't really a problem with the Color class. It is more a problem with one of its instVars (cachedBitPattern). So fixing it for Color gets me unstuck, but leaves the bigger question about Bitmap instances that might be proxied.
Slightly more general would be to add a field type something like KKAutoloadReferenceField that would always bring the full object in right away, rather than using a proxy, and mark cachedBitPattern as having this field type
What implications would this have when committing changes at a later time? It seems like you are, in essence, making the cachedBitPattern instVar an immediate value (sort of).
this would probably require adding an #allowProxy flag to KKRequestObject that would be set to false by KKAutoloadReferenceField>>referenceAt:, and then checked by KKDatabase>>objectForKey:.
So would class-side of Color somehow participate in this algorithm to declare that cachedBitPattern was one of these KKAutoloadReferenceFields (or no proxy permitted on this instVar)?
Another really specific solution would be to add Color as a new immediate value type. We did this for Character when we were having very similar problems a while back. Since object references in GOODS are 6 bytes, we have plenty of tag bits to play with...
Tell me more. How can I do this just to play with the idea? Do I need to be aware of the other immediate values and pick a number that represents my color class? I assume I have to implement writing the tagged field in this case.
Regards,
John
Hi Avi,
So I took you up on the simplest option for now. Attached is a mcz file that I hope you incorporate. It includes changes as follows:
1. Enabled you to put a class-side method called goodsFieldsToExclude where you can return an array of instVars to *not* store in GOODs. Of course, the default implementation on Object class is to return an empty list.
2. Added a class-side implementation of goodsFieldsToExclude on Color to exclude the two cached instVars of Color. This makes sense for Color anyways since the cached instVars are exactly that -- cache!.
3. Of course, fixed up Object class>>goodsFieldDescriptors to exclude any instVars in the exclude list.
4. Added ActiveWorld as a specialObject in KKDatabase's special objects array. This enables morphs that are currently being display to be stored in GOODs.
While changes #1 - 3 above constitute additional flexibility and I think are rather benign, change #4 above you may have design issues with (in principle). Let me know what you think you will include in future versions of GOODs.
Just let me re-iterate how cool of a demo I think it is to store Morphs on-screen in GOODs, then move them around, then rollback the connection, and have them all bounce back. I also want to start to use GOODs to share pre-configured morphs for simple application deployments and such. This is just a start in that direction as I will most certainly discover other issues with Bitmaps being embedded in other things besides Colors (and we all know what happens when you hand one of these proxied objects to some primitive!).
Cheers,
John
On Sep 27, 2004, at 10:31 PM, John Pierce wrote:
Hi Avi,
So I took you up on the simplest option for now. Attached is a mcz file that I hope you incorporate. It includes changes as follows:
- Enabled you to put a class-side method called goodsFieldsToExclude
where you can return an array of instVars to *not* store in GOODs. Of course, the default implementation on Object class is to return an empty list.
- Added a class-side implementation of goodsFieldsToExclude on Color
to exclude the two cached instVars of Color. This makes sense for Color anyways since the cached instVars are exactly that -- cache!.
- Of course, fixed up Object class>>goodsFieldDescriptors to exclude
any instVars in the exclude list.
- Added ActiveWorld as a specialObject in KKDatabase's special
objects array. This enables morphs that are currently being display to be stored in GOODs.
Hi John,
Looks good. I've published this versions as is on SqueakMap.
Avi
Hello, Where 3.7 can be found? On ftp site I see only Full image. Vaidas
Hi,
Someone didn't build a "basic" image. Anyone want to give it a shot?
cheers
bruce
On Tue, Sep 28, 2004 at 04:40:31PM +0200, Vaidas Didzbalis wrote:
Hello, Where 3.7 can be found? On ftp site I see only Full image. Vaidas
Sorry about that, while working on the Full image issues, I neglected to provide the "final" Basic image to Bruce... I will track down the proper image and send that to Bruce sometime tonight.
- Doug
On Wed, 29 Sep 2004 11:22:45 +0000, "Bruce O'Neel" edoneel@sdf.lonestar.org said:
Hi,
Someone didn't build a "basic" image. Anyone want to give it a shot?
cheers
bruce
On Tue, Sep 28, 2004 at 04:40:31PM +0200, Vaidas Didzbalis wrote:
Hello, Where 3.7 can be found? On ftp site I see only Full image. Vaidas
-- edoneel@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org
On Sep 27, 2004, at 6:12 PM, John Pierce wrote:
Slightly more general would be to add a field type something like KKAutoloadReferenceField that would always bring the full object in right away, rather than using a proxy, and mark cachedBitPattern as having this field type
What implications would this have when committing changes at a later time? It seems like you are, in essence, making the cachedBitPattern instVar an immediate value (sort of).
It wouldn't have any implications. Not all objects get proxied anyway - when you request an object, the GOODS server usually sends back a cluster of related objects that it thinks might be useful to you, and so the ones you didn't directly request never go through the proxy stage. And the client lib is designed so that proxies are only involved at the very highest level (nothing below KKDatabase in the stack ever sees them). So forcing a few more objects to be directly loaded rather than proxied isn't a big deal it all.
So would class-side of Color somehow participate in this algorithm to declare that cachedBitPattern was one of these KKAutoloadReferenceFields (or no proxy permitted on this instVar)?
Yes, it would implement
Color class>>cachedBitPatternDescriptor ^ KKSingleFieldContainer autoloadReferenceField
Tell me more. How can I do this just to play with the idea? Do I need to be aware of the other immediate values and pick a number that represents my color class? I assume I have to implement writing the tagged field in this case.
Yes. Currently tags 1 through 5 are used, so we could designate 6 as being for Color. You would have to add two methods to Color for encoding to work, and modify the decoding method to match:
Color>>goodsIsImmediateValue ^ true
Color>>goodsWriteTaggedOn: aStream aStream nextPut: 6; nextInt32Put: rgb
KKReferenceField>>fieldFromTag: sid value: oid ^ sid = 0 ifTrue: [nil] ifFalse: [(sid bitAnd: 16r00FF) caseOf: {[1] -> [self decodeUnsignedInteger: oid]. [2] -> [oid = 1]. [3] -> [Float fromIEEE32Bit: oid]. [4] -> [self referenceAt: oid * -1]. [5] -> [Character value: oid] [6] -> [Color basicNew setRGB: oid}]
squeak-dev@lists.squeakfoundation.org
-
Andreas Raab -
Avi Bryant -
Bruce O'Neel -
Doug Way -
John Pierce -
Vaidas Didzbalis