Re: [Vm-dev] VM crash in the Squeak3D plugin

Hi Nicolas, Hi Stéphane,

On Dec 28, 2019, at 2:24 PM, Nicolas Cellier nicolas.cellier.aka.nice@gmail.com wrote:

 I forgot to say that I had an assert failure report:

leftEdge && rightEdge 1398

Looking around line 1398:

                                    /*-- search for the next top edge which will be the right edge --*/
                                    assert(aetStart < aetSize);
                                    if(!fillList->firstFace)
                                            rightEdge = aetData[aetStart++]; /* If no current top fill just use the next edge */
                                    else while(aetStart < aetSize) { /* Search for the next top edge in the AET */
                                            rightEdge = aetData[aetStart];
                                            /* If we have an intersection use the intersection edge */
                                            if(nextIntersection->xValue <= rightEdge->xValue) {
                                                    rightEdge = nextIntersection;
                                                    break;
                                            }
                                            aetStart++;
                                            /* Check if this edge is on top */
                                            assert(fillList->firstFace);
                                            {
                                                    double xValue = rightEdge->xValue * B3D_FixedToFloat;
                                                    B3DPrimitiveFace *topFace = fillList->firstFace;
                                                    if( rightEdge->leftFace == topFace ||
                                                            rightEdge->rightFace == topFace ||
                                                            rightEdge->zValue < zValueAt(topFace, xValue, yValue))
                                                            break; /* rightEdge is on top */
                                            }
                                            /* If the edge is not on top toggle its (back) fills */
                                            b3dToggleBackFills(fillList, rightEdge, yValue, nextIntersection);
                                            rightEdge = NULL;
                                    }
                                    /*-- end of search for next top edge --*/

                                    /*-- Now do the drawing from leftEdge to rightEdge --*/

#if 1 /* This assert fails in rare cases; the fix is not understood. eem */ assert(leftEdge && rightEdge); #else if(!leftEdge || !rightEdge) FAIL_UPDATING(B3D_NO_MORE_EDGES); // another segfault //FAIL_PAINTING(B3D_NO_MORE_EDGES); // blow up in allocating edges #endif if(fillList->firstFace) { /* Note: We fill *including* leftX and rightX */ int leftX = (leftEdge->xValue >> B3D_FixedToIntShift) + 1; int rightX = (rightEdge->xValue >> B3D_FixedToIntShift);

So I'm pretty sure that we went thru:

                                            /* If the edge is not on top toggle its (back) fills */
                                            b3dToggleBackFills(fillList, rightEdge, yValue, nextIntersection);
                                            rightEdge = NULL;

if intention of #if 1... assert(...) was probably to debug the case of failure... But it was left in the release. Maybe it would be safer to replace with #ifdef DEBUG or something like that? Eliot?

The intent of asserts is that they’re compiled out of the production (fully optimized) vm. So the assert should stay. If however, the plugin isn’t pulling in platforms/Cross/vm/sqAssert.h, it won’t be using the right definitions. I’ll take a look soon.

...

Le sam. 28 déc. 2019 à 23:04, Nicolas Cellier nicolas.cellier.aka.nice@gmail.com a écrit : Running SqueakDebug.app in lldb, I get:

Process 61332 stopped

• thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x18) frame #0: 0x09f17c07 Squeak3D`b3dMainLoop(state=0x09f1928c, stopReason=0) at b3dMain.c:1407:32 1404 if(fillList->firstFace) { 1405 /* Note: We fill *including* leftX and rightX */ 1406 int leftX = (leftEdge->xValue >> B3D_FixedToIntShift) + 1;

-> 1407 int rightX = (rightEdge->xValue >> B3D_FixedToIntShift); 1408 B3DPrimitiveFace *topFace = fillList->firstFace; 1409 1410 if(leftX < 0) leftX = 0; Target 0: (Squeak) stopped. (lldb) print rightEdge (B3DPrimitiveEdge *) $0 = 0x00000000 (lldb) print leftEdge (B3DPrimitiveEdge *) $1 = 0x0617a49c (lldb) bt

• thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x18)
  • frame #0: 0x09f17c07 Squeak3D`b3dMainLoop(state=0x09f1928c, stopReason=0) at b3dMain.c:1407:32 frame #1: 0x09f0cb23 Squeak3D`b3dStartRasterizer at Squeak3D.c:1704:12 frame #2: 0x05d01398 frame #3: 0x00002f52 Squeak`interpret at gcc3x-cointerp.c:2749:3

So why rightEdge is a null pointer? If you already examined the code, you may have an idea...

...

Le sam. 28 déc. 2019 à 22:55, Nicolas Cellier nicolas.cellier.aka.nice@gmail.com a écrit : Hi Stephane, I confirm that I can reproduce the crash with an up-to-date VM (on OSX).

...

Le sam. 28 déc. 2019 à 19:20, Stéphane Rollandin lecteur@zogotounga.net a écrit :

Hello all,

For my first post on this list, I would like to submit the bug that prevents one of my game (this one: http://www.zogotounga.net/comp/guardians.htm) to work reliably.

I spent quite some time on this, and I ended up with a nice and concise way to crash the Squeak3D plugin. The image is available at:

http://www.zogotounga.net/swap/crashlab3.zip

Instructions to crash the VM are detailed therein.

The 3D scene is very simple, there are only three objects. The crash seems related to a race condition, because inserting a simple delay in the code prevents it - this is detailed in the image itself.

I have spent days studying the Smalltalk code, and could not see anything wrong with what is sent to the plugin. It looks to my uninformed eyes as a problem of shared memory, where one rasterizing operation messes up with another happening in parallel (although no processed is being forked on the Smalltalk side).

Stef