Vm-dev November 2017

vm-dev@lists.squeakfoundation.org

16 participants
44 discussions

VM Maker: VMMaker.oscog-cb.2282.mcz
by commits＠source.squeak.org 26 Nov '17

26 Nov '17

ClementBera uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog-cb.2282.mcz ==================== Summary ==================== Name: VMMaker.oscog-cb.2282 Author: cb Time: 26 November 2017, 11:07:31.121242 pm UUID: 8e91c174-9775-46dd-8e9d-3d06f08827f3 Ancestors: VMMaker.oscog- nice.2281 fix primStringReplace in the JIT. =============== Diff against VMMaker.oscog- nice.2281 =============== Item was changed: ----- Method: CogObjectRepresentationForSpur>>genPrimitiveStringReplace (in category 'primitive generators') ----- genPrimitiveStringReplace "replaceFrom: start to: stop with: replacement startingAt: repStart. The primitive in the JIT tries to deal with two pathological cases, copy of arrays and byteStrings, which often copies only a dozen of fields and where switching to the C runtime cost a lot. Based on heuristics on the method class, I generate a quick array path (typically for Array), a quick byteString path (typically for ByteString, ByteArray and LargeInteger) or no quick path at all (Typically for Bitmap). The many tests to ensure that the primitive won't fail are not super optimised (multiple reloading or stack arguments in registers) but this is still good enough and worth it since we're avoiding the Smalltalk to C stack switch. The tight copying loops are optimised. It is possible to build a bigger version with the 2 different paths but I (Clement) believe this is too big machine code wise to be worth it." |arrayReg startReg stopReg replReg repStartReg jumpNotSmi1 jumpNotSmi2 jumpNotSmi3 jumpImm jumpEmpty jumpImmutable jumpOutOfBounds1 jumpOutOfBounds2 jumpOutOfBounds3 jumpOutOfBounds4 jumpIncorrectFormat1 jumpIncorrectFormat2 jumpIncorrectFormat3 jumpIncorrectFormat4 result jmpDestYoung jmpAlreadyRemembered instr jumpFinished adjust| <var: #jumpImm type: #'AbstractInstruction *'> <var: #jumpEmpty type: #'AbstractInstruction *'> <var: #jumpFinished type: #'AbstractInstruction *'> <var: #jumpNotSmi1 type: #'AbstractInstruction *'> <var: #jumpNotSmi2 type: #'AbstractInstruction *'> <var: #jumpNotSmi3 type: #'AbstractInstruction *'> <var: #jmpDestYoung type: #'AbstractInstruction *'> <var: #jumpImmutable type: #'AbstractInstruction *'> <var: #jumpOutOfBounds1 type: #'AbstractInstruction *'> <var: #jumpOutOfBounds2 type: #'AbstractInstruction *'> <var: #jumpOutOfBounds3 type: #'AbstractInstruction *'> <var: #jumpOutOfBounds4 type: #'AbstractInstruction *'> <var: #jumpIncorrectFormat1 type: #'AbstractInstruction *'> <var: #jumpIncorrectFormat2 type: #'AbstractInstruction *'> <var: #jumpIncorrectFormat3 type: #'AbstractInstruction *'> <var: #jumpIncorrectFormat4 type: #'AbstractInstruction *'> <var: #jmpAlreadyRemembered type: #'AbstractInstruction *'> "Can I generate a quick path for this method ?" ((cogit seemsToBeInstantiating: objectMemory arrayFormat) or: [cogit seemsToBeInstantiating: objectMemory firstByteFormat]) ifFalse: [^UnimplementedPrimitive]. "I redefine those name to ease program comprehension" arrayReg := ReceiverResultReg. startReg := Arg0Reg. stopReg := Arg1Reg. replReg := ClassReg. repStartReg := SendNumArgsReg. "Load arguments in reg" cogit genStackArgAt: 0 into: repStartReg. cogit genStackArgAt: 1 into: replReg. cogit genStackArgAt: 2 into: stopReg. cogit genStackArgAt: 3 into: startReg. "start,stop,repStart Smis or fail the primitive" jumpNotSmi1 := self genJumpNotSmallInteger: repStartReg scratchReg: TempReg. jumpNotSmi2 := self genJumpNotSmallInteger: stopReg scratchReg: TempReg. jumpNotSmi3 := self genJumpNotSmallInteger: startReg scratchReg: TempReg. "repl non immediate or fail the primitive" jumpImm := self genJumpImmediate: replReg. "if start>stop primitive success" cogit CmpR: startReg R: stopReg. jumpEmpty := cogit JumpLess: 0. "If receiver immutable fail the primitive " self cppIf: IMMUTABILITY ifTrue: [jumpImmutable := self genJumpImmutable: ReceiverResultReg scratchReg: TempReg]. "0 >= start, fail" cogit CmpCq: (objectMemory integerObjectOf: 0) R: startReg. + jumpOutOfBounds1 := cogit JumpLessOrEqual: 0. - jumpOutOfBounds1 := cogit JumpLess: 0. "0 >= replStart, fail" cogit CmpCq: (objectMemory integerObjectOf: 0) R: repStartReg. + jumpOutOfBounds2 := cogit JumpLessOrEqual: 0. - jumpOutOfBounds2 := cogit JumpLess: 0. "--- Pointer object version ---" (cogit seemsToBeInstantiating: objectMemory arrayFormat) ifTrue: ["Are they both array format ?" self genGetFormatOf: arrayReg into: TempReg. self genGetFormatOf: replReg into: startReg. cogit CmpCq: objectMemory arrayFormat R: startReg. jumpIncorrectFormat1 := cogit JumpNonZero: 0. cogit CmpCq: objectMemory arrayFormat R: TempReg. jumpIncorrectFormat2 := cogit JumpNonZero: 0. "Both objects are arrays," self genGetNumSlotsOf: arrayReg into: TempReg. self genConvertSmallIntegerToIntegerInReg: stopReg. "arr size < stop" cogit CmpR: TempReg R: stopReg. jumpOutOfBounds3 := cogit JumpGreater: 0. "rep size < repStart - start + stop" self genGetNumSlotsOf: replReg into: TempReg. cogit genStackArgAt: 3 into: startReg. self genConvertSmallIntegerToIntegerInReg: startReg. self genConvertSmallIntegerToIntegerInReg: repStartReg. cogit SubR: startReg R: stopReg. cogit AddR: repStartReg R: stopReg. "stopReg: stop - start + repStart" cogit CmpR: TempReg R: stopReg. jumpOutOfBounds4 := cogit JumpGreater: 0. "Everything in bounds" "PossibleRemembered object" cogit MoveCw: objectMemory storeCheckBoundary R: TempReg. cogit CmpR: TempReg R: arrayReg. jmpDestYoung := cogit JumpBelow: 0. self checkRememberedInTrampoline ifFalse: [jmpAlreadyRemembered := self genCheckRememberedBitOf: arrayReg scratch: TempReg]. self callStoreCheckTrampoline. jmpDestYoung jmpTarget: cogit Label. self checkRememberedInTrampoline ifFalse: [jmpAlreadyRemembered jmpTarget: cogit Label]. "Copy the array" cogit genStackArgAt: 2 into: stopReg. self genConvertSmallIntegerToIntegerInReg: stopReg. "shift by baseHeaderSize and then move from 1 relative to zero relative" adjust := (objectMemory baseHeaderSize >> objectMemory shiftForWord) - 1. adjust ~= 0 ifTrue: [ cogit AddCq: adjust R: startReg. cogit AddCq: adjust R: stopReg. cogit AddCq: adjust R: repStartReg. ]. instr := cogit CmpR: startReg R: stopReg. jumpFinished := cogit JumpBelow: 0. cogit MoveXwr: repStartReg R: replReg R: TempReg. cogit MoveR: TempReg Xwr: startReg R: arrayReg. cogit AddCq: 1 R: startReg. cogit AddCq: 1 R: repStartReg. cogit Jump: instr. jumpFinished jmpTarget: (jumpEmpty jmpTarget: cogit genPrimReturn). "CANNOT REACH by falling though" jumpIncorrectFormat1 jmpTarget: (jumpIncorrectFormat2 jmpTarget: cogit Label)]. "--- Byte object version ---" (cogit seemsToBeInstantiating: objectMemory firstByteFormat) ifTrue: ["Are they both byte array format ? CompiledMethod excluded" self genGetFormatOf: arrayReg into: TempReg. self genGetFormatOf: replReg into: startReg. cogit CmpCq: objectMemory firstByteFormat R: startReg. jumpIncorrectFormat1 := cogit JumpLess: 0. cogit CmpCq: objectMemory firstCompiledMethodFormat R: startReg. jumpIncorrectFormat2 := cogit JumpGreaterOrEqual: 0. cogit CmpCq: objectMemory firstByteFormat R: TempReg. jumpIncorrectFormat3 := cogit JumpLess: 0. cogit CmpCq: objectMemory firstCompiledMethodFormat R: TempReg. jumpIncorrectFormat4 := cogit JumpGreaterOrEqual: 0. "Both objects are byte arrays" self genGetNumSlotsOf: arrayReg into: startReg. (cogit LogicalShiftLeftCq: objectMemory shiftForWord R: startReg). cogit AndCq: objectMemory wordSize - 1 R: TempReg R: TempReg. cogit SubR: TempReg R: startReg. self genConvertSmallIntegerToIntegerInReg: stopReg. "arr size < stop" cogit CmpR: startReg R: stopReg. jumpOutOfBounds3 := cogit JumpGreater: 0. "rep size < repStart - start + stop" cogit MoveR: startReg R: TempReg. "TempReg holds repl format" cogit genStackArgAt: 3 into: startReg. self genConvertSmallIntegerToIntegerInReg: startReg. self genConvertSmallIntegerToIntegerInReg: repStartReg. cogit SubR: startReg R: stopReg. cogit AddR: repStartReg R: stopReg. self genGetNumSlotsOf: replReg into: startReg. (cogit LogicalShiftLeftCq: objectMemory shiftForWord R: startReg). cogit AndCq: objectMemory wordSize - 1 R: TempReg R: TempReg. cogit SubR: TempReg R: startReg. "stopReg: stop - start + repStart" cogit CmpR: startReg R: stopReg. jumpOutOfBounds4 := cogit JumpGreater: 0. "Everything in bounds" "Copy the bytes" cogit genStackArgAt: 3 into: startReg. self genConvertSmallIntegerToIntegerInReg: startReg. cogit genStackArgAt: 2 into: stopReg. self genConvertSmallIntegerToIntegerInReg: stopReg. "shift by baseHeaderSize and then move from 1 relative to zero relative" adjust := objectMemory baseHeaderSize - 1. adjust ~= 0 ifTrue: [ cogit AddCq: adjust R: startReg. cogit AddCq: adjust R: stopReg. cogit AddCq: adjust R: repStartReg. ]. instr := cogit CmpR: startReg R: stopReg. jumpFinished := cogit JumpBelow: 0. cogit MoveXbr: repStartReg R: replReg R: TempReg. cogit MoveR: TempReg Xbr: startReg R: arrayReg. cogit AddCq: 1 R: startReg. cogit AddCq: 1 R: repStartReg. cogit Jump: instr. jumpFinished jmpTarget: (jumpEmpty jmpTarget: cogit genPrimReturn). "CANNOT REACH by falling though" jumpIncorrectFormat4 jmpTarget: (jumpIncorrectFormat3 jmpTarget: (jumpIncorrectFormat2 jmpTarget: (jumpIncorrectFormat1 jmpTarget: cogit Label)))]. (result := cogit compileInterpreterPrimitive) < 0 ifTrue: [^result]. jumpImm jmpTarget: (jumpNotSmi1 jmpTarget: (jumpNotSmi2 jmpTarget: (jumpNotSmi3 jmpTarget: cogit Label))). jumpOutOfBounds1 jmpTarget: (jumpOutOfBounds2 jmpTarget: (jumpOutOfBounds3 jmpTarget: (jumpOutOfBounds4 jmpTarget: jumpImm getJmpTarget))). self cppIf: IMMUTABILITY ifTrue: [jumpImmutable jmpTarget: jumpImm getJmpTarget]. ^ CompletePrimitive!

1 0

[OpenSmalltalk/opensmalltalk-vm] 78df78: Fix bogus primitiveAdd primitiveSubtract
by GitHub 26 Nov '17

26 Nov '17

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 78df786a9d27ee36bb087931452c573d6b2ee7ad https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/78df786a9d27ee36bb… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2017-11-26 (Sun, 26 Nov 2017) Changed paths: M nsspur64src/vm/cogit.h M nsspur64src/vm/cogitX64SysV.c M nsspur64src/vm/cogitX64WIN64.c M nsspur64src/vm/cogmethod.h M nsspur64src/vm/cointerp.c M nsspur64src/vm/cointerp.h M nsspur64src/vm/gcc3x-cointerp.c M nsspur64src/vm/interp.h M nsspur64src/vm/nssendcache.h M nsspur64src/vm/vmCallback.h M nsspursrc/vm/cogit.h M nsspursrc/vm/cogitARMv5.c M nsspursrc/vm/cogitIA32.c M nsspursrc/vm/cogitMIPSEL.c M nsspursrc/vm/cogmethod.h M nsspursrc/vm/cointerp.c M nsspursrc/vm/cointerp.h M nsspursrc/vm/gcc3x-cointerp.c M nsspursrc/vm/interp.h M nsspursrc/vm/nssendcache.h M nsspursrc/vm/vmCallback.h M nsspurstack64src/vm/gcc3x-interp.c M nsspurstack64src/vm/interp.c M nsspurstack64src/vm/interp.h M nsspurstack64src/vm/vmCallback.h M nsspurstacksrc/vm/gcc3x-interp.c M nsspurstacksrc/vm/interp.c M nsspurstacksrc/vm/interp.h M nsspurstacksrc/vm/vmCallback.h M spur64src/vm/cogit.h M spur64src/vm/cogitX64SysV.c M spur64src/vm/cogitX64WIN64.c M spur64src/vm/cogmethod.h M spur64src/vm/cointerp.c M spur64src/vm/cointerp.h M spur64src/vm/gcc3x-cointerp.c M spur64src/vm/interp.h M spur64src/vm/vmCallback.h M spurlowcode64src/vm/cogit.h M spurlowcode64src/vm/cogitX64SysV.c M spurlowcode64src/vm/cogitX64WIN64.c M spurlowcode64src/vm/cogmethod.h M spurlowcode64src/vm/cointerp.c M spurlowcode64src/vm/cointerp.h M spurlowcode64src/vm/gcc3x-cointerp.c M spurlowcode64src/vm/interp.h M spurlowcode64src/vm/vmCallback.h M spurlowcodesrc/vm/cogit.h M spurlowcodesrc/vm/cogitARMv5.c M spurlowcodesrc/vm/cogitIA32.c M spurlowcodesrc/vm/cogitMIPSEL.c M spurlowcodesrc/vm/cogmethod.h M spurlowcodesrc/vm/cointerp.c M spurlowcodesrc/vm/cointerp.h M spurlowcodesrc/vm/gcc3x-cointerp.c M spurlowcodesrc/vm/interp.h M spurlowcodesrc/vm/vmCallback.h M spurlowcodestack64src/vm/gcc3x-interp.c M spurlowcodestack64src/vm/interp.c M spurlowcodestack64src/vm/interp.h M spurlowcodestack64src/vm/vmCallback.h M spurlowcodestacksrc/vm/gcc3x-interp.c M spurlowcodestacksrc/vm/interp.c M spurlowcodestacksrc/vm/interp.h M spurlowcodestacksrc/vm/vmCallback.h M spursista64src/vm/cogit.h M spursista64src/vm/cogitX64SysV.c M spursista64src/vm/cogitX64WIN64.c M spursista64src/vm/cogmethod.h M spursista64src/vm/cointerp.c M spursista64src/vm/cointerp.h M spursista64src/vm/gcc3x-cointerp.c M spursista64src/vm/interp.h M spursista64src/vm/vmCallback.h M spursistasrc/vm/cogit.h M spursistasrc/vm/cogitARMv5.c M spursistasrc/vm/cogitIA32.c M spursistasrc/vm/cogitMIPSEL.c M spursistasrc/vm/cogmethod.h M spursistasrc/vm/cointerp.c M spursistasrc/vm/cointerp.h M spursistasrc/vm/gcc3x-cointerp.c M spursistasrc/vm/interp.h M spursistasrc/vm/vmCallback.h M spursrc/vm/cogit.h M spursrc/vm/cogitARMv5.c M spursrc/vm/cogitIA32.c M spursrc/vm/cogitMIPSEL.c M spursrc/vm/cogmethod.h M spursrc/vm/cointerp.c M spursrc/vm/cointerp.h M spursrc/vm/gcc3x-cointerp.c M spursrc/vm/interp.h M spursrc/vm/vmCallback.h M spurstack64src/vm/gcc3x-interp.c M spurstack64src/vm/interp.c M spurstack64src/vm/interp.h M spurstack64src/vm/vmCallback.h M spurstacksrc/vm/gcc3x-interp.c M spurstacksrc/vm/interp.c M spurstacksrc/vm/interp.h M spurstacksrc/vm/vmCallback.h M src/vm/cogit.h M src/vm/cogitARMv5.c M src/vm/cogitIA32.c M src/vm/cogitMIPSEL.c M src/vm/cogmethod.h M src/vm/cointerp.c M src/vm/cointerp.h M src/vm/cointerpmt.c M src/vm/cointerpmt.h M src/vm/gcc3x-cointerp.c M src/vm/gcc3x-cointerpmt.c M src/vm/interp.h M src/vm/vmCallback.h M stacksrc/vm/gcc3x-interp.c M stacksrc/vm/interp.c M stacksrc/vm/interp.h M stacksrc/vm/vmCallback.h Log Message: ----------- Fix bogus primitiveAdd primitiveSubtract The problem with code generated with VMMaker.oscog-eem.2278, was that primitiveAdd and primitiveSubtract would test for failure before generating failure/success code, leaving the behavior undefined. The code generation has been fixed in VMMaker.oscog-eem.2280, so use latest VMMaker revision to regenerate vm code

1 0

Something wrong in latest code generation
by Nicolas Cellier 26 Nov '17

26 Nov '17

Hi Eliot, if i look at spur64src/vm/cointerp.c generated from VMMaker.oscog-eem.2277 i see something wrong in primitiveAdd: static void primitiveAdd(void) { DECL_MAYBE_SQ_GLOBAL_STRUCT sqInt integerResult; char *sp; /* begin pop2AndPushIntegerIfOK: */ ====================================== See below: we test if primitive failed before we tried... ====================================== if (!GIV(primFailCode)) { integerResult = (stackIntegerValue(1)) + (stackIntegerValue(0)); if ((((((usqInt) integerResult) >> 60) + 1) & 15) <= 1) { /* begin pop:thenPush: */ longAtput((sp = GIV(stackPointer) + ((2 - 1) * BytesPerWord)), (((usqInt)integerResult << 3) | 1)); GIV(stackPointer) = sp; } else { /* begin success: */ /* Don't overwrite an error code that has already been set. */ if (!GIV(primFailCode)) { GIV(primFailCode) = 1; } } } } Note that I could comment the commit in github API. Unfortunately, diff are too big, and comments uneasy to find/view/scan.

1 1

[OpenSmalltalk/opensmalltalk-vm] b57b36: Fix bad inference of variable type in MiscPrimitiv...
by GitHub 26 Nov '17

26 Nov '17

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: b57b36bf75fa0da39a03ec629ff7dca92a2a329b https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/b57b36bf75fa0da39a… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2017-11-26 (Sun, 26 Nov 2017) Changed paths: M src/plugins/MiscPrimitivePlugin/MiscPrimitivePlugin.c Log Message: ----------- Fix bad inference of variable type in MiscPrimitivePlugin The generated code requires the fix from Squeak trunk Graphics-nice.385

1 0

snafu for e013b6766a05037812d48960702c0910
by Eliot Miranda 24 Nov '17

24 Nov '17

Hi All, mea culpa. Yesterday I managed to generate completely broken sources and broke CI servers etc. I've realized what went wrong. For the past week I've been doing VM development in a 64-bit Squeak image (*) and yesterday for the first time I generated sources from the 64-bit image. There was old very broken code in VMMaker: VMMaker>>initialize logger := Transcript. inline := true. forBrowser := false. internalPlugins := SortedCollection new. externalPlugins := SortedCollection new. platformName := self class machinesDirName. >> is64BitVM := Smalltalk wordSize == 8. interpreterClassName := Interpreter name. optionsDictionary := Dictionary new. >> optionsDictionary at: #BytesPerWord put: Smalltalk wordSize. VMStructType voidStructTypeCache which caused mayhem. In particular, invalid types were inferred for plugin files. These files ahem to be generated choosing types such that the generated files work on both 64 and 32-bit. That means /not/ initializing the VMMaker instance as above. I'm not sure when things will be back to normal but it should be soon, given that I finally understand what went wrong. Apologies for all the breakages. (*) I was looking at Juan Vuletich's 64-bit scavenger crash due to remember set overflow that Clément fixed recently. I was adding logging to the scavenger so we can better analyse failures like this and better tune the GC. Simulation of a 64-bit image is somewhat quicker in 64-bits (less work synthesizing 64-bit values from the heap) and so I thought I'd try developing in 64-bits. _,,,^..^,,,_ best, Eliot

4 4

[OpenSmalltalk/opensmalltalk-vm] 0cf830: Fix type inference of unskew variable in BitBltPlu...
by GitHub 24 Nov '17

24 Nov '17

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: 0cf83033b87b96235dd7286d287482c886076005 https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/0cf83033b87b96235d… Author: Nicolas Cellier <nicolas.cellier(a)sirehna.com> Date: 2017-11-24 (Fri, 24 Nov 2017) Changed paths: M src/plugins/BitBltPlugin/BitBltPlugin.c Log Message: ----------- Fix type inference of unskew variable in BitBltPlugin

1 0

VM Maker: VMMaker.oscog- nice.2281.mcz
by commits＠source.squeak.org 24 Nov '17

24 Nov '17

Nicolas Cellier uploaded a new version of VMMaker to project VM Maker: http://source.squeak.org/VMMaker/VMMaker.oscog- nice.2281.mcz ==================== Summary ==================== Name: VMMaker.oscog- nice.2281 Author: nice Time: 24 November 2017, 10:59:40.883312 am UUID: 4beeaee7-567e-1a4b-b0fb-bd95ce302516 Ancestors: VMMaker.oscog-eem.2280 Fix type inference of variable unskew in BitBlt >> copyLoop unskew is a signed bitShift and must remain signed. =============== Diff against VMMaker.oscog-eem.2280 =============== Item was changed: ----- Method: BitBltSimulation>>copyLoop (in category 'inner loop') ----- copyLoop | prevWord thisWord skewWord halftoneWord mergeWord hInc y unskew skewMask notSkewMask mergeFnwith destWord | "This version of the inner loop assumes noSource = false." <inline: false> <var: #mergeFnwith declareC: 'unsigned int (*mergeFnwith)(unsigned int, unsigned int)'> <var: #prevWord type: #'unsigned int'> <var: #thisWord type: #'unsigned int'> <var: #skewWord type: #'unsigned int'> <var: #halftoneWord type: #'unsigned int'> <var: #mergeWord type: #'unsigned int'> <var: #destWord type: #'unsigned int'> <var: #skewMask type: #'unsigned int'> <var: #notSkewMask type: #'unsigned int'> mergeFnwith := self cCoerce: (opTable at: combinationRule+1) to: 'unsigned int (*)(unsigned int, unsigned int)'. mergeFnwith. "null ref for compiler" hInc := hDir*4. "Byte delta" "degenerate skew fixed for Sparc. 10/20/96 ikp" skew == -32 + ifTrue: + ["Beware: separate skewMask initialization to avoid bad type inference. + Indeed, unskew is a bitShift and MUST remain signed, while skewMask is unsigned" + skew := unskew := 0. + skewMask := 0] - ifTrue: [skew := unskew := skewMask := 0] ifFalse: [skew < 0 ifTrue: [unskew := skew+32. skewMask := AllOnes << (0-skew)] ifFalse: [skew = 0 ifTrue: [unskew := 0. skewMask := AllOnes] ifFalse: [unskew := skew-32. skewMask := AllOnes >> skew]]]. notSkewMask := skewMask bitInvert32. noHalftone ifTrue: [halftoneWord := AllOnes. halftoneHeight := 0] ifFalse: [halftoneWord := self halftoneAt: 0]. y := dy. 1 to: bbH do: "here is the vertical loop" [ :i | halftoneHeight > 1 ifTrue: "Otherwise, its always the same" [halftoneWord := self halftoneAt: y. y := y + vDir]. preload ifTrue: ["load the 64-bit shifter" prevWord := self srcLongAt: sourceIndex. self incSrcIndex: hInc] ifFalse: [prevWord := 0]. "Note: the horizontal loop has been expanded into three parts for speed:" "This first section requires masking of the destination store..." destMask := mask1. thisWord := self srcLongAt: sourceIndex. "pick up next word" self incSrcIndex: hInc. skewWord := ((prevWord bitAnd: notSkewMask) bitShift: unskew) bitOr: "32-bit rotate" ((thisWord bitAnd: skewMask) bitShift: skew). prevWord := thisWord. destWord := self dstLongAt: destIndex. mergeWord := self mergeFn: (skewWord bitAnd: halftoneWord) with: destWord. destWord := (destMask bitAnd: mergeWord) bitOr: (destWord bitAnd: destMask bitInvert32). self dstLongAt: destIndex put: destWord. self incDestIndex: hInc. "This central horizontal loop requires no store masking" destMask := AllOnes. combinationRule = 3 ifTrue: [(skew = 0) & (halftoneWord = AllOnes) ifTrue: ["Very special inner loop for STORE mode with no skew -- just move words" hDir = -1 ifTrue: ["Woeful patch: revert to older code for hDir = -1" 2 to: nWords-1 do: [ :word | thisWord := self srcLongAt: sourceIndex. self incSrcIndex: hInc. self dstLongAt: destIndex put: thisWord. self incDestIndex: hInc]] ifFalse: [2 to: nWords-1 do: [ :word | "Note loop starts with prevWord loaded (due to preload)" self dstLongAt: destIndex put: prevWord. self incDestIndex: hInc. prevWord := self srcLongAt: sourceIndex. self incSrcIndex: hInc]]] ifFalse: ["Special inner loop for STORE mode -- no need to call merge" 2 to: nWords-1 do: [ :word | thisWord := self srcLongAt: sourceIndex. self incSrcIndex: hInc. skewWord := ((prevWord bitAnd: notSkewMask) bitShift: unskew) bitOr: "32-bit rotate" ((thisWord bitAnd: skewMask) bitShift: skew). prevWord := thisWord. self dstLongAt: destIndex put: (skewWord bitAnd: halftoneWord). self incDestIndex: hInc]] ] ifFalse: [2 to: nWords-1 do: "Normal inner loop does merge:" [ :word | thisWord := self srcLongAt: sourceIndex. "pick up next word" self incSrcIndex: hInc. skewWord := ((prevWord bitAnd: notSkewMask) bitShift: unskew) bitOr: "32-bit rotate" ((thisWord bitAnd: skewMask) bitShift: skew). prevWord := thisWord. mergeWord := self mergeFn: (skewWord bitAnd: halftoneWord) with: (self dstLongAt: destIndex). self dstLongAt: destIndex put: mergeWord. self incDestIndex: hInc] ]. "This last section, if used, requires masking of the destination store..." nWords > 1 ifTrue: [destMask := mask2. thisWord := self srcLongAt: sourceIndex. "pick up next word" self incSrcIndex: hInc. skewWord := ((prevWord bitAnd: notSkewMask) bitShift: unskew) bitOr: "32-bit rotate" ((thisWord bitAnd: skewMask) bitShift: skew). destWord := self dstLongAt: destIndex. mergeWord := self mergeFn: (skewWord bitAnd: halftoneWord) with: destWord. destWord := (destMask bitAnd: mergeWord) bitOr: (destWord bitAnd: destMask bitInvert32). self dstLongAt: destIndex put: destWord. self incDestIndex: hInc]. self incSrcIndex: sourceDelta. self incDestIndex: destDelta]!

1 0

[OpenSmalltalk/opensmalltalk-vm] ea5794: openssl: Fix compability with OpenSSL 1.1 as used ...
by GitHub 24 Nov '17

24 Nov '17

Branch: refs/heads/Cog Home: https://github.com/OpenSmalltalk/opensmalltalk-vm Commit: ea5794b715dcae7db5116e5cdc70813a93061c7f https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/ea5794b715dcae7db5… Author: Holger Hans Peter Freyther <holger(a)moiji-mobile.com> Date: 2017-11-24 (Fri, 24 Nov 2017) Changed paths: M platforms/unix/plugins/SqueakSSL/sqUnixOpenSSL.c Log Message: ----------- openssl: Fix compability with OpenSSL 1.1 as used by debian Norbert reported that a build for Debian9 fails to open outgoing TLS connections. In OpenSSL the semantic of BIO_write for zero sized writes has been changed. Do not attempt to write if the length is zero. Check for the length before calling BIO_write. I have updated the outgoing connection code and updated the incoming connection one as well (didn't manually test it though) and left the decrypt routine as is. The Pharo testcase: ./products/phcogspurlinuxht/pharo ~/Pharo-60520.image eval "ZnEasy getPng: 'https://pharo.org/files/pharo.png'" OpenSSL-issue: https://github.com/openssl/openssl/issues/4782 Commit: 834da7852235b8739f05fbf1e30506fd40e5a3bf https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/834da7852235b8739f… Author: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> Date: 2017-11-24 (Fri, 24 Nov 2017) Changed paths: M platforms/unix/plugins/SqueakSSL/sqUnixOpenSSL.c Log Message: ----------- Merge pull request #174 from zecke/openssl-1.2 openssl: Fix compability with OpenSSL 1.1 as used by debian Compare: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/21ad7cb6b78a...83…

1 0

Re: [Vm-dev] [Pharo-users] FFI 64 bit libClang issue on Sierra.
by Ben Coman 17 Nov '17

17 Nov '17

On Nov 14, 2017, at 4:34 PM, Ben Coman <btc(a)openinworld.com> wrote: On 15 November 2017 at 00:07, Todd Blanchard <tblanchard(a)mac.com> wrote: > I've got it loaded, have fixed up the library path and fixed the test for > version (built in lib returns very different version string). > > I have many tests green. However, as before, tests involving > CXSourceRange that return CXSourceLocations return all zero'd data. The > CXSourceRange struct looks sane. > > The calls to CXSourceLocation clang_getRangeStart(CXSourceRange range); > > and its twin RangeEnd result in zero'd structs. > > Any idea on where I can look to try to figure out where this is going > wrong? > > I have compiled little C programs and verified that both struct sizes > agree in FFI and native code (24 bytes each). > > Not sure what to try next. > c > -Todd Blanchard > Compile your own debug-VM** and make a little C wrapper MyCXSourceRange to forward to Clang's CXSourceRange, and set a gdb a breakpoint in MyCXSourceRange and compare 64 bit and 32 bit versions. That still leaves differences in the callout before your breakpoint in MyCXSourceRange is reached, so just do two callouts in a row and trace from the first to the second. ** https://github.com/OpenSmalltalk/opensmalltalk- vm/tree/Cog/build.linux64x64/pharo.cog.spur/build.debug cheers -ben On 17 November 2017 at 14:57, Todd Blanchard <tblanchard(a)mac.com> wrote: > Right, so I have my own debug VM (64 bit Pharo Cog) and a debug version of > libclang. > > I can attach with lldb and set a breakpoint on clang_getRangeStart > > Then I debug the test that gets diagnostics and I see that the > CXSourceRange has data. Then I step into 'range start' and I land in my > breakpoint for clang_getRangeStart and I can see that the argument to the > function - supposedly a CXSourceRange passed by value - is all zero'd > memory. > > So it appears that in the 64bit VM, structs passed by value doesn't work > right. > > I've traced back up the stack and I confess the marshaling code is totally > beyond my comprehension. This is what I was able to get: > > Process 1126 stopped > * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 > 2.1 > frame #0: 0x000000011e33136b libclang.dylib`::clang_ > getRangeStart(range=CXSourceRange @ 0x00007fff55f4e380) at > CXSourceLocation.cpp:89 > 86 > 87 CXSourceLocation clang_getRangeStart(CXSourceRange range) { > 88 // Special decoding for CXSourceLocations for > CXLoadedDiagnostics. > -> 89 if ((uintptr_t)range.ptr_data[0] & 0x1) { > 90 CXSourceLocation Result = { { range.ptr_data[0], nullptr }, 0 > }; > 91 return Result; > 92 } > Target 0: (Pharo) stopped. > (lldb) p range > (CXSourceRange) $0 = { > ptr_data = ([0] = 0x0000000000000000, [1] = 0x0000000000000000) > begin_int_data = 0 > end_int_data = 0 > } > (lldb) bt > * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 > 2.1 > * frame #0: 0x000000011e33136b libclang.dylib`::clang_ > getRangeStart(range=CXSourceRange @ 0x00007fff55f4e380) at > CXSourceLocation.cpp:89 > frame #1: 0x0000000109e20b5a Pharo`primitiveCalloutWithArgs at > X64SysVFFIPlugin.c:6041 > frame #2: 0x0000000109cb964c Pharo`primitiveExternalCall at > gcc3x-cointerp.c:77188 > frame #3: 0x0000000109cb1629 Pharo`interpretMethodFromMachineCode at > gcc3x-cointerp.c:19719 > frame #4: 0x0000000109cb77fc Pharo`ceSendsupertonumArgs(selector=4713336760, > superNormalBar=0, rcvr=4692466872 <(469)%20246-6872>, numArgs=1) at > gcc3x-cointerp.c:17767 > frame #5: 0x000000011774a135 > frame #6: 0x0000000109c76227 Pharo`interpret at gcc3x-cointerp.c:2703 > frame #7: 0x0000000109dbbf31 Pharo`-[sqSqueakMainApplication > runSqueak](self=0x00006080000d7370, _cmd="runSqueak") at > sqSqueakMainApplication.m:201 > frame #8: 0x00007fff81e5a8dd Foundation`__NSFirePerformWithOrder + 368 > frame #9: 0x00007fff803b1d37 CoreFoundation`__CFRUNLOOP_IS_ > CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23 > frame #10: 0x00007fff803b1ca7 CoreFoundation`__CFRunLoopDoObservers + > 391 > frame #11: 0x00007fff803926d9 CoreFoundation`__CFRunLoopRun + 873 > frame #12: 0x00007fff80392114 CoreFoundation`CFRunLoopRunSpecific + > 420 > frame #13: 0x00007fff7f8f2ebc HIToolbox`RunCurrentEventLoopInMode + > 240 > frame #14: 0x00007fff7f8f2bf9 HIToolbox`ReceiveNextEventCommon + 184 > frame #15: 0x00007fff7f8f2b26 HIToolbox`_ > BlockUntilNextEventMatchingListInModeWithFilter + 71 > frame #16: 0x00007fff7de89a54 AppKit`_DPSNextEvent + 1120 > frame #17: 0x00007fff7e6057ee AppKit`-[NSApplication(NSEvent) > _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796 > frame #18: 0x00007fff7de7e3db AppKit`-[NSApplication run] + 926 > frame #19: 0x00007fff7de48e0e AppKit`NSApplicationMain + 1237 > frame #20: 0x0000000109db5baf Pharo`main(argc=1, > argv=0x00007fff55f8bca8, envp=0x00007fff55f8bcb8) at main.m:52 > frame #21: 0x00007fff95b18235 libdyld.dylib`start + 1 > frame #22: 0x00007fff95b18235 libdyld.dylib`start + 1 > (lldb) up > frame #1: 0x0000000109e20b5a Pharo`primitiveCalloutWithArgs at > X64SysVFFIPlugin.c:6041 > 6038 result2 = floatObjectOf(floatRet1); > 6039 goto l14; > 6040 } > -> 6041 intRet1 = dispatchFunctionPointerwithwithwithwithwithwith(((SixteenByteReturn > (*)(sqIntptr_t, sqIntptr_t, sqIntptr_t, sqIntptr_t, sqIntptr_t, > sqIntptr_t)) (((void *) address2))), ((calloutState1->integerRegisters))[0], > ((calloutState1->integerRegisters))[1], ((calloutState1->integerRegisters))[2], > ((calloutState1->integerRegisters))[3], ((calloutState1->integerRegisters))[4], > ((calloutState1->integerRegisters))[5]); > 6042 /* begin maybeOwnVM:threadIndex: */ > 6043 > 6044 # if COGMTVM > > The struct that should have shown up appears in Pharo as > > "CXSourceRangeStruct ( > ptr_data1: (void*)@ 16r7FDF356C77B0 > ptr_data2: (void*)@ 16r7FDF356D0C70 > begin_int_data: 21 > end_int_data: 28 > )" > > Open to ideas at this point but I think to progress I would require deep > knowledge of Intel calling conventions and I don't really know where to go > next. > > Try to isolate the functions into a little dynamic lib and file a bug? > As you say, to minimize confounding factors the next step would be to try to reproduce the "structs passed by value" problem in a minimal library, and also someone trying to reproduce it doesn't need to compile the whole libclang Also now is a good time to move discussion to vm-dev. cheers -ben

1 0

Re: [Vm-dev] [Pharo-users] UFFI C basic types passed-by-reference for output
by Ben Coman 16 Nov '17

16 Nov '17

On 15 November 2017 at 20:47, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > On 15 Nov 2017, at 09:36, Ben Coman <btc(a)openInWorld.com> wrote: > > On 15 November 2017 at 17:27, Esteban Lorenzano <estebanlm(a)gmail.com> > wrote: > >> >> >> > On 15 Nov 2017, at 02:05, Ben Coman <btc(a)openInWorld.com> wrote: >> > >> > What is the recommended way for a C basic type to be >> passed-by-reference to function wanting to use it for output. >> > For example 'width' & 'height' here in this library unction... >> > >> > In Pharo... >> > >> > FFIOpaqueObject subclass: #FPDF_DOCUMENT >> > >> > FPDF_GetPageSizeByIndex__document: document >> > page_index: page_index >> > width: width >> > height: height >> > ^self ffiCall: #(int FPDF_GetPageSizeByIndex( >> > FPDF_DOCUMENT *document, >> > int page_index, >> > FFIFloat64 * width, >> > FFIFloat64 * height)) >> >> what you need to do here is to pass a “buffer” to contain the width and >> height: >> >> testGetPageSizeByIndex >> | document page_index widthBuffer heightBuffer width height >> result| >> PDFium FPDF_InitLibrary. >> document := PDFium FPDF_LoadDocument__file_path: helloPdf >> password: ''. >> widthBuffer := ByteArray new: (FFIFloat64 typeSize). >> heightBuffer := ByteArray new: (FFIFloat64 typeSize). >> > > Thanks Esteban. That worked. However I needed a minor tweak... > widthBuffer := ByteArray new: (FFIFloat64 new typeSize). > or... > widthBuffer := ByteArray new: (FFIFloat64 externalTypeSize). > > > Now it would be nice to do... > widthBuffer := FFIFloat64 newBuffer. > > > you know, I thought the same :) > but since I’m officially on holiday I didn’t do it (yet). > > If you want to add an issue and send a PR, it would be perfect :) > https://pharo.fogbugz.com/f/cases/20699/ btw, how do we write tests for FFI? Do we (or can we) deliver as part of standard opensmalltalk-vm a small shared library "libFFITestFrame.so" to write ffi tests against? With functions like... void passByRefOutputs(int *p_int, double *p_double) { *p_int = 42; *p_double = 4.2; } cheers -ben

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev November 2017