Hi All,
mea culpa. Yesterday I managed to generate completely broken sources and broke CI servers etc. I've realized what went wrong. For the past week I've been doing VM development in a 64-bit Squeak image (*) and yesterday for the first time I generated sources from the 64-bit image. There was old very broken code in VMMaker:
VMMaker>>initialize logger := Transcript. inline := true. forBrowser := false. internalPlugins := SortedCollection new. externalPlugins := SortedCollection new. platformName := self class machinesDirName.
is64BitVM := Smalltalk wordSize == 8.
interpreterClassName := Interpreter name. optionsDictionary := Dictionary new.
optionsDictionary at: #BytesPerWord put: Smalltalk wordSize.
VMStructType voidStructTypeCache
which caused mayhem. In particular, invalid types were inferred for plugin files. These files ahem to be generated choosing types such that the generated files work on both 64 and 32-bit. That means /not/ initializing the VMMaker instance as above.
I'm not sure when things will be back to normal but it should be soon, given that I finally understand what went wrong. Apologies for all the breakages.
(*) I was looking at Juan Vuletich's 64-bit scavenger crash due to remember set overflow that Clément fixed recently. I was adding logging to the scavenger so we can better analyse failures like this and better tune the GC. Simulation of a 64-bit image is somewhat quicker in 64-bits (less work synthesizing 64-bit values from the heap) and so I thought I'd try developing in 64-bits. _,,,^..^,,,_ best, Eliot
On 16. Nov 2017, at 07:18, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi All,
Hey!
I was looking at a OpenSSL issue and master with spur seems to be broken, is it because of the same issue? What would it take to have everyone use Pull Requests and benefit from pre-merge testing? These days computing time is so cheap... (think of a reference to the mythical man month)
cheers
holger
mea culpa. Yesterday I managed to generate completely broken sources and broke CI servers etc. I've realized what went wrong. For the past week I've been doing VM development in a 64-bit Squeak image (*) and yesterday for the first time I generated sources from the 64-bit image. There was old very broken code in VMMaker:VMMaker>>initialize logger := Transcript. inline := true. forBrowser := false. internalPlugins := SortedCollection new. externalPlugins := SortedCollection new. platformName := self class machinesDirName.
is64BitVM := Smalltalk wordSize == 8.
interpreterClassName := Interpreter name. optionsDictionary := Dictionary new.
optionsDictionary at: #BytesPerWord put: Smalltalk wordSize.
VMStructType voidStructTypeCache
which caused mayhem. In particular, invalid types were inferred for plugin files. These files ahem to be generated choosing types such that the generated files work on both 64 and 32-bit. That means /not/ initializing the VMMaker instance as above.
I'm not sure when things will be back to normal but it should be soon, given that I finally understand what went wrong. Apologies for all the breakages.
(*) I was looking at Juan Vuletich's 64-bit scavenger crash due to remember set overflow that Clément fixed recently. I was adding logging to the scavenger so we can better analyse failures like this and better tune the GC. Simulation of a 64-bit image is somewhat quicker in 64-bits (less work synthesizing 64-bit values from the heap) and so I thought I'd try developing in 64-bits. _,,,^..^,,,_ best, Eliot
On 24 November 2017 at 12:18, Holger Freyther holger@freyther.de wrote:
On 16. Nov 2017, at 07:18, Eliot Miranda eliot.miranda@gmail.com
wrote:
Hi All,
Hey!
I was looking at a OpenSSL issue and master with spur seems to be broken, is it because of the same issue? What would it take to have everyone use Pull Requests and benefit from pre-merge testing? These days computing time is so cheap... (think of a reference to the mythical man month)
cheers
holger
I haven't used them, but there seem a few options for issuing PRs from the command-line, which may make this more palatable. Thus the CI should run without needing to visit github's web-UI.
https://github.com/jd/git-pull-request https://hub.github.com/ (search for "pull") https://git-scm.com/docs/git-request-pull
cheers -ben
mea culpa. Yesterday I managed to generate completely brokensources and broke CI servers etc. I've realized what went wrong. For the past week I've been doing VM development in a 64-bit Squeak image (*) and yesterday for the first time I generated sources from the 64-bit image. There was old very broken code in VMMaker:
VMMaker>>initialize logger := Transcript. inline := true. forBrowser := false. internalPlugins := SortedCollection new. externalPlugins := SortedCollection new. platformName := self class machinesDirName.
is64BitVM := Smalltalk wordSize == 8.
interpreterClassName := Interpreter name. optionsDictionary := Dictionary new.optionsDictionary at: #BytesPerWord put: Smalltalk wordSize.
VMStructType voidStructTypeCachewhich caused mayhem. In particular, invalid types were inferred for
plugin files. These files ahem to be generated choosing types such that the generated files work on both 64 and 32-bit. That means /not/ initializing the VMMaker instance as above.
I'm not sure when things will be back to normal but it should be soon,
given that I finally understand what went wrong. Apologies for all the breakages.
(*) I was looking at Juan Vuletich's 64-bit scavenger crash due to
remember set overflow that Clément fixed recently. I was adding logging to the scavenger so we can better analyse failures like this and better tune the GC. Simulation of a 64-bit image is somewhat quicker in 64-bits (less work synthesizing 64-bit values from the heap) and so I thought I'd try developing in 64-bits.
_,,,^..^,,,_ best, Eliot
On 24. Nov 2017, at 13:27, Ben Coman btc@openinworld.com wrote:
Hey!
I haven't used them, but there seem a few options for issuing PRs from the command-line, which may make this more palatable. Thus the CI should run without needing to visit github's web-UI.
and one can probably use the github API to auto-merge PRs that have a successful build and come from a trusted developer. It is nice to live in a time computing time is so cheap. ;)
holger
+1 for using generous features of github. creating branches is a cheap action for developpers
I also see another problem: we use two different paths for developping the VM - via VMMaker (slang) changes and VM simulator tests - via code generation from VMMaker, compilation of VM, and non regression testing by passing some image SUnit test cases
Only the latest form is on Opensmalltalk-vm (github) right now. This can lead to long periods without github/travis tests. In our case see: https://github.com/OpenSmalltalk/opensmalltalk-vm/commits/Cog/spur64src/vm/c... https://github.com/OpenSmalltalk/opensmalltalk-vm/commits/Cog/spur64src/vm/g... there's been no travis regression tests from VMMaker.oscog-eem.2262 to VMMaker.oscog-eem.2277 for the JIT and from VMMaker.oscog-eem.2266 to VMMaker.oscog-eem.2277 for the interpreter... That's a lot of changes at once.
In particular, each and every change to CCodeGeneration SHALL be tested individually on github/travis, because it is precisely what can make a difference between the VM simulator and the C code.
So having the code generated from VMMaker committed into a specific branch as already proposed might help bissecting the problems...
2017-11-24 7:47 GMT+01:00 Holger Freyther holger@freyther.de:
On 24. Nov 2017, at 13:27, Ben Coman btc@openinworld.com wrote:
Hey!
I haven't used them, but there seem a few options for issuing PRs from
the command-line, which may make this more palatable.
Thus the CI should run without needing to visit github's web-UI.
and one can probably use the github API to auto-merge PRs that have a successful build and come from a trusted developer. It is nice to live in a time computing time is so cheap. ;)
holger
vm-dev@lists.squeakfoundation.org
-
Ben Coman -
Eliot Miranda -
Holger Freyther -
Nicolas Cellier