New subject: Squeak VM Speed Centre - validity and basis of improvements 5 Dec

25 Dec 2016

  Levente

  On Sun, 25 Dec 2016, Ben Coman wrote:

  > On Sat, Dec 24, 2016 at 8:13 PM, Tim Felgentreff
  > <timfelgentreff@gmail.com> wrote:
  >> We run benchmarks every day on
  >> http://speed.squeak.org/.
  >
  > Reviewing at the timeline   http://speed.squeak.org/timeline/
  > I am curious about some of the performance improvements.
  >
  > Several significant improvements seem aligned with Cog commit 2016120519
  > for example AStar...
  > http://speed.squeak.org/timeline/#/?exe=2,4,1,5,6,7,8,9&ben=AStar&env=2&revs=50&equid=off&quarts=on&extr=on
  >
  > which seems to be "Merge pull request #105 from estebanlm/Cog"
  > https://github.com/OpenSmalltalk/opensmalltalk-vm/network
  >
  > But then also aligned with the same Cog commit, there is a
  > corresponding improvement in the rsqueak performance, for example
  > ArrayAccess...
  > http://speed.squeak.org/timeline/#/?exe=2,4,1,5,6,7,8,9&ben=ArrayAccess&env=2&revs=50&equid=off&quarts=on&extr=on
  >
  > ...which seems to indicate a common cause from an in-Image
  > improvement, for which between 2016120322 and 2016120519 I see "The
  > various scanFor: and scanForEmptySlotFor: implementations only need to
  > access the size of their array once."
  > * Trunk: Kernel-eem.1050.mcz  (MethodDictionary)
  >   http://forum.world.st/The-Trunk-Kernel-eem-1050-mcz-td4925618.html
  > * Trunk: System-eem.920.mcz (SystemDictionary)
  >   http://forum.world.st/The-Trunk-System-eem-920-mcz-td4925619.html
  >
  >
  > So I'm curious do the benchmarks track both Image and VM changes?
  > Perhaps it would be useful to also benchmark Pharo to control for
  > Image changes (now that its returned to the fold using the mainline
  > opensmalltalk-vm)
  >
  > Now I'm further curious, the benchmarks below see a massive jump down
  > for 2016120519 for all data series, but all results are relatively
  > very close to zero, so I wonder are these valid results?
  >  ByteStringHash
  >  ClassVarBinding
  >  Compiler
  >  EqualBytes
  >  Fib
  >  FillArray
  >  FillByteArray
  >  FillString
  >  Graphsearch
  >  HashBytes
  >  HashWords
  >  InstVarAccess
  >  IntLoop
  >  IntegerByteCodes
  >  ModularConvolutionBytes
  >  ModularConvolutionWords
  >  ModularDotProductBytes
  >  ModularDotProductWords
  >  ModularSumBytes
  >  ModularSumWords
  >  PermutationCompositionArray
  >  PermutationCompositionWords
  >  Richards
  >  Send
  >  SendPrimitive
  >  SendWithManyArguments
  >  Slopstone
  >  WideStringHash
  >
  > Here all series jump down, and the result range seems valid...
  >  FloatLoop
  >
  > Here all series jump, and the result range seems valid. Rsqueak improves more...
  >  ShootoutSpectraNorm
  >
  > Here cog32, cog64 & rsqueakvm32 have a small jump down, but its is
  > very close to zero, so are they valid?  rsqueakvm64 shows no change...
  >  Blowfish
  >
  > Here only Cog jumps down, RSqueak stays much higher, seems valid..
  >  OrderedCollectionRandomInsert
  >  Nbody
  >
  > Here only Cog jumps down, RSqueak being already pretty low. The
  > results seem valid
  >  AStar
  >  ArrayAccess
  >  BinaryTree
  >  BitBltExampleOne
  >  DeltaBlue
  >  DoesNotUnderstand
  >  Json
  >  Mandala
  >  MandelbrotIterative1Thread
  >  MandelbrotIterative2Thread
  >  MandelbrotIterative4Thread
  >  MandelbrotIterative8Thread
  >  MandelbrotRecursive1Thread
  >  MandelbrotRecursive2Thread
  >  MandelbrotRecursive4Thread
  >  MandelbrotRecursive8Thread
  >  OrderedCollectionInsertFirst
  >
  > Here only Cog jumps down, Rsqueak is unchanged or not present, seems valid...
  >  Smopstone
  >  SplayTree
  >  ToolInteraction
  >
  > Here only cog32 jumps down, cog64, rsqueakvm32 & rsqueakvm64 no
  > change, seems valid...
  >  Fannkuck
  >
  > Here RSqueak improves, Cog stays the same, seems valid...
  >  ShootoutMandelbrot3
  >  ShootoutNBody
  >
  > The follow have no significant change around 5 Dec...
  >  BitBltColorMapping  - all already low
  >  DSAGen - all already low
  >  KMeans
  >  LRUCachePrintString
  >  Mandelbrot
  >  Polymorphy
  >  RaiseToLargeNumber
  >  RenderFont
  >  ShaLongString
  >  ShootoutBinarytrees
  >  ShootoutChameneosRedux
  >  ShootoutFannkuchRedux
  >  ShootoutFasta
  >  ShootoutFastaRedux
  >  ShootoutKnucleotide
  >  ShootoutMeteor
  >  ShootoutPidigits
  >  ShootoutRegexDNA
  >  ShootoutReverseComplement
  >  ShootoutThreadring
  >
  >
  > I also see around that time on 2 Dec Fabio says "I have fixed the
  > Squeak-trunk pipeline and we finally get daily updates again." So
  > maybe there were suddenly a bundle of improvements that showed up in
  > one go - but it seems the 2016120322 build should have picked those up
  > and didn't.
  > http://forum.world.st/Squeak-trunk-images-td4925570.html