Yes, big benchmarks would be nice. Those on speed.squeak.org or in VMMaker are all somewhat small.

Note the Ruby community, for example, has benchmarks such as a NES emulator (optcarrot) that can run for a few thousand frames with predefined input as benchmarks. It's definitely possible.

Maybe some of the projects from HPI students could be made to work, there was a Chip8 emulator in Squeak, for example, that seems big enough. Or maybe the DCPU emulator at github.com/fniephaus/BroDCPU without a frame limit would work as a decent CPU bound benchmark.

Cross - dialect could be hard. Pharo and Squeak are fairly easy to do, but with larger programs staying compatible across different dialects is harder.