On 29 July 2017 at 18:31, Eliot Miranda <eliot.miranda@gmail.com> wrote:
The success criterion is that they all succeed.  By extension this implies that the vm doesn't crash.

Does this mean that the Test Runner should report 100% pass rate? I have never seen that even with the stock image+vm from pharo.org. So I have a bootstrapping problem if I don't have a baseline that passes the CI test.

Just now I ran - randomly - a clean Pharo 5.0 image that I have on my Mac and the result I see is: 9098 run, 7408 passes, 8 skipped, 81 expected failures, 22 failures, 587 errors, 0 unexpected passes. If resolving those 22 failures and 587 errors is a blocker for running a CI then that is quite an obstacle from my perspective.

are other people really seeing 100% pass rates?