Given some possible inconveniences with the TestRunner, I am moving this to the treated inbox for now.
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2019-08-14T19:14:56+00:00, christoph.thiede@student.hpi.uni-potsdam.de wrote:
What are the specific information in the test runner you are referring to?
It is "reentrant", you do not need to re-run all tests after debugging one. A DialogBox would need to be manually set as non-exclusive.
If the test runner would be a better option than the dialog, wouldn't we want to show it even if only a single test fails (result defects size = 1)?
You're right, we could :)
Basically, I proposed these change because I did not really found a DialogBox comfortable for this purpose.
It is modally exclusive but not reentrant.
(I believe long test selector were displayed abbreviated, but I can't reproduce that.)
Also, DialogBoxes are not resizable.
On the other hand, I actually avoided using TestRunner in the last time, as it has a few convenience drawbacks:
I repeatedly faced an issue where a test, after debugging it, was not tear-downed correctly. Will try to reproduce that.
You cannot select a test without running it (impractical if you would like to open the menu, for example to browse it). Vice versa, you cannot run a test without selecting it, so the keyboard handling (such as navigation using arrow keys) is quite restricted ...
TestRunner is not aware of any additions to the test list (maybe we need something like #didCodeChangeElsewhere)
But your words "specific information" sound interesting to me. Would it be an option to turn the TestRunner upside down, aiming to display the last error information for each test?
Some wild ideas :-)
- allow selection of single test cases; instead of separately displaying all errors/failures, display their success with an icon
- add another panel to display information for the selected test
- this might include the failing line of the test method, the full error message (displayed in a big TextMorph, not as a one-liner with insufficient space), maybe the context stack - basically something like a collapsed debugger view
- I had an idea in the back of my mind for a few months: One could set up a small extension-based framework that provides additional information about the failing assertion (if you're interested, I can tell you more) - we could display these information here
- for example, in [self assert: [string allSatisfy: #isDigit]], instead of "AssertionFailed", the failure description could be something like "expected all elements of <'123a5'> to satisfy <#isDigit>, but <'a'> at index <4> didn't"
- ideally, the information would not be restricted to textual representation (why not import a diff view or an Inspector into the TestRunner? :) )
- It would be also possible to display *all* failures, not only the first one (I once read about resumable TestFailure implementations, why is TestFailure not marked as resumable by default?)
- If there are only few tests, maybe the test runner could suspend the failing/erroneous ones (actually their threads) instead of shutting them down? The user could then resume a test and directly jump to the raised Exception in debugger, without needing to wait for the test to re-run. Especially in case of heavy tests this might be a great help to some users.
- By always displaying the source of the selected test method, we could improve the overall liveness of SUnit
It's not a bunch, it's a forest of ideas, I know :) What do you think about them? Or are you already happy with the current TestRunner?
Best, Christoph
Von: Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org> im Auftrag von Taeumel, Marcel Gesendet: Mittwoch, 14. August 2019 17:03 Uhr An: John Pfersich via Squeak-dev Betreff: Re: [squeak-dev] The Inbox: SUnitTools-ct.5.mcz
Hmmm.... What are the specific information in the test runner you are referring to? If the test runner would be a better option than the dialog, wouldn't we want to show it even if only a single test fails (result defects size = 1)?
Best, Marcel
Am 31.05.2019 14:15:43 schrieb commits at source.squeak.org <commits at source.squeak.org>:
A new version of SUnitTools was added to project The Inbox: http://source.squeak.org/inbox/SUnitTools-ct.5.mcz
==================== Summary ====================
Name: SUnitTools-ct.5 Author: ct Time: 31 May 2019, 2:15:36.714053 pm UUID: 397d3b85-7ac5-3645-baa9-b9f848d90880 Ancestors: SUnitTools-tcj.4
Change CodeHolder>>testRunSuite: -- if multiple tests fail, instead of a UIManager dialog, open a TestRunner window that provides specific information.
Depends on SUnitGUI-ct.70.
=============== Diff against SUnitTools-tcj.4 ===============
Item was changed: ----- Method: CodeHolder>>testRunSuite: (in category '*SUnitTools-running') ----- testRunSuite: suite
- | runner result |
- (suite isKindOf: TestCase)
- ifFalse: [
- runner := TestRunner newForSuite: suite.
- result := runner runAll; result]
- ifTrue: [result := suite run].
- | result |
- result := suite run.
(result respondsTo: #dispatchResultsIntoHistory) ifTrue: [result dispatchResultsIntoHistory].
result hasPassed ifTrue: [^ self].
- result defects size = 1
- ifTrue: [result defects anyOne debug]
- ifFalse: [runner open]!
- (result defects size = 1
- ifTrue: [result defects anyOne]
- ifFalse: [UIManager default
- chooseFrom: (result defects collect: [:each | each class name , '>>' , each selector printString])
- values: result defects
- title: ('{1} passes, {2} failures, {3} errors\Debug a failure or error?' format: {
- result runCount . result failureCount . result errorCount}) withCRs]
- ) ifNotNil: [:defect | defect debug].
- !