Re: SUnit disappointment

"TimM" <dolphin.news.macta@xxxxxxxxxxxxxxx> wrote in message
news:ddl8g1$qg6$1@xxxxxxxxxxxxxxxxxxxxxxxx
> Coming back to Smalltalk from Java, I have to say that I am rather
> disappointed that the "grand-daddy" of testing frameworks isn't very
> helpful at tracking down why tests fail. I've never spent so much time in
> a debugger working out why tests fail, when all of the information that
> led up to the error was around but gets buried in a stack trace.

Well I won't comment on SUnit here as its a community project, except to
point out that the community is rather smaller than you might be used to,
and perhaps somewhat harder pressed. This does mean there is less effort
available to address issues that are probably less painful in Smalltalk if
you work with it. And that is key. If you use the same techniques, you'll
certainly find it less helpful. In static languages you need more help
because there is typically less you can do post mortem.

Here are a couple of techniques I use:
1) I always press the 'Debug' button in the SUnitBrowser when I'm
developing. Mind you this requires a fix that we put in D6 beta 1 so that
the success/failure indicators update in debug runs as well as normal runs.
2) Don't bother trying to work out why something failed by inspection, just
run it again. For example when examining a failure in the debugger rather
than sherlocking around in the stack trace, or even rather than restarting
and stepping through it, just select the text and evaluate it, or parts of
it. If the assertion says it should be equal, evaluate both sides
independently and see what you get.
3) Following on from (2), select the failing assertion expression itself (or
the part of it that you've determined is giving the wrong answer) and debug
through it using another debugger (Debug It command).
4) Following on from (3), I'll often be able to fix the failure in my second
debugger and then return back to the first, restart, and continue the test
run.

>
> It seems to me that for starters - should:raise: could at least tell you
> what exception was raised and what was expected. E.g. I would change the
> implemtnation to something like:
>

Please send any such suggestions to Joseph Pelrine. I'm sure he'll listen.

>... Furthermore - tests that everyone seems to write like:
> self should: [ object printString = 'test' ]

Why is "everyone" using that form? The block seems gratuitous. I would
write it:

self assert: object printString = 'test'

> Should have the infrastructure to allow a much more explicit failure so
> that you don't have to restart the relevant #should: method and step
> through the block evaluate code to find the problem.
>
> I would propose that SUnit should come bundled with some useful Contraint
> objects that can report failures more meaningfully -e.g.
> Equals verify: object printString with: 'test'
>
> // or - add some loose methods to dispatch things and make it more
> readable:
>
> object printString shouldEqual: 'test'.
>
> Internally the constraint can generate and internal #assert:description:
> with the relevent information to form a decent description when things
> fail. e.g. TestFailure signal: 'Contraint error: ',expectedValue, ' not =
> ', actualValue
>
> I've played with it for about 15 minutes and its definitely a big time
> saver - I just don't understand why this kind of stuff isn't in place
> already?
>

Like I say, not enough time and more pressing issues, depending on your
personal itch. If this is something you feel is important, please do go
ahead and put something together and run it by Joseph. If he is no longer
dealing with it, he'll know who is.

Regards

Blair


.