Re: Automated Test Scripts

"." wrote:

> Not necessarily. Some companies hire what they call Business Analysts.
> These people work closely with the customer and understand what the
> customer is going to do with the system. In some cases they help create
> the procedure manuals for the customer to train its staff. The BA will
> take these procedure manuals (or user scenarios) and make sure the end
> product works as expected.

Then that is a Testor role, regardless what's written on the business card.

We need to find a way to make tests very easy to write, so such people can
write them as easily as filling out a spread***.

> There will also be certain business rules
> (government regulated companies for example have very specific procedures
> they must do to be compliant to a law or act) the BA will test for. These
> requirements come from the customer to the BA and programmer. The BA does
> not need to interact with the programmer to confirm the requirement has
> been met.

Communication is good.

> > Given module X which drives module Y, should I test Y by driving X?
> > Indirectly?
>
> If you are testing from a customer's perspective you would test X and
> indirectly test Y.

If I want test failures to instantly implicate only the correct module...

> Take for example a function that if I pass a NULL
> reference will crash the entire system. If interface X makes it impossible
> to send a NULL reference to method Y then I don't care that Y has failed
> to validate its inputs.

If I want a module to be _more_ tested than the module's clients' protocols
need...

(Recall, per all my other posts, that more tested is good. It does _not_
mean "more bugs in the bugbase".)

Your objections are illustrations why to test more, not less.

> > Most business GUIs don't need any custom controls. All their standard
> > controls (list box, edit field, etc.) have in-memory representations
that
> > respond to test code without simulating raw inputs through the GUI
platform.
>
> In theory. Sometimes programmers find ways to mess things up. Only takes
> one programmer to do that.

That is why programmers should write tests for every module, to intercept
all the extra ways to mess things up.

In the specific case of a GUI, here's an absurdly simple GUI test:

TEST_(TestCase, setCaption)
{
OurDialog aDialog;
aDialog.setCaption("yo");
CHECK_EQUAL("yo", aDialog.getCaption());
// reveal(aDialog);
}

(Other guidelines indicate there's no need to test getters and setters.
Substitute some more complex window logic for my example.)

The infernal thing about GUI Toolkits is their vendors experience no
pressure to provide a perfect system for a program to query visual
appearances. In many GUIs you can Set a value but can't Get the correct
value back from the internal state.

The last line in my test is a commented-out call to a test-side function
called 'reveal()'. Write that function for each GUI you test, and make it
very easy to call. Then, while developing this test case, de-comment the
reveal(), and run the tests. reveal() will raise your window in its tested
state.

Look at it, and visually confirm that the test is supporting visual state.

Then comment out the reveal.

If my window had some other caption, I would invent a new test-side function
to extract that real caption, and my tests would use it instead of the buggy
getCaption(). So the odds of needing to use reveal() go down, and I can
develop GUI code (and tests) longer without displaying the GUI. Even as I
add new features.

If my window has the correct caption, getCaption works, so I leave it in
this test case, and future test cases have an incentive to use it. So the
odds of needing to use reveal() go down, and I can develop GUI code (and
tests) longer without displaying the GUI. Even as I add new features.

> Yes, that is exactly it. The test team (outside the development team) is
> there to catch all the little details that are too time consuming for a
> programmer to deal with. Case in point, Microsoft claimed that Windows 98
> Second Edition and Windows 98 Service Pack 1 were identical. Turns out
> that is not true. There is a limit in the environment varaibles on Second
> Edition that did not exist on Service Pack 1.

There are always more roles for testers. I'm not sure what the emphasis
"outside the development team" is for. If they work the same project, then
aren't they the same team?

> Another reason for testers who don't interact with programmers would be
> that programmers tend to think differently then some of their customers.

That sounds like a _major_ reason for them to work as much as possible
_with_ programmers!

> A
> tester or BA that does not interact with the programmers but does interact
> with the customer will tend to think more like the customer and do things
> the programmer might not think to try. I don't know how many times I've
> told a programmer about a problem only to get the response, "Why would
> they do that?"

Correct. It is a test-side role to track customer awareness and needs.

Ah, I see. Too much exposure to those naughty programmers will soon have the
testers playing with bluetoothed Furbies, wearing propeller beanies, popping
bubble wrap, drinking addictive carcinogens, and picking up all those
disturbing programmer habits that will disconcert their customer liaisons,
and lead to blind spots in their test coverage. Point.

--
Phlip
http://www.c2.com/cgi/wiki?ZeekLand


.