Re: Unit Testing a API library
- From: "H. S. Lahman" <h.lahman@xxxxxxxxxxx>
- Date: Mon, 23 Apr 2007 15:54:44 GMT
Responding to Test70...
I am new developer. One of my first tasks is to develop Unit Tests
for an existing library. The developers of the library had some minimal
unit tests working once upon a time but the tests are not upto date, some
of the APIs have changed, new APIs have been added existing APIs
have been removed etc. So my task is to develop a full set of APIs
for the APIs.
This is a library written in C.
I need some guidance on unit tests. How do I differentiate between
what kind of stuff needs to be tested by the Unit Tests & what kind
need to be tested by the QA team.
The short answer is that you both test to the same requirements. If QA were tasked with testing the library (as opposed to its use in an application), then they would test exactly the same way you do. You and QA just view things is a somewhat different way (black box vs. while box) and at a different scale (system vs. unit). But in the end you both provide test screens versus the same requirements, so I would not get hung up on what QA might do.
Testing libraries is usually pretty easy, especially C function libraries. That's because each function should be executable standalone. In that case all you need to do is provide known inputs, invoke it, and validate the outputs. However, ...
For eg. There a function
RetVal LibFunc1(sometype handle, someparam xxx, someoutparam * pOut);
( Desc - does stuff
Error Return Values - Err1, Err2 & Err3
If successfull, pOut could then be passed as input to LibFunc2.
)
Before Calling any APIs in the library a function called LibInitWhatever
needs to be called. So my unit test for LibFunc1 would also essentially
include a call to LibFunc1. sometype handle is essentially obtained by
calling a different function LibGetHandleFunc. So I would also have to
call this function in my unit test for LibFunc1 would also call LibGetHandleFunc.
So if I write a unit test which calls LibInitWhatever & then LibGetHandleFunc
& then proceed to test LibFunc1, then I get a feeling that I am blurring the
lines between unit testing & integration testing? Is this true?
Also in order to check if pOut was initialized correctly by LibFunc1,
I may have to call LibFunc2. Same concern here.
This library does not seem to be well-formed as a library because what the FUT does depends on what other functions that have already been called do. There also seems to be some internal state data that is preserved between function calls. Doing a proper unit test will be very difficult if you do not have access to that internal state data. For example, if you can't look at internal values that the FUT changes, how can you tell the FUT changed them properly?
What this means is that the "library" is actually an application that you are functionally testing in situ. IOW, you aren't doing unit tests. What that, in turn, means is that when writing test cases for any given function you need to answer two questions:
What do I need to do to properly initialize the "library" for the test case in hand? The answer to this will result in the appropriate calls to other library functions prior to testing the FUT. As you suggest, that would probably involve invoking LibInitWhatever and LibGetHandleFunc.
What do I need to do to see the results of what the FUT did? The answer to this will result in the appropriate calls to other library functions after the FUT has executed. That might involve calling LibFunc2, but one would only do that if one needed to evaluate something other than pOut. Since pOut is just data returned by LibFunc1, you can evaluate it for correctness directly.
As you have observed, this all implies that to test any given function you need a very good knowledge of how the library as a whole works and what its internal dependencies are. Good luck with that if it is a third party library. B-)
This is what I think my unit tests should do
- test LibFunc1 with correct input params & check if it works if it returns
success
- check if it returns Err1, Err2 & Err3 in the error conditions
That's fine; this is pretty basic.
Is this exhaustive? Am I missing something?
To exhaustively test one needs to apply every possible combination of every possible input parameter value. Even for simple functions that can get out of hand very quickly unless one has automatic test case generation software.
In this situation things are worse because that would include every possible way LibInitWhatever, etc. can be invoked prior to calling LibFunc1. That's why exhaustive testing of even modestly complex applications tends to be infeasible.
Much of the testing literature is about providing a subset of exhaustive tests that have the best fault coverage for their cost of development and execution. Any book on testing will give you basic techniques like bounds checking for designing test cases for non-exhaustive testing.
Am I adding stuff which shouldn't be in the Unit tests?
No, what you are adding is forced upon you because the library functions are not self-contained.
*************
There is nothing wrong with me that could
not be cured by a capful of Drano.
H. S. Lahman
hsl@xxxxxxxxxxxxxxxxx
Pathfinder Solutions
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
"Model-Based Translation: The Next Step in Agile Development". Email
info@xxxxxxxxxxxxxxxxx for your copy.
Pathfinder is hiring: http://www.pathfindermda.com/about_us/careers_pos3.php.
(888)OOA-PATH
.
- Follow-Ups:
- Re: Unit Testing a API library
- From: James Dennett
- Re: Unit Testing a API library
- From: Matthias Wolpers
- Re: Unit Testing a API library
- References:
- Unit Testing a API library
- From: Test70
- Unit Testing a API library
- Prev by Date: The Day Best Practices Died
- Next by Date: Re: Unit Testing a API library
- Previous by thread: Re: Unit Testing a API library
- Next by thread: Re: Unit Testing a API library
- Index(es):
Relevant Pages
|
