Re: BUG or FEATURE
- From: "Michael Bolton" <google@xxxxxxxxxxxxxxxxx>
- Date: 4 Jul 2006 03:09:41 -0700
How you decide if this is a bug or a feature.
You don't. The project owner does that.
Of course, it's easy
when you get application crash or program does not follow the
specification - but what do you do when is about all small things that
makes program look /feel /work better /easier.
In case it's a company product for sale you can get a lot of
arguments, as it is described in "An Introduction to Scenario
Testing" by Cem Kaner, to make the product clean and nice - but what
if you were hired just for "do this".
As a tester, you're not being hired to design the product, to determine
the features, or to make ultimate decisions about product quality.
You're hired to provide information to management and to the rest of
the project community that will help them to make those decisions.
Having a set of specification the product is created. Now, when it
comes to testing and reporting issues how you categorize "this is a
bug" and "this is a feature".
It helps when you have a definition of "bug". A bug, the way James
Bach identifies it and as I teach it, is "anything that threatens the
value of the product to some person"; a bug is something that bugs
someone. That's a subjective definition; what is a bug to me may not
be a bug to someone else. So whose opinion is to prevail--whose
opinion matters? The person who matters is really the person with the
authority to make decisions about the project, and that's the project
owner. So the final definition of "bug" is "anything that threatens
the value of the product to someone who matters."
For example you have an application that manage OBJECTS and you have a
feature were you can select and delete all of them. No confirmation
message, is this a bug or a feature that needs to be implemented.
I've been arguing on this kind of stuff (small basics issues
/functionalities that are not described in specs) with developers on
many projects for to long. For the above example they'll say "there
is no message request in specification" and they are right there
isn't.
In the long run, you're arguing with the wrong people (unless the
developers are also the project owners).
The problem that we run into is the oracle problem. An oracle (again,
as we teach it) is "a principle or mechanism by which we recognize a
problem". In more traditional parlance, an oracle is typically a
program or table that has the right answer. In our vision, rightness
is subjective, so at best an oracle has a potentially right answer.
Some oracles are stronger or more persuasive than others, but all
oracles are examples of heuristics; a heuristic is "a fallible method
for solving a problem". No oracle is completely reliable in all
contexts, and there is no oracle that can tell us with 100% certainty
that the application is behaving properly in all circumstances.
Some people say that testers use "intuition", or "common sense", or
"good design principles", or some other term in order to identify
potential problems in a product. We say that testers use oracles.
Many of these oracles are consistency heuristics. Generally a product
should be consistent with
- its own history; the new version should work the same as the old one;
- the image that the company wants to project;
- comparable products; products that perform the same tasks or
functions as this one;
- claims that people make about it;
- reasonable user expectations;
- the purposes for which that the program could be used;
- the product itself (that is, if the product displays or maintains the
same thing in multiple ways or in multiple places, then each instance
of the thing should be consistent with the other)
- standards, statutes, or specified requirements.
A product should be consistent with each of these things /unless there
is a compelling reason for the product to be inconsistent/. For
example, we would want a feature in product to be inconsistent with the
older version if the new version represents an improvement or a bug
fix. Some of the oracles may conflict with one another, too. The
question becomes this: which ones (does management think) are the most
important in a given context?
I remember last year was a thread width kind of the same problem (an
existing feature that was not requested), but in that case, the feature
was already implemented. In this case, I ask about the stuff that is
not done yet and solving these issues will take time which of course
will delay the product release.
Is that your responsibility, though? Testers don't create these
problems; they merely identify the circumstances that cause them. To
put it another way, we don't light the fires; we smell the smoke and
pull the alarm. It's up to management to determine whether the fire
needs to be fought, or whether it will go out on its own.
What is the solution? Test only using specs as a check list and ignore
all that is bothering me. I couldn't find in any (by any I mean IEEE
and ISO) standards that will describe exactly and very detailed quality
demands of a software product.
You could test this way (and many do), but your value as a tester would
be vastly lower. Since standards are only one kind of oracle, you
implicitly ignore all the other oracles,
Another solution would be to create an internal software quality
demands standard that would be approved by the management. But this is
very expensive solution; it will take a lot of time since this would
have to be adjusted for every project.
In fact, this is the only solution there is--have you noticed that this
is exactly what happens on every project? Now: there are some ways
that management tries to hand that responsibility off to other agents
for convenience. But ultimately management makes decisions about when
the product will ship, and what's in it when it does.
---Michael B.
.
- Follow-Ups:
- Re: BUG or FEATURE
- From: Shrinik
- Re: BUG or FEATURE
- From: Shrinik
- Re: BUG or FEATURE
- References:
- BUG or FEATURE
- From: dumitru.corobceanu@xxxxxxxxx
- BUG or FEATURE
- Prev by Date: can you recommend an application which generate tests?
- Next by Date: Re: Testing Automation/Test Managers in Linux
- Previous by thread: Re: BUG or FEATURE
- Next by thread: Re: BUG or FEATURE
- Index(es):
Relevant Pages
|
