Re: Metrics for automation ...

Hello ,

think Process, Metrics, Management and control is the solution for any
software problem on the Earth - missing completely about the problem,
context and people who are part of problem. Your ideas and suggestions
echo well with them.

To tell you the truth, in a well organised group, collecting metrics
is not a problem!

I would not consider few of the things like Gannt chart, Goals etc. I
use a simple excel spread *** to allocate and track the work. I do

If you use excel 2002 to track the work, then you can easily create
gantt chart from the excel *** you use for tracking.

not like to give goal like 100 test cases per week etc - as these are
precisely the problem areas or challenging things for me to articulate.

Explain what is the nature of work/basic unit of work. If the basic
unit of work is to create testcases, the you have some targets (number)
that should be achieved and you can monitor that. In this case, I will
assume that a testcase development time will be max 1 day.

If the testcase development time is more that 2 days, then you
should monitor the progress (ie. collect metrics once a month). If you
think a testcase can be created in 1 hour but your team members are
taking days, then there is a serious problem in your team. Consider
sending them for some training or educate them, in additional to their
work.

If you are using any testing tool, even creation of testcase can be
a good metric.

We don't have set up standard for a "Test case" and while working
with clients - we get variety of styles and shapes/forms of test case.
So, 100 test cases can mean a 1 day or a month for me for automating. I
see a huge variation. That is my problem. Moreover I work with Human

"we get variety of styles and shapes/forms of test case" is not very
clear.

Since my team does not do regular Testing ( in addition to Automation -
that is something like Drink and Drive - we don't mix both in one) -

You should maintain the energy of your team. If you feel they aer
free/bored with testcase development, consider giving them some testing
work.

I can not measure number of bugs logged by each Team member - I am
quite sure that your testers will mislead you with all cooked up data
for defects.

I guess you file defects using a defect tracking tool. Being a
manager, you should be capable enough to dig out the defects filed by
your team. That's your teams assert, dont forget that. (This should be
a simple query!!)

I forgot to mention that we are typical CMM shop and all these things
are part of general project management framework. I am trying to bring
some kind of change in the thinking of the way we work.

What kind of change? This might not be acceptable by your
superiors/management. Are you talking about improving the efficency and
effectiveness of your team?

I have read all classic and available literature on metrics - including
those of Caper Jones. I have not found any of them for suitable in my
context. Have you read - Cem Kaner's paper on Metrics?

Metrics is colected and analysed to improve the process/efficenfy of
working. In this situation if the metric that you colect does not meet
your objective, you should not consider it at this point of time.

Best Regards,
Vivekanandan M

.