Re: HardBound and SoftBound (was "The State of Software")

In article <4A7BAF51.9040502@xxxxxxxxxxxxxxx>,
Andy \"Krazy\" Glew <"ag-news AT patten DASH my-last-name DOT net"> wrote:

Sigh. I will try to explain to you one last time what the problem
area is. Yes, OF COURSE, there are cases where checking is possible;
but being able to check only the cases that are trivially checkable
is not an interesting property, either theoretically or practically.

Sigh.

I will explain to you one last time that the problem is not whether a
compiler can generate checks for all buffer overflows.

That is the first time, as well. As least you have now admitted
that the technique doesn't do a proper job of checking even a single
category of errors.

The question is what fraction of buffer overflows can be detected.

What is the fraction of true positives (real bugs, that can be turned
ibto security holes) are detected?

As those of us who have experience in this area know very well, those
are NOT the right questions. There is a LOT of mathematical theory
and practical experience, and they confirm each other's conclusions.

Firstly, the question should be what fraction of the expected cost
(i.e. the probability of the problem occurring multiplied by the
cost if it does). Standard cost-benefit analysis, backed up by
game theory. Mere frequency counting is a politician's approach.

Secondly, there is a lot of experience that such partial checkers
are often (even usually) counter-productive. Programmers, auditors
and managers start to trust them and stop looking for bugs (or
blame the innocent) when the checker comes up clean.

Thirdly, there is 35+ years of experience with such techniques,
mostly in Fortran. Given what you now admit, SoftBound/HardBound
do roughly the equivalent of the 'common' form of Fortran bounds
checking (i.e. against the declared sizes). Since you seem to be
unaware of this experience, let me explain.

It is tricky to pass array bound information for assumed size (or
even explicit size) arrays in Fortran 77, and most compilers don't.
So most debugging compilers' bounds checks compare only against
the locally declared dimensions - i.e. there is effectively no
checking across procedure calls.

Experience is that this picks up most of the errors made by people
just learning to program, but very few of those made by people with
even a few months' experience. And, worse, it picks up primarily
the errors that are easier to find by hand! So few people bother
to enable it, even if they have it.

This is comparable with the frequent claim that syntax checking
editors increase productivity by 3 times - who spends almost all
of their time writing incorrect syntax? - kiddies learning to
program and senior executives playing at programming, that's who.

NAG does better, as did/(does?) Fujitsu Fortran, WATFOR/WATFIV and
some others. All experience is that the better tools found only
a few more bounds errors by count, but saved between ten and a
hundred times more debugging time than the other ones. Yes, THAT
much.

What is the fraction of false positives (code that is not buggy, but
that is incorrectly indicated as having a bug)?

No, it isn't. Again, you are using a political and not a technical
measure. It is the probability of such positives times the cost of
bypassing them or ignoring the problem.

For example, if you get ONE such problem that causes failure and is
infeasible to bypass, it renders the whole mechanism infeasible for
any program that uses the offending technique.

Similarly, if you just produce warnings and have ONE problem that
produces thousands of false positives, each of which has to be
analysed for whether it is genuine, few people will waste time using
the checker.

[ And please don't waste our time saying that such things can be
eliminated by a script - if they could be, SoftBound could do it. ]


Look, Andy, I am prepared to accept that you know a hundred times
as much about hardware design as I do. In this area, the ratio is
likely to be reversed - I have spent most of my working life in
and around it.


Regards,
Nick Maclaren.
.

Relevant Pages

  • Re: "Visual" Fortran?
    ... > developer and needed this type of tool, I would not complain too much ... > about the cost.] ... I use Fortran for), that is a huge cost savings. ... re-writes for the new platform). ...
    (comp.lang.fortran)
  • Re: .sig change
    ... Using the patented Mavis Beacon "Hunt&Peck" Technique, ... preparation of cost we consider the machine is not economically ... £4050 + VAT on parts, including frame, front forks, radiator, front ...
    (uk.rec.motorcycles)
  • Re: Why these original FORTRAN quirks?
    ... But WHOS's money will pay for these add-ons to a stunted Fortran? ... But the cost of Fortran programming has changed vastly since the early ... I remember that a good Fortran compiler would cost a little over $130, ... Make the programming masses want to use Fortran, the market size will increase and the prices will come down. ...
    (comp.lang.fortran)
  • Re: The Unexpected Cost of a Cup of Coffee & Millionaires
    ... Buy one million dollars worth of double-your-money-back-guaranteed products and send them back. ... schoolteachers who make less money than most white-collar workers. ... For this technique to work, ... Another way to look at this is to consider the true cost of the little ...
    (alt.coffee)
  • Re: Yorkshire Air Ambulance
    ... Using the patented Mavis Beacon "Hunt&Peck" Technique, porl typed ... you never know when you might be in their capable hands, the cost of ...
    (uk.rec.motorcycles)
Loading