|
Student's Opinions
This page shows feedback from Professor Laski's former
students.
Date: Fri, 22 Sep 2000 13:39:45 -0400
From: Jeff_Moore@Guardian.com
To: laski@oakland.edu
Subject: Former student applies principles from your class
Jeff Moore Wrote:
You probably don't remember me, but I took your class about 3-4 years ago.
If you do remember me, it would be because I kept falling asleep in class
and did rather poorly. Anyway, you will be happy to know that I was later
diagnosed with hypothyroidism, which was probably the cause of my sleeping,
not your lectures. before I was diagnosed, I used to jokingly say that I
had "lecture induced narcolepsy," but I never considered that I had a real
medical condition.
The reason that I am writing is that I am now one of the administrators on
an open source unit testing tool written in pascal. (It can be found at
http://dunit.sourceforge.net.) During the normal course of the project, I
wrote the attached email to the group of developers on the project. It
draws heavily on principles learned in your class. I thought you might be
interested in seeing what I came away with from your class after several
years.
Jeff Moore
jeff@procata.com
----------------------------------------------------------------
To: dunit-interest
Subject: assert and check (a comparison of test cases vs design by
contract)
OK, I've read up on the assert discussion. Here are a variety of notes (I
hope I remember them all)
I like the CheckXXX syntax because it distinguishes between design by
contract failures and test failures.
I think the GUI Should distinguish between a failed Check and a failed
assert.
Microsoft's new C# language has native support for design by contract,
including asserts. Their .net strategy also supports this. Eiffel is
available as a .net language and M$ trotted out Bertrand Meyer when they
announced .net. (To read about Bertrand Meyer and open source, visit my
web site at www.procata.com)
There is a proposal currently in the Sun community process to add a native
assert statement to java. I imagine the existence of C# and .net will put
this on the fast track. What this means to JUnit's assert, I do not know.
I treat AVs as an indicator of a lack of assertions. When people complain
to me that their program has AVs, I have absolutely no sympathy and
generally refuse to help them if they have not done their due dilligence
with assertions in the code. Then, my help usually consists of suggesting
or helping to find additional assertions. In my experience, AVs are rare
when DbC style assertions are used.
DbC assertions and test cases complement each other.
The purpose of the test cases is to excersize the code and provide a
certain level of confidence in the code under test. A given segment of
code has a nearly infiinite number of possible test cases. Most of these
test cases provide overlapping coverage of the code under test. The level
of coverage can be measured to give an objective level of confidence in the
correctness of the code under test. When you measure your results with
coverage tools, you will find It is EXTREMELY difficult to get anything but
very shallow coverage with test cases. Here are a list of coverages that I
have actually measured and tried to achieve for small segments of code from
shallow to deep: Instruction, branch, data chain, U-context, L-context.
Commercial tools typically only measure instruction coverage. Here is the
tool that I used: http://www.cse.secs.oakland.edu/edslabs/about/stad.asp
The purpose of DbC style assertions are to document the assumptions and
expected results of the code. This is generally a finite list
You need both.
Here is a simple example showing how branch coverage is more rigorous than
instruction coverage.
if (A) then X.
if (B) then Y
This simple code has 4 instructions: X, Y and the two ifs.
There are four possible paths through the code based on the coditions (A
and B), (NOT A AND B), (A AND NOT B), (NOT A AND NOT B).
It takes only one test case to acheive 100% instruction coverage, (A AND B)
would execute all four instructions.
The acheive full branch coverage, we would actually need four test cases.
One for each combinations of conditions. There is a metric, known as
cyclomatic complexity which essentially measures the complexity of a method
in terms of how many test cases would be required to acheive branch
coverage. A good rule of thumb is that any method with a cyclomatic
complexity greater than 10 should be refactored. This fits well with
refactoring practices. This rule has never failed me.
Cyclomatic complexity is easy to approximate in delphi. (Taken from our
internal style guide)
"Count the metric by counting the number of decision points in a routine and
adding one.
Decision points in Pascal are: if, while, repeat, for, and, or, break,
continue, exit, and each limb of a case statement.
A routine with a cyclomatic complexity greater than 6 is suspicious.
A cyclomatic complexity greater than 10 indicates that the routine should be
refactored to make it easier to understand, test, and maintain."
It gets worse.
Imagine that statement X in our example above was
temp := Q;
And statement Y was
P := temp +1;
These two statements form what is called a data chain. The start of the chain
is the assignment to a variable.
The end is the read. Data chain coverage measures how many of the possible,
reachable combinations of assignments and reads have been executed.
It is possible (in fact likely) that full branch coverage does not acheive
full data chain coverage. There are generally many more data chains in a
program than branches.
One reason why we try to limit the scope of variables is to limit the
proliferation of data chains. Also, the more limited the scope, the eaiser it
is to determine the reachability of a given data chain.
So, in in a formal sense, no matter how many test cases you write, you can
never be sure that the code under test is free of bugs.
In fact the returns of testing diminish incredibly fast after the first few
test cases.
This is because test cases require setting up combinations of conditions to
achieve the different levels of more rigorous coverage.
DbC style pre and post conditions can accomplish some of the same goals as
testing, but avoid this combinatorical explosion of test cases.
Without test cases to excersize the assertions, you cannot say anything
meaninful about the quality of the code.
Both test cases and pre and post conditions can be thought of as documentation.
The test cases document by example. The DbC assertions document by
description.
Both are better than comments because they are executable and thus more easily
validated. However, writing comments can often suggest assertions and
test cases.
I use all three: Comments, test cases and assertions in mutually supporting
ways to document and assure the quality of the code I am responsible for.
|
