Bill Bither's Insights

Our software developers at Atalasoft spend a lot of time writing unit tests, and making sure these tests pass.  It's an integral part of our development.  We have unit tests written in NUnit that run whenever code is checked into our CruiseControl.NET continuous integration server, and we have even more tests that are run each night.  In fact, we have so many tests, our nightly build of our DotImage product can take a few hours to run on a very fast dedicated machine.  We try to keep our continuous builds under 10 minutes.  If any of these tests fail, we receive an email notifying us of the failure, and what code change might have caused this failure.  The system works great, and as a result, we can be confident that our products are very robust, and that changes to our code will unlikely cause additional bugs.  We strive for very high coverage in the code we write.  When ever a bug is spotted, we write a failing unit test before the bug is actually fixed in our code.  We even have UI tests that are run on our products using Automated QA's Test Complete. 

One might think that as long as unit test coverage is high, there's little need for additional testing.  That just simply isn't true!  It's been a while since I've been a core developer at Atalasoft, but I'll always be a computer geek so I keep my hands dirty.  One of my many jobs is to write stuff that uses our products; eating our own dogfood.  Today I wrote a utility that corrects the resolution of TIFF FAX images that have non-square pixel aspect ratios without having to re-encode the entire multi-page TIFF File.  I used our TiffFile class to do this which can be used among many things to replace individual pages in a multi-page TIFF without having to re-construct the entire TIFF.  The process is simple:

1. Open the TIFF file in a read-only FileStream
2. Read the file using TiffFile
3. Loop through each page in the TIFF, and inspect the X and Y resolution TIFF Tags
4. If the X and Y resolutions are different, and the image is 1-bit, decode the image, resample it by the X / Y ratio, and replace the page with this new image
5. Save the TiffFile to a new file

If you would like the source code to this, see the Knowledge Base article I wrote on the subject.

Each page in a TIFF file has a set of TIFF Tags.  Most of these tags can be represented as a standard .NET object such as uInt, uShort, String, etc.  However for the Rational data type, there is no equivalent .NET type, so we created our own.  A Rational is simply a fraction with a numerator and a denominator, both as 32-bit Integers.  Our Rational class has 100% coverage, meaning every bit of that code is hit with our unit tests.  I was very surprised when the following bit of code didn't work as expected.

This is what the debugger told me the values were for xres and yres (both Rational objects).

xres.ToString() = "{1342177280 / 16777216}"
yres.ToString() = "{1291845632 / 16777216}"

However this simple equality operator returned false! 

if (xres != yres)
{
    //pixel aspect ratio is non-square
}

I knew this was wrong, so the first thing I did was write a failing unit test in the RationalTest.cs class located in our test assembly. 

[Test]
public void EqualityFailingTest()
{
    Rational rat1 = new Rational(1342177280, 16777216);
    Rational rat2 = new Rational(1291845632, 16777216);
    Assert.IsFalse(rat1 == rat2);
}

I just copied this code from a previous test that did the same thing (but the numerical values weren't quite as large).

Now to the source of the bug.  Stepping through the above test brought me to this code in the Rational class.

public override bool Equals(object obj)
{
    Rational rat = (Rational)obj;
    if (rat.Numerator == _numerator && rat.Denominator == _denominator)
        return true;
    else
    {
        //do the calculation
        return rat.Numerator * _denominator == rat.Denominator * _numerator;
    }
}

Come to find out, the calculation was overflowing returning 0 for both sides of the comparison.  Casting the values to 64-bit longs fixes the problem and passes the unit test.

return (long)rat.Numerator * (long)_denominator == (long)rat.Denominator * (long)_numerator;

The point is that the author of the Rational class did an excellent job writing unit tests with a remarkable 100% coverage.  However this unobvious bug made it's way through.  Only through additional testing, was this bug found.  Luckily we found it before a customer did and it will be in our next update!

This is exactly why we spend time in every development cycle writing sample applications using our SDK's.  For DotImage 5.0, which is currently in development, we've taken it a step further.  Our engineering team has adopted the SCRUM development process.  The standard SCRUM processes doesn't exactly address this form of QA, so we've modified it a bit.  The entire development team is spending 3 weeks in a "development" sprint and 1 week in a "QA" sprint testing other developer's code.  Keep in mind that we expect full coverage during the 3-week development sprint, so the 1-week QA sprint is for additional black box testing.  This process should help eliminate bugs like the one outlined in this article from ever seeing the light of day.  If anyone else has modified their their SCRUM process similarly, or is willing to try it themselves, please let me know!  Comments welcome.