Welcome to Atalasoft Community Sign in | Help

Stomping on Memory

Way back when, I wrote of some of the problems with varargs.  We had a bug show up that I created in fixing a problem with a function that was a client of a varargs function.

The issue was that when called in one mode, it was assumed that a pointer to memory was pointing to bytes and in another mode it was pointing to long words.  I had to put in some special case code to make sure that the number of elements being passed in was in the right scale and that if the target varargs function expected long words that the byte-oriented data was padded out to the next long word.

In doing so I inadvertently created a new bug whereupon one of my local variables wasn't being initialized consistently.  This galls me because I remember going over the code pretty thoroughly before I checked it in.  I guess I'm still human.

At any rate, this particular bug was dependent upon the stack contents and in many cases the stack was providing values that worked just fine.  Had this code been written in C# it would've never happened, since the C# compiler disallows unitialized locals.  The default compiler settings in VisualC++ don't make this check.  Ouch.

To track this down I used the PageHeap which aligns all allocations on page boundaries so that if you step off the end an exception is thrown.  In addition I had heap checking turned on for all allocation related calls.  The problem is that these doesn't really do anything for you unless you have managed debugging on and nunit was just disappearing when I ran it through VisualStudio before it actually got to any of my tests.

Eventually I was able to divide and conquer it down to a point where it was debuggable.

Published Friday, July 14, 2006 3:30 PM by Steve Hawley


No Comments
Anonymous comments are disabled