If you build it, They will come

A Visual Studio command prompt environment in NAnt

2008-05-14T13:46:00Z

Released in March, Atalasoft's toolkit v6.0 had a bug release v6.0a and we've branched to begin building new functionality for our new release while being able to maintain/bugfix v6.0. In addition, we're internally branched to have one codebase as a sandbox, of sorts. To a build engineer, this may mean lots of work! First, you have to branch (which on our codebase takes a long time in itself) not once, but twice. Each time you branch, the engineers need to be able to build, which means all those pesky files with version numbers need updating (check out the AtalaFileRegex posts). But mostly, it means new integration targets and projects! Seeing this coming, I wanted to breakout redundancy and use build files like objects, and build targets like object methods.

The first step to this is making the environment work like it does if you open the Visual Studio command prompt. This command prompt is really just a regular console with one of a handful of batch files called automatically. Depending on the hardware and version of VS, mostly its responsibility is setting the PATH, LIBPATH and framework environment variables. Once this is set, you can call all the VS tools without thought. It'd be nice to have that setup before any NAnt script is called so that any engineer can write a build task knowing they'll get the right environment for the job. Let's have a look at how this is done.

Realizing that one would want to include this in all NAnt projects that require a good environment setup, we'll call this file environment.include instead of environment.build since it doesn't really build anything, anyway.

Next, a nifty trick most probably know about, but it's worth mentioning. NAnt has built-in properties that you can edit, like the nant.onsuccess and nant.onfailure properties, which take as a value the name of a target to run in the appropriate case. They're kind of like destructors in that they'll be called before leaving your script no matter what.

What you do with those 'events' is up to you. We simple use them to output some extra information to make logs easier to read:

</target>

</target>

Our environment script needs a bootstrap of sorts. The Visual Studio command prompt is hard coded with the path to its installation directory, but I don't want to hardcode that into the script. Instead, we'll let the user tell us where their VS install dir is with an environment variable and begin following the Visual Studio batch file. That way this script can be moved around without editing:

Here, we tell the user that they must have the VSINSTALLDIR variable set. Instead of using the default set VSxxCOMNTOOLS, we want to be a bit more flexible and not hardcode a particular version, either.

In Atalasoft, we build our binaries for a few different .NET frameworks. It's expected when the environment.include file is included that a particular property, 'Framework.Version,' be set. To ensure that, we'll let the user know with a failure message:

and ensure that the set framework is available to work with:

Notice, we also use another property, 'Atala64Bit' which is also set by the including script. This flag makes building easier later on, since we use that to trigger which build configuration we use.

Finally, we get ready to set some environment variables, as they are in the vcvars batch file:

</setenv>

I ran into a problem with %LIBPATH%, and found a way around the environment script from halting when this fails. Perhaps there's a better way of doing this, but this is good enough for now. We start by getting the current settings from the environment, and saving them into properties:

and then, as the vcvars batch file does, we tack on the settings we've discovered:

</setenv>

(Keep in mind, these are all going to be different if you're running 64-bit; I've chosen not to paste the whole file for brevity).

Finally, as an example of what we do at Atalasoft, we set the Build.Suffix property. This is a convenience thing I use later on in the build script, however, it shows that we have a few different build configurations to maintain depending on the target:

</if>

</if>

That's pretty much it! A vcvarsall.bat file in NAnt language. From here you can imagine a nice clean call to devenv.exe, without the need for discovering the path. Best of all, put this in a central location on your build server, and include it in all your build scripts like this:

and your calls to devenv are simple, like this:

I hope that streamlines some build processes out there. If anyone has suggestions on how I can make this better, please let me know!

AtalaFileRegex: Putting it together

2008-05-09T15:38:00Z

In my recent blog posts Writing NAnt tasks with nested types and Extending NAnt with your own type I went through adding nested types to custom task, and also making your own nested type to show how easy it is to get NAnt to do something it doesn't do already (it does so much, new tasks are rarely needed). In this case, I thought the generic Regex/Replace task in an entire file, or set of files would be nice.

I explained before that I wanted to make our builds parallel while maintaining their consistent version numbers. To do this, I needed to be able to hand out version numbers from CC.NET to my NAnt scripts. In an out-of-order fashion, I showed you the carriage but not the horse, but the point is the same. We'll assume NAnt has a build number coming in, somehow, so here I'll show you how I applied this to our build process.

Before I begin, I want to add to our task one small difference from the code I showed (but it's included in the posted source). I make a simple assumption in our builds that read only files have been gotten from version control and never edited. This is a safe assumption, since the source files should only be changed from the source control version by the build script. The next time the build script runs, we don't want it touching the version number (for continuous builds) because that causes a total rebuild of the project (nightly deletes the folder and starts from scratch). To get around this, I want the version part of the build script (using the AtalaFileRegex task) to only work on read only files (files that were freshly gotten from version control). To accomplish that, I added another boolean attribute to the AtalaFileRegex task called "onReadOnly." It's assumed false (that is, work on all files in the fileset). With this attribute, using the AtalaFileRegex task to apply version information to our assemblies only happens when the file has been updated in source control, or at least once per day in the nightly build. The result is our projects will be considered up-to-date when they should be!

I'm sure everyone is familiar with the AssemblyInfo.[cs,cpp] file. This file contains information used by the compiler to set assembly attributes, like version number, company name, trademark, etc. As part of my job, I cleaned up all of the AssemblyInfo files and separated out the fields that would be updated by the build server. I commented them to indicate to the editing engineers that certain fields will be edited by the build server (and that they don't have to maintain them manually).

So here's my (shaved down) task:

Everybody love regex, so I'll break it down a bit (this one's pretty easy). The regex/replace will find any lines like:

[assembly: AssemblyVersion("1.23.45.67890")]

and replace it with a line like:

[assembly:AssemblyVersionAttribute("9.0.0.45000")]

Which will work for both the semi-colon terminated, and the non-semi-colon terminated lines (.cpp and .cs respectively). I'm sure you could put more checking in there, like ensuring there wasn't comment-type characters preceding the text, but this is a destructive regex; if it's in comments, who cares? It only matters on the one line where the compiler sees it.

Next time, I'll show you how I've used Cruise Control and Team Foundation Server to feed NAnt some interesting information.

Get the source!

Extending NAnt with your own types

2008-05-08T10:51:00Z

Last time I talked about how to nest existing elements in your custom NAnt tasks. Now that you've had a good look at that, and Jacob's custom task, there's only one other thing to cover in this subject: a new type.

Types end up looking a lot like a new task, but throwing this all together should be a lot of fun.

Last time, I explained how I envisioned this working; a neat little task that I would give a FileSet to, and a regular expression plus a replacement string on match. Considering this is a build process, you're going to be manipulating a lot of files with the same regex, that we have covered. But what if I want to apply more than one regex to each file that's processed; I'd like to pass a collection of regex elements to my AtalaFileRegex task along with my FileSet. For that you need the BuildElementCollection decorator on one of your field types, like this:

         /// <summary>
        /// A collection of regex/replace pairs that will be used on the file
        /// </summary>
        [BuildElementCollection("regexs", "regex", Required = true)]
        [StringValidator(AllowEmpty = false)]
        public AtalaRegexCollection Regexs
        {
            get { return _regexs; }
        }

As Jacob explained, we're requiring this field exist in our task, and that it cannot be empty. But what about those first two arguments to BuildElementCollection? Those define the outer element's name, and the inner elements name, respectively. This will make your task have the following form:

</regexs>

</fileset>

</task>

Now, all we're missing is the two types we said we're going to use in the BuildElementCollection.

First, we need our regex type so that we can work with it in our collection. For a type, you decorate the class itself, extend the Element type, and then decorate elements within the class as we've done before:

using System;
using NAnt.Core;
using NAnt.Core.Attributes;

namespace Atalasoft.NAnt.Types
{
    /// <summary>
    /// Represents an regex pattern
    /// </summary>
    [ElementName("regex")]
    [Serializable()]
    public class AtalaRegex : Element
    {

        #region Public Instance Constructors

        /// <summary>
        /// Initializes a new instance of the <see cref="AtalaRegex" />
        /// class.
        /// </summary>
        public AtalaRegex()
        {
        }

        #endregion Public Instance Constructors

I threw in the empty constructor since it's uninteresting.

Cool, now, we said we want the regex type to accept both a matching pattern and a replacement string, so let's get a couple of instance fields going on to hold those:

        #region Private Instance Fields

        private string _value;
        private string _replace;

        #endregion Private Instance Fields

And finally, our getters and setters, properly decorated:

        #region Public Instance Properties

        /// <summary>
        /// The actual regex string
        /// </summary>
        [TaskAttribute("value")]
        [StringValidator(AllowEmpty = false)]
        public string Value
        {
            get { return _value; }
            set { _value = value; }
        }

        /// <summary>
        /// The replacement string on match
        /// </summary>
        [TaskAttribute("replace")]
        [StringValidator(AllowEmpty = true)]
        public string Replace
        {
            get { return _replace; }
            set { _replace = value; }
        }
        #endregion Public Instance Properties
    }
}

Notice on each, we've named the attribute (I called the regex string 'value' because I didn't want an element like <regex regex...). Also note, the replace string can be empty.

That's all there is to making your own, simple type. We'll move on to the more "complicated" part of the task at hand.

Second, realize that we told NAnt it's a collection, so when we make our regexs type it's going to need to extend the CollectionBase abstract class. You need nothing from the NAnt core. Let's set up the class, and the default constructors:

using System;
using System.Collections;

namespace Atalasoft.NAnt.Types
{
    /// <summary>
    /// Contains a strongly typed collection of <see cref="AtalaRegex" /> objects.
    /// </summary>
    [Serializable()]
    public class AtalaRegexCollection : CollectionBase
    {

        #region Public Instance Constructors

        /// <summary>
        /// Initializes a new instance of the <see cref="AtalaRegexCollection"/> class.
        /// </summary>
        public AtalaRegexCollection()
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="AtalaRegexCollection"/> class
        /// with the specified <see cref="AtalaRegexCollection"/> instance.
        /// </summary>
        public AtalaRegexCollection(AtalaRegexCollection value)
        {
            AddRange(value);
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="AtalaRegexCollection"/> class
        /// with the specified array of <see cref="AtalaRegex"/> instances.
        /// </summary>
        public AtalaRegexCollection(AtalaRegex[] value)
        {
            AddRange(value);
        }

        #endregion Public Instance Constructors

From here, there's a lot of things to implement from the CollectionsBase class, and an enumerator. Suffice it to say, you're going to end up filling in a lot of stubs for insert, add, copyto, remove, etc. There's not much to cover here, it's a stong-typed collection.

Next time we'll put these things to work in our task.

Writing NAnt tasks with nested types

2008-05-07T20:04:00Z

If you follow the Atalasoft Blogs by whatever means, and you're reading this, it's quite possible you've also read Jacob's blog post about extending NAnt. If you haven't already, go check it out since he's covered a huge amount of ground related to what I'm going to show in this and subsequent posts. In fact, I'll assume you've either read his post, or have some general idea about extending NAnt.

First, let me give some background on why I did this task.

At Atalasoft, our builds (built by Jacob) can run in parallel, but our .NET 2.0 and 64-bit builds cannot. Before our installer is created we want the .NET 2.0 and 64-bit assemblies to have the same version numbers. In order for that to happen, we introduced a dependency on the .NET 2.0 build by the 64-bit build (the version number is a guaranteed increasing, randomly generated number that, in the case of the 64-bit build, is obtained from the built .NET 2.0 assemblies). For continuous integration, the version numbering is internal, and not as important to our engineering department. With that in mind, it should be easy to break the dependency just for a nightly build, but how? Enter ChangeSetId, LastChangeLabeller, and a special Atalasoft NAnt task called AtalaFileRegex (the "Atala" prefix is somewhat of a running joke internally, but helps future build engineers realize it's not a built-in task for NAnt, yet). The ChangeSetId and LastChangeLabeller topics will be covered in another, upcoming post.

Now, you ask, why the FileRegex task when we're dealing with version numbers? Ah, well, assembly version numbers are gotten a few different ways, and I wanted to alleviate a pain in our current build setup (preceding Jacob) where we use no less than 3 different approaches for updating version numbers for 3 different file types (AssemblyInfo.cs/.cpp, and any .rc files). Enter AtalaFileRegex!

I imagined a task that I could give a FileSet to, a regular expression to match, and a string to replace that match with. It's a sharp (no pun intended) tool that you can easily break things with if you're not careful, but incredibly powerful if you are careful!

So we set out to make our task (this is where you read Jacob's blog). And in our shiny new task, we'd like to have a FileSet. Easy, peasy! In your public instance properties, make a field that is the NAnt.Core.Types.FileSet, and add some getter/setter code, like the following:

        /// <summary>
        /// All the matching files in this set will be updated according
        /// to the given regex/replace
        /// </summary>
        [BuildElement("fileset")]
        [StringValidator(AllowEmpty = false)]
        public FileSet RegexFileSet
        {
            get { return _fileSet; }
            set { _fileSet = value; }
        }

Notice I use the BuildElement() decorator here, instead of the already familiar TaskAttribute() decorator. This means that it's a nested element. The way you'd write your NAnt task with this is

</taskname>

Sweet! That was easy...

Next time I'll write a walk through on how to make your own type, and add that to your new task.

Update on another post

2008-05-07T11:43:00Z

You may have seen my post in the 31 Apps in 31 Days blog. If you haven't already seen the 31 Apps in 31 Days blog, definitely check it out! There's already a week's worth of applications you can make using the Atalasoft family of tools in one way or another.

I got some requests to expand the functionality of my autorotater application, namely that it have some additional rotational functionality. Sometimes you'll stumble on those pictures that were taken with an orientation 90 degrees to the right or left, or even upside-down. Some viewers will read this orientation information and rotate the picture for you while viewing, but at the same time they don't update the orientation flag of the image. As a result the image may continue to be rotated by other viewers, or in this case, it'll be rotated by the AtalaAutoRotater.

To fix this, I've added two new buttons to the autorotater. This functionality was pretty easy to add with these few steps (technical stuff here, skip to MultiSelect if you don't like techie stuff).

Modify the selectedIndexChanged event handler to enable/disable the rotation buttons. This is just an added visual to let users know they must select one or more images before rotating
Add two buttons, and give them appropriate names btnRotLeft and btnRotRight.
Add a wrapper function rotate_() that will gather a collection of thumbnails into a new ICollection<Thumbnail>() before calling rotate()
Add a new enum that contains Left, Right, All
Have the btnRotLeft and btnRotRight (modify btnRotate) call the wrapper with its direction
Modify rotate() so that it accepts an ICollection<Thumbnail> and a Rotation enum. Check, before the switch, whether it's rotating all images, or a selection and update the orientation int if it's a selection Left/Right.
Let the rest of the code do the work!

MultiSelect

But wait! You don't want your users having to select individual files and hit rotate buttons each time. So we'll make it possible to select a whole set of images to rotate. To select multiple images, users can either hold down the Ctrl key while selecting individual pictures, or the Shift key to select two endpoints of a range of photos.

For this functionality, the FolderThumbnailViewer needs to be updated as well. This might be a bit confusing, because there are two spots where you need to change options to get multiselect working, both in the FolderThumbnailView properties matrix. First, you have to enable MultiSelect, then you have the change the SelectionMode to MultiSelect from Single. Once that's done, the app is ready to go!

To make this a bit easier, I've included the source in the installer this time so there's only one file to download.

Get the installer

CI Hurts!

2008-04-03T18:19:00Z

I've only recently begun working as the Build Czar here at Atalasoft, and immediately there are pains. The builds (from what I see) are really complicated. True: building a very complicated product is going to be complicated; however, it's thought that I can work myself out of this job, making it everyone's responsibility not just one person. How can I do this?

Currently, Atalasoft employs Cruise Control.net, CCTray, NUnit and NAnt to make our CI ship go. Each piece of this puzzle is extraordinarily powerful, the problem is getting them to shine in the way they shine on their own. If I put the pieces in the wrong places, I'm going to overextend one of these applications, and end up paying the price for sure.

Welcome to my blog! For a while I'll be concentrating on overhauling an enterprise build process, the trials, tribulations, pains and glories therein. At then end of this process I can only hope the title and content of my blog will change, as that signals one of two things:

I accomplished what I set out to do, and the builds are no longer owned by me. My Build Czar title will be retired, and the entire engineering team will work together to ensure the CI process continues.
The builds break me.

Let's hope it's the former.

Dave Terrell

Build Engineer