Atalasoft Community

Why It Helps to Understand Pixel Formats

Steve Hawley — Tue, 16 Mar 2010 15:53:27 GMT

A pixel format in dotImage is a description of the depth, color space and layout of the memory used to represent a pixel. Mathematically, there are an infinite number of possible pixel formats as color space is as much a space as Cartesian space. You can create any number of basis sets that map onto any particular color space. However, only a small subset are actually useful. In dotImage we represent 12 formats, 3 paletted, 3 gray, and the rest color.

It helps to understand the meaning of pixel formats in terms of quantity of information rather than quality of information. Having this understanding will help you in avoiding possible problems when processing images or converting from one format to another.

All ImageCommand classes in dotImage advertise a list of all PixelFormats that they operate in. The property SupportedPixelFormats gives you this list. You can also call IsPixelFormatSupported() to ask if a specific format is supported.

I would like to say that IsPixelFormatSupported returns true if and only if an ImageCommand natively supports a given PixelFormat. In most cases this is true, but some older commands (before I got my hands on them) were written to operate on some formats as a convenience, but without letting the caller know. They still behave exactly as before for compatibility. ScribbleCommand is an example of this – it will strip out the alpha channel into a temporary image, before performing the operation.

What is true, is that every implementation of SupportedPixelFormats will return the “best” pixel format for the command as the first entry in the list. This is by convention, but we enforce it strictly. There’s a reason for this: all ImageCommands have the ability to operate on any pixel format. This is done by setting the property ApplyToAnyPixelFormat to true. Then, when the command is executed, if the source image is not in SupportedPixelFormats, the command will create a new version of the image in a better PixelFormat for the command. This is done internally by the command by calling SelectBestAlternatePixelFormat. The default implementation is to return the 0th entry in SupportedPixelFormats.

In short, if ApplyToAnyPixelFormat is true, the command will create a new image in the pixel format SupportedPixelFormats[0]. The default value for ApplyToAnyPixelFormat is false.

Wait a minute – this sounds like a tremendously useful feature! Why would we have it off by default?

There are two reasons: (1) backwards compatibility – where possible we try to assure that all commands in new versions of dotImage operate as they did in the past. (2) “Do No Harm” – changing the PixelFormat of an image may be a lossy operation. If the amount of information in the source image’s pixel format is greater than the amount of information in the target pixel format, then information has to be discarded in the translation. How that information is discarded makes all the difference in the world, and the choice that is made should be dependent on the source image domain. That is, if the image is a photograph, you should use one technique (dithering or another type of halftoning) and if it is a document, you should use another (dynamic or adaptive threshold).

When ApplyToAnyPixelFormat is turned on, ImageCommands will use AtalaImage.GetChangedPixelFormat() to create the new image. By default (and again, to match legacy behavior), when dotImage reduces a image to 1-bit per pixel, it uses either dithering or color quantization to generate the target image. This is fine for photographic images, but lousy for documents. Fortunately, you can change how this change will happen.

How can I know if changing a pixel format would be lossy?

You can do that by comparing the number of bits per pixel:

public bool IsPixelFormatChangeLossy(PixelFormat source, PixelFormat target)
{
    return PixelFormatUtilities.BitsPerPixel(source) >
        PixelFormatUtilities.BitsPerPixel(target);
}

and while the accuracy of this predicate is arguable, it will be good enough for most cases. For the nit pickers, conversion from color to 8 or 16 bpp gray, while lossy is well-defined enough to let it go – although you should be aware that the color conversion is done through a perceptual model, rather than a linear model. Also, conversions of 24 bit color to 8 bit paletted may be lossless in some cases, but that is more computationally and memory intensive than I’d like to put in such a simple predicate.

As a side node, PixelFormatUtilities is my favorite grab bag of useful functions and predicates based around PixelFormats. They are akin to the functions in the C routines defined in ctype.h in that they are almost always table driven and operate in constant time.

When Optimization Pays Off

Steve Hawley — Fri, 12 Mar 2010 21:38:48 GMT

“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil” – Don Knuth

If you are doing image processing, you are firmly in that remaining 3%. I had some code that I had factored into two major cases: 1 bit per pixel and n bits per pixel (where n is a multiple of 8). Each chunk of code was more or less parallel in that it did something along the lines of this:

bool PredicateInRow(BYTE *imagePtr, int x, int bytesPerPixel, fPredicate predicate, void *predicateData)
{
    BYTE *pixelPtr = imagePtr + x * bytesPerPixel;
    return predicate(pixelPtr, bytesPerPixel, predicateData);
}
void ProcessRange(BYTE *imagePtr, int height, int width, int bytesPerPixel, int bytesPerRow, fPredicate predicate, void *predicateData)
{
    for (int y=0; y < height; y++) {
        for (int x=0; x < width; x++) {
            if (PredicateInRow(imagePtr, x, bytesPerPixel, predicate, predicateData)) {
                // perform processing
            }
        }
        imagePtr += bytesPerRow;
    }
}

This does some image processing across the whole image based on a predicate using the current pixel’s data. The predicate has to be vectored out to other code instead of being done inline (technically, there are other ways I can do this, but way of doing it avoids a lot of duplication of effort). I liked the factoring in that it made the code fairly easy to read, but I knew when I wrote it that I would regret this decision.

After noting that client code of this routine was slower than I preferred it to be, I fired up a profiler and found that PredicateInRow was chewing up a fair amount of the total time and it was being called 50 million times over the comparatively short life of my test application.

I refactored it to this instead:

void ProcessRange(BYTE *imagePtr, int height, int width, int bytesPerPixel, int bytesPerRow, fPredicate predicate, void *predicateData)
{
    for (int y=0; y < height; y++) {
        BYTE *pixelPtr = imagePtr;
        for (int x=0; x < width; x++) {
            if (predicate(pixelPtr, bytesPerPixel, predicateData)) {
                // perform processing
            }
            pixelPtr += bytesPerPixel;
        }
        imagePtr += bytesPerRow;
    }
}

I’ve now lost the PredicateInRow routine, took a hit in readability but the code runs twice as fast now that I eliminated 50 million function calls and 50 million multiplies.

I should note that I tried inlining PredicateInRow which resulted in a negligible increase in performance. In my actual code, the structure of ProcessRange is much more complex – there are three small state machines that run inside of the y-based for loop and each state machine ended up looking at rows above and below in addition to the current row. The refactoring was more complicated than the example above, which resulted in two bugs, both of which I found by reading the code closely.

The code of a multiply is tiny, but when you multiply it by 50 million, it becomes significant. When I wrote this, I knew that the per-pixel accessing was going to slow it down. Nonetheless, I took the high road of “make it work, then make it fast(er)”.

If need be, I will change the code the work with specific byte per pixel values, removing the predicate call entirely, but that will take a 2x duplication of code (1-bit and n-bit) and turn it into a pentiplication (1 bit, 8 bit, 16 bit, 24 bit, 32 bit)

F# Discoveries This Week 03/12/2010

RickM — Fri, 12 Mar 2010 21:03:07 GMT

Tons this week. Vladimir Matveev’s is my favorite new F# blogger with very well written data structure posts, Ashley Feniello continues his fantastic FScheme series, and Jomo Fisher posts some great Freebase and DGML examples. That’s just the tip of the F# iceberg, do come inside.

Ashley Feniello’s FScheme Parts Twelve, Thirteen and Fourteen

The basic idea is to run a simulation by iterating a pure function from world state to world state. We’ll add a new ‘run’ primitive which will expect several user-defined functions to have been set up. The world state is initially produced by an ‘init’ function. Then every 30th of a second a ‘tick’ function is called to produce a new world state from the current state. Finally a ‘draw’ function will be called to render the world.

Luca Bolognese’s Updated Stock Prices, Divs and Splits Example

I’m working on a program to keep track of paired trades with trailing stops. I need to download stock prices, so I thought I might reuse some old code of mine. Here is the updated framework.

David Carlisle’s NAG F# Examples

NAG (Numerical Algorithms Group) is currently running a beta test of a NAG Library for .NET. One noticeable feature of the comments received so far is the relatively large number of users interfacing to the library from F# rather than C# or VB.NET.

Phillip Trelford’s The Associative Model of Data

But what if you wanted to extend the web store to have features like the online retailer Amazon, e.g. multiple sellers, recommendations, etc.? Answer: serious table and relationship proliferation. Enter an alternative model: the Associative model of data, a dynamic model where data is defined simply as items and links.

Jon Harrop’s F# vs Unmanaged C++ for Parallel Numerics

We obtained a surprising performance result when comparing optimized parallel ray tracers written in F# and C++ recently. The following two programs render the same highly complex scenes containing over a million objects. Surprisingly, the 136-line managed F# program runs slightly faster at 17s than the 168-line unmanaged C++ which takes 18s.

Vladimir Matveev’s F# and Iron Pyton

Today’s post will be devoted to various ways of integration between Iron Python and F#. I’ll try to skip the details of DLR configuration, because this is vast topic that worth separate post (maybe even a few posts). Instead I’ll focus on questions of integration.

Vladimir Matveev’s Data Structures: Finger Tree (Part 1)

What we’ll try to do in this post is to create the structure (based on 2-3 trees) with following characteristics. Immutable (modification returns new instance of structure with changes applied), Enqueue/Dequeue both in start and end in amortized constant time, and Concatenation support.

Vladimir Matveev’s Data Structures: 2-3 Tree

There are many special types of trees that perform insert/remove operation in intelligent way ensuring that result tree is small but branchy :). This trees are called self-balanced, most well-known of them are AVL trees, Red-black trees, 2-3 trees. This post is dedicated to the latter ones.

Vladimir Matveev’s Overview of F# Async Module and Event-based Async Pattern in F#.

This post I’d like to dedicate to reviewing functionality of Async module – creating and manipulating async computations.

Luis Diego Fallas’s Basic Image Processing Operations with F#

The previous post presented a way to access the image data from the Webcam using DirectShow.Net and F#. We can manipulate this data to do some basic image processing operations with it.

Jomo Fisher’s Extend your F# program with MEF and MEF in F# Scripts

The Managed Extensibility Framework is an interesting new technology in .NET 4.0. This is a simple example in F#. This code sets up MEF hosting and asks for all extensions in the c:\extensions folder.

Jomo Fisher’s Neat Samples: F#, Freebase, DGML

I recently posted about the freebase web service here. This sample reads biological classifications and renders them in DGML. The result is a huge graph, here’s a little piece of it…

Jomo Fisher’s Neat Samples: F# and Bing API

Here’s another F# web service sample. This one uses the Bing Phone API to do a query. This time the code uses Xml instead of JSON and XmlDocument instead of a DataContract deserializer. This is pretty much a straight transliteration of one of the Bing SDK samples.

Jomo Fisher’s F# and Freebase

The web service at Freebase.com lets you access all sorts of structured data from a web service. Here’s a sample that shows you how to access this data from F#. It uses DataContract and the JSON serializer. The code below reads and prints the elements of the periodic table.

Julien Ortin’s BitTorrent in F# series: I/O Operations and Bitfield

One important thing is that a BitTorrent transfer is considered as a stream of pieces. So, if you have a 100-byte file, and a 400-byte one, and if the piece size is 200-byte long, the data from the piece need to be appropriately split (both when reading and when writing). In this library, we use a reference to the AsyncWorker described on Don Syme’s blog.

Matt Moloney’s Drag and Drop using Rx and WPF in F#

I have recently been experimenting with combining Reactive X, WPF, and F# and have found the combination to be very palatable. I chose drag and drop as the test case because it is both non trivial and generally deeply stateful. The resulting Rx turns out to be one fifth the code of my original C#, much easier to read and has fewer errors.

Kean Walmsley’s Using jig from F# to create Spirograph patterns in AutoCAD

After my initial fooling around with turning AutoCAD into a Spirograph using F#, I decided to come back to this and bolt a jig on the front to make the act of making these objects more visual and discoverable.

Joh’s Thoughts about F# and Xbox games

I have been working for quite some time now on Asteroid Hunter. This has not left me much time for exploration with F#, but there is quite a bit a learned during the process anyway.

Richard Minerich’s Abstract Thoughts about F# Abstractions

My recent work on Professional F# 2.0 has left me thinking a lot about the nature of abstractions.

Oliver Strum’s Creating a Lazy Sequence of Directory Descendants

I thought these code examples all look rather verbose – in the case of Clojure because in that way rather typical for Java, the APIs are pretty verbose to use, and in the case of C# because of all the syntactic, well, ahem, necessities, as well as the fact that there’s no language feature for integrating nested sequences seamlessly. Keeping it nice and simple, in F# that example can look like this…

Tormod Fjeldskar’s Tail Recursion in C# and F#

Tail recursion is essential in functional languages like F#, where iterative solutions are often implemented using recursion.

I’m not only the Hair club president, i’m also a client.

christina — Thu, 11 Mar 2010 18:47:17 GMT

(Remember that commercial? It always makes me laugh. It’s related. I promise.)

Lately I’ve been having some computer drama. When I say “drama” I mean “expenses,” and when I say “expenses” I mean $1,400 poorer.

You know what a new computer means—the transfer of your ENTIRE LIFE to another machine. It’s scary and painful, and I don’t know why anyone buys a new computer ever.

Because this isn’t my personal blog, I won’t go into the reasons why I had to do that twice. Twice! It’s like 2 root canals in 3 weeks. Ugh.

Part of that includes updating wireless logins and other various important things.

Now—my memory for passwords is amazing. I am a whiz with 6-7 character phrases. For whatever reason, I can never remember the wireless password at work.

Here at Atalasoft, we use a product called SharePoint. Heard of it? (Ha.) Stored in a document in SharePoint is our wireless internet password. I remember the first time I found the document—I had to 1. Find it through various folders and sub-folders, 2. Click to open it, and 3. Launch MS Word to view it. It was annoying and I groaned at the thought of going through that again.

BUT. BUT! We have a little something called Vizit SP. So, I 1. Searched for the document using SharePoint’s search feature and 2. Clicked a little “Vizit Preview” button next to the document. How about this? A PREVIEW OF THE DOCUMENT APPEARED. I did not have to launch Word. I did not have to remember the hierarchy of folders. The password was in front of me.

It was one of those moments where I was so tickled that the company I work for makes COOL STUFF THAT WORKS.

To sum this up: I’m not only the Vizit SP Marketing Manager, I’m also a User.

Solving Problems, aka, Nobody Codes Like This Anymore

Steve Hawley — Wed, 10 Mar 2010 22:04:24 GMT

When I was in college, I had a class in machine architecture/assembly language. At the time that I took it, I knew 6502 already and had published two video games for the Apple II written entirely in assembly. I love the drunken power of having access to the entire machine and making it do things it wasn’t precisely designed for – so the class was easy to me. We started off with a fake Von Neumann architecture machine called MANIAC. MANIAC was a basic accumulator-model machine – not too different from the 6502, so coding for it was familiar ground.

We then transitioned to VAX assembly and were writing code to run directly on the machine. Compared to the barren wasteland of 6502, VAX was like stepping into Paris. 16 registers? What the hell am I going to do with them all? Multiply? Divide? Floating point? SOBGTR (subtract one and branch on greater)? There was even an instruction to insert into a queue. Yikes.

In class, the final assignment was to write a program to maintain a family tree. The program needed to read a line and parse out a set of commands:

CREATE name – create a new person named name
MARRY name1 name2 – mark persons named name1 and name2 as married
CHILD name1 name2 – mark person named name1 as a child of name2
GRANDCHILDREN name – prints a list of name’s grandchildren
COUSIN name1 name2 – prints TRUE if person named name1 is a cousin of name2

I looked at this task and thought, family tree? Really? In assembly? No. Wrong approach.

So I first wrote it in Pascal and got it working. Then I hand compiled each procedure (I didn’t know how compilers worked yet, but I had a pretty good idea). I noted that many of my peers were struggling with the argument passing in VAX, so I avoided the whole problem and adopted register passing as the protocol where the caller, not the callee was responsible for saving registers. Every subroutine was clear what it used as arguments and I set aside some registers that were free to be destroyed.

Once I had my “architecture” in place, it was a matter of writing little subroutines that matched the Pascal. I even included the Pascal in the source as documentation. The only documentation. Why would you need more?

It wasn’t without bugs, but it eventually worked just fine and I was done well before most of my peers, even though I technically wrote the program twice.

So, nobody really writes code like this anymore. Compilers are good enough that unless you doing something like image processing, you don’t really need to drop to the machine level. That doesn’t mean that we can’t learn something from the process.

This particular problem lent itself to being solved in a particular language domain – Pascal (or your favorite procedural or OOP language). So much so that writing it in another language was trivial. It became divide and conquer to factor the tasks out to reasonable pieces: a simple parser, a REPL, a structure definition, and five procedures. Then I picked a mapping strategy to go from my problem language domain into my solution language domain and followed through on it. This is still a valid way to solve problems, even though this particular solution language domain is obsolete. This particular approach is how we end up with domain specific languages and small virtual machines to run them.

Writing a Composite ImageCommand – Gamma Corrected Resampling

Steve Hawley — Wed, 03 Mar 2010 16:05:52 GMT

We have a customer who expressed some concern over how our resample command works (and how many similar commands work, in general). He pointed us to an article here which describes the problem. The short of it is that the energy in a pixel is an expression of power (Watts) over the area of the pixel and although the scale of the pixel component intensities is represented as linear, it is really exponential. Averaging a set of power measurements linearly produces results which don’t effectively represent the average power. The trick is compensate for this is to make the power scale linear by applying an inverse gamma function to the image, then resampling it and finally applying the original gamma function to restore it.

The gamma function looks like this:

pv’ = pv^γ

where pv is the normalized value for the channel of a pixel – this means it is a value between 0 and 1 inclusive. γ is a value > 0, typically 2.2 for most monitors. So the typical process of converting an 8-bit per channel pixel is something like this:

foreach pixel
   foreach channel
        double normalizedChannel = channel / 255.0;
        double channelPrime = Math.Exp(normalizedChannel, gamma);
        channel = Math.Floor(channelPrime * 255);

DotImage has an ImageCommand for performing this transformation efficiently. We can do the transformation I’ve described as a sequence of steps, but I thought it would be nice to combine everything into one command.

The first thing to do is to make a subclass of ImageCommand. ImageCommand is an abstract class which is, more or less the a collection of GoF template methods. This is to make it easy to implement ImageCommands from scratch. We’re not doing that – we’re making a composite command. We’ll implement the abstract methods, but they won’t be needed. So let’s start off with an outline of what we want:

public class GammaResampleCommand : ImageCommand
{
    GammaCommand _gammaCommand = new GammaCommand();
    ResampleCommand _resampleCommand = new ResampleCommand();
    public GammaResampleCommand()
    {
    }
    public GammaResampleCommand(double gammaLevel, Size destSize, ResampleMethod method)
    {
        GammaLevel = gammaLevel;
       DestSize = destSize;
        Method = method;
    }
    protected override AtalaImage PerformActualCommand(AtalaImage source, AtalaImage dest, System.Drawing.Rectangle imageArea, ref ImageResults results)
    {
        throw new NotImplementedException();
    }
    public override PixelFormat[] SupportedPixelFormats
    {
        get { return _gammaCommand.SupportedPixelFormats; }
    }
    protected override void VerifyProperties(AtalaImage image)
    {
       ImageCommand.ProxyVerifyProperties(_gammaCommand, image);
        ImageCommand.ProxyVerifyProperties(_resampleCommand, image);
    }
    public double GammaLevel
    {
        get { return _gammaCommand.GammaLevel; }
        set { _gammaCommand.GammaLevel = value; }
    }
    public Size DestSize
    {
        get { return _resampleCommand.DestSize; }
        set { _resampleCommand.DestSize = value; }
    }
    public ResampleMethod Method
    {
        get { return _resampleCommand.Method; }
        set { _resampleCommand.Method = value; }
    }
}

You’ll note that rather than implement the properties or VerifyProperties, we’re just acting as a proxy to existing instances of GammaCommand and ResampleCommand. This means we don’t need to implement error checking on the properties – that will happen for free. In fact, to implement VerifyProperties, we’re taking advantage of a feature of ImageCommand – ImageCommand has a set of methods named ProxyXXXX, which will call the method XXXX for us, even though it is protected. This breaks encapsulation, but it is there just for this purpose – creating composite commands. Finally, you’ll note that PerformActualCommand throws an exception. That’s OK – we’re going to ensure that it is never called.

What remains is to do the actual work, and for that we’ll override the method Apply().

    public override ImageResults Apply(AtalaImage image)
    {
        using (AtalaImage gammaCorrected = image.Clone() as AtalaImage)
        {
            double gamma = _gammaCommand.GammaLevel;
            double inverseGamma = 1 / gamma;
            GammaLevel = inverseGamma;
            _gammaCommand.Apply(gammaCorrected);
            AtalaImage finalImage = _resampleCommand.Apply(gammaCorrected).Image;
            GammaLevel = gamma;
            _gammaCommand.Apply(finalImage);
            return new ImageResults(finalImage, false);
        }
    }

GammaCommand is InPlaceProcessing = true, which means that it side-effects the source image. I don’t want this, so I’ll make a copy of the source image using Clone, then apply the inverse gamma to the source, resample it, then apply the normal gamma.

Here is some quick test code for this:

using (AtalaImage image = new AtalaImage(@"..\..\gamma_dalai_lama_gray.jpg"))
{
     GammaResampleCommand resampler = new GammaResampleCommand(2.2,
           new Size(image.Width / 2, image.Height / 2),
           ResampleMethod.LanczosFilter);
     using (AtalaImage smaller = resampler.Apply(image).Image)
     {
         smaller.Save("dalailama.jpg", new JpegEncoder(), null);
     }
}

The input file is this:

and the subsequent output is here:

I’m looking for the Lord high Executioner. Is he/she available?

christina — Tue, 02 Mar 2010 21:52:32 GMT

I recently put out the call that we’d be ordering new business cards, and that interested people should forward me their preferred name/email/title.

Here are some of the specific title requests received:

THE MAN

Sales God

The Sellinator

The Dude

Steve noted that Gilbert & Sullivan had a knack for coming up with titles, such as The Lord High Executioner, Wicked Baronet, etc.

Coming up with titles is always tricky, especially when your employees are quite versatile. What especially awesome titles have you held in the past?

F# Discoveries This Week 02/25/2010

RickM — Thu, 25 Feb 2010 21:31:07 GMT

Between Atalasoft, Professional F# 2.0 and the MVP Summit I’ve been completely swamped and ended up with quite a backlog of posts. Between the Brian-Chris F# Code Battle, Luca’s LChart, Ashley’s continuing FScheme series and Matthew’s MongoDB stuff I’m not sure I can pick a favorite. That’s not true, I’d go right for the DAWG-fight series. The arcane secrets of F# optimization await inside.

C9 Lectures: Dr. Don Syme - Introduction to F#, 3 of 3

In Part 3 of this 3-part lecture series, Dr. Don Syme elaborates further on: Patterns, Object Basics, [and] Imperative Programming.

Dr. Don Syme’s Async and Parallel Design Patterns in F#: Agents

In part 3 of this series, we explore lightweight, reactive agents in F# and look at some typical design patterns associated with these agents, including isolated internal state.

Chris Smith’s DAWG-fight, Optimizing text search in F# II and Dr. Brian McNamara’s DAWG-Gone (F# Language Team Code Battle Summary)

So today I’ll walk you through what Chris did right (shortest… subsection… ever!), as well as what he did wrong (get comfy in your chair), and then show my own implementation.

Luca Bolognese’s LChart: Displaying Charts in F# Parts One, Two and Three

I want to use F# as a exploratory data analysis language (like R). But I don’t know how to get the same nice graphic capabilities. So I decided to create them. Here is a library to draw charts in F#.

Dr. Brian McNamara’s An RSS Dashboard in F# Parts Four, Five and Six.

You can see from the colors in the screenshot from part one what I desired, but this requires knowing which links are ‘visited’ and which are not, so I can color each link appropriately. It turns out, this information can be had via unusual means…

Ashley Feniello’s FScheme Parts Ten and Eleven

We’re now taking the first small step into the world of nondeterministic logic programming (chapter 16 of Bill Hails’ book). Hopefully you enjoyed the last post about continuation passing and found the idea to be ripe with potential power; indeed so powerful that we’re going to use it now to bifurcate the universe! No really, we are…

Luca Bolognese’s A simpler F# MailboxProcessor

I always forget the pattern to use to create a functioning MailboxProcessor in F#. I mean, which piece has to be async and how to structure the recursive loop. When I find myself in that kind of a situation situation, my instincts scream at me: “Wrap it and make it work how your mind expects it to work”. So here is a simplification of the paradigm.

Matthew Podwysocki’s The F# PowerPack Released on CodePlex

As announced yesterday, the new February 2010 release of F# is out. This release is much more of a stabilization release instead of adding a lot of features including improvements in tooling, the project system and so on.

Matthew Podwysocki’s Exploring MongoDB with F#

I just want a quick answer with the data I have. There was one issue of course that nagged me which was the ubiquitous use of strings for everything from databases, collections, and keys. With a language such as F#, could we do any better than this approach?

Matthew Podwysocki’s F# and the Dynamic Lookup Operator ala C#

In the F# language, we have the ability to define two “dynamic” operators, a get member operator denoted by the ( ? ), and the set member operator denoted by the ( ?<- ). The F# language and its associated libraries do not have an actual implementation of these operators, but instead allow you to implement them as you see fit.

Steve Gilham’s Updating F# posts to 1.9.9.9 Parts One and Two

Possibly a few more issues will come out as I reconstitute my analysis tool, which is currently undergoing a retooling of its build system to automate some of the manual checks for whether the operational tests succeeded or not.

Steffen Forkmann’s FAKE – F# Make 0.29 Released

Last week I released version 0.29 of my build automation tool “FAKE – F# Make”. The new version comes along with a couple of changes which I will now describe.

Luis Diego Fallas’s Using a Webcam with DirectShowNET and F#

In this post I'm going to show a small F# example of using DirectShowNET to access a webcam and manipulate the image data.

Mark Needham’s F#: Passing an argument to a member constraint

I've written previously about function overloading in F# and my struggles working out how to do it and last week I came across the concept of inline functions and statically resolved parameters as a potential way to solve that problem.

Julien Ortin’s BitTorrent Bencoded values and BitTorrent Metainfo file handling in F#

Byte strings can be strings represented as byte strings (such as the description of the exchanged data), as well as an array of bytes (such as a hash). Representing them with the .Net strings can thus lead to errors. Hence we shall keep the byte array representation.

Julien Ortin continues his series on Purely Functional Data Structures with a Skew binomal heap, a Banker’s double-ended queue, a Alternative binary random access list, a Bootstrapped queue, and a Implicit queue.

This post describes the F# implementation of the <insert data structure here> from Chris Okasaki’s “Purely functional data structures”.

Scott Seely’s F# is Changing My Style

The types I’m interested are concrete (no abstract methods) and have a zero-argument constructor. A few years ago, I would have done a bunch of looping constructs to look at each element. However, I’ve been doing a lot more work with F#. While doing this experiment in C# for a project, I wound up writing the following instead:

Steve Gilham’s It almost feels like cheating…

Having renamed all the .c files as .cpp, getting a first building assembly was simple, and then making it refer to an initially empty F# library, no problem; and then setting up an (again, initially empty) unit test assembly depending on both (where vice-tests to match up new and old implementations, and their eventual unit test replacements can accumulate), just as normal.

Brad Clow’s F# Examples talk at BFG

Earlier this week I gave a short talk at the Brisbane Functional Group on F#. The idea was to give an introductory feel for the language by way of some simple examples.

Phillip Trelford’s F# Talk at Edge UG: Slides and Demos

I presented a 1 hour talk introducing F# with 4 demos (attached to this post): Full-screen WPF POS Checkout sample with Barcode scanner integration (in 100 lines). Twitter WPF client script (200 lines). Matermind board game in WPF (300 lines). Lunar Lander XNA game (>400 lines).

Matt Jackson’s AssertWasCalled in F# (or how I learned to stop worrying and hate extension methods)

Unfortunately, this upcasting led me on a short goose chase to find out which interface in the Rhino Mocks codebase was responsible for AssertWasCalled only to realise that it was an extension method sitting hiding away somewhere else.

Libpng – You’re Doing It Wrong

Steve Hawley — Tue, 23 Feb 2010 17:24:28 GMT

I took a three day hiatus from developing DotImage 9.0 to work on an experiment: how would a 100% managed PNG decoder perform, compared to libpng?

Using only C#, I knocked together a very basic chunk of code that has a very strong .NET feel/smell to it to decode PNG images. Internally, I build a map of the available chunks in the file and then I ask for them by type. This is different from most PNG consuming code – the spec encourages you to read chunks in possible expected order, which requires a very state machiney layout. State machines aren’t bad, per se, but they do lead to code that is not always easy to read or maintain. So instead of waiting for a chunk of the right type to arrive, I just ask for it by name.

For the required zlib compression, I used DotNetZip, in particular the Zlib assembly. The main reason for this choice was that Ionics implementation is a filter stream. I give their stream another stream to work with and it will decompress it while I read from it. This is awesome, because I don’t need to worry about the underlying implementation.

Well, that’s actually a lie. I do have to worry about it. PNG allows compressed image data to be spread across multiple PNG chunks in the file. So I created a ChunkStream that give a stream a List<ChunkDescriptors> that describe where image data chunks reside, it will read/seek through the chunks as needed.

In sum, my reading model for image data is this Stream->ChunkStream->ZLibStream.

So here is my code for decoding a non-interlaced PNG image:

List<ChunkDescriptor> chunks = _chunkMap[ChunkType.IDAT];
ChunkStream chunkstm = new ChunkStream(_stm, chunks);
ZlibStream stm = new ZlibStream(chunkstm, CompressionMode.Decompress);
for (int y = 0; y < Header.Height; y++)
{
    int filter = stm.ReadByte();
    if (filter < 0)
        throw new Exception("unexpected EOF");
    if (!FilterIsValid(filter))
        throw new Exception("invalid filter");
    int count = stm.Read(currLine, 0, bytesPerLine);
    Unfilter(currLine, prevLine, (PngAdaptiveFilter)filter, bytesPerLine);
    OnScanlineRead(new ScanlineEventArgs(y, currLine));
    byte[] temp = prevLine;
    prevLine = currLine;
    currLine = temp;
}

That’s all that it takes – this code is easy to read, easy to maintain, and nicely abstracted. Do you see the total lack of special case handling? All that is wrapped up in the stream model and instead, I can concentrate on the decoding.

I did some testing and found that compared with a libpng, this code takes a little less than twice as long. With some tweaking, I can make it better. In my case, the bottleneck is not where I expected. It’s not in I/O. It’s not in reordering color channels. It’s not in using an event driven model for reporting scanlines read. It’s not in mapping out the chunks at the start. In my particular test case, the bottleneck is the Paeth predictor that was used on the test file to improve compression.

Here is the Paeth predictor as I originally implemented it:

private static int PaethPredictor(int a, int b, int c)
{
    int p = a + b - c;
    int pa = Math.Abs(p - a);
    int pb = Math.Abs(p - b);
    int pc = Math.Abs(p - c);
    if (pa <= pb && pa <= pc)
        return a;
    else if (pb <= pc)
        return b;
    return c;
}

This is very simple code, so it’s hard to imagine how you might make it better. I took this approach:

private static int PaethPredictor(int a, int b, int c)
{
    int bmc = b - c;
    int pa = bmc < 0 ? -bmc : bmc;
    int amc = a - c;
    int pb = amc < 0 ? -amc : amc;
    int apbmcmc = a + b - c - c;
    int pc = apbmcmc < 0 ? -apbmcmc : apbmcmc;
    if (pa <= Math.Min(pa, pc))
        return a;
    else if (pb <= pc)
        return b;
    return c;
}

I eliminated the calls to Math.Abs, which helps a little. I also broke apart the expression p which also helped a little bit. When I say “a little”, I mean that it shaves off maybe a few hundredths of a second. To read my source image 100 times took 1500ms and after these basic rearrangements, I got it down to 1100ms. These tiny changes help because the Paeth Predictor gets called 7.5 million times in my test code. If I inline it (seriously hurting readability), the result is slightly faster. At this point the pure managed code runs at 1.5x libpng, which is way better than I expected.

I looked at the image decoding in libpng and while the authors clearly went to a great deal of effort to handle errors and special cases and to make the code as readable as possible, ultimately I think they’re doing it wrong. My code is memory efficient, easy to read, easy to maintain, and will never leak memory

A personal note.

Elaine — Mon, 22 Feb 2010 16:35:47 GMT

Page 1, paragraph 3, sentence 1 of Seth Godin’s Tribes reads “A tribe is a group of people connected to one another, connected to a leader, and connected to an idea.”

I could not get this out of my head Saturday.

I play ice hockey, and at 3:10 PM this past Saturday, I found myself playing goalie for a team that was not my own, under the worst set of circumstances.

“How did I get here?” was the second thought going through my mind. I was on the ice because of the multitude of tribes to which I belong.

First, Atalasoft. Our small tribe all working together to be leaders in our industry. Part of that tribe includes Steve, our chief architect. Because of Steve, I’m involved with another tribe, the Massachusetts Down Syndrome Congress. I play hockey for them in a charity game to raise money for awareness.

Because we play against former NHL players, I joined a women’s team out of Boston, the Raging Storm, to get in playing shape. So there are more tribes – athletes, women athletes, hockey players, goalies – to name a few. Through this tribe, someone passed my name to another team who was looking for a goalie to play for them this past weekend in the Fitchburg Have A Heart tournament. I joined another tribe, the Panthers.

While dressing for our first game, we heard devastating news. The goalie for the tournament host team had chest pains in the second period of her game. She refused to leave the ice, not recognizing the seriousness of the situation. She finished the game, returned to the locker room, and had a massive heart attack. She did not survive.

Her team decided to continue in the tournament, as that is what she would have wanted. When I heard they were playing, I approached the captain and let her know I would play for them if they needed me.

She informed me that they did need me for their third game of the tournament, and so I joined their tribe.

While dressing for that game, the team let me know that they rarely won, and that I shouldn’t expect them to score more than one goal, if that even. Their faces were heartbreaking to look at – they were in grief and in shock, eyes red and puffy – but they put on brave faces. They bonded together over the tragedy, and connected with each other over their (and their goalie’s) love of hockey.

We took the ice. I have never wanted to win so badly in my life. I have played at the national level, USA Hockey B division, but that was nothing compared to the pressure I felt playing behind these women.

We won 5-3.

It was the hardest game I have ever played, but will go down in my mind as the best game I have ever played, both physically and mentally.

Everything we do in life affects everyone we know, even if the details are so small not to be noticed directly. Every one of my customers will interact with someone who is improved. Each tribe I am a member of is now stronger, because I am stronger for the experience.

Thank you, Shelly Leclier. I would say rest in peace, but I know you’ll find ice time in heaven.

42 things you can do to get bad customer service

Elaine — Wed, 17 Feb 2010 21:35:52 GMT

It’s been a while since I posted anything campers. I actually only have five suggestions…

1. Be unprofessional.

We have had voicemails that made us cringe (like the customer who flushed the toilet – we don’t think he realized he had dialed the phone… at least, we hope he didn’t know), and people with children screaming in the background. No excuse for the first case. The second case – we sometimes work from home as well, and understand that those things can happen sometimes. In fact, I have two dogs, and one of them hates when I am on the phone and tries to talk to me himself. If you hear “A-ROO-ROO-ROO” in the background when talking to me, please know it’s not me howling into the speaker. However, when he does that, I make sure to quiet him as soon as possible with a rawhide or a biscuit. Point is, it happens only once.

Secondly, ending your support case descriptions with “U giv code, plz? KTHXBAI.” Really? Show me your resume’. Tell me how you got a job. I expect to be greeted by English (and I will be quite lenient with non-English speakers using a translator), but text speak should be used on the WOW forums or with your teenage little brother over IM.

2. Complain, complain, complain.

When I first started with Atalasoft, my eyes glazed over when trying to read the documentation. Now, it’s the easiest thing in the world for me to find what I’m looking for. (Practice, practice, practice.) I understand that a lot of people have trouble with it. But, if over the course of a 20-minute conversation, you spend 19 minutes complaining about the documentation, then we’ve wasted 18 minutes. You could have complained for 1 minute and I could have helped you in 1 minute. Which one sounds more productive?

Don’t tempt me to send you to a different support group.

3. Lie, cheat, steal…

Software piracy is an unfortunate way of life. From Windows to Adobe to DotImage, people are always on the hunt to get around paying for products they use. We can’t stop everybody, and we’ve put in quite a few measures to prevent people from taking advantage of us. However, people do what they do, and then expect to get the same service as people who legitimately pay for the product and service. When they get caught, they ask why they’re not receiving timely service. I think the answer is obvious – you get what you pay for.

4. Refuse to help us help you.

We are not mind readers. If we were, we would not be engineers. In fact, I would be going on game show after game show (“No whammies… no whammies… STOP!”) When we ask you questions, when we make suggestions, it is because we are trying to determine the issue, not waste your time. We do not know the answer to every single problem immediately – sometimes we need to investigate down to the real cause of the problem. Never say “this doesn’t work” and leave it at that. If I ask you a question, and you do not answer it, I’m going to repeat the question until you do. If I make a suggestion, and you don’t try it, I’m going to repeat the suggestion until you do. We don’t follow a script, and no one develops the exact same application. We treat you like an individual, just like everyone else.

5. Expect us to debug your code.

As an engineer for a software company, my family can afford more than ramen noodles every night. As an engineer for a SMALL software company, we’re not eating prime rib every night, either. So when a contract programmer making 5 times more than me per hour expects me to pore over their code to find where they forgot to set something to true, I tell them politely that that’s not what I do. Give a college kid $20, and I’m sure he’ll do it for you. Be careful, though – he might take your place on the next contract.

Which leads to… one thing you can do to get great customer service.

Be nice.

If you’re nice, we might overlook the fact that you’re popping gum into the phone. If you’re nice, we might step you through the documentation and show you how we find things quickly. If you’re nice, we might hint that you can find your answer in a particular demo. If you’re nice, we might make other suggestions for you to try. If you’re nice, we might start a go-to meeting and give you a second set of eyes.

It seems that a little over a year ago I posted something similar with the same conclusion, but it can’t hurt to repeat this message until everyone gets it.

Be nice. (I was going to link to a YouTube clip of Patrick Swayze’s “Be nice” speech in Roadhouse, but, uh, the language wasn’t very… nice.)

Get Started with F# via Higher order IEnumerable extension methods in C#

RickM — Fri, 12 Feb 2010 16:03:05 GMT

The more I use F# the more I want to write my every day production C# code in a functional way. To this end, I’ve written a few higher order extension methods as the need arose. I wanted to take a moment and share them with you. I think that in seeing an implementation of these functions in C#, what they do becomes easy to understand.

Filter provides a new collection with unwanted items removed.

public static IEnumerable<T> Filter<T>(this IEnumerable<T> list, Predicate<T> predicate)
 {
     List<T> newList = new List<T>();
     foreach (var item in list)
     {
         if (predicate(item))
             newList.Add(item);
     }
     return newList;
 }

Reduce boils down a collection into exactly one item of the collection’s containing type.

public static T Reduce<T>(this IEnumerable<T> list, Func<T, T, T> function)
 {
     var enumerator = list.GetEnumerator();
     enumerator.MoveNext();
     var last = enumerator.Current;
     while (enumerator.MoveNext())
     {
         last = function(enumerator.Current, last);
     }
     return last;
 }

Fold is much like reduce except that it boils the collection down into an instance of any type.

public static U Fold<T, U>(this IEnumerable<T> list, Func<T, U, U> function, U initial)
 {
     var enumerator = list.GetEnumerator();
     var last = initial;
     while (enumerator.MoveNext())
     {
         last = function(enumerator.Current, last);
     }
     return last;
 }

Map takes one collection and returns another of the same length by applying a function to each element.

public static IEnumerable<U> Map<T, U>(this IEnumerable<T> list, Func<T, U> mapFunction)
 {
     List<U> newList = new List<U>();
     foreach (var element in list)
     {
         newList.Add(mapFunction(element));
     }
     return newList;
 }

While these four are a good subset to begin with, F# provides a much broader range of IEnumerable functionality. Also, while examples here build new lists, their F# counterparts often evaluate lazily and so are usable with very large, parallelized or even infinite sequences.

For a more detailed view with some F# and Linq code check out Matthew Podwysocki’s post on learning Functional C# from F# and LINQ. His example are actually lazy when appropriate and have precondition checks.

DataMatrix Described

Steve Hawley — Tue, 09 Feb 2010 16:33:50 GMT

Saw this great description of Datamatrix encoding via Trivium.

Very good your us visual thinkers.

And if you need it, Atalasoft has tools for both reading and writing Datamatrix 2D barcodes (and other barcodes as well).

F# Discoveries This Week 02/08/2010

RickM — Mon, 08 Feb 2010 19:28:41 GMT

Back again with another F# community roundup. There’s been a ton of great content this week, almost too much. To try to combat this I’ve attempted to sort posts roughly in terms of how interesting I found them. All were worth the read though.

Also, I’ll be speaking this Wednesday on F# at the Boston .NET User Group. If you find yourself there, be sure to say hello!

Don Syme’s Introduction to F# (Video 1 of 3)

Dr. Don Syme is a principal researcher in MSR Cambridge. He has a rich history in programming language research, design, and implementation (C# generics being one of his most recognized implementations), and is the principle creator of F#.

Ashley Feniello’s FScheme Part 8: Language vs. Library

Perhaps this post should have gone along with the one about macros and how Lisp is a “programmable programming language.” The common tension in any language or runtime design is how much to build in as primitives and how much to implement as libraries within the language or atop the runtime.

Luke Hoban’s F# for Parallel and Asynchronous Programming

Last November at PDC 2009 in Los Angeles I gave a talk on F# for Parallel and Asynchronous Programming. The talk begins by covering basic F# concepts, and then focuses on four challenging issues related to concurrency and the tools F# brings for addressing these - immutability, async workflows, and agents.

Andrew Brehaut’s On Iteration (Could just as well be entitled “Why Programmers Leave Python/C#/Java")

There is an observable trend in Python programmers that results in a reasonable section of them moving to functional programming languages. This trend is encouraged by the Python language, and has a couple of temporal considerations.

Matthew Podwysocki’s Using and Abusing the F# Dynamic Lookup Operator

Much like C# 4.0 has the ability to do dynamic lookup, F# also has the same capability, although in a different capacity. The language has support for a dynamic lookup get operator ( ? ) and set operator ( ?<- ), but note that I said support and not actual implementation. The actual implementation is up to you and how you want to use it.

Matthew Podwysocki’s A Kick in the Monads – Writer Edition

In the past couple of Monads posts, we’ve talked briefly about the State and Reader Monads and their potential uses and misuses. Before this series completes, I have a few more to cover including the Writer, Continuation and eventually Observable monad. Today, we’ll get started looking at the Writer Monad and what it can do for us.

Ashley Feniello’s Recursion Is the New Iteration

Not all recursive expressions represent recursive behavior. In fact, iteration can be expressed recursively. One trick when considering the process that an expression represents is to think of functions as reducing to values rather than returning them. In a pure functional style (precisely because of referential transparency) you can do analysis by successive substitution.

Julien Ortin continues his series on Purely Functional Data Structures with a Real-time double-ended queue, a Skew binary random access list and a Binary random access list

This post describes the F# implementation of the <insert data structure here> from Chris Okasaki’s “Purely functional data structures”.

Flying Frog Consultancy Presents - John Conway’s Game of Life in 32 lines of F#

John Conway's Game of Life is a famous example of a simple cellular automaton that produces remarkably diverse results. The game can be implemented in only 32 lines of F# including real-time visualization using Windows Presentation Foundation as follows:

Brian McNamara’s An RSS Dashboard in F# (Part 3)

Last time I covered IObservables and we created a useful ObservableSource class. Today I’ll cover the next technology piece of the app: reading RSS feeds. I’ll discuss the design considerations regarding how to poll feed for updates and publish feed items as IObservables, and walk through one implementation.

Nick Gravelyn’s Pong in F# with XNA Game Studio

A few of my colleagues were discussing F# today and when/where/how it is/isn’t better than C#. I haven’t ever really used F# beyond a very, very brief look at the syntax, so tonight I decided to see what it was all about. As a little project, I decided to make Pong with XNA Game Studio using F# with Visual Studio 2010.

Jim Burger’s Maybe F# isn’t for you…

If you aren’t excited about new ways to tackle the concurrency problem, or new approaches to handling generic and mathematical problems, and if the thought of breaking out the shiny new lexer and parser don’t give you a tingle, then maybe F# isn’t actually marketed at you at all.

Alex Turner’s Calling F# from COBOL and Back Again (CodeProject)

Running languages on .NET is ultra-powerful. Using managed COBOL (from Micro Focus), it is possible to use F# code to work with COBOL code. Imagine a Cloud based F# map reduce system consuming legacy COBOL - yes, that really is on the horizon.

Steffen Forkmann’s New syntactic sugar for “FAKE – F# Make”

The new version 0.27 of “FAKE – F# Make” comes with new syntactic sugar for build targets and build dependencies. Don’t be afraid the old version is still supported – all scripts should still work with the new version.

Jared Parsons’s Having fun with events in F#

Recently I ran into a situation where I needed to handle some events in F# in a special way. In this particular case I wanted to be able to disable and re-enable my handler based on changes in the program. Essentially the C# equivalent of continually adding and removing the handlers.

Steve Gilham’s “Hello Glade#” from F#

Another bit of spiking, a rather tardy follow up from raw GTK#, starting from the C# example at the Mono Project site, but incorporating the earlier example, so as to build in a clean application exit, for one thing.

Luis Diego Fallas’s Using a Webcam with WIA and F#

In this post I'm going to show a small example of taking a picture using a Webcam with Windows Image Adquisition 1.0 . This API seems to have changed in Vista and above, the following code only applies to XP.

Flying Frog Consultancy’s WPF Tic-tac-toe demo

The article walks through the design and implementation of a multithreaded program that uses logic programming to create an unbeatable computer opponent and Windows Presentation Foundation to provide a graphical user interface in only 115 lines of elegant F# code!

A (Relatively Reasonable) Discussion of F# on Slashdot

OCatenac passes along an interview with Don Syme, chief designer of F#, which is Microsoft Research's offering for functional programming on the .Net platform. Like Scala, which we discussed last fall, F# aims at being an optimal blend of functional and object-oriented languages.

What is /?

Steve Hawley — Fri, 05 Feb 2010 21:21:01 GMT

I learned something today about the humble slash character: /.

First, let me say that to me, / is always pronounced ‘slash’. This is due to my first encountering of this character as a path separator in UNIX when I worked at Bell Labs. The other slash, \, I only knew as back slash. When I encountered it in DOS paths, I was appalled that people used the name ‘slash’ interchangeably with ‘back slash’ and called ‘/’ ‘forward slash’. Slash/back slash made sense to me because it fit cleanly with tick/back tick (‘ and ’).

What I learned is that the character that we think of as slash (or forward slash to you heathens) is really called ‘solidus’.

I also found out that the paragraph symbol, ¶, is called ‘pilcrow’ or ‘alinea’.

How about that?

I’ll take “Punctuation Names” for $800, Alex.

F# Discoveries This Week 01/29/2010

RickM — Fri, 29 Jan 2010 19:15:34 GMT

Many apologies if I missed your post. Having skipped last week due to some uncontrollable circumstances left me with quite a large number to sort through. My personal favorite this week is Ashley Feniello’s series. SICP eat your heart out.

Richard Morris’s Geek of The Week is Don Syme

“It came as a surprise to many of us when Microsoft pulled from it's hat a rabbit in the form of an exciting, radical, language that offers an effective alternative to the Object-oriented orthodoxy. The creative force behind this language, F#, turns out to be a brilliant Cambridge-based Australian called Don Syme, already well known for his work on generics in .NET.”

Ashley Feniello’s extremely cool eight part (so far) series FScheme:

“One of my New Year’s goals is to re-read Lisp in Small Pieces and implement all 11 interpreters and 2 compilers. As much as I like the "Lisp in Lisp" idea and enjoyed the eureka moment in SICP when Sussman writes the metacircular interpreter on the board to the music from Space Odyssey, I don't want to do Lisp in Lisp itself. Lisp in F# sounds like more fun.”

Part 1: FScheme - Scheme in F#
Part 2: Just ‘let’ Me Be Already!
Part 3: Lambda the Ultimate!
Part 4: Rinse and Recurse
Part 5: What ‘letrec’ Can’t Do
Part 6: What's Lisp Without Lists?!
Part 7: No Wait, Macro the Ultimate!
Part 8: Oh, The Humanity!

Matthew Podwysocki’s Series - Creating Extended Builders Part 1, Part 2 and Part 3

“But, what we lack is an imperative programming model on top to allow for such things as if statements, for and while loops, and try/catch or try/finally blocks. Luckily, there is a programmatic model to follow to make these things possible inside of our expressions. Let’s cover each of these functions in turn and see what each one does and in the process implement them for the Reader Monad.”

Chris Smith’s Being an Evil Genius with F# and .NET

“Rather than doing the prototypical “Intro to F# Talk” I figured I go with something a bit more fun and relevant to the every day developer. Sure F# is neat and everything – but why bother to learn a new programming language unless you can use it to do something meaningful. Well, in addition excelling at both functional and object-oriented programming, F# is ideal for world domination.”

Tomas Petricek’s Accelerator and F# (IV.): Composing computations with quotations

“In this part of the series, we're going to look at working with quotations explicitly. We'll use meta-programming techniques to work with Accelerator. Meta-programming means writing programs that manipulate with other programs or pieces of code.”

Luke Hoban’s Tuning a Parallel Ray Tracer in F#

“One of the samples that is included with the Parallel Programming Samples for .NET 4 is a simple Ray Tracer. This ray tracer provides a nice visual way of seeing the benefits of .NET 4 parallelism features, as well as giving insights into the way work stealing happens under the hood.”

Julien Ortin continues his series on Purely Functional Data Structures with a Scheduled binomial heap, Scheduled bottom-up merge sort and a Hood-Melville queue

“This post describes the F# implementation of the <insert data structure here> from Chris Okasaki’s “Purely functional data structures”.”

Julien Ortin’s Rate meter in F#

“The following code allows us to measure (and keep track of) the exchange rate (whether upload or download), and the time needed to exchange n additional bytes (assuming the rate is stable).”

Julien Ortin’s Two-way traffic control in F# – bucket approach

“The following approach is based on a per-request basis. That is, an actors asks for n tokens to read (or write), and the manager sends it an answer when the actor is allowed perform the action. In effect, if the actor asks for 100 bytes and only 50 bytes can be exchanged per second, it will get a green light only after two seconds have elapsed.”

Stasyan continues his Red and Black tree Series with Insertion, Deletion, Extras and a Post-Mortem.

“But it works nevertheless. I did some random testing of 10000 elements. Insertion and deletion worked. The cool thing is that the Black height of the tree with 10000 elements does not exceed 10.”

Granville Barnett Review’s F# for Technical Computing

“An F# book by Jon brings with it excitement and promise: his work is well known for being clear and concise, and the examples he uses to help the reader familiarise themselves with the application of F# are highly stimulating.”

David Worthington’s WebSharper platform seeks to broaden F# use

“Microsoft's F# language is best suited for financial and scientific applications, but a startup wants to broaden its usage to building mainstream Web applications.”

Ade Miller’s Implementing a Parallelized Octree in F#

“What’s an octree? An octree is a tree structure where each node has up to eight children, one for each of the octants. You can think of them as a three dimensional variation on a binary tree.”

Diego Echeverri’s Data Visualization with Websharper

“In this post I'll show you how to make a nice visualization similar to the ones used by Hans Rosling. For this, we'll use the WebSharper Google Visualization bindings available as an extension package to the core platform.”

Matt Manela’s Regex based Lexer with F#

“This lexer allows you to define your regular expression based rules in a very declarative way using F# computation expressions.”

Phillip Trelford’s Sorted with F# custom operators

“F# lets you define your own operators, and like a man with a new hammer hunting for nails :) I’ve found an application of F# custom operators for sorting multiple columns.”

Chris Rizzuto’s Http Requests in F# using a TCPClient

“Obviously a few ways to do this using WebClient, HttpRequests, but in this case I decided to use a TcpClient object as I liked the control, and the ability to easily access the textual outputs.”

Chris Rizzuto’s F# WCF Service to calculate StdDev and Keep Real-Time KPIs, F#, WCF, KPIs, ConcurrentDictionary, First Class Events and Message Queues in F# / Building into the KPI Service

“First, below is the functions and events to calculate StdDev. Thanks again LukeH for doing all the work for this part. I only made a small change by adding a new method for handling the event when it is raised, and to make Mean and StdDev accessible from a WCF web service so that if polled for the current values, it is able to return them to the user.”

Cameron Taggart’s List Transactional WCF Bindings in F#

“The code uses LINQ to print a list of WCF binding that have a default constructor and support transactions. I think it actually is a whole lot more readable in F#”

Cameron Taggart’s Hello World for Bing Maps + Silverlight + F#

“The Bing Maps Silverlight Control was released in November. […] I’ve committed code for this blog that shows you how to get started with it using an F# Silverlight application. I did not need to write any C# or XAML.”

Scott Seely’s F# is Changing My Style

“A few years ago, I would have done a bunch of looping constructs to look at each element. However, I’ve been doing a lot more work with F#. While doing this experiment in C# for a project, I wound up writing the following instead:”

DelegatedImageSource

Steve Hawley — Thu, 28 Jan 2010 21:24:07 GMT

One common task in working with images is, given a file or a set of files, read in each page, perform some operation on it and write it as a page in a target file.

This is tricky to do out of the box in dotImage in that a lot of the onus of memory management falls onto client code. The client needs to know how to append images onto a possibly existing file in a way that won’t use up a ton of memory.

Let me give you an example:

TiffDocument doc = new TiffDocument();
for (int i = 0; i < totalPages; i++) {
    AtalaImage page = GetSomePage(i);
    Watermark(page);
    doc.Pages.Add(new TiffPage(page, TiffCompression.Default));
    page.Dispose();
}
doc.Save(someOutputFile);

This is a chunk of code that is meant to loop through some set of pages, watermark each one, add it to a TiffDocument and then save it to a TIFF. We’re being good in that we’re calling Dispose() on the AtalaImage to let it go. The problem is that TiffPage allocates memory for the image (in most cases compressed) so if you’re working with a dozen pages or less, you’ll be fine, but if you work with hundreds, you will bring your machine to a crawl.

In thinking about this, I decided that what we needed was a flavor of ImageSource and RandomAccessImageSource that could adapt an existing ImageSource and allow the user to inject code into the process.

To do this, we’ll create a subclass of ImageSource that takes another ImageSource as an argument. It will let the provided ImageSource do all the heavy lifting, allowing code injection at the appropriate time:

using System;
using Atalasoft.Imaging;
namespace DelegatedImageSource
{
    public class DelegatedImageSource : ImageSource
    {
        Func<AtalaImage, AtalaImage> _imageProc;
        ImageSource _source;
        public DelegatedImageSource(ImageSource source, Func<AtalaImage, AtalaImage> imageProc)
        {
            if (source == null)
                throw new ArgumentNullException("source");
            if (imageProc == null)
                throw new ArgumentNullException("imageProc");
            _imageProc = imageProc;
            _source = source;
        }
        protected override ImageSourceNode LowLevelAcquireNextImage()
        {
            AtalaImage image = _source.AcquireNext();
            AtalaImage processed = _imageProc(image);
            processed = processed ?? image;
            if (processed == image)
            {
                processed = (AtalaImage)image.Clone();
            }
            _source.Release(image);
            return new ImageSourceNode(processed, null);
        }
        protected override void LowLevelDispose()
        {
            _source.Dispose();
        }
        protected override bool LowLevelFlushOnReset()
        {
            return true;
        }
        protected override bool LowLevelHasMoreImages()
        {
            return _source.HasMoreImages();
        }
        protected override void LowLevelReset()
        {
            _source.Reset();
        }
        protected override void LowLevelSkipNextImage()
        {
            _source.Release(_source.AcquireNext());
        }
        protected override int LowLevelTotalImages()
        {
            return _source.TotalImages;
        }
        protected override bool LowLevelTotalImagesKnown()
        {
            return _source.TotalImagesKnown;
        }
    }
}

The main meat of the code is in LowLevelAcquireNextImage, which calls AcquireNext on the provided source, then hands the resulting AtalaImage off to the delegate. If the delegate returns null or the original image, we return a clone, otherwise we return the image created by user code.

Here is an example snippet which combines a set of images into a single TIFF:

FileSystemImageSource source = new FileSystemImageSource(@"\Images\Documents", "*", true);
DelegatedImageSource dis = new DelegatedImageSource(source,
            (image) => { return null; });
TiffEncoder encoder = new TiffEncoder();
using (FileStream fs = new FileStream("output.tif", FileMode.Create))
{
    encoder.Save(fs, dis, null);
}

which is a sweet little chunk of code to combine image files into one TIFF. Further, there won’t be the same memory issue as with our naive example because ImageSource handles resource management for you via the Acquire/Release model. You might ask, “so why do need the TiffDocument class when I can do all the work like this?” The answer is that when an AtalaImage is created from a TIFF document, the metadata associated with the page (EXIF, etc) is shed. TiffDocument maintains that information.

Now let’s change our task to be “mark each page with a red X and save as a PDF”. The code doesn’t change much:

DelegatedImageSource dis = new DelegatedImageSource(source, (image) =>
{
    AtalaImage rgb = null;
    if (image.PixelFormat == PixelFormat.Pixel24bppBgr || image.PixelFormat == PixelFormat.Pixel32bppBgr ||
        image.PixelFormat == PixelFormat.Pixel32bppBgra)
    {
        rgb = image;
    }
    else
    {
        rgb = image.GetChangedPixelFormat(PixelFormat.Pixel24bppBgr);
    }
    using (Graphics g = rgb.GetGraphics())
    {
        using (Pen p = new Pen(Color.Red, 25))
        {
            g.DrawLine(p, 0, 0, rgb.Width, rgb.Height);
            g.DrawLine(p, 0, rgb.Height, rgb.Width, 0);
        }
    }
    return rgb;
});
PdfEncoder encoder = new PdfEncoder();
using (FileStream fs = new FileStream("output.pdf", FileMode.Create))
{
    encoder.Save(fs, dis, null);
}

Note that the save code is identical. This is because both PdfEncoder and TiffEncoder are MultiFramedEncoder objects which can take an ImageSource in one flavor of Save().

Let’s say that you want instead to overlay a watermark onto each image – the code, again, doesn’t change much:

AtalaImage watermark = new AtalaImage(@"C:\Users\shawley\Pictures\bunny.png");
OverlayCommand overlay = new OverlayCommand(watermark, new Point(20, 20));
overlay.Opacity = 0.50;
overlay.ApplyToAnyPixelFormat = true;
DelegatedImageSource dis = new DelegatedImageSource(source, (image) =>
{
    return overlay.Apply(image).Image;
});
TiffEncoder encoder = new TiffEncoder();
using (FileStream fs = new FileStream("output.tif", FileMode.Create))
{
    encoder.Save(fs, dis, null);
}

In this case, I’m using the captured variable, overlay, to write onto the image, turning the actual watermarking code into a one-liner.

And here, finally, is the same code modified to work specifically for RandomAccessImageSource:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Atalasoft.Imaging;
namespace DelegatedImageSource
{
    public class DelegatedRandomAccessImageSource : RandomAccessImageSource
    {
        Func<AtalaImage, int, AtalaImage> _imageProc;
        RandomAccessImageSource _source;
        public DelegatedRandomAccessImageSource(RandomAccessImageSource source, Func<AtalaImage, int, AtalaImage> imageProc)
        {
            if (source == null)
                throw new ArgumentNullException("source");
            if (imageProc == null)
                throw new ArgumentNullException("imageProc");
            _imageProc = imageProc;
            _source = source;
        }
        protected override ImageSourceNode LowLevelAcquire(int index)
        {
            AtalaImage image = _source[index];
            AtalaImage processed = _imageProc(image, index);
            processed = processed ?? image;
            if (processed == image)
            {
                processed = (AtalaImage)image.Clone();
            }
            _source.Release(image);
            return new ImageSourceNode(image, null);
        }
        protected override ImageSourceNode LowLevelAcquireNextImage()
        {
            AtalaImage image = _source.AcquireNext();
            AtalaImage processed = _imageProc(image, -1);
            processed = processed ?? image;
            if (processed == image)
            {
                processed = (AtalaImage)image.Clone();
            }
            _source.Release(image);
            return new ImageSourceNode(processed, null);
        }
        protected override void LowLevelDispose()
        {
            _source.Dispose();
        }
        protected override bool LowLevelFlushOnReset()
        {
            return true;
        }
        protected override bool LowLevelHasMoreImages()
        {
            return _source.HasMoreImages();
        }
        protected override void LowLevelReset()
        {
            _source.Reset();
        }
        protected override void LowLevelSkipNextImage()
        {
            _source.Release(_source.AcquireNext());
        }
        protected override int LowLevelTotalImages()
        {
            return _source.TotalImages;
        }
    }
}

So we see that by using delegates in tandem with ImageSource objects, we can make repetitive tasks much easier and remove a lot of issues in resource management at the same time.

Speaking at The Architect Factory Event

loufranco — Mon, 25 Jan 2010 21:11:24 GMT

On Wednesday, I’ll be leading a session at The Architect Factory event in Cambridge, MA. My topic is Persuading Product/Project Managers and is in the Architecture Leadership track.

I’ll be covering:

What is a Product Manager?
How Product Managers think
How technical should Product Managers be?
What motivates Product Managers
How Architects and Product Managers work together
How to describe technical concepts to non-technical managers
How to communicate estimates
How to communicate design choices
How to describe implementation details like unit testing, refactoring, and infrastructure

More than a third of the time will be devoted to discussion.

Vignettes in DotImage

Steve Hawley — Fri, 15 Jan 2010 18:38:35 GMT

On Stack Overflow, a user asked how to create vignettes in .NET. I wrote up this answer, which explains the math and the general approach.

To give you an idea of what this looks like, here is the output of an application which applies the vignette to an image. This is different from the Stack Overflow answer in that I chose to do this in an ImageRegionCommand from within dotImage.

To do this, I created a new class called VignetteCommand, overrode four members and added one utility method. This command works on 24 bit RGB, 32 bit RGB, and 32 bit RGBA which a limitation imposed by GDI+. Since this command works by painting on top of an existing image, InPlaceProcessing will be overridden to return true always. In theory, we could do without, but we’d just be copying the existing image onto the new image, then painting. We override VerifyProperties to make sure that any user set properties are right. Finally, we override PerformActualCommand to do the actual work. All the rest of the infrastructure is handled by dotImage.

Here is the code:

using System;
using Atalasoft.Imaging;
using Atalasoft.Imaging.ImageProcessing;
using System.Drawing;
using System.Drawing.Drawing2D;
namespace Vignette1
{
    public class VignetteCommand : ImageRegionCommand
    {
        private PixelFormat[] _supportedPixelFormats = new PixelFormat[] {
            PixelFormat.Pixel24bppBgr, PixelFormat.Pixel32bppBgr, PixelFormat.Pixel32bppBgra
        };
        public override PixelFormat[] SupportedPixelFormats { get { return _supportedPixelFormats; } }
        public override bool InPlaceProcessing { get { return true; } }
        protected override void VerifyProperties(AtalaImage image)
        {
            if (Positions == null && Factors == null)
                return;
            if (Positions == null || Factors == null)
                throw new ArgumentException(Positions == null ? "Positions" : "Factors");
            if (Positions.Length != Factors.Length)
                throw new ArgumentException("position and length arrays must be the same length", "Positions");
        }
        protected override AtalaImage PerformActualCommand(AtalaImage source, AtalaImage dest, System.Drawing.Rectangle imageArea, ref ImageResults results)
        {
            using (Graphics g = source.GetGraphics())
            {
                PaintVignette(g, imageArea);
            }
            return null;
        }
        private void PaintVignette(Graphics g, Rectangle bounds)
        {
            Rectangle ellipsebounds = bounds;
            ellipsebounds.Offset(-ellipsebounds.X, -ellipsebounds.Y);
            int x = ellipsebounds.Width - (int)Math.Round(.70712 * ellipsebounds.Width);
            int y = ellipsebounds.Height - (int)Math.Round(.70712 * ellipsebounds.Height);
            ellipsebounds.Inflate(x, y);
            using (GraphicsPath path = new GraphicsPath())
            {
                path.AddEllipse(ellipsebounds);
                using (PathGradientBrush brush = new PathGradientBrush(path))
                {
                    brush.WrapMode = WrapMode.Tile;
                    brush.CenterColor = Color.FromArgb(0, 0, 0, 0);
                    brush.SurroundColors = new Color[] { Color.FromArgb(255, 0, 0, 0) };
                    Blend blend = new Blend();
                    float[] positions = Positions;
                    float[] factors = Factors;
                    if (positions == null || factors == null)
                    {
                        positions = new float[] { 0.0f, 0.2f, 0.4f, 0.6f, 0.8f, 1.0F };
                        factors = new float[] { 0.0f, 0.25f, 0.5f, 0.75f, 1.0f, 1.0f };
                    }
                    blend.Positions = positions;
                    blend.Factors = factors;
                    brush.Blend = blend;
                    Region oldClip = g.Clip;
                    g.Clip = new Region(bounds);
                    g.FillRectangle(brush, ellipsebounds);
                    g.Clip = oldClip;
                }
            }
        }
        public float[] Positions { get; set; }
        public float[] Factors { get; set; }
    }
}

F# Discoveries This Week 01/13/2010

RickM — Wed, 13 Jan 2010 16:59:05 GMT

Back again this week with a fresh batch of F# Posts, Videos and Events. I’ve been enjoying Matthew Podwysocki’s “Much Ado About Monads” series quite a lot. They are well worth checking out for beginner and advanced alike.

Events

If you would like to see your event here, send me an email via the link at the top of the page.

Rick Minerich - Charleston SC Technology Users Group on the 27th of January (check out the awesome flier)

Steffen Forkmann - Frankfurt .NET Usergroup on the 21st of January

Posts

Don Syme’s Async and Parallel Design Patterns in F#: Parallelizing CPU and I/O Computations

One simple way to write parallel and reactive programs is with F# async expressions. In this and future posts, I will cover some of the basic ways in which you can use F# async programming - roughly speaking, these are design patterns enabled by F# async programming.

Don Syme’s F# Interactive Tips and Tricks: Visualizing Data in a Grid

The demos in my F# talks use a number of coding snippets to acquire, generate and display data interactively. Some of these little snippets are not so well known, but they are useful :-)

Don Syme’s F# Interactive Tips and Tricks: Formatting Data using AddPrinter, AddPrintTransformer and %A in sprintf/printf/fprintf

Here are some tips and tricks for formatting data in F# Interactive. This is not meant to be a comprehensive guide, just enough to get you started. Please let me know if you need more examples.

Matthew Podwysocki’s Much Ado About Monads – Reader Edition

So, our ultimate goal would be instead to have our environment set once and then read from it implicitly. We still want to keep what we have here in terms of our script, but change the underlying mechanism for how it happens.

Anton Schwaighofer’s SkillsMatter.com talk: F# and Units-of-measure for Technical Computing

I will start by giving an introduction to units-of-measure and their implementation in F#. I'll work through smaller and larger code examples that make use of units-of-measure.

Nancy Strickland’s MSDev.com Training Session on F#

Visual Studio 2010 includes a new programming language, F#. This session explains and provides a walk-through demonstration of basic programming in F#.

Julien Ortin continues his series on Purely Functional Data Structures with a Lazy Pairing Heap, a Real-time Queue and Bottom-up Merge Sort

This post describes the F# implementation of the <insert data structure here> from Chris Okasaki’s “Purely functional data structures”.

Mauricio Scheffer’s Translating Haskell to F# and other considerations

BUT, syntactic similarity does not imply that they're semantically the same. There's a fundamental difference from the original code: Haskell is a lazy language, while F#, coming from the ML-family, does eager evaluation.

Mark Needham’s F#: Refactoring to pattern matching

I was looking through some of the F# code I've written recently and I realised that I was very much writing C# in F# with respect to the number of if statements I've been using.

Mark Needham’s F# attempt at Roy Osherove’s TDD Kata

As I've mentioned in a few of my recent posts I've been having another go at Roy Osherove's TDD Kata but this time in F#.

Holoed’s Weak Subscribe to an IObservable Source

A code snippet describing how to subscribe to an event via a weak reference.

Erik Schulz’s Resource Pool in F#

I’ve been toying around with F# recently, it’s good to see an example that you can easily compare and contrast with the C# version.

Ade Miller’s Gotchas: Common Traps for the F# n00b

I spent a bunch of time over the holidays getting to know F# a bit better. I think I now consider myself to be truly dangerous with it. A couple of things which repeatedly bit me as I stumbled through learning F# as a n00b.

SharePoint Document Management Checklist: Get Things Done

loufranco — Sat, 09 Jan 2010 15:56:21 GMT

Windows IT Pro published a checklist for SharePoint Document Management. It covers some of the things you need to think about in addition to just adopting SharePoint (storage, interoperability, backup, security, etc)

Those things are important, but instead of concentrating on the infrastructure mechanics of document management, it’s better to talk about the purpose. If you are sitting in front of your computer looking at SharePoint and thinking about folders, Documentum integration, your SAN, and backup procedures, then something has gone completely wrong.

The reason to use any tool is to get stuff done – pay invoices, process claims, answer inquiries. The more you can get SharePoint and whatever you add to it to put your work in front of you and help you do it, the more you will get out of it.

Here’s my checklist of what to look for in Document Management that will help you get work done.

Are all the things you need to process a document organized and easy to get to while viewing it (Annotations, meta-data, document editing)?
Are related documents readily accessible?
Can you compare two documents easily? For example, the purchase order and the related invoice or two versions of your newsletter.
Is search integrated so that I don’t need to leave my management window to bring up another document?
Can I correct a badly scanned page with an ad-hoc page rescan right from my viewer?

Of course everything needs to be backed up, versioned correctly, integrated, etc. But if that’s all you get, then you won’t get the full benefit. In the end, the ROI will be driven by the business getting it’s work done, so don’t stop at just the infrastructure requirements.

F# Discoveries This Week 01/05/2010

RickM — Tue, 05 Jan 2010 21:10:05 GMT

It’s 2010, the year of F#, and the quantity of posts in this last week reflects it. Not to pick favorites but the work Tomas Petricek has been doing with Accelerator is amazing, be sure to check it out.

Also, I’d like to thank everyone who supported me on the road to becoming a Microsoft MVP. I couldn’t have done it without you.

Tomas Petricek parallelizes Quotations with Accelerator

In this article, we'll look at more sophisticated way of using Accelerator from F#. We'll introduce F# quotations and look at translating 'normal' F# code to use Accelerator.

Don Syme Posts updates his F# JAOO tutorial code

I've now updated this tutorial code for the F# Visual Studio 2010 Beta2 release (with matching CTP release for Visual Studio 2008). We've also added some more content and explanatory comments.

Matthew Podwysocki investigates the State Monad

In the past, I’ve had a series on Much Ado About Monads where I look at the basic Monads such as Maybe and List, but this time, let’s look at what we can do with the State Monad.

Adam Granicz shows off WebSharper at the Hungarian Web Conference

In this talk, I will present our new web development platform, WebSharper(TM), that leverages the power and expressiveness of F# to allow .NET developers to write robust web applications in mere hours.

Chance Coble and Ted Neward release DZone Refcardz for F#

This DZone Refcard will lead you through the basic essentials so that you can quickly move on to using this Functional Programming Language for creating some mind-bending code.

Talbott Crowell posts his FSUG Parallel Programming Talk

In this video, I discuss some of the advantages of .NET 4 and F# for asynchronous programming plus some of the new analysis tools built into Visual Studio 2010 for visualizing concurrency.

Steffen Forkmann creates a ‘FAKE – F# Make’ group on BitBucket and releases version 0.17

Due to its integration in F#, all benets of the .NET Framework and functional programming can be used, including the extensive class library, powerful debuggers and integrated development environments like Visual Studio 2008 or SharpDevelop, which provide syntax highlighting and code completion.

Julien Ortin continues his series on Purely Functional Data Structures with a Physicist Queue, a Lazy Binomal Heap and a Bankers Queue

This post describes the F# implementation of the <insert data structure here> from Chris Okasaki’s “Purely functional data structures”.

Mathias Brandewinder StackOverflows on Unit Testing with F#

So far, my process with F# has been to write some functions, play with them with the interactive console until I am "reasonably" sure they work, and tweak & combine. This works well on small-scale problems like the Euler Project, but I can't imagine building something large that way.

Chas Emerick demonstrates the problem with mutable objects

316 arguments to a method (which I don't think is actually possible in the jvm, but bear with me)? "That's absurd!", you'd say. The problem, of course, is that the 3-arg doSomething actually has far more arguments than its signature implies:

A Mysterious Stranger implements a Red-Black Tree

Both 2-4 and Red and Black tress are self balancing trees. They are kind of like cousins, since one could be converted into the other quite easily. Hence all performance analysis that apply to 2-4 trees applies to Red and Black trees.

Mark Needham discusses expressing intent with application operators

While trying out Roy Osherove's TDD Kata I realised that perhaps the choice of which of these to use or whether to use them at all depends on what intent we're expressing.

12 Pink Highlighters

christina — Wed, 30 Dec 2009 21:48:00 GMT

(Alternate Title: An Obvious Lesson in Customer Service.)

We use W.B. Mason for our office supply company. My predecessor used them here and there, and I started using them more regularly when price shopping at B.J.’s and CostCo became a pain.

They matched prices, and give us great deals on K-Cups for coffee (an engineer’s life blood). Plus they have great prices on recycled products (like biodegradable plastic cups! Corn Cups—the official beverage container of Atalasoft.)

Amidst the coffee shipment today, I ordered an Air Wick air freshener system and 2 refills. I received 1 refill and a box of 12 pink highlighters.

This hasn’t really happened before, but I can understand how—the boxes are hand packaged. Human error. No big deal.

And for some reason, I found a box of 12 pink highlighters to be hilarious. What’s wrong with green and yellow? A variety pack can really spice up your document annotation, you know? (I like to think I know a little something about document annotation, too.)

So, as I do when certain things strike me as funny, I tweeted about it.

And, shortly after, I mean, WITHIN MINUTES, I received a voice message from my W.B. Mason rep. I called him back, he apologized for the error, and shipped out the missing Air Wick refill. Even though I had an inkling, I asked him how he knew. “Oh,” he replied, “My marketing department forwarded me your Twitter message.”

Man. Twitter is powerful stuff.

I assured him that we are happy with their business and thanked him for the speedy response.

The bonus of all of this? He’s letting us keep the 12 pink highlighters.

Now, faithful readers—my challenge to you—what would you do with 12 pink highlighters?

Leave a comment here or drop us a line on twitter (we’re @Atalasoft. Use the tag #12PH).

The Elements of Programming Style

Steve Hawley — Wed, 30 Dec 2009 17:01:29 GMT

Many years ago, I had the pleasure of working at Murray Hill Bell Labs. My job, of all things, was making violins, but that’s a story for another time. My office was in section D of the 5th floor of building 2, which was also where the UNIX room was. I learned a lot from looking at the code written by the creators of C and UNIX.

It is terrific to watch Brian Kernighan giving this talk about clarity in code. A lot of what he shows as bad examples are grist for the mill of the Daily WTF and such sites. And while I don’t always meet all of the goals of clarity (who does?), I do try to write things so that they can be read.

That’s one of the reasons why I was so pleased when Rick showed me his hill climbing code before he wrote his blog article: it’s beautiful code. It reads like a textbook algorithm. Nice job, Rick.

Atalasoft 2009 Blog Retrospective

loufranco — Wed, 30 Dec 2009 14:01:27 GMT

Last year, around this time, I did a 2008 retrospective with one blog post from each of the blogs hosted by Atalasoft. Here is my 2009 version.

Atalasoft Office Blog

This year, Christina (our Vizit Marketing Manager among other things) started an office blog to let our customers, partners, etc get to know us a little better. No post does that better than the chronicle of Bacon Day:

The 3 of us sat around my desk and thought, "What is something FUN we can do for the handful of staff members in the office? Something that will make the rest of those guys, enjoying the night life in Philadelphia, CRINGE with JEALOUSY?"

Steve's answer to everything is "bacon." And in this case, he was 100% spot on.

I’m not sure how you name something “Bacon Day” when you do it three times in a year.

Rick Minerich’s Development Wonderland

Rick has been speaking more and more about F#, and his blog has become a popular destination for the community. In June, he wrote this concise history of programming languages – my favorite part is the quotes he chose for each part:

“Real programmers can write assembly code in any language.”
- Larry Wall

Steve’s Tech Talk

Staff meetings at Atalasoft became epic in 2009. In May, we changed our format so that a rotating producer would be in charge of organizing and running the meeting. One aspect of the meeting is a topical presentation on something relevant to the company – in July, Steve, our Chief Architect, explained our development process to the rest of the company with an educational video.

Bill Bither’s Insights

Bill, our CEO and founder, wrote this entry just ahead of the Vizit 1.0 release with some practical advice on introducing a new product on a limited budget:

We researched the market, talked to potential customers, built relationships with industry recognized players, and are now in the midst of turning Atalasoft from a one product company to a two product company – all without any external financing, and during a recession.

I believe that now IS a good time to start a new company, or launch a new product. This is the year for small agile companies to innovate while larger companies cut back and wait it out.

David Cilley’s Ajax Imaging Blog

While Dave was working on a demo using YUI, he whipped up a way of converting YUI Button Groups into Image Buttons.

I poked around YUI’s documentation a bit and came up with simple function that can convert a YUI Button into an image only button, much like a button on a tool bar. These converted YUI Buttons still maintain the same original properties internally, so you can use them interchangeably.

Jake Opines

When Jacob told me he was going to move his personal projects from WordPress to SharePoint, I was shocked. I personally use WordPress, but have been unhappy with it for years – it’s just the best of a lot of bad alternatives to me. This entry explains some of his rationale for using SharePoint 2010 over LAMP:

SharePoint offers a framework that abstracts away databases and security design, it gives you a complete content management system, a plug-in architecture, and for the price, unmatched scalability. This doesn't even begin to take into account the growing ecosystem of third party features available both for free and fee.

If you can’t be a developer, be a developer supporter

Elaine runs our support and training. On her blog, you’ll find a mixture of information about our products and posts like this one, an account of her experience as a customer using some other company’s support:

I could end all this with some philosophical advice and parallels of my experiences as a support engineer in the software industry, but, really, everything is pretty clear in four rules:

1. Do it right.
2. If you can’t, communicate.
3. Do what you can do.
4. Do it quickly.

If you build it, They will come

Dave named this blog when he was in charge of builds – now, he’s a Vizit engineer (but still the most knowledgeable about our builds). When these worlds collide, you get a very thorough post on using TypeMock for SharePoint to mock SharePoint SPSecurity Delegates.

The main problem I found was that SPSecurity had to be mocked, otherwise it would throw a NullReferenceException […] for obvious reasons, there’s no SharePoint to test against! Let’s flex those Mock muscles and handle calls to SharePoint’s static objects.

Insert Quality Here

In 2008, I discovered LLVM for Cocoa/Objective-C static analysis, which, unlike Lint, lists problems with no false positives. Adam did this round-up of static analysis tools last year that gives a few good reasons to use them even if their output is unwieldy.

It has seemed to me that most folks' (including mine) first impression of the massive amounts of output that a static analysis tool generates is, “Oh, dear Lord. What's all this crap about code style?”, and “Where're the useful results?”.

Dan Barvitsky

Our newest engineering blog is from Dan. I’m a sucker for sequence diagrams and unit testing techniques, so I love this post he wrote about unit testing to find concurrency bugs.

When I run this test it passes on my laptop. Why? Because it is fairly old and has single CPU. Once I move it to my desktop (TADA) it still passes inconsistently. If I uncomment priority setting line it messes everything up badly. It turns out threading introduces some randomization which makes test non-deterministic. Bad test. We’d better fix it.

He thought about the problem some more and eventually documented a solution.