Rick Minerich's Development Wonderland : fp

F# and You! - New Hampshire User’s Group and Beyond

RickM — Mon, 26 Oct 2009 16:24:49 GMT

I’ve been working for a while on a new presentation which I was finally able to give last week at the New Hampshire .NET User Group. F# and You! focuses on painting the big picture about F# instead of the off-putting details like having to learn new syntax.

For this new presentation I start by discussing the adoption of functional programming on other platforms by using Scala, Erlang and Haskell as examples. I then continue on to how the algorithmic programmers are moving to functional languages while UI developers are moving towards declarative.

This then naturally raises the question of why programmers would choose a specific style of programming for a specific task. The answer, of course, is that when you work close to a domain you can build things more quickly and with a lower error rate.

This then transitions easily into the specific beneficial properties of functional programming (and F#).

General Overview:

The Separation of UI and Back End
What is Functional Programming? (Functional Concepts)

Pure functions
Immutability
Lambda Expressions
Higher Order Functions
Recursion

The Benefits of Functional Programming

Code Compression
Parallelism
Robustness

Proof is in the Pudding

TrueSkill
Grange Insurance Rating Engine
MSN adCenter

Wrap-up

I’ve also found that it is useful to show the latest circulating version of the Ant Colony simulation I wrote over a year ago. I found a version in Don Syme’s JAOO Talk code samples but am not sure who has been keeping it up to date. A big thanks to whoever that person is. It provides some sweet eye candy to dull the bitterness of the technical Functional Concepts section.

Planned (and Past) “F# and You!” Locations:

11/10/2009 – CT .NET Developers Group (Farmington, CT)
10/28/2009 – Technology Users Group (Charleston, SC)
10/21/2009 – New Hampshire .NET User Group (Nashua, NH)

See the new Other Events section of our F# User Group site for information on other upcoming F# talks.

Downloads:

Latest Slides
VS2008 Code Samples

Image Processing as Sets of Transformations

RickM — Tue, 28 Apr 2009 00:21:00 GMT

In the image processing world, like most computational problems, we often think our work is composed of only two basic ideas: representation and transformation. Of course, one may have many layers of both representations of transformations and transformations of representations which can make things appear quite complex at times.

However, the problem is much more simple than it appears. This is because a representation can be considered as a transformation from a zero or identity state. Thus, in writing a symbolic language for image processing, we are left with only a single idea to consider: transformations. By composting layers of transformations we can apply image processing techniques in way which is not only bidirectional and platform agnostic but also comes along with a host of other benefits.

Let us consider a simplified example of processing an image:

1) We read in a file (representation) and use a codec (transformation) to convert it into a format understood by our API (representation).

2) We then perform some type of algorithm on that data (transformation) which results in some type of output (representation).

3) Finally, via another codec (transformation), another file is saved to disk (representation).

In most cases there are a great number of intermediate representations. Each is a full copy of the previous iteration with whatever changes have been so far applied. Essentially, the same information is copied over and over again in memory. We do allow for some kinds of in-place processing, however, this is bad as when the operation has been completed, the previous representation has been destroyed.

Instead, what if we batched up sets of transformations? This could have many benefits:

1) The most obvious benefit is that of parallelization. Even at the simplest level of functional composition, these transformations could be handed off to a cluster for asynchronous processing or saved for a later batch processing job.

2) With an intermediate symbolic transformation language, processing algorithms could potentially be combined and reduced to produce a single transformation out of many. This would significantly reduce the processing overhead as well as the number of intermediate memory representations.

3) An intermediate symbolic language which encompassed both codec and processing may make it possible to push the processing transformation through the codec transformation and in so doing no longer need to have any intermediate memory representation. This could provide significant memory and processing speed time benefit.

4) The intermediate symbolic language could be saved into the files themselves thus removing the need for the codec to be present on the end machine. Admittedly, the user would also need the image language interpreter.

5) Instead of applying simple image processing algorithms to an image, the symbolic representation could be appended to the end of the file. This would be quite similar to layers in practice. In this way it would be possible to view the image at all stages of transformation.

6) For large or proprietary transformations, the representation could be kept on the internet and either be downloaded or, in the case where the owner did not want to expose their algorithm, a flattened representation could be sent out and a processing delta could be sent back.

Conclusion

Of course, when I speak of data I don’t only mean the image itself. This technique could also be applied to many classes of data or algorithm. Most notably for us, image metadata.

My initial goal is to build a basic codec representation along with some simple transformations. Currently, I am researching bidirectional, reversible and declarative languages as examples. With F# as a base language I believe it will be possible to build something portable to other ML variants.

F# for Testing and Analysis at Code Camp 11 New England

RickM — Fri, 27 Mar 2009 22:36:00 GMT

At this Saturday’s Code Camp I’ll be giving a brand new presentation on using F#. The goal of this presentation is to have those attending leave with an idea of what F# can help them achieve today as well as instill a desire to know more. This post contains my presentation materials as well as links to additional information on the topics covered.

Presentation Details

F# for Testing and Analytics will be held in the TBC (Technical Briefing Center) at 10:30am.

New England Code Camp 11 Information

Download Presentation Slides: F# for Testing and Analysis.pdf

I’ve been thinking quite a lot lately on the best way to get developers excited about F#. By focusing less on the language’s details and more on what the language can help achieve I hope to impress upon those attending why F# is worth the effort. The preliminary structure of the talk is as follows::

Introduction
- Announcing the New England F# User Group
Why F#?
- Time Savings
- Readability
- Code Exploration
- Robust Software
- Power
F# Testing Toolbox
- xUnit.NET
- TestDriven.NET and NCover
- NaturalSpec
- FsCheck
A Real World Example
- Exploring the Problem Space
- Collect, Analyze, Report
- Visualizing Data

Presentation Influences
John Hughes’s Paper: Why Functional Programming Matters
John Hughes’s Talk: FP – A Secret Weapon for Software Testing
Matthew Podwysocki’s 7 Part Series: Functional Programming Unit Testing

Announcing the New England F# User Group

I’m proud to announce that we will be holding our first New England F# User Group meeting on Monday April 6th, from 6:30pm to 8:30pm. It will be held at the aptly named Microsoft NERD center. I hope to see you there.

Visit the F# User Group Homepage for more information.

xUnit.NET

“About a year ago it became clear to myself and Brad Wilson that there were some very clear patterns of success (and failure) with the tools we were using for writing tests. Rather than repeating guidance about "do X" or "don't do Y", it seemed like it was the right time to reconsider the framework itself and see if we could codify some of those rules.”

xUnit.NET Homepage

Learn More about xUnit.NET
Matthew Podwysocki’s The Unit Testing Story in F# Revisited
Jim Burger’s Unit testing in F# with xUnit.NET
Harry Pierson’s Practical F# Parsing: Unit Testing

TestDriven.NET and NCover

“TestDriven.NET is a zero friction unit testing add-in for Microsoft Visual Studio .NET. The current release of TestDriven.NET supports multiple unit testing frameworks including NUnit, MbUnit and MS Team System and is fully compatible with all versions of the .NET Framework.”

TestDriven.NET Homepage (Includes a free version of NCover)
NCover Homepage

Learn More about TestDriven.NET and NCover
Matthew Podwysocki’s F# + TestDriven.NET + xUnit = Win
Matthew Podwysocki’s FP Unit Testing Part 4 – Code Coverage

NaturalSpec

“The idea of NaturalSpec is to give domain experts the possibility to express their scenarios directly in compilable Unit Test scenarios by using a Domain-specific language (DSL) for Unit Tests. NaturalSpec is completely written in F# – but you don’t have to learn F# to use it. You don’t even have to learn programming at all.”

NaturalSpec Homepage

Learn More about NaturalSpec
Steffen Forkmann’s “Getting Started” with NaturalSpec
Steffen Forkmann’s Testing QuickSort with NaturalSpec
Steffen Forkmann’s Using NaturalSpec to create a spec for C# projects

FsCheck

“FsCheck is a tool for testing .NET programs automatically. The programmer provides a specification of the program, in the form of properties which functions, methods or objects should satisfy, and FsCheck then tests that the properties hold in a large number of randomly generated cases.”

FsCheck Homepage

Learn More about FsCheck
Claudio Cherubino’s Random testing in F# with FsCheck
Matthew Podwysocki’s FP Unit Testing Part 2 – QuickCheck and FsCheck
Kurt Schelfthout’s FsChecking dnAnalytics Part 1, Part 2 and Part 3

FsStory

“FsStory is a library for writing executable user stories in F#. FsStory enables the developer to write user story scenarios (in Given/When/Then form) in F# code.”

FsStory Homepage

Learn More about FsStory
Gustaf Nilsson Kotte’s Fluent language in FsStory
Gustaf Nilsson Kotte’s FsStory, executable stories in F#

Discoveries This Week 03/20/2009

RickM — Sun, 22 Mar 2009 20:09:43 GMT

This week there seems to be quite a bit of video and audio content to share. We have a dramatic depiction of the benefits of Units of Measure, Luke Hoban on F#’s feature, Ted Neward talks about F# under the hood and finally a discussion on operator type inference.

Video Blog – Could F# have saved the Mars Climate Orbiter?

A fantastic example of how static dimensional analysis of scalar values could prevent an entire class of bugs. In the past, I’ve talked about what a huge fan I am of the Units of Measure package for F#. I’m glad that others also see how revolutionary units of measure are.

Would you want to work in a building or live in a house in which the engineers who built it did not keep track of units? I don’t think anyone would answer yes. Similarly, we should not entrust our lives or livelihoods to software in which these very same things are not kept track of.

Podcast – SER Episode 129 - F# with Luke Hoban

In this podcast Luke Hoban, the F# program manager, discusses about various aspects of F#.

Some highlights:

Mutability comes along with .NET Framework support.
Statically Typed means strong guarantees and so robust systems.
FP design in the small for concurrency
OO design in the large for scalability
Explorative programming via interactive window
Influenced heavily by Haskell
FP as a technique, not a class of languages

Video Blog – dnrTV Show #136: Ted Neward Introduces F#

Ted Neward gives an in-depth presentation on both how F# works under the hood and the general benefits that stem from FP in general. The examples used are simple but Ted covers them in a very exhaustive way.

Beyond the language and concepts therein, he also has a number of insightful comments about teaching people to use F#. In particular the idea that F# syntax will be easy to teach but functional programming technique will take years really struck home with me.

Blog – Kalani Thielen’s The F# overload-o-phone

In his post Kalani points out a small but significant logical inconsistency in F#’s type inference system. The issue seems to stem from the way type inference is handled for basic operators. Instead of working generically, they each have default types.

From his post:

> (+);;
val it : (int -> int -> int)
> 3.2 + 5.4;;
val it : float = 8.6
> let twice x = x + x;;
> twice 9.4;;
stdin(2,5): error FS0001: This expression has type
float but is here used with type int.

To get around this, he then goes on to build a function which takes an operator and both arguments. However, this workaround is completely unnecessary. You need only to annotate the function’s input and it works without issue.

> let twice (x: double) = x + x
val twice : double –> double
> twice 9.4;;
val it : double = 18.8

Now, it’s important to note that this version of the twice function won’t accept an integer. In fact, it was a conscious design decision to disallow function overloading in F#. In both F# and C# all kinds of overloading must be qualified by type. That is, type methods support argument overloading but free floating functions do not.

As a FP Ambassador - F# for Testing and Analysis at Code Camp 11

RickM — Thu, 19 Mar 2009 16:00:00 GMT

Over the last six months I’ve given variations on the same F# presentation on six separate occasions. Each time I’ve gotten a similar response: enthusiasm from a small percentage of the audience and glazed-over stares from the majority. In retrospect it seems obvious that I’ve been missing one of the major tenets of making great presentations, know thy audience. It’s time for a new approach.

Information Overload

When thinking about how to get people excited about F# I immediately jumped to the learning format that I found the most exciting. There was no time in my life when I was as overwhelmed by information like I was working on my Computer Science degree at UMass Amherst and I loved every minute of it. When I started out, it seemed that using a university lecture as a model was the best possible format for sharing this kind of information.

I’ve since come to realize that this style of soaking the audience with “a fire hose of information” is a poor format for Code Camps. While using this technique, I could see people turning off one by one as they became overloaded with information. This is not in the spirit of code camp which aims to be a much more laid back kind of experience.

As a FP Ambassador

Let’s invert perspectives and think about what an audience member would want. When a person comes to a Code Camp presentation, it’s likely to be one of their first times being exposed to the topic being presented. They don’t want to be taught as if they were in a classroom. Instead, they want to be exposed to new and exciting information. If their interest is piqued, they will seek details later.

So, instead of focusing on the details of F#, this new presentation will be built around the goal of instilling five key ideas to the audience:

It must be understood that almost every Code Camp attendee will have had little or no exposure to functional programming. To many attendees, to be presenting on F# is to be an ambassador from the strange and nebulous functional programming world. Instead of building walls of information, the goal should be to instead focus on connecting with the audience on the benefits of functional programming.

Planned Talk Structure

Depending on an Audience Poll, Broad Concepts
Testing with F#

nUnit Examples (as a baseline)
xUnit Examples
FsStory Examples

Analysis with F# (A Real Example From Atalasoft)

Using The F# Interactive Window To Build Tests
Data Collection
Visualizing Data

Conclusion

Why F#?

More details will follow soon.

What Would You Want From a F# User Group?

RickM — Thu, 12 Mar 2009 13:39:00 GMT

Michael de la Maza, Talbott Crowell and I are all very excited about finally getting our F# User Group off the ground. However, as F# groups are currently few and far between, we are very interested in finding out what attendees would want from such a group.

Where and When?

In terms of broad time frame, we are looking at having our first meeting in April. We currently have a few options open in terms of which day of the month. If you are interested in attending, head on over to our doodle poll and tell us which days would be best for you.

The group will be held at the new Microsoft NERD center in Cambridge, Massachusetts. A big thanks to Chris Bowen for helping us find such an exciting place to hold our meetings. I know that I’m not the only one hoping that some of the Microsoft research NERDs will have time to come down to our meetings.

Online Content?

We are planning on recording each meeting and putting the videos up online. We don’t have the hosting solution for this firmly pinned down yet, however there are a large number of different options available. Our criteria are support for video of at least two hours in length and high resolution. Ideally, we would like to use a Microsoft-based solution.

Types of Meetings?

While most .NET user groups seem to be primarily presentation based, they are often much more broadly scoped and targeted at an audience with deeper topical knowledge. For these reasons we are considering a number of different formats for the group.

F# Boot Camp - The focus of this type of session would be getting developers who were new to F# and/or FP started. This might include a small presentation, some group activity work and, finally, an open discussion forum.
Presentations - We are working on bringing in F# experts to present on related topics. This will be via both physical presence and remote conferencing.
Competitions - I love competitive coding and I think that there’s no better way to get people excited about a topic. These competitions would likely be web-based and open to all F# users.

Our goal is provide interesting content for both those new to F# as well as those with some experience. As F# is a fairly new language, it’s reasonable to assume that news attendees will have a low level of previous exposure to the language. Due to this, our current thinking is that it would generally be best to start with the basics and work our way up to more advanced topics over time.

Why OO may not suck, or, Take a ride on the Falsus Omnibus

RickM — Wed, 11 Feb 2009 11:00:00 GMT

This post is in response to Joe Armstrong’s Why OO Sucks. While I feel that Joe’s post reads more like an sermon than a stream of rational thought, it does bring up a number of misconceptions I feel many people in the functional programming world share and which need to be discussed openly if it and the object oriented world are to come to terms and work together.

The main issue here is that, in the minds of many functional programming advocates, the concept of an object is inseparable from the amalgamation of 30 years of different, and sometimes contradictory, object oriented programming ideologies.

Perhaps they don’t understand that when binding these ideas together for the sake of argument they are simply constructing a straw man. Similarly, to say that we must either have all of the history of object oriented design or pure functional programming, is a false dichotomy. We can have either, neither, both separately, or any number of combinations of ideas from both groups.

Now, lets address each of Joe’s objections from a somewhat more moderate perspective.

In Response to “Objection 1 – Data Structure and functions should not be bound together.”

I disagree with Joe’s logic here. It does not follow that because functions and data structures are “different animals” that they should “not be kept in the same cage”. Wouldn’t having functions be first class citizens be locking them in the same cage with data structures as well? The fact that they do share some fundamental similarity is manifest in that most functional programming languages do not treat function and data structure in a fundamentally different way.

Also, I have seen no compelling reason as to why it is wrong to couple transformations with the data structure that they are able to be applied to. When transformations often have to be tailored specifically for a particular data structure, why would we not keep them strongly associated with said data structure.

In Response to “Objection 2 – Everything has to be an object.”

As I discussed in my introduction, this is simply not true in all languages which support objects.

In Response to “Objection 3 – In an OOPL data type definition are spread out all over the place.”

First of all, it should be immediately obvious to anyone who works on even medium sized projects that having many types defined in the same file is a very bad thing. The core of the issue is that it makes group collaboration quite difficult.

Secondly, any system that has ubiquitous data structures can’t be very large otherwise the namespace would become quite cluttered. In large systems this cannot be mitigated by simply saying “more functions, fewer data structures”.

Finally, to state that any object which operates on another must inherit from it is a completely incorrect statement. In the modern object oriented world it is common to have objects which exist solely to build, process or interface with other objects. Also, as I mentioned above, it’s entirely possible to have stateless objects which are simply data structures which are bundled with a collection of predefined transformations.

In Response to “Objection 4 – Objects have private state.”

First of all, state is not the root of all evil. While it may be harmful to concurrency and static analysis, there are many stateful algorithms which perform much better than their stateless counterparts.

Secondly, it’s entirely possible to have immutable objects. Perhaps in these objects some state is hidden for the sake of representing ideas in a simple manner, perhaps not.

Finally, as to information hiding in general, in large systems it’s very easy to be overwhelmed by the quantity of information. If you can say via testing “I can trust this structure over here” it means it’s entirely possible to ignore that and focus work on another piece of the system. It’s also important to note that functional programming languages often have information hiding systems of their own. In the most base case, calling other precompiled functions amounts to information hiding and comes along with all of the negative things stated in this section.

In Response to “Conclusion – Why OO was popular.”

It seems to me here that if you removed all instances of “It was thought” this would be a somewhat correct summarization.

In reality, the industry has been using object oriented programming for many more reasons than what has been listed here. A particularly strong reason is that it supplies an organizational framework for data structures and functions which simply does not have an equivalent in the pure functional programming world.

Conclusion

I must apologize to Joe here. I don’t intend to pick on him in particular. However, I had never previously seen a post in which the the most common misconceptions around object oriented programming were so plainly stated.

To say that the concept of an object and that the collections of ideologies used commonly to construct programs using objects are somehow inseparable is completely false. To further state that the only reasonable choice is to instead move to pure functional programming bears no additional truth. Yet they are both pieces of constantly perpetuated misinformation.

The fundamental idea that data and function can be bound for the sake of organization still holds and does not inherently violate any of the tenants of functional programming. In terms of the organizational structure of our programs we have gained much from objects. Why then should we abandon them without sufficient reason? It seems to me that it would be a mistake to dismiss all of what we have gained for baseless ideological reasons alone.