Monday, December 31, 2007

Rhino Mocks Callbacks with Lambda expressions

I recently wanted to write a test for this piece of code using Rhino Mocks:

public void Register(string username, string password, string posterName)
{
    User user = new User()
    {
        Name = username,
        Password = password,
        Role = Role.Poster
    };

    userRepository.Save(user);

    Poster poster = new Poster()
    {
        Name = posterName,
        ContactName = username,
        PosterUsers = new System.Collections.Generic.List<PosterUser>() { new PosterUser() { User = user } }
    };

    posterRepository.Save(poster);
}

As you can see, it simply takes some parameters: username, password, posterName and then constructs a User object which it passes to the Save method of a userRepository and a Poster object which is passed to the Save method of a posterRepository. Both userRepository and posterRepository are created using Inversion of Control. Here's my initial attempt at a unit test for the Register method:

[Test]
public void ReigsterShouldSaveNewUserAndPoster()
{
    string username = "freddy";
    string password = "fr0dd1";
    string posterName = "the works";

    using (mocks.Record())
    {
        // how do get the user object that should be passed to userRepository ???
        //userRepository.Save(????);

        //posterRepository.Save(????);
    }

    using (mocks.Playback())
    {
        posterLoginController.Register(username, password, posterName);
        
        // here I need to assert that the correct User and Poster objects were passed to the 
        // repository mocks
    }
}

I couldn't see how to get the object that was passed to my mock instances of userRepository and posterRepository. I needed some way of capturing the arguments to the mock objects so that I could assert they were correct in the playback part of the test.

Alternatively, I briefly considered using abstract factories to create my user and repository:

public void Register(string username, string password, string posterName)
{
    User user = userFactory.Create(username, password, Role.Poster);

    userRepository.Save(user);

    Poster poster = posterFactory.Create(posterName, username, user);

    posterRepository.Save(poster);
}

Then I could have written the tests like this:

[Test]
public void ReigsterShouldSaveNewUserAndPoster()
{
    string username = "freddy";
    string password = "fr0dd1";
    string posterName = "the works";

    using (mocks.Record())
    {
        User user = new User();
        Expect.Call(userFactory.Create(username, password, Role.Poster)).Return(user);
        userRepository.Save(user);

        Poster poster = new Poster();
        Expect.Call(posterFactory.Create(posterName, username, user)).Return(poster);
        posterRepository.Save(poster);
    }

    using (mocks.Playback())
    {
        posterLoginController.Register(username, password, posterName);
    }
}

It makes for a very clean test and it's forced me to separate out the concerns of creating new instances of User and Poster and the co-ordination of creating and saving them which is what the controller method 'Register' is really about. But in this case I just felt it was overkill to create abstract factories just to make my tests neat.

A bit more digging in the Rhino Mocks documentation revealed Callbacks. They allow you to create a delegate to do custom validation of the mocked method call. It effectively allows you to override Rhino Mocks' usual processing and plug in your own. The delegate has to have the same input parameters as the method it's validating and return a boolean; true if your test passes; false if it fails.

It's quite easy to use callbacks with the Linq Func delegate and Lambda expressions, although it took me a few attempts to work it all out. Here's the test:

[Test]
public void ReigsterShouldSaveNewUserAndPoster()
{
    string username = "freddy";
    string password = "fr0dd1";
    string posterName = "the works";

    User user = null;
    Poster poster = null;

    using (mocks.Record())
    {
        userRepository.Save(null);
        LastCall.Callback(new Func<User, bool>( u => { user = u; return true; }));

        posterRepository.Save(null);
        LastCall.Callback(new Func<Poster, bool>(p => { poster = p; return true; }));
    }

    using (mocks.Playback())
    {
        posterLoginController.Register(username, password, posterName);

        Assert.AreEqual(username, user.Name);
        Assert.AreEqual(password, user.Password);
        
        Assert.AreEqual(posterName, poster.Name);
        Assert.AreEqual(user, poster.PosterUsers[0].User);
    }
}

It's quite awkward syntax and it's not at all obvious how you'd do this from the Rhino Mocks API. I don't like the LastCall syntax either, but from this blog post it looks like that something that Oren is working on. There also Moq, which is a new mocking framework that fully leverages the new C# language features. Definitely something to keep an eye on.

Friday, December 07, 2007

RESTful Web Services by Leonard Richardson & Sam Ruby

restfull_web_services

I've had some experience creating web services. I've been involved in a couple of major SOA projects and even had a go at writing my own web service test tool: WsdlWorks, although I've sadly not had enough time to finish it off. So I've been soaked in SOAP (sorry) for quite some time, especially the Microsoft way of doing it. I've been aware of an alternative web service universe, REST, for a while too, but I dismissed it for lacking tooling and widespread adoption. However, in response to a comment on my 'REST as Betamax' blog, I went out and bought RESTful Web Services by Richardson & Ruby. Yes, it's one of those books, you know, the ones that totally kick you out of your rut. After reading it I've had my mind comprehensively changed. OK, so I'm of weak and easily lead by a good argument, but R&R's arguments for REST are pretty compelling. The core goodness is the linkability of RESTful services; the way that every resource (think object) has an individual URL which allows true hyper-linking of machine readable information. It's the restoration of HTTP to it's throne as the core of the World Wide Web. They show that SOAP is not only needless complexity, but actually a step backward.

As for lack of tooling, I was impressed by how nicely Ruby on Rails supports REST and it leads to the obvious conclusion that the new MVC Framework will make an excellent REST platform, especially with its powerful URL Routing engine. However, for the time being 99% of MS shops will not embrace REST no matter how powerful the arguments are for it. If they're keeping up to date they'll be writing RPC based web services with WCF, if not, they'll still be annotating stuff with [WebMethod]. But, if you hired me and gave me a free hand, I'd definitely go with REST.

Reading this book has just reinforced my view that Microsoft spends most of its time playing catchup with the rest of the software world. Thinking about it, none of the books that have had the most influence on the way I write software mention Microsoft tools.

Exposing generic methods to frameworks that don't understand generics.

How do you wrap a generic method like this:

public string WriteHtml<T>(string target, T source)

so that frameworks which don't understand generics can understand it, like this:

public string WriteHtml(string target, object source)

I've been writing a web application with Monorail. It's been loads of fun, and the speed at which I've been able to get a fully featured application up and running is awesome. I'll be blogging more about this soon, but today I just want to make a quick 'howto' about exposing generic APIs to frameworks that don't understand generics.

The problem is this: Monorail's default view engine is NVelocity, a port of the java Velocity template engine. I've found it pretty nice so far. To use it you just add stuff into a 'PropertyBag' in your controller and then reference in your view using the $parameterName syntax. You can invoke methods on your types and indeed Monorail comes with a nice collection of helpers to use within the view. I wanted to create a new helper to display a tree of objects (in this case a location hierarchy) in a combo box like this:

LocationSelect

So in my favorite TDD style I quickly got a class (HierarchicalSelect) up and running that took any object of type t with a property of IList<t> and spat out the HTML for my desired combo box. I then referenced my new helper in my template like this:

$hierarchicalSelect.WriteHtml("location.id", $location)

But it didn't work. Unhappily, NVelocity doesn't throw, it simply outputs your placeholder when it can't bind. I eventually worked out that the reason it didn't like WriteHtml() was because it's an generic method:

public string WriteHtml<T>(string target, T source)

What I needed was a way of exposing WriteHtml with the source argument typed as object rather than T. It turned out to be quite an effort. This is the obvious way:

public string WriteHtml(string target, object source)
{
    return WriteHtml(target, source);
}

But of course it doesn't work because T becomes System.Object rather than the actual type of 'source'.

Instead you have to use reflection to create a MethodInfo for WriteHtml<T> and then create a typed version before invoking it.

public string WriteHtml(string target, object source)
{
    Type sourceType = source.GetType();
    
    // get our unambiguous generic method
    MethodInfo writeHtmlMethod = typeof(HierarchicalSelect).GetMethod("WriteHtmlUnambiguous", 
        BindingFlags.NonPublic | BindingFlags.InvokeMethod | BindingFlags.Instance);

    // create a typed version
    MethodInfo typedWriteHtmlMethod = writeHtmlMethod.MakeGenericMethod(sourceType);
    
    // invoke it.
    return (string)typedWriteHtmlMethod.Invoke(this, new object[] { target, source });
}

// need to provide this so that GetMethod can unambiguously find it.
private string WriteHtmlUnambiguous<T>(string target, T source)
{
    return WriteHtml<T>(target, source, null);
}

Just for your entertainment. Here's the full code for HierarchicalSelect:

using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Reflection;
using System.Web;
using System.Web.UI;
using System.Linq;

namespace Hadlow.PropertyFinder.HtmlHelpers
{
    public class HierarchicalSelect
    {
        // NVelocity doesn't support generic methods :(
        // we have to take an object argument and then construct one of the generic methods first
        // before calling it.
        public string WriteHtml(string target, object source)
        {
            Type sourceType = source.GetType();
            
            // get our unambiguous generic method
            MethodInfo writeHtmlMethod = typeof(HierarchicalSelect).GetMethod("WriteHtmlUnambiguous", 
                BindingFlags.NonPublic | BindingFlags.InvokeMethod | BindingFlags.Instance);

            // create a typed version
            MethodInfo typedWriteHtmlMethod = writeHtmlMethod.MakeGenericMethod(sourceType);
            
            // invoke it.
            return (string)typedWriteHtmlMethod.Invoke(this, new object[] { target, source });
        }

        // need to provide this so that GetMethod can unambiguously find it.
        private string WriteHtmlUnambiguous<T>(string target, T source)
        {
            return WriteHtml<T>(target, source, null);
        }

        public string WriteHtml<T>(string target, T source)
        {
            return WriteHtml<T>(target, source, null);
        }

        public string WriteHtml<T>(string target, T source, IDictionary attributes)
        {
            string id = CreateHtmlId(target);
            string name = target;

            StringBuilder sb = new StringBuilder();
            StringWriter sbWriter = new StringWriter(sb);
            HtmlTextWriter writer = new HtmlTextWriter(sbWriter);

            writer.WriteBeginTag("select");
            writer.WriteAttribute("id", id);
            writer.WriteAttribute("name", name);
            writer.Write(" ");
            writer.Write(GetAttributes(attributes));
            writer.Write(HtmlTextWriter.TagRightChar);
            writer.WriteLine();

            if (source != null)
            {
                HierarchicalThing<T> root = new HierarchicalThing<T>(source);
                WriteOptions(writer, root, 0);
            }

            writer.WriteEndTag("select");

            return sbWriter.ToString();
        }

        private void WriteOptions<T>(
            HtmlTextWriter writer, 
            HierarchicalThing<T> item, 
            int level)
        {
            writer.WriteBeginTag("option");

            //if (item.IsSelected)
            //{
            //    writer.Write(" selected=\"selected\"");
            //}

            writer.WriteAttribute("value", HttpUtility.HtmlEncode(item.Id));
            writer.Write(HtmlTextWriter.TagRightChar);
            writer.Write(GetLevelString(level) + HttpUtility.HtmlEncode(item.ToString()));
            writer.WriteEndTag("option");
            writer.WriteLine();

            level++;
            foreach (HierarchicalThing<T> child in item.Children)
            {
                WriteOptions(writer, child, level);
            }
        }

        private string GetLevelString(int level)
        {
            StringBuilder sb = new StringBuilder();
            for (int i = 0; i < level; i++)
            {
                sb.Append("    ");
            }
            return sb.ToString();
        }

        private string CreateHtmlId(string name)
        {
            StringBuilder sb = new StringBuilder(name.Length);

            bool canUseUnderline = false;

            foreach (char c in name.ToCharArray())
            {
                switch (c)
                {
                    case '.':
                    case '[':
                    case ']':
                        if (canUseUnderline)
                        {
                            sb.Append('_');
                            canUseUnderline = false;
                        }
                        break;
                    default:
                        canUseUnderline = true;
                        sb.Append(c);
                        break;
                }

            }

            return sb.ToString();
        }

        private string GetAttributes(IDictionary attributes)
        {
            if (attributes == null || attributes.Count == 0) return string.Empty;

            StringBuilder contents = new StringBuilder();

            foreach (DictionaryEntry entry in attributes)
            {
                if (entry.Value == null || entry.Value.ToString() == string.Empty)
                {
                    contents.Append(entry.Key);
                }
                else
                {
                    contents.AppendFormat("{0}=\"{1}\"", entry.Key, entry.Value);
                }
                contents.Append(' ');
            }

            return contents.ToString();
        }

        /// <summary>
        /// Represents a hierarchial object with one property that has a type of 
        /// IList<sametype>
        /// </summary>
        /// <typeparam name="T"></typeparam>
        private class HierarchicalThing<T>
        {
            readonly T source;
            readonly PropertyInfo childrenProperty;
            readonly string id;

            public string Id
            {
                get { return id; }
            }

            public HierarchicalThing(T source)
            {
                this.source = source;
                Type sourceType = source.GetType();

                childrenProperty = FindChildProperty(sourceType);
                id = FindIdValue(source);

                if (childrenProperty == null)
                {
                    throw new ApplicationException("The source object must have a property that " +
                        "represents it's children. This property much have a type of IList<source type>. " +
                        "No such property was found in type: " + sourceType.Name);
                }
            }

            private string FindIdValue(T source)
            {
                return source.GetType().GetProperties().First(p =>
                    p.GetCustomAttributes(true).Any(a =>
                        a is Castle.ActiveRecord.PrimaryKeyAttribute)
                        ).GetValue(source, null).ToString();
            }

            private PropertyInfo FindChildProperty(Type sourceType)
            {
                return sourceType.GetProperties().First(p => p.PropertyType == typeof(IList<T>));
            }

            public HierarchicalThing<T>[] Children
            {
                get
                {
                    List<HierarchicalThing<T>> children = new List<HierarchicalThing<T>>();
                    IList<T> childValues = (IList<T>)childrenProperty.GetValue(source, null);
                    foreach (T childValue in childValues)
                    {
                        children.Add(new HierarchicalThing<T>(childValue));
                    }
                    return children.ToArray();
                }
            }

            public override string ToString()
            {
                return source.ToString();
            }
        }
    }
}

Sunday, November 25, 2007

Developer Developer Developer Day 6

I had a great time at DDD6 on Saturday. The nicest thing for me was getting to meet and talk to some of the most highly regarded people in the UK developer community like Dave Sussman, Dave Verwer, James Winters and Paul Lockwood. There were also a few familiar faces from Brighton there: Simon Harriyot (of Sussex Geek Dinners fame), Jane Dallaway from Madgex and Steve Mason from Cubeworks.

My talk 'Why do I need an Inversion of Control Container' got off to a bad start when I discovered that there was no way of attaching my Macbook's DVI plug to the projector. I should have known there would be some bad voodoo bringing a Mac into the heart of Microsoft, or thinking again, maybe I just should have brought the right plug. Duh and double duh! I was saved by Phil Winstanley who very kindly jumped in and lent me his laptop and helped me get up and running, thanks Phil! Ten minutes of transferring files and installing Testdriven.net later,everything worked and I was able to start. I must admit the panic meant that I was more than a bit flustered, I missed a whole load of good points I wanted to make and my tendency to talk too fast was probably attenuated somewhat.

Luckily it was an excellent crowd with some great questions. By the end of the talk I was really enjoying myself. Thanks to everyone who came up to talk afterwards, I really appreciated your feedback, especially the lady who pointed out that I hadn't said where you can get the Castle Windsor download from. Duh again! In order to rectify that oversight, you need to install the entire Castle Project which you can get here:

http://www.castleproject.org/castle/download.html

Here are the slides and demo code. The code requires the castle project and NUnit to work.

http://static.mikehadlow.com/Mike.IocDemo.zip

There should also be a video at some point; watch this space.

Tuesday, November 20, 2007

Hammett on Microsoft's MVC Framework

Here's an interesting post by Hammett, the leader of the Castle Project and writer of the Monorail MVC framework. He was invited up to Redmond to play with the MS MVC Framework. His attitude is very mature; he says that he won't bother with Monorail 2.0 if everything he wants from an MVC framework is supplied by MS. There's obviously a great deal of mutual respect going on. I wonder how long before Hammett's working for The Beast?

Installing Visual Studio 2008

Visual Studio 2008 is now available to MSDN subscribers. I downloaded it over night and have just started the install on Vista Ultimate. It's taking a long time and is currently stuck on installing Framework 3.5 with the progress bar at 0% for more than ten minutes.

... Later, it's done. But that was the longest install I've experienced for a long long time! And I have to restart Vista.

Thursday, November 15, 2007

Ian Cooper on MVC

Ian Cooper (of the London .NET User Group fame) has a very thoughtful series of posts on Model View Controller architecture:

Part 1

Part 2

Part 3

He's also doing a session on Monorail at DDD6. Which should be most interesting.

It's funny, but Microsoft's announcement of their MVC framework has raised more interest in Monorail that it could ever have on it's own. Previously it was a well regarded, but slightly obscure web development framework, but now it's got the 'copied by Microsoft' endorsement, everyone's talking about it.

Sunday, November 11, 2007

An important new .NET language: LOLCODE

I've just come across this fantastic new langauge: LOLCODE. I'm probably a little too old to ever be an expert at it because it's designed specifically for people who have grown up with texting; people who can succinctly express their ideas in abbreviated form. In true Code Rant 'love of fibonacci' fashion, here's a fib sample from John Lam's blog:

HAI
CAN HAS STDIO?
I HAS A FISH ITZ "Yummy"
VISIBLE FISH
VISIBLE "HAI WORLD!"

I HAS A FIB
I HAS A A ITZ 1
I HAS A B ITZ 0

GIMMEH FIB

IM IN YR LOOP
 VISIBLE B

 IZ FIB SMALR 1?
 YARLY
   GTFO
 NOWAI
   VISIBLE "NOWAI"
 KTHX

 I HAS A TEMP ITZ A UP B
 LOL A R B
 LOL B R TEMP

 NERFZ FIB!!
KTHX
KTHXBYE

No, it's not a joke (well not entirely), this is a real .NET language written with the DLR.

The MVC framework and new C# 3.0 features

You can't get hold of the MVC framework itself yet, but Scott Hanselman has posted the demo code used to demonstrate it at DevConnections and the PNPSummit. Browsing the routing demo code initially left me quite confused until I realized that it was using a couple of new C# 3.0 features: object initializers and anonymous types to initialize the routing engine.

using System;
using System.Web.Mvc;

namespace HelloWorld {
    public class Global : System.Web.HttpApplication {
        protected void Application_Start(object sender, EventArgs e) {

            // Custom "pretty" route
            RouteTable.Routes.Add(new Route {
                Url = "products/[category]",
                Defaults = new {
                    controller = "sample",
                    action = "ShowProducts",
                    category = "beverages"
                },
                RouteHandler = typeof(MvcRouteHandler)
            });

            // Default route
            RouteTable.Routes.Add(new Route {
                Url = "[controller]/[action]/[id]",
                Defaults = new { action = "Index", id = (string)null },
                RouteHandler = typeof(MvcRouteHandler)
            });
        }
    }
}

It initially looks like named arguments, but in fact it's creating a new anonymous type with the named properties. But how does the Route object parse its Defaults property? Does it have to use reflection to find the properties of the Defaults type, or am I missing something?

Friday, November 09, 2007

T-SQL Fibonacci and the power of 'With'

It's amazing how things can pass you by. I only found out about the recursive power of the T-SQL 'with' statement today. With it you can easily write a Fibonacci sequence generator:

with fib(a, b) as
(
 select 0, 1
 union all
 select b, a+b from fib where b < 100
)
select a from fib

Here's the output:

a
-----------
0
1
1
2
3
5
8
13
21
34
55
89

It really comes into its own when you've recursive structures such as a tree in your schema. Take this extremely typical example: a hierarchy of locations. Each record has a primary key 'id' and a foreign key 'parentId' that references another record in the same table.

locationTable

Here are some records:

locationRecords

Say I wanted to output all the locations with their level in the hierarchy. Easy:

with locationLevel(id, [name], [level]) as
(
 select id, [name], 0 from location where parentId is null
 union all
 select location.id, location.[name], locationLevel.[level]+1
 from location
 join locationLevel on location.parentId = locationLevel.id
)
select * from locationLevel

id          name                 level
----------- -------------------- -----------
0           World                0
1           UK                   1
2           France               1
5           Paris                2
6           Rouen                2
3           London               2
4           Brighton             2

Or when I search for a particular location, I want to find all its ancestors:

with locations(id, parentId, [name], targetName) as
(
 select id, parentId, [name], [name] from location
 union all
 select location.id, location.parentId, location.[name], locations.[targetName]
 from location
 join locations on location.id = locations.parentId
)
select id, [name] from locations where targetName = 'Rouen'

id          name
----------- --------------------
6           Rouen
2           France
0           World

Sunday, November 04, 2007

My new MacBook

My chum Richard just got back from a jaunt over the pond to New York. Before he left he very kindly asked if there was any shopping he could do for me. Not wanting to pass up the chance to get some cool tech at US prices, I asked him to pop into the Apple store and swag me a MacBook. For my US readers, you have to understand that as the dollar sinks into oblivion there's always an opportunity for US companies to make a few quid (bucks to you) by not slavishly following the exchange rate. Anyway Richard was true to his word and here it is...

macbook

It's the bottom of the range Apple laptop, but it's still considerably more powerful than my two year old dell desktop. Richard picked it up for almost exactly £600 which is £100 less than I'd pay for one in the UK. Not an incredible saving, but not at all bad either.

I love Apple hardware, they can actually do design; unlike every other PC manufacturer. Of course, as a windows software developer, I could never justify buying one no matter how much I like OSX, but now that they run on Intel processors and you can install windows it's a no brainer. I think a lot of other people share the same view because Apple's sales have gone through the roof. I would even consider buying a mac pro next time I need a desktop, although I was quite inspired to build my own after reading about Jeff Altwood building Scott Hanselman's from scratch.

I really like OSX, its eye-candy is so much better than Vista's, that there's really no competition. But I must have Vista, so after a quick play with Leopard, I fired up boot camp and got to work. Like most Apple stuff, boot camp is very nicely thought out and easy to use. You simply decide how much space you want to give your windows partition and click 'go' (or whatever); it partitions your hard drive, asks you to insert your Vista DVD and reboots into the Vista installer. The install went without any problems apart from once, when it came up from a reboot and stuck on a blank screen with the fan going full blast. I thought that was the end of it, but after doing a hard reboot, it came back to life into the last section of the Vista install. After Vista comes up, you insert the OSX DVD and it automatically installs all the drivers for the MacBook's hardware. The driver support is pretty impressive. Everything works: The keyboard's hardware control keys, the trackpad with it's double-touch scroll feature, the iSight camera and even the little remote control. I especially like the double touch scrolling on the touch pad. You just drag two fingers across and it acts just like It's a scroll wheel. Very intuitive. You right click by holding two fingers on the pad while clicking, which isn't quite as nice as having a button and requires too much hand contortion for my liking. All together it's a  pretty compelling package, especially if, like me, you've got an MSDN subscription and don't have to pay for a copy of Vista.

Now of course young Richard didn't go all the way to New York and not get a shiny new gadget for himself. Oh no. He returned to blighty the proud owner of an iPhone. Since they're not on sale in the UK until the end of next week, it was the first time I've seen one in the flesh. I was very impressed. The UI is superb; easily the best I've seen on a mobile device. The web browser is really cool. It's actually useable, unlike every other mobile device I've seen. My son has a PSP, which has wifi and a browser, but it's so hard to use that apart from one brief experiment I've never done any surfing with it. The iPhone's browser is, if anything, maybe more intuitive to use than a desktop browser. Being able to scroll around the page with your fingers is really nice and the screen is just big enough that you can a see a decent amount of text at once. I want one.

Friday, November 02, 2007

Inversion of Control, Unit Tests and Mocks

This is the second part of an on-going series about Inversion of Control. The first was:

What is Inversion of Control?

And I plan to do a few more digging deeper into the wonderful world of IoC. Hmm, nice.

OK, so now I want to talk about how IoC can help you to test your classes / components. As an aside, I'm really not sure whether to say 'class' or 'component' these days. We are essentially talking about individual classes, but by leveraging IoC you can make your classes more component like. I'll try and expand on this train of thought in one of my future posts. But, getting back to the point, I want to show how IoC is almost essential if you are going to do meaningful unit testing and how you can use Mock objects to completely isolate the class under test.

Do you remember this code from the last post, a little reporter class that gets some reports from a ReportBuilder and then sends them with a ReportSender:

public class Reporter
{
    public void Send()
    {
        ReportBuilder reportBuilder = new ReportBuilder();
        List reports = reportBuilder.GetReports();

        ReportSender reportSender = new ReportSender();

        // send by email
        reportSender.Send(reports, ReportSendType.Email);

        // send by SMS
        reportSender.Send(reports, ReportSendType.Sms);
    }
}

Now what if we want to write a unit test for Reporter? How about this:

[Test]
public void OReporterTest()
{
    Reporter reporter = new Reporter();
    reporter.Send();
}

What happens when we run this test? In my little demo application it outputs some stuff to the console. In a real world scenario the ReportBuilder class might get it's list of reports from a relational database via a data access layer. The data access layer might need configuration. The ReportSender might rely on an email library and an SMS library. They might also need configuration. The logging might rely on a logging library, also configurable, it might write to the machine's application log. There might be helper functions from other places in our application.

If we run this test, unless we do some very sophisticated work hooking up email and SMS clients and poking into in the application log we are probably going to rely on manually checking our email in-box, our mobile phone's text messages and visually inspecting the application log to see if our little Reporter class has done its job. All that work just for a class method five lines long.

We might do this once or twice while we're writing Reporter, but we're unlikely to bother every time we change some other part of the application. But say we change something in the data access layer, who's to say that won't break Reporter, or how about when some other class wants to use ReportSender, but in a slightly different way that requires a change that breaks Reporter? We wont find out that Reporter is broken until some time later in System testing, and that's if we're lucky.

And there's more craziness. When we test Reporter, we're not just testing reporter, we're testing the data access layer, the relational database, the email library, the SMS library, the configuration, the logging... the list goes on. What if it doesn't work? What do we do? That's right, we fire up the debugger and spend the next half an hour stepping line by line through our application scratching our head wondering why the hell it doesn't work.

All for five lines of code.

Now remember our other reporter that used Inversion of Control. It took a ReportBuilder, a ReportSender and a ReportSendLogger (yuck, did I really make up that name) in its contructor:

[Test]
public void ReporterTestWithLogging()
{
    IReportBuilder reportBuilder = new ReportBuilder();
    ILogger logger = new Logger();

    // send by email
    IReportSender emailReportSender = new ReportSendLogger(new EmailReportSender(), logger);
    Reporter reporter = new Reporter(reportBuilder, emailReportSender);
    reporter.Send();

    // send by SMS
    IReportSender smsReportSender = new ReportSendLogger(new SmsReportSender(), logger);
    reporter = new Reporter(reportBuilder, smsReportSender);
    reporter.Send();
}

Now in a real world scenario, the way this test is currently written we have exactly the same problems as our simpler non-IoC-reporter I was complaining about above. If we run the test we're testing a large chunk of our application and there's no way for the test itself to know if it's passed or not; we'd have to manually check it.

This is where mock objects come to the rescue. I've written about them before here and here. They are basically replacement instances of our dependencies that can tell us what has happened to them. When I first started to do unit testing I used to write my own, but in the last couple of years I've become a convert to using a mock object library, my favorite now is Rhino Mocks. Here's the same test as above but using mock objects:

[Test]
public void ReporterTestWithMocks()
{
    // create mock objects
    IReportBuilder reportBuilder = mocks.CreateMock<IReportBuilder>();
    IReportSender reportSender = mocks.CreateMock<IReportSender>();

    // create the reports
    Report report1 = new Report("Report 1");
    Report report2 = new Report("Report 2");
    List<Report> reports = new List<Report>();
    reports.Add(report1);
    reports.Add(report2);

    // record expectations
    Expect.Call(reportBuilder.GetReports()).Return(reports);
    reportSender.Send(report1);
    reportSender.Send(report2);

    // run the test
    mocks.ReplayAll();

    Reporter reporter = new Reporter(reportBuilder, reportSender);
    reporter.Send();
    
    mocks.VerifyAll();
}

First we create mock versions of IReportBuilder and IReportSender. 'mocks' is an instance of the Rhino mock's MockRepository. Then we create some reports for our IReportBuilder to return. The next three lines of code are the really interesting ones. We record what we expect to happen to our reportBuilder and reportSender inside our Reporter instance. When we use our mock objects before the call to mocks.ReplayAll(), the mock object framework simply records what happened. Last of all we create our Reporter and call the Send() method. When we call VerifyAll(), Everything that happened to the mock objects after ReplayAll() is compared to what happened before. If the before and after series of events are different an exception is thrown which causes the test to fail.

The really cool thing about this test is that it only tests Reporter. Nothing else. If the test fails the reason will be obvious. The test is entirely automated. We can run it with hundreds; thousands; of other tests quickly enough that there is no overhead to execute all our unit tests frequently. certainly quickly enough to make sure we never check in any code that causes any of the tests to fail.

If you adopt Test Driven Development, you'll notice all kinds of unexpected side effects. One that I didn't expect was that I've almost stopped using the debugger.

Another really interesting side effect is that we don't actually need concrete versions of IReportBuilder or IReportSender to run this test. We can write Reporter before we write our ReportBuilder or ReportWriter. This is counter to the usual bottom-up way of writing applications where you have to have your lower level code in place before you can write the high level stuff. It lets you adopt a top down style; thinking about high level concerns first and then filling in the details.

In my next post on the amazing world of IoC, I'm going to talk about IoC patterns other than the constructor dependency-injection I've shown so far.

Tuesday, October 30, 2007

How big should a function be? Uncle Bob explains.

As I said recently, one of my favorite programming books is Agile Software Development by Robert C. Martin AKA Uncle Bob. The Object Mentor blog that he writes along with others is essential reading. Today he posted a subject very dear to my heart: How big should a function be? Now this topic has been covered many times, not least of all by Steve McConnell in Code Complete, but it's amazing that wherever I work the one constant seems to be huge classes and huge functions and lots and lots of duplicated code. What's with it guys? What's stopping you from creating that new class or breaking that huge function into smaller ones? If you're not sure about how to go about splitting large classes and functions into smaller ones you really should read Martin Fowler's Refactoring.

There you go, I've managed to cram references to all three of my favorite programming books into one rant.

OK, mea-maxima-culpa, I know you don't have to browse far through this blog to see some quite large functions, but I'd like to defend myself by saying that's not how I tend to write production code and when one is explaining a technique or a simplest-thing-that-could-possibly-work it makes more sense to present it as a single block of code. Dig dig, is my hole big enough now?

using Rhino.Mocks

As you might have realized from reading this blog, I'm a big fan of Test Driven Development (TDD). A core part of writing good tests is the use of mock objects. Usually these are instances of objects that implement the the interfaces that the class under test has dependencies on. When I started doing TDD I used to write my own mock objects and it took me a while to realize the power of a good mock object framework. When I first discovered NMock it was a major revelation and took the ease and power of my tests to a new level. NMock has a real limitation though; its use of string literals to describe methods when you set your expectations. Any framework that uses string literals to reference types or members looses most of the advantages of working with a statically typed language.

Recently there's been quite a buzz about a new mocking framework: Rhino Mocks. Its main advantage is that it uses real invocations of the mock object's methods to set up expectations for your tests. Here's a little example:

 

rhinomocks

 

It's a very natural style of writing expectations because they mirror what the class under test should be doing. You can also refactor to your heart's content safe in the knowledge that your tests will pick up any symbol name changes.

The guy who wrote Rhino Mocks, Oren Eini AKA Ayende Rahien is a bit of a phenomena. Subscribing to the feed of his blog is like drinking from the proverbial fire-hose; eight posts last Saturday alone. He's also an active contributor to the Castle Project which I'm very interested in at the moment. Respect!

Monday, October 29, 2007

What is Inversion of Control

Inversion of Control is one of the core principles of object oriented software design. I first came across it about five years ago from reading Robert Martin's wonderful book Agile Software Development. Uncle Bob calls it 'The Dependency Inversion Principle', but these days 'Inversion of Control' seems to be the preferred term. it's what Martin Fowler calls it in this article.

I like to think of Inversion of Control as the ghostly opposite to subroutines, but instead of making it possible to break out sub pieces of code from a larger piece (the literal meaning of 'sub-routine'), it enables you to break out the surrounding code; the framework that's left when all the little sub-routines are removed. Being able to break out that remaining skeleton and reuse it is a very powerful tool.

However, subroutines are built into pretty much every modern programming language but IoC is something you have to spin yourself using OO building blocks. In this post I'll show you how.

Even the most junior programmers are aware of the idea of splitting their code into separate functions or subroutines. Using functions gives us our most important tool to avoid repeating ourselves, because repeating code is very very bad.

Take this code as an example. In true pedagogical fashion it is overly simplified, but if you can imagine that where I've simply put Console.Writeline(something), there's actually the real code to do that thing. Anyway, you get the idea.

[Test]
public void InlineReporterTest()
{
  // create some reports
  List<Report> reports = new List<Report>();
  for (int i = 0; i < 3; i++)
  {
      reports.Add(new Report(string.Format("Report {0}", i)));
  }

  // send reports by Email
  foreach (Report report in reports)
  {
      // pretend to send an email here
      Console.WriteLine("Sending by email: {0}", report.Title);

      // pretend to log here
      Console.WriteLine("[Log Message] Sent Report: {0}", report.Title);
  }

  // send reports by SMS
  foreach (Report report in reports)
  {
      // pretend to send an SMS message here
      Console.WriteLine("Sending by SMS: {0}", report.Title);

      // pretend to log here
      Console.WriteLine("[Log Message] Sent Report: {0}", report.Title);
  }
}

You can see that there's plenty of repeated code here. The two foreach loops are practically identical. Any sane developer would factor out the common code into a subroutine. Here's a possibility:

[Test]
public void ProceduralReporterTest()
{
  // create some reports
  List<Report> reports = BuildReports();

  // send reports by Email
  SendReports(reports, ReportSendType.Email);

  // send reports by SMS
  SendReports(reports, ReportSendType.Sms);
}

private static List<Report> BuildReports()
{
  List<Report> reports = new List<Report>();
  for (int i = 0; i < 3; i++)
  {
      reports.Add(new Report(string.Format("Report {0}", i)));
  }
  return reports;
}

private static void SendReports(List<Report> reports, ReportSendType reportSendType)
{
  foreach (Report report in reports)
  {
      switch (reportSendType)
      {
          case ReportSendType.Sms:
              // pretend to send an SMS message here
              Console.WriteLine("Sending by SMS: {0}", report.Title);
              break;
          case ReportSendType.Email:
              // pretend to send an email here
              Console.WriteLine("Sending by email: {0}", report.Title);
              break;
      }
      // pretend to log here
      Console.WriteLine("[Log Message] Sent Report: {0}", report.Title);
  }
}

Now we only have one copy of the foreach loop and one copy of the logging code. We've made the foreach loop a little more complex by inserting a switch statement, but it's probably worth it to remove the significant amount of duplication we had before. Also, if there are other places in our program that need to send reports they can use that same SendReports subroutine. I've also factored out the creation of the reports list into a subroutine called BuildReports. What we are left with is a skeleton: some coordinating code that calls BuildReports and then passes the reports to SendReports twice. Once to send them as Emails and secondly to send them as SMS messages.

Let's take a little interlude now and talk about object orientation. In simple terms... no very simple terms, this is the idea that we can split out separate concerns into separate classes or components that know how to do one thing and one thing only. They carry their data around with them encapsulated from interference from the outside world. We're now going to refactor our example again, this time using classes:

[Test]
public void OOReporterTest()
{
  ReportBuilder reportBuilder = new ReportBuilder();
  List<Report> reports = reportBuilder.GetReports();

  ReportSender reportSender = new ReportSender();

  // send by email
  reportSender.Send(reports, ReportSendType.Email);

  // send by SMS
  reportSender.Send(reports, ReportSendType.Sms);
}

public class ReportBuilder
{
  public List<Report> GetReports()
  {
      List<Report> reports = new List<Report>();
      for (int i = 0; i < 3; i++)
      {
          reports.Add(new Report(string.Format("Report {0}", i)));
      }
      return reports;
  }
}

public class ReportSender
{
  public void Send(List<Report> reports, ReportSendType reportSendType)
  {
      foreach (Report report in reports)
      {
          switch (reportSendType)
          {
              case ReportSendType.Sms:
                  // pretend to send an SMS message here
                  Console.WriteLine("Sending by SMS: {0}", report.Title);
                  break;
              case ReportSendType.Email:
                  // pretend to send an email here
                  Console.WriteLine("Sending by email: {0}", report.Title);
                  break;
          }
          // pretend to log here
          Console.WriteLine("[Log Message] Sent Report: {0}", report.Title);
      }
  }
}

Now we've got two separate classes, one that's responsible for building reports and one that's responsible for sending them. In this simple example these classes have no state, so there's no benefit from encapsulation, but they can participate in inheritance hierarchies which would allow us to extend them without having to alter them, the famous open-closed principle.

But going back to the skeleton again; the client code; it is hard coded to take our ReportBuilder and call our ReportSender, once for emails and once for SMSs. Although we can reuse the ReportBuilder and the ReportSender we can't reuse the coordinating code. Another problem worth noting is that the ReportSender has an intimate knowledge of sending emails and SMS messages, if we wanted to add a third type of sender, we would have to add an extra member to the ReportSendType enumeration and alter the switch statement inside the ReportSender. This is where IoC comes in. With it we can factor out the calling code and remove the tight coupling between the ReportSender and the different sending methods.

The basic technique of IoC is to factor out the public contracts of our classes into interfaces. The public contracts being the public properties and methods of our classes that the outside world interacts with. Once we've factored the public contracts into interfaces we can make our client code rely on those interfaces rather than concrete instances. We can then 'inject' the concrete instances in the constructor of our coordinating class; this is known as Dependency Injection. In the example below we've factored the coordinating code into the Reporter class and then injected a concrete IReportBuilder and IReportSender.

[Test]
public void IoCReporterTest()
{
  IReportBuilder reportBuilder = new ReportBuilder();

  // send by email
  IReportSender emailReportSender = new EmailReportSender();
  Reporter reporter = new Reporter(reportBuilder, emailReportSender);
  reporter.Send();

  // send by SMS
  IReportSender smsReportSender = new SmsReportSender();
  reporter = new Reporter(reportBuilder, smsReportSender);
  reporter.Send();
}

public interface IReportBuilder
{
  List<Report> GetReports();
}

public interface IReportSender
{
  void Send(Report report);
}

public class EmailReportSender : IReportSender
{
  public void Send(Report report)
  {
      Console.WriteLine("Sending by email: {0}", report.Title);
  }
}

public class SmsReportSender : IReportSender
{
  public void Send(Report report)
  {
      Console.WriteLine("Sending by SMS: {0}", report.Title);
  }
}

public class Reporter
{
  IReportBuilder reportBuilder;
  IReportSender messageSender;

  public Reporter(IReportBuilder reportBuilder, IReportSender messageSender)
  {
      this.reportBuilder = reportBuilder;
      this.messageSender = messageSender;
  }

  public void Send()
  {
      List<Report> reports = reportBuilder.GetReports();

      foreach (Report report in reports)
      {
          messageSender.Send(report);
      }
  }
}

Notice how we can reuse the Reporter to send first emails then SMSs without having to specify a ReportSendType. The Reporter code itself is much simpler because we don't need the switch statement any more. If we wanted to add a third sending message, we would simply implement IReportSender a third time and inject it into ReportBuilder's constructor.

But, wait a minute, I've forgotten about the logging somewhere along the line! Never fear! Without having to recode Reporter I can use the Decorator pattern to create a logger that implements IReportSender and gets injected with another IReportSender in its constructor:

class ReportSendLogger : IReportSender
{
  IReportSender reportSender;
  ILogger logger;

  public ReportSendLogger(IReportSender reportSender, ILogger logger)
  {
      this.reportSender = reportSender;
      this.logger = logger;
  }

  public void Send(Report report)
  {
      reportSender.Send(report);
      logger.Write(string.Format("Sent report: {0}", report.Title));
  }
}

Now I can simply string a logger and an IReportSender together and I have logging again without having to change a thing in Reporter.

[Test]
public void ReporterTestWithLogging()
{
  IReportBuilder reportBuilder = new ReportBuilder();
  ILogger logger = new Logger();

  // send by email
  IReportSender emailReportSender = new ReportSendLogger(new EmailReportSender(), logger);
  Reporter reporter = new Reporter(reportBuilder, emailReportSender);
  reporter.Send();

  // send by SMS
  IReportSender smsReportSender = new ReportSendLogger(new SmsReportSender(), logger);
  reporter = new Reporter(reportBuilder, smsReportSender);
  reporter.Send();
}

So there we have it, the power of Inversion of Control. In my next post I'll show how IoC makes real unit testing possible: Inversion of Control, Unit Tests and Mocks.

Friday, October 26, 2007

Who uses this stuff?

I was ranting about my some of my favorite subjects to a colleague recently: Test Driven Development, Continuous Integration, Component Oriented Design, Transacted Source Control; and he asked, quite reasonably, if any of my clients had actually used any of these techniques. The sad answer is "very few", but it started me thinking. I've had 7 clients since I started doing .NET commercially in 2002, ranging from huge corporations to tiny four man web development shops. I thought it would be illustrative to summarize their use of tools and practices. So here we go.

When I first started this blog, my second post was a list of development essentials. For each organization I've listed the (very approximate, OK it's me guessing most of the time) budget and team size for the project and whether they used any of the essentials I mentioned in that post: Dedicated Build Server, Automated Deployment, Source Control, Test Driven Development, Tool Enforced Coding Standards and Bug Tracking Databases. Yeah, it's not the best list and it's mostly focused around my developer level outlook and things I've personally found useful. I'm also a bit of an agile fanboy and I could just as easily made a list of clients that follow agile practices, but that would have been a very short list of none. There are also plenty of analysis and project management tools and techniques, but that's for another time.

A large cable company (£100,000,000 - 100+ people)

Dedicated Build Server: yes

Automated Deployment: yes

Source Control: yes (source safe)

Test Driven Development: no

Tool Enforced Coding Standards: no

Bug Tracking Database: yes

This was a huge project. At the time they said it was the largest .NET development in the UK. We had a team from Microsoft on board and a lot of the development was off-shored. When I joined this project they had no centralized build or deployment process and it was in deep deep pain. It took a change in management and the lucky fact that they'd inadvertently employed a build expert or two without intending it, before a centralized build and deployment process was put in place. This allowed the project to hobble on for a while, but initial poor management and quality control meant that the code base was a monster and even surrounding it with the best processes and the cleverest people couldn't make it agile enough to be anything but a hindrance for the business. It's now being replaced with a third party product.

A tiny web development shop (£10,000 - 2 people)

Dedicated Build Server: yes

Automated Deployment: no

Source Control: yes (subversion)

Test Driven Development: yes

Tool Enforced Coding Standards: yes

Bug Tracking Database: no

These guys were on the ball. They were doing a lot of things right: centralized builds, decent source control (subversion) and actually doing TDD for once. The simple reason was that the lead technical guy was very good and really cared about following good practice.

One of Europe's largest software service companies (£1,000,000 - 10 people)

Dedicated Build Server: yes

Automated Deployment: yes

Source Control: yes (source safe)

Test Driven Development: yes

Tool Enforced Coding Standards: yes

Bug Tracking Database: yes

This was a large government contract. When I joined the project they were in a whole world of pain six months had passed with almost nothing of value being delivered. Maybe because of this they actually listened to my incoherent rantings and allowed me to re-architect the entire application as well as introduce TDD and have a big input on the build system.

One of the world's largest insurers (£100,000 - 4 people)

Dedicated Build Server: yes

Automated Deployment: yes

Source Control: yes (source safe)

Test Driven Development: yes

Tool Enforced Coding Standards: yes

Bug Tracking Database: yes

I was the single developer on this integration piece, so I used all my favorite tools and techniques. I was aided by having one of the best architects I've ever worked with on the team. Another very strong developer developed the build and deployment process. A very pleasant experience.

One of the UK's biggest banks (£1,000,000 - 10 people)

Dedicated Build Server: no

Automated Deployment: no

Source Control: yes (source safe)

Test Driven Development: no

Tool Enforced Coding Standards: no

Bug Tracking Database: yes

This was similar to the previous project, another integration piece, but almost at the opposite extreme in terms of their understanding of modern development practices. I left.

One of the UK's busiest web sites (?? - 1 person)

Dedicated Build Server: no

Automated Deployment: no

Source Control: yes (source safe)

Test Driven Development: no

Tool Enforced Coding Standards: no

Bug Tracking Database: no

These guys were pretty good in terms of software architecture, but had almost no process. It showed how far you can get with some clever people even if you're just hacking, but they were starting to feel some scale pain. They were working on implementing centralized build and deployment but hadn't considered TDD.

A small fast growing software company (£20,000 - 3 people)

Dedicated Build Server: no

Automated Deployment: no

Source Control: yes (source safe)

Test Driven Development: no

Tool Enforced Coding Standards: no

Bug Tracking Database: yes

These guys are very similar to the previous project. I have to watch what I say because I'm still working with them, but I think they'd agree that they're starting to feel a little scale pain and starting to talk about putting more process around what they do. On the positive side there are plenty of people there interested in writing good software and it will be interesting to see what practices they adopt as the company grows. I've ranted about TDD at one of our development meetings and got a mostly positive response.

 

OK, so what have I learnt from the last five years (I try not to think about my former life as a VB Mort :). Well, the same pattern seems to repeat itself. A successful small product, initially often written by one person will grow and attract more and more resources. As the project grows more people are hired and then it's a race between how quickly the team realizes they have to adopt larger scale techniques and how quickly the software decays. A lot depends on the quality of the initial product and how well the lead developer passes on his design vision and how consistent the code stays. But even with well architected code and good developers, once the team gets past a certain size, things like having centralized build and deployment become essential if there is to be any confidence that the software can be integrated and tested. Without continuous refactoring code decays and you can't refactor with confidence if you don't have unit tests. I think that TDD is the most important technique to emerge in enterprise software development, certainly in the ten years I've been working in the field. It provides a step change in any teams ability to build great software. It's still mostly unknown in the Microsoft development world, but I think it makes such a great improvement that it will inevitably see progressively wider adoption.

Of course the quality of the people on the team is the most important factor, but then good developers adopt good techniques so they tend to go together. I think it's important that everyone agrees about what needs to be done. It's always a terrible mistake to try and impose techniques from above. To a certain extent you can enforce centralized build and deployment, but insisting on TDD without proper buy-in is almost always counter-productive. Trying to enforce design practices is a waste of time; nobody can do good software design except good developers.

Of course you can make successful software without worrying about architecture, clean design or testing of any kind, so long as it has one essential property: it's small. By small I mean maybe something written by one person over a month or two. But as soon as the software grows over a certain scale, if you don't start to listen to the lessons of people who have built large scale software and suffered for it, you are going to feel pain.

Tuesday, October 23, 2007

Developer Developer Developer Day. I've been selected!

Wow, I got an email this morning from zi makki telling me that my talk on 'Why do I need an Inversion of Control Container?' has been selected for the 6th Developer Developer Developer Day. Thanks to everyone who voted for me! I now have only a couple of weeks to get the first draft of my talk together so that I can submit the slides for the 9th November.

Monday, October 22, 2007

The Hollywood Principle

As you might know if you read this blog, I've been getting very excited by Inversion of Control containers recently. One of the things I keep coming across is 'The Hollywood Principle':

"don't call us, we'll call you."

Basically the idea is that a class says what it does by implementing an interface and what it needs by requesting interfaces, then a framework decides when to create it and what concrete instances to give it.

Friday, October 19, 2007

Breaking a list into columns using custom iterators

I've been working on a web application that has lots of lists of items in columns. The existing code was pretty ugly; it repeated the same pattern over and over again, iterating through the list of whatever items while keeping a count that triggered the move over to the next column. Lots of error prone state variables and conditionals. In obedience to the programming god DRY, I decided to factor out the column building. It turned out to be a great application for the custom iterators that came with .NET 2.0.

First of all, here's the test that shows how it's going to work. Yes, yes, I know this isn't  a proper unit test, there are no assertions and it simply outputs the results to the console, but it serves its purpose here.

using System;
using System.Collections.Generic;
using NUnit.Framework;

namespace Mike.ColumnDemo
{
    [TestFixture]
    public class ColumnTests
    {
        [Test]
        public void BreakIntoColumns()
        {
            // create a list of strings
            List<string> lines = new List<string>();
            for (int i = 0; i < 20; i++)
            {
                lines.Add(string.Format("item {0}", i));
            }

            foreach (Column<string> column in ColumnBuilder.Get(3).ColumnsFrom(lines))
            {
                Console.WriteLine("\nColumn\n");
                foreach (string line in column)
                {
                    Console.WriteLine(line);
                }
            }
        }
    }
}

Isn't that much nicer? Do you like the line segment 'ColumnBuilder.Get(3).ColumnsFrom(lines)'? I've become a big fan of DSL-ish APIs, they're much more readable, even if you do need to do slightly more work up front to make them happen.

Here's the ColumnBuilder class. The nested class ColumnCounter actually does most of the work of working out the column boundaries. It yields a Column object for each column (of course). By the way, I love the Math class, it's well worth digging into even if you don't do a lot of maths in your applications.

using System;
using System.Collections.Generic;

namespace Mike.ColumnDemo
{
    public static class ColumnBuilder
    {
        // return a ColumnBuilder from Get() so that we can write this:
        // foreach(Column<Person> column in ColumnBuilder.Get(4).ColumnsFrom(People)) { ... }
        internal static ColumnCounter Get(int numberOfColumns)
        {
            return new ColumnCounter(numberOfColumns);
        }

        public class ColumnCounter
        {
            int numberOfColumns;

            public ColumnCounter(int numberOfColumns)
            {
                this.numberOfColumns = numberOfColumns;
            }

            // Break the items into the given number of columns
            internal IEnumerable<Column<T>> ColumnsFrom<T>(IList<T> items)
            {
                int itemsPerColumn = (int)Math.Ceiling((decimal)items.Count / (decimal)this.numberOfColumns);
                for (int i = 0; i < this.numberOfColumns; i++)
                {
                    yield return new Column<T>(items, i * itemsPerColumn, ((i + 1) * itemsPerColumn) - 1);
                }
            }
        }
    }
}

Finally, here's the column class. It simply iterates through the list of items and only yields the ones in the start-end range.

using System;
using System.Collections.Generic;

namespace Mike.ColumnDemo
{
    // represents a single column
    public class Column<T> : IEnumerable<T>
    {
        IEnumerable<T> items;
        int start;
        int end;

        public Column(IEnumerable<T> items, int start, int end)
        {
            this.items = items;
            this.start = start;
            this.end = end;
        }

        public IEnumerator<T> GetEnumerator()
        {
            int index = 0;
            foreach (T item in items)
            {
                if (index >= start && index <= end)
                {
                    yield return item;
                }
                index++;
            }
        }

        System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
        {
            return (System.Collections.IEnumerator)GetEnumerator();
        }
    }
}

Thursday, October 18, 2007

F# to join C# and VB.NET as a core .NET language

Functional programming is very cool and I've been following the development of F#, a functional language from Microsoft Research in Cambridge for the last year or so. A while back I had a go a writing Life in F# and really enjoyed it. I'm now slowly working my way through The Little Schema which is blowing my mind with recursion. Building up basic mathematical building blocks from Add1 and Subtract1 using recursion is great brain food. OK, so it's Scheme and not F#, but the functional principles still apply. But I digress.  Apparently, yesterday, Soma announced that F# will included as an official Microsoft language with full tool support from Visual Studio. It's already pretty mature, definitely good enough for some serious playing, so download it and get currying!

Tuesday, October 16, 2007

Installing Vista

I finally got around to installing the freebee Vista Ultimate I got at Mix last night. I started to sweat a bit when the installation got to the 'expanding files' bit and stuck at 27% for about 15 minuets. After googling, it looked like a few people's Vista installations had failed at the same point and I was just about roll everything back when my wife distracted me. Five minutes later I went back to the installation and it had started moving again. I wonder if the people who were posting about the installation freezing had waited long enough? My Dell dimension 8400 only gets a score of 2.4, mainly because of the low end video card, so I don't get all the wizzy aero graphics; maybe it's time I invested in some new hardware :)

Monday, October 15, 2007

A custom SSIS PipelineComponent

I've recently been doing a spot of data migration. It's not my favorite task mostly because it involves the painful manual task of mapping one set of tables and columns to another. Drudge work of the worst kind. I've been using SQL Server Integration Services (SSIS) which is the replacement for Data Transformation Services (DTS). Data migration is a job I've mostly managed to avoid in recent years, so I never really had a reason to play with DTS and so I can't comment on how much, if at all, SSIS improves on it. SSIS basically gives you a design surface that you can drag various data sources and destinations and then connect them up via a whole stack of tools such as 'Derived Column', 'Data Conversion' and 'Lookup'. It's designed to support a variety different data manipulation task, including data mining and it's certainly capable of handing most data migration requirements.

SSIS

However, one of my data migration tasks was to take a simple list of locations in the source database that looked a bit like this:

ID     Location_Name
------ --------------------------
1      All UK
2      -England
3      --South
4      ---South East
5      ----Bedfordshire
6      ----Berkshire
7      ----Buckinghamshire
8      ----East Sussex
9      ----Essex
10     ----Hampshire
11     ----Hertfordshire
12     ----Isle of Wight
13     ----Kent
14     ----London & M25
15     -----London
16     ------Central London
17     -------City of London
18     ------East London
19     ------North London
20     ------South London
21     ------West London

... and import it into a table that looks like this:

id          name                 parentId
----------- -------------------- -----------
0           All                  NULL
1           All UK               0
2           England              1
3           South                2
4           South East           3
5           Bedfordshire         4
6           Berkshire            4
7           Buckinghamshire      4
8           East Sussex          4
9           Essex                4
10          Hampshire            4
11          Hertfordshire        4
12          Isle of Wight        4
13          Kent                 4
14          London & M25         4
15          London               14
16          Central London       15
17          City of London       16
18          East London          15
19          North London         15
20          South London         15

Notice how a location hierarchy is represented by the number of hyphens before the location name in the source table, but is represented as a correctly normalized relationship in the destination table. In order to map these two tables I wanted to write some C# code. You can write VB script directly into a script component, but it wasn't really what I wanted. The other obvious alternative was to write my own custom SSIS pipeline component. A brief look at the MSDN documentation on this suggested that it should be quite easy. You just extend a base class PipelineComponent and override a few methods. But in true MSDN style the documentation had large gaps in it leaving out some essential information, eventually I discovered CalendarTransform by Grant Dickinson, also on MSDN which pointed me in the right direction. The SSIS extensibility API could be much easier to use. It's not impossible, but there's far too much digging around for column ids that really should be provided by the toolkit.

Anyway, for your amusement, here's my very simple pipeline component that takes the source table and pumps out the destination.

[ComVisible(false)]
[DtsPipelineComponent(DisplayName="Location Mapper", ComponentType = ComponentType.Transform)]
public class LocationMapper : PipelineComponent
{
    int parentIdColumnIndex;
    int locationDescriptionColumnIndex;

    int idColumnIndex;
    int locationNameColumnIndex;

    // this override provides the columns at design time to allow you to wire up you components
    // on the design surface.
    public override void ProvideComponentProperties()
    {
        base.ProvideComponentProperties();

        //Support resetting the component, this is straight out of CalendarTransform   
        this.RemoveAllInputsOutputsAndCustomProperties();
        this.ComponentMetaData.RuntimeConnectionCollection.RemoveAll();
        this.ComponentMetaData.UsesDispositions = false;
        this.ComponentMetaData.ValidateExternalMetadata = true;

        // Add the input collection.
        IDTSInput90 input = ComponentMetaData.InputCollection.New();
        input.Name = "Input";
        input.ExternalMetadataColumnCollection.RemoveAll();
        input.ExternalMetadataColumnCollection.IsUsed = false;

        // Add the output collection.
        IDTSOutput90 output = ComponentMetaData.OutputCollection.New();
        output.Name = "Output";
        output.ExclusionGroup = 0;
        output.SynchronousInputID = input.ID;
        output.ExternalMetadataColumnCollection.RemoveAll();
        output.ExternalMetadataColumnCollection.IsUsed = false;

        // add the output columns, this is the bit that the MSDN documentation doesn't tell you :P

        // parent id
        IDTSOutputColumn90 parentIdColumn = this.InsertOutputColumnAt(output.ID, 0, "ParentId", "Parent Id");
        parentIdColumn.SetDataTypeProperties(DataType.DT_I4, 0, 0, 0, 0);

        // location description
        IDTSOutputColumn90 locationDescriptionColumn = this.InsertOutputColumnAt(output.ID, 1, "LocationDescription", "Location Description");
        locationDescriptionColumn.SetDataTypeProperties(DataType.DT_WSTR, 500, 0, 0, 0);
    }

    // this runs first at runtime.
    public override void PreExecute()
    {
        // input columns, assumes that the correct columns have been mapped on the design surface.
        // a bit of a hack!
        IDTSInput90 input = ComponentMetaData.InputCollection[0];
        IDTSInputColumnCollection90 inputColumns = input.InputColumnCollection;

        IDTSInputColumn90 idColumn = inputColumns[0];
        if (idColumn == null)
        {
            throw new ApplicationException("id column is missing");
        }

        IDTSInputColumn90 locationNameColumn = inputColumns[1];
        if (locationNameColumn == null)
        {
            throw new ApplicationException("location name column is missing");
        }

        // this is the really wacky stuff, you have to discover the column indexes inside the buffer
        // using this convoluted syntax. I could never have worked this out for myself!!
        idColumnIndex = BufferManager.FindColumnByLineageID(input.Buffer, idColumn.LineageID);
        locationNameColumnIndex = BufferManager.FindColumnByLineageID(input.Buffer, locationNameColumn.LineageID);

        // output columns
        IDTSOutput90 output = ComponentMetaData.OutputCollection[0];
        IDTSOutputColumnCollection90 outputColumns = output.OutputColumnCollection;

        IDTSOutputColumn90 parentIdColumn = outputColumns[0];
        IDTSOutputColumn90 locationDescriptionColumn = outputColumns[1];

        // do the crazy column index lookup again, this time for the output columns.
        parentIdColumnIndex = BufferManager.FindColumnByLineageID(input.Buffer, parentIdColumn.LineageID);
        locationDescriptionColumnIndex = BufferManager.FindColumnByLineageID(input.Buffer, locationDescriptionColumn.LineageID);
    }

    // this is the bit that actually does all the work.
    public override void ProcessInput(int inputID, PipelineBuffer buffer)
    {
        base.ProcessInput(inputID, buffer);

        if (buffer != null)
        {
            if (!buffer.EndOfRowset)
            {
                // level is a little class that I wrote to manage the parent/child mapping.
                Level level = new Level();

                while (buffer.NextRow())
                {
                    short id = (short)buffer[idColumnIndex];
                    string locationName = (string)buffer[locationNameColumnIndex];

                    // location is another helper class that simply describes a row in the destination table
                    Location location = new Location(id, locationName);
                    level.Set(location);

                    buffer[parentIdColumnIndex] = location.ParentId;
                    buffer[locationDescriptionColumnIndex] = location.Name;
                    buffer[orderColumnIndex] = location.Order;
                }
            }
        }
    }
}

Friday, October 12, 2007

Vote For Me!

Would you like to see me humiliate myself in front of a technical audience at one of the UK’s biggest .NET developer events? If the answer is "nothing would make me happier", go to the following link and choose my session, ''Why do I need an Inversion of Control Container?" (search for ‘Mike Hadlow’) and 9 others.

http://www.developerday.co.uk/ddd/votesessions.asp

The event is Developer Developer Developer Day 6 at the Microsoft campus in Reading on Saturday 24th November. I’ve volunteered to talk about Inversion of Control containers for an hour. It should be fun so come along!

Tuesday, October 09, 2007

Microsoft's new MVC web development framework

I've just watched the funeral of Web Forms. On Scott Hanselman's blog is a video of ScottGu presenting an initial prototype of Microsoft's new MVC rails-a-like web development framework at the ALT.NET conference. Gone is the event driven, postback, impossible to test old Web Forms. I guess Microsoft would have had to have had it's collective head buried firmly in the sand not have been closely watching all the excitement surrounding rails and more recently MonoRail, both of which were heavily name checked by Scott during his presentation. The framework has some very neat stuff that heavily leverages many of the new language features of C# 3.0 such as lambda expressions and anonymous types. Scott was particularly keen to show off the rather nice URL processor that makes it very easy to do SEO. It still uses the aspx templating engine for the views, so it will support all the nice wysiwyg support as well as the AJAX update panel stuff. Also nice is the way that it supports IoC containers and dependency injection.

Someone in the audience asked how Microsoft will be positioning this vis-a-vis Web Forms. He said that it wasn't a replacement for Web Forms, which will still be supported, but the enterprise development message will be to use the new framework. I expect Web Forms will share a similar fate to the Dataset, gradually melting away as one by one us Morts get the message. Probably around 4 years.

Monday, October 08, 2007

Windsor Container Resources

Here a few Windsor Container resources that I've stumbled across recently. I'm planning to maintain this list as a reference for my potential presentation at DDD6.

Jeremy Jarrell of Digital Blasphemy does a good overview in his 'Windsor IoC Container in a Lunch Break'. The comments are worth reading too.

Alex Henderson aka Bitter Coder has a great tutorial in his Wiki.

Oren Eini has a very cool article showing the power of Windsor when working with generic interfaces on MSDN: Inversion of Control and Dependency Injection: Working with Windsor Container.

Thursday, October 04, 2007

My Developer Developer Developer Day session

I've just offered to present a session on Inversion of Control containers at the Developer Developer Developer Day at the Microsoft campus on the 24th November. It's titled 'Why do I need an Inversion of Control Container?' and I intend to present an expanded version of my recent post about the Castle Project's Windsor Container. I think there's enough material to easily fill an hour, especially if I do a hands-on coding session.

Although I'm an ex English teacher, spending two years on the JET program, and I've done plenty of presentations and workshops for my clients, this will be the first time I've presented at a developer event like this. It should be a lot of fun, I just hope my session gets enough votes to be put forward. So if you're reading this and thinking, "I'd really like to see Mike humiliate himself in front of a technical audience', don't fail to vote for me when registration opens for DDD :)

Tuesday, October 02, 2007

Ben on MonoRail

As you'd probably know from reading any of my recent posts, I've become very curious about the Castle Project recently. One of the it's elements is the MonoRail Rails-a-like web development toolkit that sits on top of ASP.NET. 'Ben' (I couldn't find his full name on his blog anywhere) provides a great critique of WebForms here and the pros and cons of MonoRail here. I recently ran through the getting started guide for MonoRail and it has really intrigued me, especially the separation of concerns that an MVC framework provides that allows you to finally be able to test your web application logic properly.

Why are you still hand coding your data access layer?

At last it seems that the dreaded DataSet is dead. There are many reasons why you should always think twice before choosing the DataSet as the core of your application architecture, I covered most of them a couple of years ago here. In my freelancing work I've found that none of my recent clients have used DataSets, preferring instead some kind of Domain Model or Active Record data access mechanism, with Active Record becoming by far the favorite. It's also worth noting that the terminology in most Microsoft shops calls the Active Record class a 'business object' or 'data object', almost nobody says 'Active Record'.

A core part of an Active Record based architecture is some kind of Data Access Layer that does Object Relational Mapping (ORM). Everyone writes their own one of these, and that's the main point of this post; you shouldn't need to do this. If you are like the majority of my clients, your application features thousands of lovingly hand crafted lines of code like this:

DataAccessLayer

These hand written data access layers are the commonest source of bugs in most business applications. There are several reasons for this, the most obvious being that there are hand coded string literals representing stored procedure names, stored procedure parameter names and column names. People do use an  enumeration instead of the string literal column names, mainly for performance reasons, but it doesn't stop the enumeration and the stored proc select statement's columns getting out of sync. There's also the overhead of matching up the SQL Server types to .net types and dealing with null values.  But the worst offense of all is the tedium and the waste of time. Writing and maintaining these Data Access Layers is the programmer's equivalent of Dante's inner ring of hell, and you don't have to do it.

If you're like me, you resent any kind of repetitive coding that the computer could do just as easily, but much much faster and more accurately. At some time in your career you've probably written a little program that queries the database schema and generates your Active Record classes and Data Access Layer. Yes, I've done this twice, once back in the days of Visual Basic client server systems and more recently for .NET. The second attempt got quite sophisticated, handling object graphs and change tracking, but I never really got it to the stage of a real Data Access Framework, one that could look after all my persistence needs. I used it to generate the bulk of the code and then hand coded the tricky bits, such as complex queries and relationships. I'm not the only one who's been  down this road, a whole army of very clever people, cleverer than you or me, have devoted large amounts of time to this problem, which is great because it means that you and me don't have to bother any more. These tools are now robust and mature enough that it's more risky to do it yourself than use one of them.

But how  do you choose which one to use? There are two basic approaches, code generators and runtime ORM engines. Code generators, like mine, are the easiest to create, so there are more of them out there. Runtime ORM engines are a much trickier engineering problem but they will probably win in the end because they're easier to use for the end developer. Amongst the code generators, the ones I hear of the most are various Code Smith templates like net tiers, LLBLgen by Frans Bouma who's also a very active participant in community discussions around ORM, Subsonic which is attempting to be a Rails for .net, and Microsoft's very own Guidance Automation Toolkits. All are well regarded and you probably wouldn't go too far wrong with choosing any of them.

Among the runtime ORM engines, I hear NHibernate mentioned more than anything else. Hibernate is huge in the Java world so the NHibernate port has plenty of real world experience to fall back on. It's been used in a number of large scale projects and is the core of the Castle project's ActiveRecord rails-a-like data access solution. I haven't used it in anger, but my few experiments with it have been quite fun.

I haven't mentioned the elephant in the room yet, that's LINQ to SQL coming with .net 3.5. Microsoft have taken a long time to join the ORM party. A couple of years ago there was much talk of ObjectSpaces a Hibernate style ORM tool that never saw the light of  day. LINQ is a very elegant attempt to integrate declarative query style syntax into imperative .net languages like C#. LINQ to SQL makes good use of it, especially, as you'd expect, with its query syntax. In other ways LINQ to SQL is very much a traditional ORM mapper along the lines of Hibernate, it features some code generation tools to create your Active Record objects, a runtime ORM mapper that creates SQL on the fly, identity management and lazy loading; all the features you'd expect. If you're starting a new project and you're prepared to risk using the beta of Visual Studio 2008 then I would choose LINQ to SQL in favor of any of the other alternatives, not least because it's what everyone will be using in a couple of years time.

Monday, September 24, 2007

The joys of System.Diagnostics.StackFrame

I see code like this an awful lot where a logging function or whatever is passed a string literal of the current method name:
public void TheOldMethodUnderTest()
{
    Log("TheOldMethodUnderTest");
}

private void Log(string methodName)
{
    Console.WriteLine("The name of the method was: {0}", methodName);
}
I've also seen a lot of code like this where the actual method name and the string literal have become out of sync, either because of refactoring, or more commonly, the cut-and-paste method of code reuse. Fortunately there's a really easy way of getting the current stack trace from the System.Diagnostics.StackFrame class. The code below does exactly the same thing as the code above, but the Log() function works out for itself what its caller's name is:
public void TheMethodUnderTest()
{
    Log();
}

private void Log()
{
    System.Diagnostics.StackFrame stackFrame = new System.Diagnostics.StackFrame(1, false);
    System.Reflection.MethodBase method = stackFrame.GetMethod();
    Console.WriteLine("The name of the method was: {0}", method.Name);
}
Doing this kind of thing is a legitimate requirment in many applications, but if you find yourself having to put a lot of boiler-plate code in every method because of cross cutting concerns such as logging, it's worth looking at some kind of AOP framework that provides dynamic proxies that you can intercept.

Thursday, September 13, 2007

MIX 07 UK

ScottGu

I had a great time in London over the last couple of days at MIX 07 UK. Most of the excitement was about Silverlight, but there was also a lot of coverage of VS2008. I've been reading a lot about Silverlight recently, but this was the first time I'd seen a good demonstration of it in use. It's obviously Microsoft's Flash killer and it fills a big gap in RIA application development where you'd currently have to use Flash. For .net developers like me it opens up some great opportunities, so it can only be good. I guess its success will primarily hinge on how well Microsoft drives adoption. Scott Guthrie explained that they only have to persuade about 15 of the top sites to use it and it will become ubiquitous. One thing that really struck me, that I hadn't properly grepped before was how core XAML is to the current MS development stack. I really liked the way you could use the Expression * tools to move XAML projects between designers and developers. Those tools (especially Blend) are yet another thing I need to play with.

I went to the second of the VS2008 sessions, but it was pretty much covering stuff I already knew, although it was nice to see all the LINQ to SQL stuff presented. One thing that was new was the Rails style scaffolding stuff. I don't remember what it was called, but MS have blatantly got their eyes on the Ruby camp. Only for the best I think.

I was also really looking forward to the 'Why IronRuby', and 'IronPython et al' sessions. Dynamic languages are generating some real excitement these days and was very keen to get some insight into how Microsoft sees them playing with .net. 'Why IronRuby' by Dave Verwer was a bit of a disappointment, not because Dave didn't do an excellent job introducing Ruby, but I was rather hoping for a more philosophical discussion about dynamic languages in the .net world and how well they play with the existing MS technology stack. I had to leave before the questions, so maybe there was some interesting discussions? I hoped to grab Dave for a chat later on, but I couldn't track him down. Michael Foord's (aka Fuzzyman) IronPython session was much more what I wanted. Michael spent a lot of time digging into the intricacies of building dynamic languages on top of the CLR, the DLR and some of the cool stuff you could do with IronPython and Silverlight. I'm kinda seeing what the Python and Ruby people are saying about the benefits of dynamism, but I'm still not quite ready to really embrace it yet. The opportunity will  come when the IronRuby implementation has been fully developed in a year or so, but I must stay, the DSL story from the Ruby camp is very compelling.

My favorite session was the 'A nice cup of tea and a sit down', a panel discussion with Scott Guthrie. As you probably know, Scott is the general manager of the .net development group and his blog is always my first stop to keep up with what's happening at MS. It's not everyday that you get to fire some questions at one of the core .net guys, so I really enjoyed myself.

I asked Scott about TDD at Microsoft and how they saw TDD fitting in with their products. I was especially keen to hear what he thought about the problems with mocking the BCL.

He told us about a new MVC framework that's in the pipeline for ASP.NET that sounded very intriguing and how they were consciously trying to keep it interoperable with open source tools. I was especially gratified to hear him mention Windsor and RhinoMocks.

It's a shame there wasn't more time, I really wanted to ask him about functional programming and LINQ and how explicitly MS are going to encourage more declarative programming styles in the BCL and documentation. Right now the message seems to be that LINQ is query and the functional declarative stuff is only mentioned if you're prepared to dig deep.