Humblecoder

I’m sure like many exploring Dependency Injection (DI) for the first time the biggest problem I had, was how to rid the world of those pesky little new statements.  The first application I tried to use DI with in anger was DirLinker.  I had no prior knowledge of any of the DI frameworks and started looking at patterns that would help me remove all the new statements.  Unfortunately, I hit up on service locator and the rest is a mess history.  I ended up with code the looks like the following inside DirLinker:

IFile aFile = ClassFactory.CreateInstance<IFile>();
aFile.SetFile(file);

There is a couple of things that bother me about this.  First is the static, I’m not a fan of statics at all, infact, I hate them.  Next is the SetFile call, for any meaningful work to be done with this class you need to pass in a filename and it should be a constructor argument.  In all fairness the SetFile problem is more to do with the limitations of my roll your own DI framework than anything else.

In spite of my niggling doubts, it worked so I left it alone.  Fast forward to last week when I came across a blog post entitled Service Locator is an Anti-Pattern.  This confirmed my initial discomfort and highlighted a problem I hadn’t thought of, namely that the class doesn’t advertise its dependencies.  Very bad for reuse.

Factories

As the blog post suggests a better way of doing this would have been to use factories so lets go over a couple of examples on how this could be achieved. 

Virtual Instance Methods

This is a method I would use when trying to a create a seam in legacy code to inject a dependency for unit testing.  To do this you create a virtual method on the class that needs to create the object. In the unit test inherit from the class under test and override the factory method to return your mock or stub.  For example:

public class FileConsumer
{
    protected virtual IFile GetFile(String filename)
    {
        return new FileImp(filename);
    }

    public void DoSomething()
    {
        IFile fileA = GetFile("test.txt");
        IFile fileB = GetFile("test2.txt");
        //uses files
    }
}

[TestFixture]
public class FileConsumerTests : FileConsumer
{
    protected override IFile GetFile(string filename)
    {
        return new FileMock(filename);
    }

    [Test]
    public void DosomethingTest()
    {
        //Code to perform test
    }
}

This is good because it’s clear to any maintainer what the dependency is and where it is coming from. 

Abstraction Factory Pattern

This would loudly and proudly advertise the dependency on the ability to create IFile objects.  It works by creating a class with methods solely responsible for creating IFile objects and then taking this as dependency for your class.  For example:

public interface IFileFactory
{
    IFile CreateFile(String filename);
}

public class FileFactory : IFileFactory
{
    public IFile CreateFile(string filename)
    {
        return new FileImp(filename);
    }
}

public class FileConsumer
{
    IFileFactory _fileFactory;

    public FileConsumer(IFileFactory fileFactory)
    {

        _fileFactory = fileFactory;
    }

    public void DoSomething()
    {
        IFile fileA = _fileFactory.CreateFile("test.txt");
        IFile fileB = _fileFactory.CreateFile("test.txt");
        //uses files
    }
}

This is the pattern recommended by the blog post and it covers all the things I don’t like about Service Locator.  But I think we could take advantage of some of C#’s language features and the fact we are creating objects via a container to achieve something similar but with less code.

Delegate Factories

Before I go into this a little disclaimer, this is totally and utterly inspired by Autofac’s wonderful generated delegate factory functionality, you can read more about it here.

OK, honesty out of way lets look at what the container knows and what it does.  The DI container knows what Interfaces should map on to what concrete types and it has the ability to resolve constructor arguments for types that it knows about.  This is just a generic version of a factory so it makes sense to take advantage of it.

One way would be to use Func<TResult>() (and its friends).  If the class being constructed requires a constructor parameter of Func<TResult> where TResult is the required interface, the container could generate an appropriate delegate at runtime and pass it in.  For example:

   public class FileConsumer
   {
       Func<IFile> _fileFactory;

       public FileConsumer(Func<IFile> fileFactory)
       {

           _fileFactory = fileFactory;
       }

       public void DoSomething()
       {
           IFile fileA = _fileFactory();
           IFile fileB = _fileFactory();

           //uses fileA
       }
   }

This would require a slight modification to the container and could be extended by the use of Func<T, Tn, TResult>(T value, Tn valuen).  The container would match the parameters to a suitable constructor. 

This is quite a powerful and time saving concept but I don’t like the vagueness of Func<TResult>(), it’s not as expressive as it could be.  So the method I have chosen for DirLinker is to use strongly typed delegates and generate them at runtime from the container.   It works in the same manner only you declare the delegate up front and pass them in as an argument.  This, also, requires a bit of configuration but the pay off is worth it.  I have started work on the implementation for DirLinker so this example is taken directly from the unit tests and can be found here. 

interface ITestClassWithDelegateFactory { ITestClassFactory Factory { get; set; } }
delegate ITestClass ITestClassFactory();

class TestClassWithDelegateFactory : ITestClassWithDelegateFactory
{
     public ITestClassFactory Factory { get; set; }
     public TestClassWithDelegateFactory(ITestClassFactory delegateFactory)
     {
         Factory = delegateFactory;
     }
}

[Test]
public void ManufactureType_Type_delegate_factory_manufactures_correct_type()
{
	IClassFactory testClassFactory = new ClassFactory();

	testClassFactory.RegisterType<ITestClass, TestClass>()
                .WithFactory<ITestClassFactory>();

	testClassFactory.RegisterType<ITestClassWithDelegateFactory, TestClassWithDelegateFactory>();

 	ITestClassWithDelegateFactory manufacturedType =
			testClassFactory.ManufactureType<ITestClassWithDelegateFactory>();
	ITestClass instance = manufacturedType.Factory();

	Assert.IsInstanceOf(typeof(TestClass), instance);

}

Admittedly there is quite a lot going on here but the main points are we registered a type and factory in a strongly typed manner with the container.  Then pull it out and use it to create an instance of a different type.  I am going to cover this in quite some detail along with the how to create strongly typed delegates at runtime in my next post.

As a hormone ravaged teenage, I squirmed uncomfortably as parents, teachers and community health practitioners imparted the words of wisdom “Don’t be a fool, wrap your tool”.  So it is fitting, I’m equally as squeamish when coming across the same advice as an adult. 

What am I talking about?  Creating wrappers for anything at all on the boundaries of your code for the purpose of unit testing.  I’ve been struggling to think of a succinct way to explain this so I decided to go through a worked example.

Consider the following code:

   public class CommandReceiver
    {

        public void WaitForMessage()
        {
            using (NamedPipeServerStream pipeServer = new NamedPipeServerStream("testpipe", PipeDirection.In))
            {

                // Wait for a client to connect
                Trace.Write("Waiting for client connection...");
                pipeServer.WaitForConnection();

                Trace.WriteLine("Client connected.");
                try
                {
                    // Read user input and send that to the client process.
                    using (StreamReader sr = new StreamReader(pipeServer))
                    {
                        String command = sr.ReadLine();
                        DispatchCommand(command);
                    }
                }
                catch (IOException e)
                {
                    Trace.WriteLine("ERROR: {0}", e.Message);
                }
            }
        }

        private void DispatchCommand(String command)
        {
            //Knows how to deal with messages.
        }
   }

Looking at this code there is a number of problems but lets focus on the unit testing problems.  It is impossible to unit test this code because it creates a real pipe server and waits on a blocking call before continuing.  This means to test this code we need to create a pipe client, connect and all this would have to be threaded because of the blocking call.  Of course, this would make a good integration test because it tests that the pipe is connectable and receives a string message.

Before we set about making this code more unit test friendly, lets look at what we are trying to unit test.  We are trying to test that the WaitForMessage method can receive a string and pass it on.  For us to do this we need to abstract the pipe and stream out.  Also, while we are there lets remove the DispatchCommand method since it violates SRP and it would be more testable on its own.  So lets take a second stab at the code.

public interface INamedPipeServer : IDisposable
{
    void WaitForConnection();
}

public class ManagedNamedPipeServer : INamedPipeServer
{
    private NamedPipeServerStream _pipeServer;

    public ManagedNamedPipeServer(String name, PipeDirection pipeDir)
    {
        _pipeServer = new NamedPipeServerStream(name, pipeDir);
    }

    public void WaitForConnection()
    {
        _pipeServer.WaitForConnection();
    }

    public void Dispose()
    {
        _pipeServer.Dispose();
    }

}

public interface IStreamReader: IDisposable
{
    String ReadLine();
}

public class ManagedStreamReader : StreamReader, IStreamReader
{
    public ManagedStreamReader(Stream stream) : base(stream)
    {}
}

public class CommandReceiver : ICommandReceiver
{
    INamedPipeServer _NamedPipeServer;
    ICommandDispatcher _CommandDispatcher;

    public CommandReceiver(INamedPipeServer pipeServer, ICommandDispatcher dispatch)
    {
        _NamedPipeServer = pipeServer;
        _CommandDispatcher = dispatch;
    }

    protected virtual IStreamReader GetStreamReader()
    {
        //Code to create a stream reader from the pipe
    }

    public void WaitForMessage()
    {
        // Wait for a client to connect
        Trace.Write("Waiting for client connection...");
        _NamedPipeServer.WaitForConnection();

        Trace.WriteLine("Client connected.");
        try
        {
            // Read user input and send that to the client process.
            using (var sr = GetStreamReader())
            {
                String command = sr.ReadLine();
                _CommandDispatcher.DispatchCommand(command);
            }
        }
        catch (IOException e)
        {
            Trace.WriteLine("ERROR: {0}", e.Message);
        }
    }
}

This is where my uncomfortable squirm returns because my “keep-it-simple” sense is tingling.  I’ve just taken a fairly simple class that was around 30 lines and turned it into 75 lines of complicated OOP code.  I would hope that anyone reading this can follow it but in a real project this will probably be split over several files and the two styles of wrapping (because NamedPipeServerStream is sealed) can add significant cognitive burden to understanding what is going on.

What benefits does this bring?  It allows us to unit test that we read a string and pass it on to a dispatcher.  But I would go out on a limb and say that this is least likely of all the code in that class to go wrong.  The real problem area will be in connecting the pipe and reading from it.  We can make assumptions about failure conditions from the docs but as we all know docs != reality. 

Is the abstraction here a benefit? In my opinion, not to the extreme level we have. The abstraction at ICommandReceiver will allow us to swap out how the application does IPC calls making it flexible in the future and, as I eluded to above, the unit test ‘coverageability’ is of lower value in this instance. 

My point in all this rambling nonsense?  In an ideal world we would have both sets of tests and the unit tests would cover all the error conditions but in the real world we only have a finite and, usually, short amount of time with pressure from project managers and deadlines.  So we have to look at what will bring us the most value and focus on that.  I believe at application boundaries like this one we should focus our attention on writing integration tests because it will bring us more value in the long run.  I would not shun unit testing entirely and in this example the ICommandDisptacher and other supporting classes would have a full suite of unit tests.

As a side note, my final version of the code would be a mid way point between the two listings.

I posted up the Directory Linker CodePlex project a little over 2 weeks ago and it has clocked up a reasonable 103 downloads in that time.  Last night I was glancing over where the traffic came from on the stats page and a lot of referrals have come from this review site, they state that:

The app works on Windows 7 and Vista only, support for Windows XP is planned in the near future.

This is incorrect and maybe I should have been more clear that Directory Linker will not support XP unless there is some overwhelming outcry for it or someone else does it.

Why?

First for selfish reasons, I don’t use a machine that has XP installed anymore, furthermore, I would encourage anyone on XP to upgrade for security if nothing else.  At the end of the day, I created this tool for my use and I don’t use XP.

The second reason is technical, without going to far in, symbolic directory links have been in NTFS for quite a while but only got ‘first class’ support by the OS recently.  Previous to Vista they were called Junction Points and had all kinds of weird side effects (more details here).  Also, the API calls used by Directory Linker to create the symobolic link is only supported by Vista and above.  It is technically possible to do the same for XP and it is not difficult but Directory Linker simply doesn’t do it.

I hope that clears things up, also if you find a download link for Director Linker that links to SoftSea in anyway please do not use it.  If you’re not a programmer it’s safe to stop reading now.

Fork You!

Well, I rather you didn’t.  I will happily accept any patch that is submitted to the project to do this.  To help you out here is a quick overview on how to do  it.

1. Locate FolderImp.cs and make the CreateLinkToFolderAt method virtual.
2. Create a new class that inherits from FolderImp and name it to show it is for XP.
3. Override only the CreateLinkToFolderAt and add the code to create a link in XP (Tip: start at DeviceIoControl).
4. Open Program.cs and go to the FillIoCContainer method and add code to conditional select the correct IFolder class when the application starts.

Quite simple really.

I am a fairly junior developer with just short of 3 years commercial experience in an office where the average experience level is 10+ years.  I love to read and learn about software development nearly as much as I love to actually write code and I have lots of ideas and opinions on code, style, design, etc,. For several years I’ve been trundling along nervously squeaking up to more senior staff members and often being shot down or told “that’s a great idea but we don’t have time/resource/it doesn’t fit in with what we do.”.  This has all lead to a mixture of frustration and self doubt as to if what I am learning is right or even unprofessional at times. Enter NxtGenUG Manchester.

What Is It?

NxtGenUG is a .NET users group that meets once a month.  The meetings generally consist of a short member ‘nugget’ (~10 minutes), a longer ‘featured’ speaker, eating pizza, giving away swag and most importantly, yes even more important than pizza, socialising with other interested developers. 

The nugget is a short talk from a member about something they are learning about or know about.  A wide range of topics have been covered from printing in SilverLight to the Pomodoro technique.  While ten minutes is not a very long time it does give a wonderful little introduction to a topic and provide a talking point over pizza.

The main attraction, as it were, is a full hour long presentation on a topic.  There has been some quite interesting presentation from varying quality presenters.  The highlights including a crash course in TypeMock and a tour of PLINQ.  Even if the topics are not things you work with directly, it’s a great way to learn about new things and keep up to date.

Why Has It Brought Out The Best In Me?

While all the presentations and information have been great what has really helped me is the social side of things.  I do a lot of reading and learning and I form opinions based around it.  What I really want is place to voice these opinions and get a good debate going with like minded people.  I am a true believer in “Strong opinions, weakly held” and from this point of view the conversation I’ve had while at the user group have really challenged me and changed my thinking, for the better.

Don’t get me wrong people at work are interested as well, but it’s nice to see and hear other people in the community talk about their experiences and the challenges they’ve faced introducing Agile, TDD, etc,.  This type of information sharing is invaluable and really allows you to see things from a completely different point of view, than chatting with colleagues would normally do.   For example,  I’ve gotten several invaluable tips for guerrilla tactics to introduce things in a resistant culture.

More importantly for me, it’s given me the confidence to fight my corner more in the office and really push forward.  Listening to other people talk and talking to speakers, I’ve realised that you don’t have to be right all of the time, just open minded and have confidence in your own knowledge.  I have taken this to an extreme level by presenting my own member nuggets at the user group, starting this blog and running several training sessions/OpenSpaces discussions at work.  Now I’m looking toward getting on the amateur speaking circuit.

In summary, just being around other people from different work environments and cultures to mine has really driven me and given me the confidence to push forward with change.  User groups aren’t just about the presentations, they are about the community and improving yourself and workplace through other peoples experiences.   For me this has been a massive success so thank you Steve, Andy, John, Joel and countless others.

The Problem

In my haste to leave the office I will often click shutdown on my laptop then shut the lid.  The laptop will probably shutdown successfully 1 in 5 times for two main reasons:

• The laptop will hibernate or suspend before it can shutdown
• Windows will be waiting on applications to close that are waiting for save dialogs etc to be dismissed.

I have two possible solutions, create an application that modifies this behaviour or be more patient.

Software Solution

Lets face it I was never going to change so I started to put some thought in to an application to modify the behaviour and ShutdownEx was born.  Not long ago I created a shutdown timer app so I had all the interop calls required to shutdown the machine and a quick search of MSDN turned up the WM_POWERBROADCAST message for intercepting suspends.

Armed with this I created a quick plan and requirement list in my mind:

• The app should be standalone, no install, no dependencies, just work.
• A simple Winform application, the main form would have a couple of options and a button to shutdown the system.
• When shutdown was invoked from it, it would listen for the WM_POWERBROADCAST message and stop it from suspending.
• If the system didn’t shutdown within a set time out, it would recall shutdown with the force shutdown flag.

Before even writing a line of code I spotted a major flaw with this.  In order to wait for the timeout the application would have to ignore the first WM_QUERYENDSESSION message that came in, meaning it would always timeout and we would be forced to shutdown when the second message came.  This introduces two massive problems

1. We’re holding up shutdown meaning it will always take longer when we’re running.  Not cool!
2. When we have been forced to shutdown, a small window of opportunity will open up for the suspend event to come in.

The second problem is a deal breaker, the whole point of the application is to stop the suspend from taking place during shutdown.

Revised Solution and Compromising My Own Requirements

I came up with two more solutions.  The first, and ultimately still flawed approach, keeping to the simple WinForm one executable approach.  I could enumerate all process running in my session and keep polling till everything has closed then close gracefully myself allowing the shutdown to continue.  While the majority of the time, I feel, it wouldn’t send out the force shutdown there is still a sizeable (measured in seconds) gap when the suspend event could still take place.  Breaking one of the main requirements.

The second approach is to ditch the requirement of a standalone WinForm applications and go to a Windows Service.  The Windows service will not need to end when the session does so I don’t have to worry holding up the shutdow.  This, also, means I have a longer window of opportunity to block the suspend message.  With the theory being when I go down, Windows will be taking its subsystems down and the message will be ignored anyway.  The big downsides are I now have to install the service from an MSI and I’ve just added a new and very complex layer to my simple application.

Complexity from a service point of a view because I need to create a hidden Window so I can listen for Windows Messages. I will still need a UI for the user to shutdown with so I need to have comms between the service and UI, a named pipe more than likely.  Throw in any permissions problems I might have when running as system and it’s now quite far from the quick app with lots of existing code samples I first thought of.

The worst part of it is, I’ve started it now and I have to finish to prove it can be done.  Although, I will never be happy with the comprised requirements.

Also, I have a private SVN repo I work against but seriously considering moving to GitHub from the start of development, would you be interesting in seeing it take shape?

In my previous post introducing DirLinker I posted up a short to do list and I can happily check off two of the shorter items.

• Remove the dependency on Telerik and tidy up the UI –> Done, well it’s still not pretty but it doesn’t have a typo this time
• Post the source up on Codeplex –> http://dirlinker.codeplex.com/ 

Whether the rest will ever get done is another matter.  If you want to contribute then please go ahead.

I’ve been spending a bit of time lately learning Python and Pythonista ways, although a lot of the stuff I’ve read is more about the how of dynamic languages not the why but that’s another discussion.  I find the best way to learn a new language is to pick a pet project and run with it.  As random as it may seem this fits in nicely with my new years resolution of losing weight.  So I started a Diet Tracker website using the Django framework.

In my continuing quest to use and learn more about Test Driven Development one of the first things I spent time diving into (after the basic syntax) was the built-in unit test framework in Django, the Python standard library has one too but the Django one is richer.  It paid off quite quickly.

One of the key parts of the site was the ability to track your changing weight.  The first version of the weight display page is sortable in ascending or descending order and can have a selectable number of entries.  Also, weights will be stored in the model as KG but displayed in stone and pounds, this will be made user configurable at a later date.  The exceedingly simple algorithm for this page is:

1. Accept a list of weights
2. Reverse the sort order if we need it to be newest first
3. Take X number of items
4. Create a new list of the display adapter class

I would write this in C# as something like:

public List<DisplayWeight> ConvertToDisplayWeightList(List<ModelWeight> weights, int numberForDisplay, bool newestFirst)
{
	if (newestFirst)
	{
		weights.Reverse();
	}

	List<DisplayWeight> returnList = new List<DisplayWeight>();
	weights.GetRange(0, numberForDisplay).ForEach( mw => returnList.Add(new DisplayWeight(mw)));

	return returnList;
}

This does look WTF-esq but it’s not that bad once you understand how Django’s ORM works.  The Python I wrote to do this was:

def get_display_weight_list_by_date(self, numberToLoad = 5, newest_first = True):
        currentWeightList = Weight.objects.all().order_by('date')

        if newest_first:
            currentWeightList.reverse()

        finalList =[]

        for w in self.currentWeightList[:numberToLoad]:
            finalList.append(DisplayWeight(w))

        return finalList

def testLoadWeights_5items_newest_first(self):
        "Checks we have 5 weights returned and the newest is first"
        loader = DisplayWeightLoader()
        loader.get_display_weight_list_by_date(numberToLoad=5, newest_first=True)

        self.assertEquals(len(loader.currentWeightList), 5)
        self.assertEquals(loader.currentWeightList[0].date, datetime.date(2010, 01, 19))

def testLoadWeights_10items_oldest_first(self):
        "Checks we have 10 weights returned and the last is first"
        loader = DisplayWeightLoader()
        loader.get_display_weight_list_by_date(numberToLoad=10, newest_first=False)

        self.assertEquals(len(loader.currentWeightList), 10)
        self.assertEquals(loader.currentWeightList[0].date, datetime.date(2010, 01, 9))

The function is not very pythonic and a bit WTFy, I’ve rewritten it since, but the second unit test failed.  Without going into where the test data is coming from or what is going on, the test is failing because the newest date is still first.  Why is it failing? Because I made an assumption.

The assumption?  That an instance method on an instance of a mutable object affects the state of that object.  I think it’s a fair assumption and it is the way that C# works.  But it turns out not to be the case in Python, the line at fault:

currentWeightList.reverse()

It should be

currentWeightList = currentWeightList.reverse()

There is a couple of very important lessons here:

• Never make assumptions: Yet more proof that assumptions make an ass out of you and me.
• Reading language and library documentation is extremely important:  I had skimmed over the tutorials and docs for the standard Python library and I thought I knew enough to get up and running.  I should really have gone back and read the documentation properly for the functions I’m using for the first time.

The most important lesson though:

Unit testing is not an advanced skill or topic, it is essential and part of the basics 

Unit testing is not part of the standard Python docs tutorial, it is at the end of the Django tutorial (to be fair it is mentioned briefly early on) and barely gets a mention on MSDN.  Implying that it’s not that important and it’s something you can leave to later.  I think that it is extremely important, whether you’re an experienced developer trying to get to grips with a new language or a beginner trying to get to grips with programming it is the single most important thing after learning the syntax.

At every level it helps you to think about what you’re actually trying to achieve, gives you instance feedback to your understanding of a feature, language or library and helps you to write small simple functions.

Introduction

I’m a massive DropBox fanboy, I just love it, love it, love it.  Up until recently I’ve used it by dragging and dropping stuff in or saving directly into it from the application.  Then I came across this post by GrumpyDev which describes how to move Windows Live Writer draft posts into Dropbox by using symbolic directory links.

I hadn’t thought of this before, instead of remembering to move my files about or saving them to places other than the standard documents folder, which I always forget to, I could create symbolic directory links for all the common things I use.  I went a bit mad at this point and moved all kinds of folders inside my Dropbox.

The only thing about doing this is, the process is a bit clunky.  You have to:

1. Create the target
2. Copy any existing content to the target
3. Delete the folder you want to be a link
4. Start cmd.exe and then enter the mklink command along with the full paths of the folders you want to link

I was very surprised to find there is no pretty little one click GUI to do this.  So I decided to create one, I was looking for a test app to allow me to practice with TDD and MVC in Winforms and this was a perfect opportunity to.

DirLinker

So there it is, a very basic UI that allows you to enter where you want the link and where you want to link to.  You can just delete the link place or copy the contents over first.  Use these options at your own risk, I will not be held responsible for any data loss.

Requirements:

• Vista or later (I have tested it on Vista x86 and Win7 x64)
• .NET 3.5
• Admin rights

Download links:

This can now to be downloaded from CodePlex: http://dirlinker.codeplex.com/

DirLinker

DirLinker – source (I do have this in SubVersion at Unfuddled but I can’t make the repo public.  When I remove the dependency on Telerik to build, I will move it to CodePlex)

This is only the first version and I have a list of planned updates, which includes:

• A full UI review and remove the dependency on Telerik - Updated 17/01/2010:  http://www.humblecoder.co.uk/?p=59
• Add rollback support when an operation fails -
• A general tidy up
• Add a check for .NET 3.5 at start up
• XP support (maybe)

Download, enjoy and please report any bugs.

Introduction

I recently attended the Manchester NxtGenUG “Open Mic.” Night that was run as an OpenSpace style event. I had never heard of OpenSpace before and I wasn’t to sure what to expect from the open mic night but it turned out to be an extremely enjoyable experience with lots of valuable information. Thanks to Steve and Seb for making it such a great night!

I shameless stole the format and used it as a team building exercise at work. I had several worries because of the small team and other constraints that meant it couldn’t be a true OpenSpace event.

So What Is An OpenSpace Event

An OpenSpaces conference is one with no agenda, keynote or dedicated speakers, just passionate people talking about what they are interested in. When I was looking for more material to help explain it to people at work I came across this great quote about the idea behind OpenSpace.

In my experience open space is based on the belief that we humans are intelligent, creative, adaptive, meaning- and fun-seeking. It sets the context for such creatures to come together knowing they are going to treat each other well. When this happens there is no limit to what can unfold.

Alan Stewart
personal communication

source and excellent article about OpenSpace

The format of them is fairly lack. You get all the participants to sit around in a circle then go through them one by one asking for a topic they would like to talk about. Once all the subjects are collated a vote is taken as to which to talk about. When a subject is chosen the person who chose the subject starts the conversation by expanding on it. The conversation should be flowing at this point as people will have voted for it for a reason.

Each subject is talked about till it has naturally run its course then you move on to the next. It is simply over when it’s over.

How Would This Work In The Office?

All this sounded wonderful and works great when you have like minded people and lots of time but how would this work in a workplace?  In the office we have time constraints, it has to be justifiable to the business, not everyone is as passionate about software as others and more experienced members of staff can be cynical.

The time constraint was unavoidable but I was confident even with only an hour available it could still be constructive. To make it relevant to the business I positioned it as a training session that was self led, to discover the strengths and weaknesses of the team along with exposing hidden knowledge in the team. Finally, to tackle cynical/non passionate people I spoke to as many people as possible and explained the concept and made it clear it was optional.

I ran it past the team leader and asked for permission to try it during the working day to get as many as people to come. I think I got the timing right here, we are currently in a quiet spell before the start of a new project and it is the run up to Christmas so not a lot is going on so it was given the go ahead.

So with some trepidation I arrange a meeting for late Monday afternoon.

How Did It Go?

Well in a word, fantastic. All topics chosen were topical, relevant and most importantly could be applied to the current code base and processes in the team. I was truly impressed with the level of knowledge and desire to improve that was shown.

The meeting itself got off to a bit of a shaky start with people unsure about what to expect. This is where the host (me in this case) has to really take the lead and explain to people what is going to happen. We selected a subject, which just happened to be mine. This helped as it allowed me to take the lead and show others just how easy it was. Everyone of the nine people contributed something interesting.

At the end we had a five minute retrospective (We couldn’t be Agile without it ), everyone was pleased with how it went but suggested next time we take more notes or have an official collator of links. So this is something we are going to try.

Conclusion

We took the concept and made it fit within it our situation, I see this as an evolving thing inside the team and each time we will tweak it to fit the topic or mood of the team. The biggest takeaway, for me, has to be that it turned into a great session not only did it show where people’s interest and knowledge lies but it was a great team bonding exercise.

I can’t recommend trying something like this in your team enough, remember:

• Don’t stick to the format rigidly
• Trust your team to bring up the problem areas or where they want to improve
• Have fun!
• Oh, and try to make sure the cake isn’t a lie.

Introduction

Lately I’ve spent a lot of time tracking down a handle leak in a .NET application that called into a DCOM object.  In the process I had a crash course in WinDBG.  Before this I was, frankly, scared of WinDBG it was just fast scrolling text with a command interface, far from user friendly.  But I have discovered it is extremely powerful and not quite the scary monster I thought it was.

To demonstrate this I’m going to use a fictional ASP.NET app called MemoryMuncher that leaks memory with each page reload to give a quick overview of diagnosing a managed memory leak with WinDBG.

.NET Doesn’t Have Memory Leaks!

It is a common misconception that .NET can not leak memory because it has a garbage collector.  The most common situation where it would leak memory is in registering delegates with an event and not unregistering them.  Consider the following code:

private void DoSomeWork()
{
    SomeClass bigClass = new SomeClass();
    HoldOnToARef.IDoSomethingImportantMaybeAjaxy += bigClass.IHandleSomethingImportant;

    //Some other code
}

When the SomeClass instance goes out of scope the pointer, bigClass, to it will be deleted.  When the GC runs next it will walk the stack looking for references to the memory and it will find one!  The one it will find is the link between HoldOnToARef.DoSomethingImportantMaybeAjaxy event and the delegate we registered with it.  But we don’t have a reference to bigClass anymore to deregister the delegate.  So it will last until HoldOnToARef goes out of scope, which it won’t in this case as it is a static.

You maybe looking thinking “I would never write code like that” but we all have (maybe not with the statics as that’s an extreme example) with objects that live for a long time or we have all inherited code like this.

Memory Muncher is…errmm….Munching Memory

Every time a page refreshes on Memory Muncher the host process’s, w3wp.exe, memory usage raises by 5MB and when left it does not go down by itself.  At this rate, on a 32bit system, the process will run out of virtual address space in 400 page views causing the app to crash and the App pool to recycle.  If it’s a 64bit system, it will just start to really slow down the server as it grows, so no big deal there .

The first thing to do is to take a process dump when the memory usage is high.  This is relatively straight forward on Server 2008+ / Vista+, go to task manager find the offending process, right click and select “Create Dump File” (fig1). If you’re on Server 2003 / XP you need to use the built in Dr Watson tool or Process Explorer to create a dump.

fig1: Menu option

This should let you select a location to save the dump to or tell you where it has saved it to.

Ready, Steady, WinDBG…

Now we have the dump lets get WinDBGing.  WinDBG can be downloaded from here, you will need the 32 bit or 64 bit version depending on what you want to analyse and to analyse .NET dumps you need to do it on a machine with .NET installed.

First thing to do is open the dump and load the SOS extensions for .NET.  I normally find it best to start WinDBG elevated on Vista/2008 systems.  The SOS extension bring additional commands to WinDBG for dealing with the CLR and the managed memory model.  They are loaded by typing

.loadby sos mscorwks

This is used by .NET 2.0 + if you’re using .NET 1.x there is a different command to load the extension.

Next we need the symbols, symbols make the stack traces readable.  If this is the first time you have used WinDBG type .symfix at the prompt.  This will setup the symbols path to use the public Microsoft symbol server.  You will need the symbols for your application as well.  The best way to do this is get them together in a folder, they need to be the correct version for the build you just took the dump from. Go to the menu and select File –> Symbol File Path and then browse for the folder you put them in.  Check the reload option in that dialog then hit OK.  You should now be good to go

Back to Memory Muncher and we are ready to start to look for the leak.  The first thing to do is dump the heap, this done by entering the command

!dumpheap

This will give a lot of information, for Memory Muncher we ended up with:

There is an almost overwhelming amount of information here.  What we have is, from left to right, the method table address, the number of instances of the type, the amount of memory being used in total by all instances of a type in bytes and finally the type.  I’ve highlighted the two important ones here, the number of objects for a given type and the amount of memory taken up.  The list sorted by what is taking up the most memory.

I have underlined my object that is using the most amount of memory, this seems like a good place to start.  The questions I’m trying to answer at this point is why do we have so many of these objects alive?  What is the object?  What is keeping it alive?  To start answering these we need a reference to just one of them, to get this we can use !dumpheap again but this time for the type, this is achieved by running the following command:

!dumpheap -type MemoryMuncher.SomeOtherClass

This produces:

Again an awful of text flying past but the important bits are the first column, this is the address of the instance of an object on the heap.  The other information is the method table address (this is for reflection to look up metadata) and the size in memory of the instance.  At the bottom is a list of who owns the references to these objects and how many of them there are.

We know that the garbage collector is compacting so that means the lower the address the older the object is, this is pertinent because if we look at the older objects they have probably survived a few garbage collector generations and as such are more likely to show us who is holding the reference causing the leak.

So, we can at this point have a look at what the object is and its current instance values by doing !do <address>.  This would help identify individual objects and what state they are in but since there is so many it would be wiser to see who is holding a reference.  This can be done by running the following command:

!gcroot <address>

This produces:

This output shows we are being held on to by a class called SomeClass which is being held open by an object called MyCallbackDelegate. At this point we can find these objects in the code and start the looking at what MyCallbackDelegate is.  The full Memory Muncher code is listed below and we can see that a handler is registered but never deregistered.  So it should be a simple fix.

namespace MemoryMuncher
{

    public delegate void MyCallbackDelegate();

    public class HoldOnToARef
    {
        public static event MyCallbackDelegate IDoSomethingImportantMaybeAjaxy;

        public static void DoThatImportantThing()
        {
            IDoSomethingImportantMaybeAjaxy();

        }
    }

    public class SomeClass
    {
        private List<SomeOtherClass> m_SomeList = new List<SomeOtherClass>();
        public SomeClass()
        {
            for (int i = 0; i < 500; i++)
            {
                m_SomeList.Add(new SomeOtherClass());
            }
        }

        public void IHandleSomethingImportant()
        {
            foreach (SomeOtherClass aClass in m_SomeList)
            {
                Console.WriteLine(aClass.BigByteArray);
            }
        }
    }

    public class SomeOtherClass
    {
        public Byte[] BigByteArray;

        public SomeOtherClass()
        {
            BigByteArray = new Byte[1024];
            for (int i = 0; i < 1024; i++)
            {
                BigByteArray[i] = 0xA0;
            }
        }

    }

    public partial class _Default : System.Web.UI.Page
    {
        protected void Page_Load(object sender, EventArgs e)
        {
            for (int i = 0; i < 10; i++)
            {
                SomeClass bigClass = new SomeClass();
                HoldOnToARef.IDoSomethingImportantMaybeAjaxy += bigClass.IHandleSomethingImportant;
            }
        }
    }
}

Final Thoughts and Further Reading

This is a fairly noddy example that would have been caught simply by the reviewing the code and as such the dump is a quick study.  In a real world example there might be multiple objects that exist in their thousands or complex object graphs that make GCRoot’s output more cryptic.  Either way the same methodology would apply, look for suspicious objects you know should have fairly short life cycle and dig into why the reference is being held open.

This is only a tiny portion of what WinDBG can do and I haven’t even began to do it justice.  If you want to know more these are great places to start

• MSDN http://msdn.microsoft.com/en-us/library/bb190764.aspx
• Tess Ferrandez: http://blogs.msdn.com/tess/

Also, a general read for all .NET developers with lots of useful background information for this is CLR via C#.

Happy WinDBGing!