C# 4.0 New Features Part 1.1 – dynamic keyword second look

Part 1 – Dynamic Keyword

I was originally going to just update my original post with some performance data regarding the dynamic keyword versus MethodInfo.Invoke, but I started looking a bit deeper at the performance implications of the dynamic keyword. In my previous post I posted these performance numbers:

Compile Time Bound: 6 ms – yep, 2 million calls, 6 ms. Fast.

Dynamically Bound: 2106ms – so, roughly 351 times slower than the strongly typed calls

But these numbers don’t really tell the whole picture. The reality is that in the above test I was factoring in the overhead of initializing the callsites used by the dynamic invoke. Since I ran the test over and over from the start, this was included every time. So, I decided to update my testing script to look like this:

private static void Method1()
{
    Console.WriteLine("compile bound");
    var test = new TestClass();
    test.TestMethod1();
    test.TestMethod2();

    long totalMilliseconds = 0;

    for (int j = 0; j < 5; j++)
    {
        var sw = new Stopwatch();
        sw.Start();
        for (int i = 0; i < 1000000; i++)
        {
            test.TestMethod1();
            test.TestMethod2();
        }
        sw.Stop();

        Console.WriteLine(sw.ElapsedMilliseconds);
        totalMilliseconds += sw.ElapsedMilliseconds;

        sw.Reset();
    }

    Console.WriteLine("Average: " + totalMilliseconds / 5);
}

What we are doing here is calling the two methods once before we start testing them in order to get the overhead out of the way. Then we run a loop 5 times and then run our 1000000 loops through calling both methods. So, we are still doing 2,000,000 calls, but we are now doing them five times in a row. So, 10,000,000 total calls per method. I also tested the above code using the “dynamic” keyword and MethodInfo.Invoke.

The numbers that I got were pretty startling. The first method didn’t change, there is no startup time for compile time binding, so we don’t really see any difference testing it this way.

The second method (the dynamic keyword) is where the real changes started to show up. The start up time for the dynamic keyword is pretty high right now (I was recording about 3.5 seconds to make the first two calls). But once the first calls were out of the way, and our CallSites were created, they are assigned as static methods on a compiler generated class and so they are only created once when first called. The performance here was dramatically better. The calls to warm it up just looked like this:

dynamic test = new TestClass();
test.TestMethod1();
test.TestMethod2();

Third I tested MethodInfo.Invoke where we also started them up and invoked the calls once. The difference here is that each time the method is called, we have significant overhead to make the call, even though we are not retrieving out MethodInfo objects each time.

var test = new TestClass();
MethodInfo mi1 = typeof(TestClass).GetMethod("TestMethod1");
MethodInfo mi2 = typeof(TestClass).GetMethod("TestMethod2");
mi1.Invoke(test, null);
mi2.Invoke(test, null);

So, what could we do in C# 3.0 that would be similar to what is happening with the C# 4.0 dynamic keyword? Well, if you have never played around with it, lets look at the DynamicMethod class. The DynamicMethod class is a helper for creating a method at runtime that we can then invoke. It is doing something similar to what we have above, only in order to use DynamicMethod we have to emit the IL ourselves. This is no simple task, but the easiest way to do it is to compile a handwritten method yourself that does something similar, then reflect it and look at the IL. Then you will at least have a good starting point.

With DynamicMethod we have to write quite a bit of code to get this all wired up, but here it is:

MethodInfo mi1 = typeof(TestClass).GetMethod("TestMethod1");
MethodInfo mi2 = typeof(TestClass).GetMethod("TestMethod2");

var test = new TestClass();

DynamicMethod method1 = new DynamicMethod("CallMethod1", null, new Type[] {typeof(TestClass)}, typeof(TestClass).Module);
ILGenerator il = method1.GetILGenerator(128);
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Callvirt, mi1);
il.Emit(OpCodes.Ret);

DynamicMethod method2 = new DynamicMethod("CallMethod2", null, new Type[] { typeof(TestClass) }, typeof(TestClass).Module);
il = method2.GetILGenerator(128);
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Callvirt, mi2);
il.Emit(OpCodes.Ret);            

Action method1Delegate = (Action)method1.CreateDelegate(typeof(Action), test);
Action method2Delegate = (Action)method2.CreateDelegate(typeof(Action), test);

method1Delegate();
method2Delegate();

Now that we have our delegates that we can all, we can run the same loops against this method. Again, we call each method once but in this instance all of the work is being done when we emit the IL.

Our four methods are planned out and now we can lay out what you really came here for. Here are the averages:

Compile Time Bound: 6 ms

Dynamically Bound with dynamic keyword: 45ms

Dynamically Bound with MethodInfo.Invoke – 10943ms

Dynamically Bound with DynamicMethod – 8ms

Compile time still wins out, but the dynamic keyword is a much much better looking option now. The overhead of the first call is pretty high, in my tests almost 3 seconds. I’m sure that they will work on the speed of this before the final release. Every successive call is actually really fast though. The next method is MethodInfo.Invoke which is quite slow compared to the rest, but in terms of 2 million calls is still pretty fast. Finally the last method is DynamicMethod, which is practically as fast as the compile bound method because essentially we are just manually compiling it to IL.

There you go, the dynamic keyword has some overhead, but you only have to deal with it once for each dynamic call. In a long running application this isn’t really a concern. I hope you enjoyed this post and I hope that I have answered most of the performance questions that you may have for the dynamic keyword.