I had introduced a while back a ParallelMap function, and then later I updated it to make sure that it was tracking order when results were returned. I took a look at the method recently, and realized that it was ugly. I had introduced a struct that allowed me to tie the index to the result value, not really realizing that I could have done it all with closures. I guess that is the normal path of code though… write…rewrite…come back 3 months later…rewrite. Well, anyways, in my last post I mentioned a few updates that I had made to the method and promised to post it. So, here it is:
public static IEnumerable<TResult> ParallelMap<TArg, TResult>(
this IEnumerable<TArg> list,
Func<TArg, TResult> func)
{
var result = new TResult[list.Count()];
using (var resetEvent = new ManualResetEvent(false))
{
int count = result.Length;
int i = 0;
foreach (TArg item in list)
{
TArg localItem = item;
int index = i;
ThreadPool.QueueUserWorkItem(
n =>
{
TResult value = func(localItem);
result[index] = value;
if (Interlocked.Decrement(ref count) == 0)
{
resetEvent.Set();
}
}
);
i++;
}
resetEvent.WaitOne();
}
return result;
}
Beautiful! I am still using the ThreadPool class, I am going to implement a custom version in the future that will use explicit threads. The changes I have made is that we are now copying the item and index to a local variable and then using them in the lambda that we are now passing directly into the QueueUserWorkItem method. So, now we can drop our struct that we were before passing as state.
I hope that someone out there finds this method useful!
Loved the article? Hated it? Didn’t even read it?
We’d love to hear from you.
Justin,
This is good stuff. I had thought about writing something similar, but your implementation will save me the trouble.
I had started digging into PLINQ a few months ago, but a couple of major projects at work trumped my spare time. Now, I finally have my head above water again (for the moment) and I back to tinkering with PLINQ again, especially since a new CTP was just dropped.
We seem to have some similar interests in the power of parallel programming combined with functional language concepts. Very powerful and cool stuff. You just do a better job of finding time to write about all of it than I do. 🙂
Keep the posts coming.
Jeff
Yeah, I definitely need to start looking at PLinq again with the new CTP drop. Thanks for the comments!
Good stuff, Justin. Is the lock on the result array inside the statement lambda really necessary? Each ThreadPool thread will be touching a unique index within the result array. It would be nice to deserialize updates to the result.
That is interesting that you say that, because I must have pulled and updated an old version of this method. I had removed the lock once before when someone else made a comment about that. I am lock crazy! Thanks for pointing that out…updated!