C# – Is this a right case for using the Threadpool


Here's the setup: I'm trying to make a relatively simple Winforms app, a feed reader using the FeedDotNet library. The question I have is about using the threadpool. Since FeedDotNet is making synchronous HttpWebRequests, it is blocking the GUI thread. So the best thing seemed like putting the synchronous call on a ThreadPool thread, and while it is working, invoke the controls that need updating on the form. Some rough code:

private void ThreadProc(object state)
 Interlocked.Increment(ref updatesPending);

    // check that main form isn't closed/closing so that we don't get an ObjectDisposedException exception
 if (this.IsDisposed || !this.IsHandleCreated) return;
 if (this.InvokeRequired)
   if (!marqueeProgressBar.Visible)
    this.marqueeProgressBar.Visible = true;

 ThreadAction t = state as ThreadAction;
 Feed feed = FeedReader.Read(t.XmlUri);

 Interlocked.Decrement(ref updatesPending);

 if (this.IsDisposed || !this.IsHandleCreated) return;
 if (this.InvokeRequired)
  this.Invoke((MethodInvoker)delegate { ProcessFeedResult(feed, t.Action, t.Node); });

 // finished everything, hide progress bar
 if (updatesPending == 0)
  if (this.IsDisposed || !this.IsHandleCreated) return;
  if (this.InvokeRequired)
   this.Invoke((MethodInvoker)delegate { this.marqueeProgressBar.Visible = false; });

this = main form instance

updatesPending = volatile int in the main form

ProcessFeedResult = method that does some operations on the Feed object. Since a threadpool thread can't return a result, is this an acceptable way of processing the result via the main thread?

The main thing I'm worried about is how this scales. I've tried ~250 requests at once. The max number of threads I've seen was around 53 and once all threads were completed, back to 21. I recall in one exceptional instance of me playing around with the code, I had seen it rise as high as 120. This isn't normal, is it? Also, being on Windows XP, I reckon that with such high number of connections, there would be a bottleneck somewhere. Am I right?

What can I do to ensure maximum efficiency of threads/connections?

Having all these questions also made me wonder whether this is the right case for a Threadpool use. MSDN and other sources say it should be used for "short-lived" tasks. Is 1-2 seconds "short-lived" enough, considering I'm on a relatively fast connection? What if the user is on a 56K dial-up and one request could take from 5-12 seconds and ever more. Would the threadpool be an efficient solution then too?

Best Solution

The ThreadPool, unchecked is probably a bad idea.

Out of the box you get 250 threads in the threadpool per cpu.

Imagine if in a single burst you flatten out someones net connection and get them banned from getting notifications from a site cause they are suspected to be running a DoS attack.

Instead, when downloading stuff from the net you should build in tons of control. The user should be able to decide how many concurrent requests they make (and how many concurrent requests per domain), ideally you also want to offer controls for the amount of bandwidth.

Though this could be orchestrated with the ThreadPool, having dedicated threads or using something like a bunch of instances of the BackgroundWorker class is a better option.