I have run into this problem across multiple programming languages and I was just wondering what the best way to handle it is.
I have three method calls that fire off asynchronously. Each one has a callback. I want to do something only when all three callbacks have completed.
What is the best way to code this? I usually end up with all these public bool flags and as you add more calls the code gets more convoluted.
Coming from C#, I would probably use WaitHandle.WaitAll. You can create an array of ManualResetEvent objects (one for each task to be completed), and pass that array to WaitAll. The threaded tasks will get one ManualResetEvent object each, and call the Set method when they are ready. WaitAll will block the calling thread until all tasks are done. I'll give a C# code example:
private void SpawnWorkers()
{
ManualResetEvent[] resetEvents = new[] {
new ManualResetEvent(false),
new ManualResetEvent(false)
};
// spawn the workers from a separate thread, so that
// the WaitAll call does not block the main thread
ThreadPool.QueueUserWorkItem((state) =>
{
ThreadPool.QueueUserWorkItem(Worker1, resetEvents[0]);
ThreadPool.QueueUserWorkItem(Worker2, resetEvents[1]);
WaitHandle.WaitAll(resetEvents);
this.BeginInvoke(new Action(AllTasksAreDone));
});
}
private void AllTasksAreDone()
{
// OK, all are done, act accordingly
}
private void Worker1(object state)
{
// do work, and then signal the waiting thread
((ManualResetEvent) state).Set();
}
private void Worker2(object state)
{
// do work, and then signal the waiting thread
((ManualResetEvent)state).Set();
}
Note that the AllTasksAreDone method will execute on the thread pool thread that was used to spawn the workers, and not on the main thread... I assume that many other languages have similar constructs.
If you really only want to wait for all to finish:
Create volatile counter
Synchronize access to counter
Increase counter on start
Decrease on callback fired
Wait for counter to reach 0
Use a semaphore.
Futures are very easy to use. Futures look like normal functions, except that they execute asynch.
The classes:
public struct FutureResult<T>
{
public T Value;
public Exception Error;
}
public class Future<T>
{
public delegate R FutureDelegate<R>();
public Future(FutureDelegate<T> del)
{
_del = del;
_result = del.BeginInvoke(null, null);
}
private FutureDelegate<T> _del;
private IAsyncResult _result;
private T _persistedValue;
private bool _hasValue = false;
private T Value
{
get
{
if (!_hasValue)
{
if (!_result.IsCompleted)
_result.AsyncWaitHandle.WaitOne();
_persistedValue = _del.EndInvoke(_result);
_hasValue = true;
}
return _persistedValue;
}
}
public static implicit operator T(Future<T> f)
{
return f.Value;
}
}
Here I use futures to simulate a deadlock:
void SimulateDeadlock()
{
Future> deadlockFuture1 = new Future>(() =>
{
try
{
new SystemData(ConfigurationManager.ConnectionStrings["DbConnectionString"].ConnectionString)
.SimulateDeadlock1(new DateTime(2000, 1, 1, 0, 0, 2));
return new FutureResult { Value = true };
}
catch (Exception ex)
{
return new FutureResult { Value = false, Error = ex };
}
});
Future> deadlockFuture2 = new Future>(() =>
{
try
{
new SystemData(ConfigurationManager.ConnectionStrings["DbConnectionString"].ConnectionString)
.SimulateDeadlock2(new DateTime(2000, 1, 1, 0, 0, 2));
return new FutureResult { Value = true };
}
catch (Exception ex)
{
return new FutureResult { Value = false, Error = ex };
}
});
FutureResult result1 = deadlockFuture1;
FutureResult result2 = deadlockFuture2;
if (result1.Error != null)
{
if (result1.Error is SqlException && ((SqlException)result1.Error).Number == 1205)
Console.WriteLine("Deadlock!");
else
Console.WriteLine(result1.Error.ToString());
}
else if (result2.Error != null)
{
if (result2.Error is SqlException && ((SqlException)result2.Error).Number == 1205)
Console.WriteLine("Deadlock!");
else
Console.WriteLine(result2.Error.ToString());
}
}
For those using JavaScript, consider using the pattern discussed at this Stackoverflow question:
javascript: execute a bunch of asynchronous method with one callback
Related
I want to make ActionResult with run a thread. When thread are running I will return View with loading but when thread finished the ActionResult return the another View
My code look like:
private static void ExecuteScrapper()
{
ScriptEngine pythonEngine = IronPython.Hosting.Python.CreateEngine();
ScriptSource pythonScript = pythonEngine.CreateScriptSourceFromFile(HostingEnvironment.MapPath("/PythonScript/main.py"));
var searchPath = pythonEngine.GetSearchPaths();
searchPath.Add(HostingEnvironment.MapPath("/PythonScript/"));
searchPath.Add(HostingEnvironment.MapPath("/PythonScript/drivers/"));
pythonEngine.SetSearchPaths(searchPath);
var result = pythonScript.Execute();
}
private ScrapperEnum.ScrapperOperations scraperParam;
private Thread t = new Thread(new ThreadStart(ExecuteScrapper));
public ActionResult RunScrapper(ScrapperEnum.ScrapperOperations operation)
{
scraperParam = operation;
t.Start();
bool run = false;
while(t.IsAlive)
{
if(!run)
{
run = true;
return View("ScrapperLoadingView");
}
}
return RedirectToAction("Scrapper");
}
I don't know exactly how to do that, because when in this code I return ScrapperLoginView it will break while so the function don't switch the View
You just need to add an else if I'm correct
while(t.IsAlive)
{
if(!run)
{
run = true;
return View("ScrapperLoadingView");
}
else
{
return RedirectToAction("Scrapper");
}
}
}
First, here's a reference to where I read all of what I know now regarding this question: http://docs.jboss.org/netty/3.2/api/org/jboss/netty/bootstrap/ServerBootstrap.html#bind%28%29
Although not explicitly specified by the documentation, it would seem that ServerBootstrap.bind is synchronous - because it does not return a ChannelFuture, but rather a Channel. If that is the case, then I do not see any way to make an asynchronous bind using the ServerBootstrap class. Am I missing something or will I have to roll my own solution?
Best regards
I ended up rolling my own bootstrap implementation with the following addition:
public ChannelFuture bindAsync(final SocketAddress localAddress)
{
if (localAddress == null) {
throw new NullPointerException("localAddress");
}
final BlockingQueue<ChannelFuture> futureQueue =
new LinkedBlockingQueue<ChannelFuture>();
ChannelHandler binder = new Binder(localAddress, futureQueue);
ChannelHandler parentHandler = getParentHandler();
ChannelPipeline bossPipeline = pipeline();
bossPipeline.addLast("binder", binder);
if (parentHandler != null) {
bossPipeline.addLast("userHandler", parentHandler);
}
getFactory().newChannel(bossPipeline);
ChannelFuture future = null;
boolean interrupted = false;
do {
try {
future = futureQueue.poll(Integer.MAX_VALUE, TimeUnit.SECONDS);
} catch (InterruptedException e) {
interrupted = true;
}
} while (future == null);
if (interrupted) {
Thread.currentThread().interrupt();
}
return future;
}
In Netty 3.6 there is an async bind. Here's the javadoc: http://netty.io/3.6/api/org/jboss/netty/bootstrap/ServerBootstrap.html#bindAsync()
I am calling few methods on a button click.
functionA()
functionB()
functionC()
All three functions are independent from each other and they take long time to execute. I checked and found that by threading I can run all three together which will save the execution time.
As I am new to threading concept, could anyone please guide me the simplest way I can do threading in scenario or other way which will be useful in this scenario.
EDIT
One more problem in the same function:
I am binding 5 gridviews after the three functions execution. Like this
gv1.DataSource = GetData("Mill");
gv1.DataBind();
gv2.DataSource = GetData("Factory");
gv2.DataBind();
gv3.DataSource = GetData("Garage");
gv3.DataBind();
gv4.DataSource = GetData("Master");
gv4.DataBind();
They all are using the same method for getting the result and they are also taking time to load. Is there any way I can run them parallel too? I afraid, because they are using same method to get the data. Is it possible to do threading for them. How ?
I am not sure how Parallel.Invoke() decides what to execute in parallel, but if you want an assurance that they will execute in parallel, use threads:
var t1 = new Thread(MySlowFunction);
t1.IsBackground = true;
t1.Start();
var t2 = new Thread(MySlowFunction);
t2.IsBackground = true;
t2.Start();
# To resync after completion:
t1.Join();
t2.Join();
Or even better, use the ThreadPool:
ThreadPool.QueueUserWorkItem(MyWork);
Remember to handle your thread exceptions.
The simplest answer is to use MSDN: Parallel.Invoke().
You should also consider: Asynchronous Pages.
Try using System.Threading.Tasks namespace
Something like
var task1 = Task.Factory.StartNew(() => DoA());
var task2 = Task.Factory.StartNew(() => DoB());
var task3 = Task.Factory.StartNew(() => DoC());
Task.WaitAll(task1, task2, task3);
http://www.codethinked.com/net-40-and-systemthreadingtasks
Here's an example which will execute the 4 tasks in parallel:
public partial class _Default : System.Web.UI.Page
{
public class MyViewModel
{
public int Id { get; set; }
public string Name { get; set; }
}
protected void Page_Load(object sender, EventArgs e)
{
}
protected void BtnBindClick(object sender, EventArgs e)
{
// we define the input for the tasks: each element consists
// of the grid we are willing to bind the results at the end and
// some optional parameter we want to pass to the GetData function
var inputs = new[]
{
new { Grid = gv1, Input = "Mill" },
new { Grid = gv2, Input = "Factory" },
new { Grid = gv3, Input = "Garage" },
new { Grid = gv4, Input = "Master" },
};
// define the tasks we want to execute in parallel
var tasks = inputs
.Select(x => Task.Factory.StartNew(
() => new { Grid = x.Grid, Output = GetData(x.Input) })
)
.ToArray();
// define a task which will be executed once all tasks have finished
var finalTask = Task.Factory.ContinueWhenAll(tasks, x => x);
// wait for the final task
finalTask.Wait();
// consume the results
foreach (var item in finalTask.Result)
{
if (item.Exception == null)
{
// if no exception was thrown for this task we could bind the results
item.Result.Grid.DataSource = item.Result.Output;
item.Result.Grid.DataBind();
}
}
}
private MyViewModel[] GetData(string input)
{
// Simulate slowness
Thread.Sleep(1000);
return Enumerable.Range(1, 5).Select(x => new MyViewModel
{
Id = x,
Name = input
}).ToArray();
}
}
I have a Silverlight project which takes some encrypted string thru its Service Reference: DataService (service which is done in an ASP.NET project).
The method from TransactionServices.cs to get the encrypted string is:
public void GetEncryptedString(string original)
{
DataService.DataServiceClient dataSvc = WebServiceHelper.Create();
dataSvc.GetEncryptedStringCompleted += new EventHandler<SpendAnalyzer.DataService.GetEncryptedStringCompletedEventArgs>(dataSvc_GetEncryptedStringCompleted);
dataSvc.GetEncryptedStringAsync(original);
}
On completing, put the result in encodedString var (which is initialized with an empty value):
void dataSvc_GetEncryptedStringCompleted(object sender, SpendAnalyzer.DataService.GetEncryptedStringCompletedEventArgs e)
{
if (e.Error == null)
{
try
{
if (e.Result == null) return;
this.encodedString = e.Result;
}
catch (Exception ex)
{
Logger.Error("TransactionService.cs: dataSvc_GetEncryptedStringCompleted: {0} - {1}",
ex.Message, ex.StackTrace);
MessageBox.Show(ex.ToString());
}
}
}
Now I want to get the encoded string from my MainPage.xaml like:
TransactionService ts = new TransactionService();
ts.GetEncryptedString(url);
Console.WriteLine(ts.encodedString);
I do not uderstand why ts.encodedString is empty. When I do the debug I see that it actually prints out empty and AFTER that it goes to the void dataSvc_GetEncryptedStringCompleted to take the result and fill it.
Can you point me what I've done wrong? Is there a way to wait for the encodedString to be fetched and only after that to continue?
Thanks a lot.
When you call the ts.GetEncryptedString(url); you just started async operation. And therefor the value you are accessing is will be set only in the callback method.
But you access it before the value is modified by the callback.
The solution which I am using will looks similar to folowing:
Redefine the GetEncryptedString method signature.
public void GetEncryptedString(string original, Action callback)
{
DataService.DataServiceClient dataSvc = WebServiceHelper.Create();
dataSvc.GetEncryptedStringCompleted += (o,e) =>
{
dataSvc_GetEncryptedStringCompleted(o,e);
callback();
}
dataSvc.GetEncryptedStringAsync(original);
}
Call it like this:
ts.GetEncryptedString(url, OtherLogicDependantOnResult);
where
OtherLogicDependantOnResult is
void OtherLogicDependantOnResult()
{
//... Code
}
I am mocking a wrapper to an MSMQ. The wrapper simply allows an object instance to be created that directly calls static methods of the MessageQueue class.
I want to test reading the queue to exhaustion. To do this I would like the mocked wrapper to return some good results and throw an exception on the fourth call to the same method. The method accepts no parameters and returns a standard message object.
Can I set up this series of expectations on the method in Moq?
Yup, this is possible if you don't mind jumping through a few minor hoops. I've done this for one of my projects before. Alright here is the basic technique. I just tested it out in Visual Studio 2008, and this works:
var mockMessage1 = new Mock<IMessage>();
var mockMessage2 = new Mock<IMessage>();
var mockMessage3 = new Mock<IMessage>();
var messageQueue = new Queue<IMessage>(new [] { mockMessage1.Object, mockMessage2.Object, mockMessage3.Object });
var mockMsmqWrapper = new Mock<IMsmqWrapper>();
mockMsmqWrapper.Setup(x => x.GetMessage()).Returns(() => messageQueue.Dequeue()).Callback(() =>
{
if (messageQueue.Count == 0)
mockMsmqWrapper.Setup(x => x.GetMessage()).Throws<MyCustomException>();
});
A few notes:
You don't have to return mocked messages, but it's useful if you want to verify expectations on each message as well to see if certain methods were called or properties were set.
The queue idea is not my own, just a tip I got from a blog post.
The reason why I am throwing an exception of MyCustomException is because the Queue class automatically throws a InvalidOperationException. I wanted to make sure that the mocked MsmqWrapper object throws an exception because of Moq and not because of the queue running out of items.
Here's the complete code that works. Keep in mind that this code is ugly in some places, but I just wanted to show you how this could be tested:
public interface IMsmqWrapper
{
IMessage GetMessage();
}
public class MsmqWrapper : IMsmqWrapper
{
public IMessage GetMessage()
{
throw new NotImplementedException();
}
}
public class Processor
{
private IMsmqWrapper _wrapper;
public int MessagesProcessed { get; set; }
public bool ExceptionThrown { get; set; }
public Processor(IMsmqWrapper msmqWrapper)
{
_wrapper = msmqWrapper;
}
public virtual void ProcessMessages()
{
_wrapper.GetMessage();
MessagesProcessed++;
_wrapper.GetMessage();
MessagesProcessed++;
_wrapper.GetMessage();
MessagesProcessed++;
try
{
_wrapper.GetMessage();
}
catch (MyCustomException)
{
ExceptionThrown = true;
}
}
}
[Test]
public void TestMessageQueueGetsExhausted()
{
var mockMessage1 = new Mock<IMessage>();
var mockMessage2 = new Mock<IMessage>();
var mockMessage3 = new Mock<IMessage>();
var messageQueue = new Queue<IMessage>(new [] { mockMessage1.Object, mockMessage2.Object, mockMessage3.Object });
var mockMsmqWrapper = new Mock<IMsmqWrapper>();
mockMsmqWrapper.Setup(x => x.GetMessage()).Returns(() => messageQueue.Dequeue()).Callback(() =>
{
if (messageQueue.Count == 0)
mockMsmqWrapper.Setup(x => x.GetMessage()).Throws<InvalidProgramException>();
});
var processor = new Processor(mockMsmqWrapper.Object);
processor.ProcessMessages();
Assert.That(processor.MessagesProcessed, Is.EqualTo(3));
Assert.That(processor.ExceptionThrown, Is.EqualTo(true));
}