Using grpc c++‘s async server api, should I add one cq per rpc method or just one cq for all service?
I am taking over a grpc c++ service in my job. it use 10 completionqueue for a async service method , and yet another 20 completionqueue for another stream service method. The handling of the completionqueue is quite similar to the helloworld async service example, except that it use one thread worker per completionqueue.
Such a design is not very convenient for adding more service method. And I am thinking that refactoring it to using just one completionqueue, and dispatching the tags in the completionqueue to my own domain transactions handling thread pool.
Is it the right design pattern, or is there something better?
The best way to know the answer for your specific application is to create benchmarks with various scenarios and then compare the results from the different threading models.
Based on https://grpc.github.io/grpc/cpp/md_doc_cpp_perf_notes.html, the current recommendation is to use a pool of num_cpus threads with each thread polling over its own completion queue.
Related
I have a CRUD app in Blazor that simply fetches the assignment lists from a table and has an AssignmentReminderService for data access layer that has a method (async version)
public async Task<AssignmentReminder> AddAssignment(AssignmentReminder assignment)
{
_context.assignments.Add(assignment);
await _context.SaveChangesAsync();
return assignment;
}
I can also call the method with synchromus code as :
public AssignmentReminder AddAssignment(AssignmentReminder assignment)
{
_context.assignments.Add(assignment);
_context.SaveChanges();
return assignment;
}
Now it is just one database being accessed from a local server(could be hosted on cloud as well) with just one assignment table and the default authentication/authorization tables generated when one uses Individual User Account (aspnetusers, aspnetroles etc)
Can someone let me know which of the two ways I should use (between async or sync) method declaration?
In the general case, you should use asynchronous APIs if they are available. This will result in greater scalability on the server side, since asynchrony will allow the calling request thread to be used for other requests while the asynchronous operation is in progress.
There are specific scenarios where this guideline doesn't apply. E.g., if you're calling a generic asynchronous API (like Stream.ReadAsync) but you know that the implementation is actually synchronous (like MemoryStream). But in general, if there's an asynchronous API, then that's the one you should use.
You should be clear about the version of blazor you're talking about, because using async methods in the client is different from using them in the server version.
which of the two ways I should use (between async or sync) method declaration?
The first one.
The scarce resource here are the Threads. You want to keep their number down, and the first approach enables that by releasing the Thread to do other work.
In the second approach the Thread is suspended for the duration of the I/O operation. You would need more Threads to handle the same number of requests.
So using async I/O lets the same hardware handle more requests at the same time.
I have a WCF service with ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple). I want to use ThreadStatic variable to srore data.
I start worrying about is it possible two parallel requests for the same or different operationContracts get handled by the same thread serverside, because if this happens my ThreadStatic variable will get overriden.(I.e. something like the thread changing between HttpHandlers and HttpModules in ASP.NET)
I made a spike service with the same ServiceBehaviour and maxConcurrentCalls="2". After that a wcf client called the service with 50 parallel requests and my worry did not occur. However this is not a 100% proof.
Thank in advance!
Irrespective of the ConcurrencyMode, a ThreadStatic value will persist when your request terminates and the thread is returned to the thread pool. The same thread can be reused for a subsequent request, which will therefore be able to see your ThreadStatic value.
Obviously this won't be true for two concurrent requests, because by definition they will be executed on different threads.
From comments:
Also by definition MSDN says: 'The service instance is multi-threaded. No synchronization guarantees are made. Because other threads can change your service object at any time, you must handle synchronization and state consistency at all times.' So it is not so obvious:)
This means that a single instance of your service class can be accessed concurrently by multiple requests. So you would need to handle synchronization for any accesses to instance members of the service class.
However ThreadStatic members are by definition only used by one thread (and hence one request) at a time, so don't need synchronization.
The direct answer to your question is Joe's answer.
However you mention in the comments you are using an ambient design pattern. That pattern is already implemented in WCF as the OperationContext and is specifically designed to be extensible. I highly recommend using OperationContext over any custom thread storage.
See Where to store data for current WCF call? Is ThreadStatic safe?
I wanted to add to Joe's answer here because I would recommend that you use some sort of correlation for your requests if you're needing to store state. The threading model will become very convoluted and unreliable in production.
Further, now imagine you have two IIS servers hosting this service and a hardware or software load balancer forward facing so that you can consume it. To ensure that the correct state is gathered you'll need correlation because you never know which server the service will be started on. In the post below I mocked up a simplified version of how that might work. One thing to keep in mind is that the SessionState would need to be kept in a shared location to all instances of the service, an AppFabric Cache server for example.
Global Variable between two WCF Methods
We are making a web service call on some data updates to sync another database. This web service call takes up some response time. Would adding it in a thread help at all? Any downfalls of doing this? If the web service calls fails, it fails and that is it. It is like a fire and forget call.
You could use an Asynchronous Web Service call using asyncronous callbacks to prevent blocking of your main thread.
By making an asynchronous call to a Web service, you can continue to
use the calling thread while you wait for the Web service to respond.
This means users can continue to interact with your application
without it locking up while the Web service access proceeds.
From MSDN: Making Asynchronous Web Service Calls
If it's taking long enough to hang the user interface then calling it on another thread is the recommended thing to do.
In addition to Tudor's answer I would suggest that you start off by using the new Task class from .NET 4.0.from task parallel library. Example would be:
Task backgroundProcess = new Task(() =>
{
service.CallMethod();
});
I strongly advice against using Async Web Service calls (including making calls in separate threads) from a web app. Instead use alternate approach like Ajax, and make this webservice call from an Ajax Call instance. There is no easy approach in the web context to handle threading and Async calls.
I'm just wondering, if I have an ASP.net Web Application, either WebForms or MVC, is there any situation where doing stuff asynchronously would make sense?
The Web Server already handles threading for me in that it spins up multiple threads to handle requests, and most request processing is rather simple and straight forward.
I see some use for when stuff truly is a) expensive and b) can be parallelized. but these are the minority cases (at least from what I've encountered).
Is there any gain from async in the simple "Read some input, do some CRUD, display some output" scenario?
If a page is making a request to an external web service on every request, then using the asynchronous BCL APIs to fetch data from the web service will help free up resources on the server. This is because Windows can make a distinction between a managed ASP thread that is waiting for stuff (asynchronous web service call) and a managed ASP thread that is doing stuff (synchronous web service call). These asynchronous calls may end up as I/O Completion Ports behind the scenes, freeing ASP from essentially all of the burdens. This can result in the web site being able to handle more simultaneous requests. Specifically, when an ASP thread is waiting for a callback from an asynchronous operation, ASP may decide to reuse that thread to serve other requests in the meantime.
The synchronous BCL way to invoke external resources is the Get(), Read(), EndRead() etc family of methods. Their asynchronous counterpart is the BeginGet(), EndGet(), BeginRead(), EndRead() etc family of methods.
If you have a page that does two things simultaneously, and both are essentially wait for external data type operations, then the asynchronous BCL APIs will enable parallellism automatically. If they are calculate pi type operations, then you may want to use the parallell BCL APIs instead.
Here's one example when you will clearly gain from an async call: imagine that in order to render a page you need to aggregate information coming from different web services (or database calls) where each being independent and resulting in a network call. In this case the async pattern is very good because you have IO bound operations for which IO Completion Ports will be used and while waiting for the response you won't monopolize worker threads. While the operations are running no thread will be consumed on the server side.
If you have many CPU bound operations than async pattern will not bring much benefit because while you free the request worker thread from performing the operation, another thread will be consumed to do calculations.
I find this article a very useful reference.
What is the difference between Web Services Asynchronous Call and Asynchronous Task's.
We are working an a ASP.NET application that requires to make a call to a Web Service Method that will process thousand rows of data. This process usually takes between 2 to 3 minutes (maybe more maybe less it depends of the amount of Data). So we run all the time in Timeout's on that specific page.
So we decided to go in rout of calling this Web Service Method Asynchronously, but we had a conflict caused by HTTP handler of one of the UI component's that we are using. Well lucky on that case we could remove the page from the httphandler directives.
So far no issues, but here it comes the question, a coworker find out that we can use instead of Asynchronous Webs Services Call, wrap a Synchronous call in a Asynchronous Task in the ASP.NET page and be able to keep the directives to the component, and execute the Web Service Method with out getting a Timeout.
So now my concern is what kind of issues we can find using Asynchronous Task's instead of an Asynchronous Call.
Thank you in advance.
Web services should not be used in this manner by the way. There's a reason HTTP timeouts are so low. You should have the Web service trigger the task either by setting a flag in the DB that an actual service picks up on or the web service should spawn a process.
If I understand your scenario, there should be no issues. In both cases, your page is asynchronous. In both cases, you don't wait for the service to complete - you give up the request thread while the service is running. In both cases, your page takes the same amount of time to execute as it would if you had called the service synchronously.