My ASP .Net C# web application allows its users to send files from their account on my server to any remote server using FTP. I have implemented a WCF service to do this. The service instantiates a class for each user that spawns a worker thread which performs the FTP operations on the server. The client sends a command to the service, the service finds the worker thread assigned to the client and starts the FTP commands. The client then polls the service every two seconds to get the status of the FTP operation. When the client sends the "disconnect" command, the class and the worker thread doing the FTP operations is destroyed.
The FTP worker thread needed to persist between the client's queries because the FTP processing can take a long time. So, I needed a way for the client to always get the same instance of the FTP class between calls to the service. I implemented this service as a singleton, thus:
[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single)]
public class UserFtpService : IUserFtpService
{
private SortedDictionary<string, UserFTPConnection> _clients = new SortedDictionary<string, UserFTPConnection>();
...
}
Where "UserFTPConnection" is the class containing the worker thread and the user's account name is used for the indexing in the dictionary.
The question I have is this: In the books I have read about WCF, the singleton instance is called "the enemy of scalability." And I can see why this is so. Is there a better way to make sure the client gets the same instance of UserFTPConnection between queries to the WCF service other than using a singleton?
Actually here your first problem is synchronizing the access to this static object. Dictionary<TKey, TValue> is not thread safe so you must ensure that only one thread is accessing it at the same time. So you should wrap every access to this dictionary in a lock, assuming of course you have methods that are writing and others that are reading. If you are only going to be reading you don't need to synchronize. As far as singleton being the enemy of scalability, that's really an exaggerated statement and pretty meaningless without a specific scenario. It would really depend on the exact scenario and implementation. In your example you've only shown a dictionary => so all we can say is that you need to ensure that no thread is reading from this dictionary while other is writing and that no thread is writing to this dictionary while other thread is reading.
For example in .NET 4.0 you could use the ConcurrentDictionary<TKey, TValue> class which is thread safe in situations like this.
One thing's for sure though: while the singleton pattern might or might not be an enemy of scalability depending on the specific implementation, the singleton pattern is the arch-enemy of unit testability in isolation.
If you are going to use a singleton, I'd recommend also setting ConcurrencyMode to ConcurrencyMode.Multiple. For example...
[ServiceBehavior(ConcurrencyMode = ConcurrencyMode.Multiple, InstanceContextMode = InstanceContextMode.Single)]
public class UserFtpService : IUserFtpService
{
}
If you don't do this, your WCF service will be a singleton but only allow one thread to access at a time, which would certainly effect performance. Of course you will need to ensure thread safety of collections (as in previously mentioned answer).
Related
#WebListener
public class AllRequestsWebListener implements ServletRequestListener {
#Inject HttpRequestProducer producer;
public void requestInitialized(ServletRequestEvent sre) {
producer.requestInitialized(sre);
}
}
...
#RequestScoped
public class HttpRequestProducer {
...
}
I don't know howto inject request-bean as method-parameter and therefore I can guess that it will work properly when Request-bean injection is threadLocal. Can someone explain me how it's implemented in a thread-safe manner?
What you have injected in your bean is a proxy representing the real deal. The proxy will always forward the invocation to the correct bean
Intuition based answer
I believe it is thread safe, as request scope is thread safe (session and above are not, as a user can open multiple browser sessions and use the same session ID)
I tested it, although it's empiric evidence, but the injected HttpRequestProducer gets a new instance each request.
Note that the requestInitialized and requestDestroyed can be (and in practice are) different threads, so I will investigate further if you intend to use the same injected object on both methods.
Specs backed answer
The hard part was to find hard evidence for this claim in the specs.
I looked into the CDI spec and couldn't quickly find conclusive evidence that a #RequestScoped object is thread safe (e.g. using thread local) however I assume that a #RequestScoped bean is using the same scope as the scoped beans in Java EE 5: (see here)
In there this clause is interesting:
Controlling Concurrent Access to Shared Resources In a multithreaded
server, it is possible for shared resources to be accessed
concurrently. In addition to scope object attributes, shared resources
include in-memory data (such as instance or class variables) and
external objects such as files, database connections, and network
connections.
Concurrent access can arise in several situations:
Multiple web components accessing objects stored in the web context.
Multiple web components accessing objects stored in a session.
Multiple threads within a web component accessing instance variables.
A web container will typically create a thread to handle each request.
If you want to ensure that a servlet instance handles only one request
at a time, a servlet can implement the SingleThreadModel interface. If
a servlet implements this interface, you are guaranteed that no two
threads will execute concurrently in the servlet’s service method. A
web container can implement this guarantee by synchronizing access to
a single instance of the servlet, or by maintaining a pool of web
component instances and dispatching each new request to a free
instance. This interface does not prevent synchronization problems
that result from web components accessing shared resources such as
static class variables or external objects. In addition, the Servlet
2.4 specification deprecates the SingleThreadModel interface.
So in theory, it seems that the object itself is going to have one instance per request thread, however I couldn't find any hard evidence that this is supported.
I have a WCF service with ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple). I want to use ThreadStatic variable to srore data.
I start worrying about is it possible two parallel requests for the same or different operationContracts get handled by the same thread serverside, because if this happens my ThreadStatic variable will get overriden.(I.e. something like the thread changing between HttpHandlers and HttpModules in ASP.NET)
I made a spike service with the same ServiceBehaviour and maxConcurrentCalls="2". After that a wcf client called the service with 50 parallel requests and my worry did not occur. However this is not a 100% proof.
Thank in advance!
Irrespective of the ConcurrencyMode, a ThreadStatic value will persist when your request terminates and the thread is returned to the thread pool. The same thread can be reused for a subsequent request, which will therefore be able to see your ThreadStatic value.
Obviously this won't be true for two concurrent requests, because by definition they will be executed on different threads.
From comments:
Also by definition MSDN says: 'The service instance is multi-threaded. No synchronization guarantees are made. Because other threads can change your service object at any time, you must handle synchronization and state consistency at all times.' So it is not so obvious:)
This means that a single instance of your service class can be accessed concurrently by multiple requests. So you would need to handle synchronization for any accesses to instance members of the service class.
However ThreadStatic members are by definition only used by one thread (and hence one request) at a time, so don't need synchronization.
The direct answer to your question is Joe's answer.
However you mention in the comments you are using an ambient design pattern. That pattern is already implemented in WCF as the OperationContext and is specifically designed to be extensible. I highly recommend using OperationContext over any custom thread storage.
See Where to store data for current WCF call? Is ThreadStatic safe?
I wanted to add to Joe's answer here because I would recommend that you use some sort of correlation for your requests if you're needing to store state. The threading model will become very convoluted and unreliable in production.
Further, now imagine you have two IIS servers hosting this service and a hardware or software load balancer forward facing so that you can consume it. To ensure that the correct state is gathered you'll need correlation because you never know which server the service will be started on. In the post below I mocked up a simplified version of how that might work. One thing to keep in mind is that the SessionState would need to be kept in a shared location to all instances of the service, an AppFabric Cache server for example.
Global Variable between two WCF Methods
I am new to server side coding and JSP/servlets. I have a code which has 3 classes. 1st is a Serv class inherited from java httpservlet. In this i have doPost() method implemented. In doPost() i use object of the 2nd class ResourceClass. ResourceClass is a singleton class. Hence essentially to use any method is do something like ResourceClass.getInstance().readResource();
Now readResource furthur uses Java Native access library to read a resource from disk. Now my question is Since as i understand if 1000 clients connect to my server(Apache Tomcat) for each new request i will have a new servlet serving the request. But all these servlets will essentially use the same singleton object. Hence will this reading be thread safe.
I do not change any internal state. So i think it wont effect my output hence the whole stuff is Idempotent. But will all these requests be queued making the singleton class object a bottleneck. Or will each servlet has its own copy.
Also if i change the Resource state then in that case will it be thread safe.
First of all, you won't have a new servlet for each request. The same, unique instance of servlet will be used to concurrently handle all the requests. The servlet is also a singleton: the web container instantiates only one instance.
You say that the requests to your ResourceClass singleton will be queued. They won't, unless you mark the method as synchronized or use some other locking mechanism. If you don't, then the threads will invoke your singleton method concurrently.
Whether it's thread-safe or not is impossible to say without seeing the code of your singleton and the code of the JNI library. The fact that it's read-only is a sign that it could be thread-safe, but it's not guaranteed.
In a Java EE server, you only have 1 instance of each servlet.
On the other hand, each http request is processed by the server in its own thread.
There is one instance of ResourceClass because it's a singleton so you will have a bottleneck if the readResource() method is synchronized.
Hi i am creating an API using WCF. My question can be broken down into two seperate ones
1) I have quite a few calls, for instance i have calls relating to the customer, products, orders, employees.
My question is should all this go into one public interface class e.g
public interface IRestService
public class RestService : IRestService
Or should I have one for each call e.g
public interface ICustomer
public class Customer : ICustomer
public interface IProducts
public class Products: IProducts
2) If you have an API which will be accessed by tens of thousands of users and thousands of users concurrently, how would you set up, what will your web config settings be for instance in terms of throttling. Also what setting would you give your InstanceContextMode, or ConcurrencyMode. Finally what type of binding would it be, bearing in mind websites and mobile phones can access the api.
For the sake of good practice, I would break up the API into separate interfaces so you have the option of splitting them into separate implementations in the future. You can still have just one service class implement all of the interfaces, like this:
public class RestService : ICustomer, IProducts, IOrders
However, it sounds as if you'd probably want to make them separate implementations anyway.
In terms of concurrency settings, ask yourself what resources need to be used on each call. If your service class's constructor can be written without any lengthy startup, then use PerCall. If you need to initialize expensive resources, then I'd recommend InstanceContextMode.Single with ConcurrencytMode.Multiple and make sure you write thread-safe code. Eg: make sure you lock() on any class properties or other shared resources before you use them.
Database connections would not count as "expensive to initialize", though, because ADO will do connection pooling for you and eliminate that overhead.
Your throttling settings will be revealed by testing, as Ladislav mentions. You'd want to stress-test your service and use the results to get an idea of how many machines you'd need to service your anticipated load. Then you'll need a dedicated load balancer to route requests as either round-robin, or something that checks the health of each server. Load balancers can be set up to GET a "systemhealth.asp" page and check the results. If you return an "OK" then that machine stays in the pool, or can be temporarily removed from the pool if it times out or returns any other status.
Your binding would need to be WebHTTPBinding for REST. BasicHTTPBinding is meant for SOAP interfaces and doesn't support [WebGet], for example.
If it doesn't have to be a REST service, then you can get a bit more performance by using NetTcpBinding.
If you really have few operations, single service can be used. Generally services are logical collection of related operations but the number of operations should be limited. Usually if your service have more than 20 operations you should think about refactoring.
Do you plan to use REST service? I guess you do because of your first interface example. In such case you need WebHttpBinding (or similar custom binding) with default InstanceContextMode (PerCall) and ConcurrencyMode (Single) values. Only other meaningful combination for REST service is InstanceContextMode.Single and ConcurrencyMode.Multiple but it will create your service as singleton which can have impact on your service implementation. My rule of thumb: Don't use singleton service unless you really need it.
Throttling configuration is dependend on your service implementation and on performance of your servers. What does thousands concurrent users really mean for you? Processing thousands of requests concurrently requires good server cluster with load balancer or hosting in Azure (cloud). All is also dependend on the speed of processing (operation implementation) and size of messages. The correct setting for MaxConcurrentInstances and MaxConcurrentCalls (should be same for PerCall instancing) should be revealed by performance testing. Default values for service throttling have changed in WCF 4.
Normally Servlets are initiated just once and web container simple spawns a new thread for every user request. Let's say if I create my own web container from scratch and instead of Threads, I simply create Servlets as Singleton. Will I be missing anything here? I guess, in this case, the singleton can only service one user request at a time and not multiple.
Normally Servlets are initiated just once and web container simple spawns a new thread for every user request.
The first statement is true, but the second actually not. Normally, threads are been created once during applications startup and kept in a thread pool. When a thread has finished its request-response processing job, it will be returned to the pool. That's also why using ThreadLocal in a servletcontainer must be taken with high care.
Let's say if I create my own web container from scratch and instead of Threads, I simply create Servlets as Singleton. Will I be missing anything here?
They does not necessarily need to follow the singleton pattern. Just create only one instance of them during application's startup and keep them in memory throughout application's lifetime and just let all threads access the same instance.
I guess, in this case, the singleton can only service one user request at a time and not multiple.
This is not true. This will only happen when you synchronize the access to the singleton's methods on an application-wide lock. For example by adding the synchronized modifier to the method of your servlet or a synchronized(this) in the manager's method who is delegating the requests to the servlets.
JavaEE used to have a mechanism for this - a marker interface called SingleThreadModel that your servlet could implement:
Ensures that servlets handle only one request at a time. This interface has no methods.
If a servlet implements this interface, you are guaranteed that no two threads will execute concurrently in the servlet's service method. The servlet container can make this guarantee by synchronizing access to a single instance of the servlet, or by maintaining a pool of servlet instances and dispatching each new request to a free servlet.
Note that SingleThreadModel does not solve all thread safety issues. For example, session attributes and static variables can still be accessed by multiple requests on multiple threads at the same time, even when SingleThreadModel servlets are used. It is recommended that a developer take other means to resolve those issues instead of implementing this interface, such as avoiding the usage of an instance variable or synchronizing the block of the code accessing those resources. This interface is deprecated in Servlet API version 2.4.
Containers could use this to instantiate a new servlet for each request, or maintain a pool of them, if they chose to.
This was deprecated in Servlet 2.4, for the reasons documented above. Those same reasons still apply to your question.
That's basically it.
I would question the motivations for creating your own container, with so many available for a wide range of purposes.