Given a C++-application running on a linux-pc.
It has an integrated webserver associated with one thread.
Considering that, is it possible to have more than one client (active HTTP-Sessions) connected to the webserver ?
In my mind, I always though we need something like a thread pool in the webserver task. Acceptor threads on a listen socket accept new connection and put it into a connection queue. The task will then pick up connections and service requests.
Is there any possibilities to have multiple session with one thread ?
Many thanks
Related
Our application is expected to receive thousands of request every second and we are considering gRPC as one of our main service is in a different language.
My queries are
Can we use something like supervisor to spawn multiple workers (one gRPC server per service) as gRPC servers listening to the same port, Or is gRPC servers limited to only one per server/port
How would i go about the performance testing to determine maximum requests per gRPC server.
Thanks in advance
While you can certainly use supervisord to spawn multiple gRPC server processes, port sharing would be a problem. However, this is a Posix limitation, not a gRPC limitation. By default, multiple processes cannot listen on the same port. (to be clear, multiple processes can bind to the same port with SO_REUSEPORT, but this would not result in the behavior you presumably want).
So you have two options in order to get traffic routed to the proper service on a single port. The first option is to run all of the gRPC services in the same process and attached to the same server.
If having only a single server process won't work for you, then you'll have to start looking at load balancing. You'd front all of your services with any HTTP/2-capable load balancer (e.g. Envoy, Nginx) and have it listen on your single desired port and route to each gRPC server process as appropriate.
This is a very broad question. The answer is "the way you'd benchmark any non-gRPC server." This site is a great resource for some principles behind benchmarking.
I am learning Signal-R, and this is something that has been in my head during all time.
How does Signal-R fits in the IIS/ASP.NET life cycle?
How long does the Hubs live (I see they have re-connection semantics)?
Does IIS does prevent the shutdown of an AppDomain that has a persistent connection?
It is my understanding that IIS is designed to handle request-response scenarios. A request hits IIS, this finds the AppDomain, activate it, and then pass the request to it. And after an idle time, shutdown the AppDomain. If the request takes too long, a timeout exception is thrown.
Now let´s imagine that I have another application that broadcast information through a TCP socket. I want my javascript clients to get that information in real time, so I create a Signal-R web application. I can create a TCP client on application start, but what does guarantee that IIS is not going to shutdown the whole thing after some time with inactivity?
I could self host the Signal-R app in a window service, but then I would have to use a different port, enable cross domain, etc... Many problems for deployment. But, I am concerned about using an ASP.NET MVC application for this, since it looks to me like fitting a driving wheel in a motorbike.
Cheers.
SignalR in IIS/ASP.NET Lifecycle
SignalR uses Owin: http://owin.org/
A good article on Owin here: http://msdn.microsoft.com/en-us/magazine/dn451439.aspx
Hub object lifetime
From the SignalR docs: http://www.asp.net/signalr/overview/signalr-20/hubs-api/hubs-api-guide-server#transience:
You don't instantiate the Hub class or call its methods from your own code on the server; all that is done for you by the SignalR Hubs pipeline. SignalR creates a new instance of your Hub class each time it needs to handle a Hub operation such as when a client connects, disconnects, or makes a method call to the server.
Because instances of the Hub class are transient, you can't use them to maintain state from one method call to the next. Each time the server receives a method call from a client, a new instance of your Hub class processes the message. To maintain state through multiple connections and method calls, use some other method such as a database, or a static variable on the Hub class, or a different class that does not derive from Hub. If you persist data in memory, using a method such as a static variable on the Hub class, the data will be lost when the app domain recycles.
Your long running TCP client
This is not a problem with SignalR. Your TCP client can be shutdown by IIS: http://haacked.com/archive/2011/10/16/the-dangers-of-implementing-recurring-background-tasks-in-asp-net.aspx/
I would rather make the TCP client run in a windows service. The TCP client receives TCP broadcast messages and forwards the messages to the Hub using the SignalR .NET client.
Hubs are recreated on each SignalR request, so if you need a persistent connection you may have to look into using static vars or dictionary to hold state. But as you point ASP.NET can restart for a variety of reasons.
It depends on what persistancy you really need. If you have a connection that MUST stay alive at all times and cannot be torn down and reestablished then hosting in IIS is not the right choice. However, if you can re-establish the same connection after a shutdown, then maybe this can still work.
You can do quite a bit in making sure that ASP.NET apps don't shut down in recent versions of IIS:
http://weblog.west-wind.com/posts/2013/Oct/02/Use-IIS-Application-Initialization-for-keeping-ASPNET-Apps-alive
If that's not enough for you running as a separate service is an option. If you run as a service on the same IP address there are no cross domain concerns. Here's more info on running SignalR using a Windows Service:
http://weblog.west-wind.com/posts/2013/Sep/04/SelfHosting-SignalR-in-a-Windows-Service
How does a web server handle multiple incoming requests at the same time on a single port(80)?
Example :
At the same time 300k users want to see an image from www.abcdef.com which is assigned IP 10.10.100.100 and port 80. So how can www.abcdef.com handle this incoming users' load?
Can one server (which is assigned with IP 10.10.100.100) handle this vast amount of incoming users? If not, then how can one IP address be assigned to more than one server to handle this load?
A port is just a magic number. It doesn't correspond to a piece of hardware. The server opens a socket that 'listens' at port 80 and 'accepts' new connections from that socket. Each new connection is represented by a new socket whose local port is also port 80, but whose remote IP:port is as per the client who connected. So they don't get mixed up. You therefore don't need multiple IP addresses or even multiple ports at the server end.
From tcpipguide
This identification of connections using both client and server sockets is what provides the flexibility in allowing multiple connections between devices that we take for granted on the Internet. For example, busy application server processes (such as Web servers) must be able to handle connections from more than one client, or the World Wide Web would be pretty much unusable. Since the connection is identified using the client's socket as well as the server's, this is no problem. At the same time that the Web server maintains the connection mentioned just above, it can easily have another connection to say, port 2,199 at IP address 219.31.0.44. This is represented by the connection identifier:
(41.199.222.3:80, 219.31.0.44:2199).
In fact, we can have multiple connections from the same client to the same server. Each client process will be assigned a different ephemeral port number, so even if they all try to access the same server process (such as the Web server process at 41.199.222.3:80), they will all have a different client socket and represent unique connections. This is what lets you make several simultaneous requests to the same Web site from your computer.
Again, TCP keeps track of each of these connections independently, so each connection is unaware of the others. TCP can handle hundreds or even thousands of simultaneous connections. The only limit is the capacity of the computer running TCP, and the bandwidth of the physical connections to it—the more connections running at once, the more each one has to share limited resources.
TCP Takes care of client identification
As a.m. said, TCP takes care of the client identification, and the server only sees a "socket" per client.
Say a server at 10.10.100.100 listens to port 80 for incoming TCP connections (HTTP is built over TCP). A client's browser (at 10.9.8.7) connects to the server using the client port 27143. The server sees: "the client 10.9.8.7:27143 wants to connect, do you accept?". The server app accepts, and is given a "handle" (a socket) to manage all communication with this client, and the handle will always send packets to 10.9.8.7:27143 with the proper TCP headers.
Packets are never simultaneous
Now, physically, there is generally only one (or two) connections linking the server to internet, so packets can only arrive in sequential order. The question becomes: what is the maximum throughput through the fiber, and how many responses can the server compute and send in return. Other than CPU time spent or memory bottlenecks while responding to requests, the server also has to keep some resources alive (at least 1 active socket per client) until the communication is over, and therefore consume RAM. Throughput is achieved via some optimizations (not mutually-exclusive): non-blocking sockets (to avoid pipelining/socket latencies), multi-threading (to use more CPU cores/threads).
Improving request throughput further: load balancing
And last, the server on the "front-side" of websites generally do not do all the work by themselves (especially the more complicated stuff, like database querying, calculations etc.), and defer tasks or even forward HTTP requests to distributed servers, while they keep on handling trivially (e.g. forwarding) as many requests per second as they can. Distribution of work over several servers is called load-balancing.
1) How does a web server handle multiple incoming requests at the same time on a single port(80)
==> a) one instance of the web service( example: spring boot micro service) runs/listens in the server machine at port 80.
b) This webservice(Spring boot app) needs a servlet container like mostly tomcat.
This container will have thread pool configured.
c) when ever request come from different users simultaneously, this container will
assign each thread from the pool for each of the incoming requests.
d) Since the server side web service code will have beans(in case java) mostly
singleton, each thread pert aining to each request will call the singleton API's
and if there is a need for Database access , then synchronization of these
threads is needed which is done through the #transactional annotation. This
annotation synchronizes the database operation.
2) Can one server (which is assigned with IP 10.10.100.100) handle this vast amount of incoming users?
If not, then how can one IP address be assigned to more than one server to handle this load?
==> This will taken care by loadbalancer along with routetable
answer is: virtual hosts, in HTTP Header is name of domain so the web server know which files run or send to client
I'm developing a webservice that will, when called, notify another program via a tcp connection. My problem is where to store the open tcp connection. The way I understand web services, they start and end with each HTTP Request, with no room for application wide variables, like the open tcp connection.
Please correct me if I'm wrong.
Specifically, in what part of the asmx file, or outside of it, should I place the code for listening for incoming tcp traffic?
Application events in Global.asax should fire for a web service hosted as an application in IIS. You can use these. Keep in mind that they will fire even if a web page and not the web service is accessed in the same application.
You can place the tcp connection as a static member of the service class and make a static constructor that handles the instantiation.
This will create the tcp connection before the first access of the web service is handled and then persist the connection as long as the hosting process is running. The only drawback with that approach is that the tcp connection is process wide. If you host two instances of the web service within the same process (quite unlikely) they will share the same tcp connection.
Fortunately for you, you are wrong.
Application-wide events do fire plus you have the access to all asp.net containers, the Application container for application-level variables, the Session container for session-level variables (if the client side supports cookies, the session id could even be passed in a cookie) and the Items container for request-level variables.
However, whether or not this helps you to store an additional tcp listener (if I understand correcly) is another story, not obvious one.
Does stopping a BizTalk host instance also stop the Applications that run under it?
Or, what is the difference between stopping a host instance and stopping the applications under it?
No, host instances and applications are completely independent. You can stop a host instance and the application will remain in the started state. You can stop an application and the host instance will remain in the running state.
To understand the difference between stopping a host instance and stopping an application you first need to understand what these things are.
Basically, you need to think of your application as a set of assemblies plus some runtime configuration, and a set of logical subscriptions.
When you "start" an application up there are actually two steps which happen.
The parts of your application which need to receive messages (orchestrations and send ports) are enlisted. This ensures that an internal queue exists to receive the messages. Note that the application is not yet started, but it can receive and queue messages for processing later.
Then when you start the application the various parts of your application are able to process the messages.
The host instance is basically a windows service.
When you stop the host instance, all you are really doing is stopping the underlying windows service which runs the host instance. This means that all the assemblies which contain your application artifacts are unloaded, and the application will obviously stop processing. This is despite the fact that the application is still in the started state.
When you start the host instance again it loads your application assemblies back into memory and will be able to continue processing new messages. Messages which were being processed when the host instance was stopped may be in a state called suspended, but if they are can be manually resumed.
Hope this helps.
Yes, if you application run only on that host instance (meaning: application will stop to process messages). However internals of why it stopped processing is quite different. See explanation below and in hugh jadick's answer.
Stopping host instance for specified host type will stop execution of all artifacts (adapter handlers, receive locations, pipelines, orchestrations, etc.) that run on specified host. Application is a logical group of artifacts which can run on single or multiple host instances. Multiple applications can run on a single host instance, and vice versa. So, stopping an app is just shutdown of execution of artifacts, while stopping host instance is shutdown of physical instance there app artifacts are executing.