I'm currently developing an application from the ground up that requires some form of communication between the user-facing web servers, and the backend (hidden) worker servers that communicate with endpoint devices. I've diagrammed the whole environment below:
My question is, what is the best way (industry way, perhaps?) to send requests from the web servers to the worker servers with a request-response structure? My current implementation (as recommended by other answers) uses a REDIS message broker (although RabbitMQ and other solutions seem identical to how I am using it) to achieve this communication. Downside is, it's not request-response oriented.
Here is a sample scenario:
User A clicks a "PING Device E" button on the website
Webserver 1 receives this as an HTTP request. Webserver 1 knows Device E is connected to Worker Server 2 (knowing this is not a problem), and thus sends a message to Worker Server 2, telling it to send the PING request to Device E.
Device E responds to Worker Server 2 with "PONG". Worker Server 2 then completes the request from Web Server 1, telling it that it received "PONG".
Web Server 1 completes the initial HTTP request, telling the user the device responded with "PONG"
Related
I have a SignalrHub hosted in IIS in two servers which are loadbalanced. I used backplane rabbitmq. I had setup rabbitmq server on Server1 and on startup class registering with rabbitmq backplane pointing to the Server1 IP. Also as given in example (https://github.com/mdevilliers/SignalR.RabbitMq) I am trying to raise the message from server on every 10 sec to all clients. The issue is, when I run only one client which is connceted to Server 1, the client is started receiving messages from Server 2 also though its not connected physically with Server 2.
But from what I understand, if Client 1 is connected with Server 1 and Client 2 connected with Server 2, when message is sent from Server 1, it will reach to server 2 and those who are connected with Server 2 will receive the Server 1's message. But in my case when there is only one client connected with Server 1, it started receiveing from Server 2 also.
But when I tried the sample given in project (https://github.com/mdevilliers/SignalR.RabbitMq/tree/master/SignalR.RabbitMQ.Example) the same case by running a timer to send message to clients, using IIS Express running on two differnt Ports, this issue not there. I mean client1 receives only from Server 1 , not from Server 2 (though its up and running).
Let me know if anything is wrong here .
thanks
I've been looking into SignalR implementation incorporated with a load balancer
and have a few basic (if not simple sounding) questions.
I must preface this by saying I've got zero (0) experience with load balancers.
We will have 2 servers sitting behind a load balancer.
The client is an ASP .Net application.
We've been told that the load balancer maintains session affinity.
Consider the following scenario:
Client1 & Client2 -- connect to GroupA--> Server1
Client3 & Client4 -- connect to GroupA--> Server2
1) Server1 makes a client call to GroupA - this assumes that
Clients 1-4 will get the notification, correct?
2) How does the processing occur on this?
3) Is it a function of SignalR itself, or the load balancer?
4) When sending messages at the group level, do messages only get delivered to the client
apps associated with the group on that specific server, or will messages get forwarded
to all clients of that group?
Does anyone have any thoughts on this?
Thanks,
JB
I believe the scenario you're looking at requires a SignalR Backplane to be setup.
Here's a relevant selection from the article but you'll want to read the full thing to answer your specific questions:
Each server instance connects to the backplane through the bus. When a
message is sent, it goes to the backplane, and the backplane sends it
to every server. When a server gets a message from the backplane, it
puts the message in its local cache. The server then delivers messages
to clients from its local cache.
We have a requirement wherein the server needs to push the data to various clients. So we went ahead with SSE (Server-Sent events). I went through the documentation but am still not clear with the concept. I have following queries :
Scenario 1. Suppose there are 10 clients. So all the 10 clients will send the initial request to server. 10 connections are established. When the data enters the server, a message is pushed from server to client.
Query 1 : Will the server maintain the IP address of all the client? If yes is there an API to check it?
Query 2: What will happen if all the 10 client windows are closed? Will the server abort all connections after a period of time?
Query 3: What will happen if the Server is unable to send messages to client due to unavailability of client like machine shutdown. Will the server abort all connections after a period of time for those client for whom they are unable to send the message?
Please clarify?
This depends on how you implement the server.
If using PHP, as an Apache module, then each SSE connection creates a new PHP instance running in memory. Each "server" is only serving one client at a time. Q1: yes, but not your problem: you just echo messages to stdout. Q2/Q3: If the client closes the connection, for any reason, the PHP process will shutdown when it detects this.
If you are using a multi-threaded server, e.g. using http in node.js. Q1: the client IP is part of the socket abstraction, and you just send messages to the response object. Q2/Q3: as each client connection closes the socket, the request process that was handling it will end. Once all 10 have closed your server will still be running, but not sending data to any clients.
One key idea to realize with SSE is that each client is a dedicated socket. It is not a broadcast protocol, where you push out one message and all clients get exactly the same message. Instead, you have to send the data to each client, individually. But that also means you are free to send customized data to each client.
Is it possible to configure IIS in such a way that it can handle multiple HTTP requests that arrive on the same TCP socket in HTTP pipelining mode in parallel?
We have a problem where multiple requests are done by a web client in a single TCP socket, using HTTP pipelining. The client basically sends let's say 10 requests at once, and then the server sends 10 responses (in the same order as the requests). Our server takes quite some time for each request, mostly waiting for external IO. It would be much more efficient if IIS could start to work on all 10 requests in parallel, then serialize the responses in the correct order back to the client. Obviously, the server would need some way to cache responses if e.g. response 3 is available earlier than response 2.
Is that possible somehow? Maybe this is not possible in IIS, or I'm just searching for the wrong keywords... We are running IIS 7.5 and ASP.NET 4.5 on Windows Server 2008 R2.
We came across the same issue in IIS 7.5.
Our solution was to enable "Web Garden"... and it really really works well! It's just that you can't have a "session" based web site. So if you have clients "logging in", you will have to re-configure the process. (We used cookies to store an encrypted token - anyway that's besides the point).
Go to:
Internet Information Service > Applications Pools
Select the Pool being used (you should have a pool per site)
Click Advanced Settings...
Find "Maximum Worker Processes" and crank that sucker!
The amount of processes that you push it up to now depends entirely on how much RAM your system has. You can of course monitor and control this your self.
With a "Web Garden" enabled, you will notice (with Process Explorer or something similar), IIS will spawn a new instance of w3wp.exe for each request, up to the max number you specified. New requests simply get processed by the next available Worker Process available, enabling true IIS parallel request processing. If two requests come in within moments of each other, and request 2 is completed before request 1, request 2 is sends its response.
IIS uses the HTTP server api (that uses HTTP.sys); so I did a simple test -
wrote an HTTP server using this API,
wrote a Winsock client that opens a connection and sends 2 http requests
I observed that if I called HttpReceiveHttpRequest twice on the server (without sending the response for the first request), it doesn't receive the second request (basically, the second call blocks). This holds true for both PUT and GET requests.
It appears that HTTP.sys is in fact serializing requests to IIS on a single connection; I couldn't find any configuration on HTTP.sys that might modify this behavior.
As you can see while the requests from all users all over the web are just being added to the queue, and building up and up (Green) - only 1 single Request is Executing (Blue).
This doesn't really answer the question - but its an beautiful illustration of this disastrous situation.
How does a web server handle multiple incoming requests at the same time on a single port(80)?
Example :
At the same time 300k users want to see an image from www.abcdef.com which is assigned IP 10.10.100.100 and port 80. So how can www.abcdef.com handle this incoming users' load?
Can one server (which is assigned with IP 10.10.100.100) handle this vast amount of incoming users? If not, then how can one IP address be assigned to more than one server to handle this load?
A port is just a magic number. It doesn't correspond to a piece of hardware. The server opens a socket that 'listens' at port 80 and 'accepts' new connections from that socket. Each new connection is represented by a new socket whose local port is also port 80, but whose remote IP:port is as per the client who connected. So they don't get mixed up. You therefore don't need multiple IP addresses or even multiple ports at the server end.
From tcpipguide
This identification of connections using both client and server sockets is what provides the flexibility in allowing multiple connections between devices that we take for granted on the Internet. For example, busy application server processes (such as Web servers) must be able to handle connections from more than one client, or the World Wide Web would be pretty much unusable. Since the connection is identified using the client's socket as well as the server's, this is no problem. At the same time that the Web server maintains the connection mentioned just above, it can easily have another connection to say, port 2,199 at IP address 219.31.0.44. This is represented by the connection identifier:
(41.199.222.3:80, 219.31.0.44:2199).
In fact, we can have multiple connections from the same client to the same server. Each client process will be assigned a different ephemeral port number, so even if they all try to access the same server process (such as the Web server process at 41.199.222.3:80), they will all have a different client socket and represent unique connections. This is what lets you make several simultaneous requests to the same Web site from your computer.
Again, TCP keeps track of each of these connections independently, so each connection is unaware of the others. TCP can handle hundreds or even thousands of simultaneous connections. The only limit is the capacity of the computer running TCP, and the bandwidth of the physical connections to it—the more connections running at once, the more each one has to share limited resources.
TCP Takes care of client identification
As a.m. said, TCP takes care of the client identification, and the server only sees a "socket" per client.
Say a server at 10.10.100.100 listens to port 80 for incoming TCP connections (HTTP is built over TCP). A client's browser (at 10.9.8.7) connects to the server using the client port 27143. The server sees: "the client 10.9.8.7:27143 wants to connect, do you accept?". The server app accepts, and is given a "handle" (a socket) to manage all communication with this client, and the handle will always send packets to 10.9.8.7:27143 with the proper TCP headers.
Packets are never simultaneous
Now, physically, there is generally only one (or two) connections linking the server to internet, so packets can only arrive in sequential order. The question becomes: what is the maximum throughput through the fiber, and how many responses can the server compute and send in return. Other than CPU time spent or memory bottlenecks while responding to requests, the server also has to keep some resources alive (at least 1 active socket per client) until the communication is over, and therefore consume RAM. Throughput is achieved via some optimizations (not mutually-exclusive): non-blocking sockets (to avoid pipelining/socket latencies), multi-threading (to use more CPU cores/threads).
Improving request throughput further: load balancing
And last, the server on the "front-side" of websites generally do not do all the work by themselves (especially the more complicated stuff, like database querying, calculations etc.), and defer tasks or even forward HTTP requests to distributed servers, while they keep on handling trivially (e.g. forwarding) as many requests per second as they can. Distribution of work over several servers is called load-balancing.
1) How does a web server handle multiple incoming requests at the same time on a single port(80)
==> a) one instance of the web service( example: spring boot micro service) runs/listens in the server machine at port 80.
b) This webservice(Spring boot app) needs a servlet container like mostly tomcat.
This container will have thread pool configured.
c) when ever request come from different users simultaneously, this container will
assign each thread from the pool for each of the incoming requests.
d) Since the server side web service code will have beans(in case java) mostly
singleton, each thread pert aining to each request will call the singleton API's
and if there is a need for Database access , then synchronization of these
threads is needed which is done through the #transactional annotation. This
annotation synchronizes the database operation.
2) Can one server (which is assigned with IP 10.10.100.100) handle this vast amount of incoming users?
If not, then how can one IP address be assigned to more than one server to handle this load?
==> This will taken care by loadbalancer along with routetable
answer is: virtual hosts, in HTTP Header is name of domain so the web server know which files run or send to client