Can Load Balancer be only responsible for TCP handshake and selecting appropriate workers and next network flows are transferred between client and worker?I searched lots of information about Load Balance , like LVS , but seems that IP Tunneling and Direct Routing Mode can only be used for achieving the following purpose:
The response data can be directly send from worker to client.
The request data is still relayed by load balancer which means that the request data is passed from client to Load Balancer to Worker..
What I want to achieve is as follows:
Client sends request to Load Balancer.
Load Balancer assigns a proper worker to Client.
Then next Client and worker communicate directly but not going through the load balancer.
The next steps may work ,but I think it is a little stupid..
Client sends request to Load Balancer.
Load Balancer response the proper worker's IP to client.
Client disconnect from Load Balancer and establish a TCP Connection to Worker directly.
Then next Client and worker can communicate as expected.
Does anyone have other better ideas?
Related
I'm trying to get my head around what type of load (e.g. in terms of network traffic) a load balancer for HTTP traffic can be expected to receive / forward / redirect.
Consider the following two different HTTP traffic scenarios.
Single HTTP request with a payload that exceeds significantly in size what fits on a single TCP IP packet (i.e. request requires many network packets)
Multiple HTTP requests over a single persistent TCP connection
For each of them:
Would all network traffic (e.g. TCP IP packets) go through the load balancer itself? (i.e. load balancer "acting like a cable" seeing and forwarding all traffic so-to-speak)
Or would the load balancer get the client to establish a direct TCP connection with the IP address of the app servers alleviating the LB of the network traffic?
Does this depend on the protocol? (e.g. say HTTP requests vs WebSocket)? Or do these protocols require that a TCP connection is always available with the load balancer receiving and forwarding all IP traffic throughout the full duration of the WebSocket connection or request-response HTTP transmission?
It depends.
There are application layer load balancers like HAProxy, where the full HTTP request and response is passed through the proxy. There are two separate TCP connections here: one between client and load balancer and another between load balancer and upstream server. The choice of the upstream server can be done based on the contents of the HTTP request, for example the Host header and/or path but also session cookies - to make sure that the same session is always handled by the same upstream server. If the decision for the upstream server is done on the HTTP request then the connection to the upstream server can only be established after the HTTP request was read, since it is not known before. But the request does not need to be inside a single packet.
There are network or transport layer load balancers which do not act on the packet payload at all. Instead the choice of upstream server is usually done based on the client IP, so that the same client ends up on the same upstream server. In this case the decision which upstream to use is already done on the first packet (i.e. the SYN starting the TCP handshake) and the client essentially establishes the connection directly with the upstream proxy - the load balancer only forwards the packets like a router does. The size of the HTTP request does not matter here either since the TCP payload is not even inspected to make the routing decision.
With a network or transport layer load balancer there can be asymmetric routing, i.e. the response might go a different way and not pass through the load balancer. With application layer load balancing the response goes back through the load balancer instead.
I want to write a "smart" load balancer, with "smart" I mean that it should route different request on different servers and ports based on the URL.
As example somehost.com/server1 should go to the server1 while somehost.com/server3 should go to the server3.
However I don't want my load balancer to establish the connection with the client, make the requests to the backend server, and the return to the client.
The load balancer should be as much transparent as possible.
The request should arrive to the backend servers and then return immediately to the client, without the need to go through the load balancer.
How is this achievable ? Are there example in erlang ?
I know how load balancers work for http requests. A client opens a connection with the LB, LB forwards the request to the backend servers, LB gets a response and from the same connection it sends the response to the client and closes the connection. I want to know the internal details of load balancer for websockets. How the connections are maintained and how the response is sent to the client. I read many questions on stackoverflow but none of them gave a clear picture of the internal implementation of LB
the LB just route the connection to a server behind it.
so as long you keep the connection open you will keep being connected to the same server and do not communicate with the LB again.
depending on the client, on reconnection you could be routed to another server.
I'm not sure how it works when some libraries fallback to JSON-P tho
Implementations of load balancers have great variety. There are load balancers that support websockets, like F5's BIG-IP (https://support.f5.com/kb/en-us/solutions/public/14000/700/sol14754.html), and LB's that I don't think support websocekts, like AWS ELB (there is a thread where somebody says they could make it with ELB but I suppose they added some other component behind ELB: How do you get Amazon's ELB with HTTPS/SSL to work with Web Sockets?.
Load Balancer's not only act as terminators of HTTP connections, they can terminate also HTTPS, SSL, and TCP connections. They can implement stickiness based on different parameters, like cookies, origin IP, etc. (like F5) In the case of ELB's they use only cookies, and it could be application generated cookies or LB generated cookies (both only with HTTP or HTTPS). Also stickiness can be kept for certain defined time, sometimes configurable.
Now, in order to forward data corresponding to websockets, they need to terminate, and forward, connections at level of SSL or TCP (not HTTP or HTTPS). Unless they understand websocket protocol (I don't know if any does it). Additionally, they need to keep stickiness to the server with which the connetion was opened. This is not possible with ELB but yes with more complex LB's like BIG-IP.
I am new to SignalR and I have a question on SignalR communication when we introduce a load balancer.
Lets assume we want to execute a void method on server side which receives some data as a parameter from client. Server takes that data and processes further. Lets say after processing for a while, it identifies that it has to send the notification back to client.
Case 1(Between client and server): Client calls void method on server side(Hub) by passing some data. Connection gets disconnected. Server processes the client data further. When it identifies that it has to push the notification back to client, it recreates the connection and pushes back the data to client.
Case 2(Between client and server with load balancer in between): How does the above scenario(Case 1) work here?. When server sends the push notification back to load balancer after processing client data, how does it know to which client it has to send the notification back?
You should read the scaleout docs. Short version: messages get sent to all servers, so if the client reconnects (it's not the server that establishes the connection!) before the connection times out , it will get the message.
Quote from the docs:
The cursor mechanism works even if a client is routed to a different
server on reconnect. The backplane is aware of all the servers, and it
doesn’t matter which server a client connects to.
I'm trying to determine how to load balance TCP traffic. I understand how HTTP load balancing works because it is a simple Request / Response architecture. However, I'm unsure of how you load balance TCP traffic when your servers are trying to write data to other clients. I've attached an image of the work flow for a simple TCP chat server where we want to balance traffic across N application servers. Are there any load balancers out there that can do what I'm trying to do, or do I need to research a different topic? Thanks.
Firstly, your diagram assumes that the load balancer is acting as a (TCP) proxy, which is not always the case. Often Direct Routing (or Direct Server Return) is used, or Destination NAT is performed. In both cases the connection between backend server and the client is direct. So in this case it is essentially the TCP handshake that is distributed amongst backend servers. See the following for more info:
http://www.linuxvirtualserver.org/VS-DRouting.html
http://www.linuxvirtualserver.org/VS-NAT.html
Obviously TCP proxies do exist (HAProxy being one), in which case the proxy manages both sides of the connecton, so your app would need to be able to identify the client by the incoming IP/Port (which would happen to be from the proxy rather than the client). The proxy will handle getting the messages back to the client.
Either way, it comes down to application design as I would imagine the tricky bit is having a common session store (a database of some kind, or key=>value store such as Redis), so that when your app server says "I need to send a message to Frank" it can determine which backend server Frank is connected to (from DB), and signal that server to send it the message. You reduce the problem of connections (from the same client) moving around different backend servers by having persistent connections (all load balancers can do this), or by using something intrinsically persistent like a websocket.
This is probably a vast oversimplification as I have no experience with chat software. Obviously DB servers themselves can be distributed amongst several machines, for fault-tolerance and load balancing.