I am confused with the load balancer performance using Haproxy, NginX, or Service Mesh.
Assume 2 different endpoints (Endpoint 1, Endpoint 2), where Endpoint 1 can handle more requests than Endpoint 2 in a certain period of time. I use round-robin load balancing in Haproxy.
Different scenarios:
how can I achieve the 1600 completed requests in scenario C?
Related
I am trying to do routing using Nginx via TCP layer.
I have multiple different databases where I need to perform routing from API to them base on rule or the request that is received by the API
+++++ +++++++++++++ +++++++--------> mysqlServer(managerDb)
+API+ ----->+MySQLClient+---->+Nginx+--------> mysqlServer(customerDb)
+++++ +++++++++++++ +++++++--------> mysqlServer(........Db)
I already did the typical one with load balancing but I want it to be routed based on the request
thanks
Nginx mainly works at layer 7 (application), what you need is something at layer 4 (transport) for this HAProxy could better help to achieve what you need since it can do both HTTP and TCP (db load balancing with HAProxy example)
Probably by using Nginx plus you could also do the same, check this article regarding how to do layer 4 load balancing.
Also look at TCP and UDP load balancing with requires the latest open source Nginx built with the --with-stream configuration flag.
Here an article explaining how to do load balancing with HAProxy
My web app runs on AWS Ec2. I am throttling requests in my servlet doPost() based on various parameters following Protect yourself against Dos attacks answer.
How do I throttle requests before it gets to my servlet?
Maybe based on IP (or) based on tps at load balancer level.
I want to write a "smart" load balancer, with "smart" I mean that it should route different request on different servers and ports based on the URL.
As example somehost.com/server1 should go to the server1 while somehost.com/server3 should go to the server3.
However I don't want my load balancer to establish the connection with the client, make the requests to the backend server, and the return to the client.
The load balancer should be as much transparent as possible.
The request should arrive to the backend servers and then return immediately to the client, without the need to go through the load balancer.
How is this achievable ? Are there example in erlang ?
These are two phrases I hear about very often, mainly associated with Nginx. Can someone give me a laymans defintion?
Definitions are often difficult to understand. I guess you just need some explanation for their use case.
A short explanation is: load balancing is one of the functionalities of reverse proxy, and reverse proxy is one of the softwares that can do load balancing.
And a long explanation is given below.
For example a service of your company has customers in UK and German. Because the policy is different for these two countries, your company has two web servers, uk.myservice.com for UK and de.myservice.com for German, each with different business logic. In addition, your company wants there to be only one unified endpoint, myservice.com for the service. In this case, you need to set up a reverse proxy as the unified endpoint. The proxy takes the url myservice.com, and rewrites the url of incoming requests so that requests from UK(determined by source ip) go to uk.myservice.com and requests from German go to de.myservice.com. From the view of a client from UK, it never knows the response is actually generated from uk.myservice.com.
In this case, the load of request traffic to the service is actually balanced to servers on uk.myservice.com and de.myservice.com as a side effect. So we normally don't call it used as a load balancer, just say it as a reverse proxy.
But lets say if your company uses the same policy for all countries, and has 2 servers, a.myservice.com and b.myservice.com, only for the reason that the work load is to heavy for one server machine. In this case, we normally call the reverse proxy as load balancer to emphasize the reason why it is being used.
Here is the basic definition:
Reverse Proxy is a proxy host, that receives requests from a client, and sends it to one of the servers behind itself. Nginx and apache httpd are commonly used as reverse proxies. These are in the administrative network of the web server that a servers a request.
This is in contrast with a (forward) Proxy, which sits in front of a client, and sends requests on behalf of a client to a web server. As an example, your corporate network address translator is a forward proxy. These are in the administrative network of the client from where the request originates.
Load balancing is a function performed by reverse proxies. The client requests are received by a load balancer, and the load balancer tries to send that request to one of the nodes (hosts) in the server pool, in an attempt to balance the load across various nodes.
I see both of them as a functionality of a HTTP/Web Server.
Load balancer’s job is to distribute the workload between servers node in a way that makes the best use of it.
Reverse proxy is a interface for external world ,forwarding request to a server node (even when we have a single node)
Its other use cases are caching of static content ,compression etc
I'm trying to determine how to load balance TCP traffic. I understand how HTTP load balancing works because it is a simple Request / Response architecture. However, I'm unsure of how you load balance TCP traffic when your servers are trying to write data to other clients. I've attached an image of the work flow for a simple TCP chat server where we want to balance traffic across N application servers. Are there any load balancers out there that can do what I'm trying to do, or do I need to research a different topic? Thanks.
Firstly, your diagram assumes that the load balancer is acting as a (TCP) proxy, which is not always the case. Often Direct Routing (or Direct Server Return) is used, or Destination NAT is performed. In both cases the connection between backend server and the client is direct. So in this case it is essentially the TCP handshake that is distributed amongst backend servers. See the following for more info:
http://www.linuxvirtualserver.org/VS-DRouting.html
http://www.linuxvirtualserver.org/VS-NAT.html
Obviously TCP proxies do exist (HAProxy being one), in which case the proxy manages both sides of the connecton, so your app would need to be able to identify the client by the incoming IP/Port (which would happen to be from the proxy rather than the client). The proxy will handle getting the messages back to the client.
Either way, it comes down to application design as I would imagine the tricky bit is having a common session store (a database of some kind, or key=>value store such as Redis), so that when your app server says "I need to send a message to Frank" it can determine which backend server Frank is connected to (from DB), and signal that server to send it the message. You reduce the problem of connections (from the same client) moving around different backend servers by having persistent connections (all load balancers can do this), or by using something intrinsically persistent like a websocket.
This is probably a vast oversimplification as I have no experience with chat software. Obviously DB servers themselves can be distributed amongst several machines, for fault-tolerance and load balancing.