I'm trying to use the round robin (RR) load balancer (LB) with a single backend. Is it possible to have RR LB create multiple connections (sub channels) for a single backend without having to write a custom LB implementation?
Related
I have a gRPC client, not dockerised, and server application, which I want to dockerise.
What I don't understand is that gRPC first creates a connection with a server, which involves a handshake. So, if I want to deploy the dockerised server on ECS with multiple instances, then how will the client switch from one to the other (e.g., if one gRPC server falls over).
I know AWS loadbalancer now works with HTTP 2, but I can't find information on how to handle the fact that the server might change after the client has already opened a connection to another one.
What is involved?
You don't necessarily need an in-line load balancer for this. By using a Round Robin client-side load balancing policy along with a DNS record that points to multiple backend instances, you should be able to get some level of redundancy.
How can I use Network Endpoint Groups (NEGs) from other Google Cloud project as a Load Balancer backend? Is there any solution?
You have 2 solutions to achieve this
Use Internet Network Endpoint Group to reach the load balancer of the second project.
Create a peering between the 2 projects and create Zonal Network Endpoint Groups to reach the VM in the second project
All depend on the level of separation of your projects, the which to manage them independently and so on.
In my project, we already have an external load balancer. However, there are several teams within the organisation which uses our internal load balancer. I want to know why do we need internal load balancer if we already have a public-facing external load balancer? Please elaborate.
I answer here to your question in the comment because it's too long for a comment
Things are internal, other are external. For examples
You have an external TCP/UDP load balancer
Your external Load Balancer accepts connexion on port 443 and redirects them to your backend with NGINX installed on it
Your backend needs a MongoDB database. You install your database on a compute and your choose to abstract the VM IP and to use your Load Balancer
You define a new backend on your external load balancer on the port 27017
RESULT: Because the load balancer is external, your MongoDB is publicly exposed on the port 27017.
If you use an internal load balancer, it's not the case, and you increase the security. Only the web facing port is open (443), the rest is not accessible from internet, only by your in your project.
You should check the documentation and then decided if your use case requires using internal load balancer or not. Below you can find links to the Google Cloud documentation and an example.
At first, have a look at the documentation Choosing a load balancer:
To decide which load balancer best suits your implementation of Google
Cloud, consider the following aspects of Cloud Load Balancing:
Global versus regional load balancing
External versus internal load balancing
Traffic type
After that, have a look at the documentation Cloud Load Balancing overview section Types of Cloud Load Balancing:
External load balancers distribute traffic coming from the Internet to your Google Cloud Virtual Private Cloud (VPC) network.
Global load balancing requires that you use the Premium Tier of
Network Service Tiers. For regional load balancing, you can use
Standard Tier.
Internal load balancers distribute traffic to instances inside of Google Cloud.
and
The following diagram illustrates a common use case: how to use
external and internal load balancing together. In the illustration,
traffic from users in San Francisco, Iowa, and Singapore is directed
to an external load balancer, which distributes that traffic to
different regions in a Google Cloud network. An internal load balancer
then distributes traffic between the us-central-1a and us-central-1b
zones.
More information you can find at the documentation.
UPDATE Have a look at the possible use cases for internal HTTP(S) load balancer and for internal TCP/UDP load balancer and check if they're suitable for you and if using them could improve your service.
It's not required to use internal load balancer if you don't need it.
Clients are connecting to API gateway server through websocket connection. This server just orchestrates swarm of cloud functions, that are handling all of the data requesting and transforming. Server is statefull - it holds essential session data, which is defining, for example, what cloud functions are allowed to be requested by a given user.
This server doesn't use socket to broadcast data, so socket connections are not interacting between each other, and will not be doing this. So, all it needs to handle is single-client-to-server communication.
What will happen if i'll create bunch of replicas and put load balancer in front of all of them (like regular horizontal scaling)? If a user got connected to certain server instance, then his connection will stick there? or it will be switching between instances by load balancer?
There is a parameter available for load balancer that allows you to do what you are looking for: Session affinity.
"Session affinity if set attempts to send all network request from the same client to the same virtual machine instance."
Actually even if it seems to be related to load balancer you set it while creating target pools and/or backends. You should check if this solution can be applied to your particular configuration.
Docker Swarm mode achieves Inner Load Balancing, and as far as I know, nginx is called hard load balancing, zookeeper is kinda soft load balancing.
So what's the mechanism of the Inner Load Balancing coming along with Docker v1.12?
Does it embed a nginx inside or similar methods like zookeeper?
"Inner" load balancing? Not exactly.
Commit ea4fef2 documents it (docs/swarm/key-concepts.md) as
Swarm uses ingress load balancing to expose the services you want to make available externally to the Swarm.
Swarm can automatically assign the service a PublishedPort or you can configure a PublishedPort for the service in the 30000-32767 range.
External components, such as cloud load balancers, can access the service on the PublishedPort of any node in the cluster, even if the node is not currently running the service.
Swarm has an internal DNS component that automatically assigns each service in the Swarm DNS entry.
Swarm uses internal load balancing to distribute requests among services within the cluster based upon the services' DNS name.
Right now (docker 1.12 August 2016), that inner load balancing does not work consistently: issue 25325
➜ ~ time curl http://10.218.3.5:30000
I'm 272dd0310a95
curl http://10.218.3.5:30000 0.01s user 0.01s system 6% cpu 0.217 total
➜ ~ time curl http://10.218.3.5:30000
curl: (7) Failed to connect to 10.218.3.5 port 30000: Operation timed out
And swarmkit issue 1077 illustrates there is no plan yet to
provide capabilities for session stickiness (cookie-based etc.) in this router mesh.
As awesome as it would be, not all apps are stateless, and we need to route users to the proper container in certain cases
Because:
since we do load balancing at L3/L4 it cannot be bases on things like session cookie.
The best that can be done is to have Source IP based stickiness.
And source IP is not always good enough:
That wouldn't work for our case.
We would have an upstream load balancer (F5) which would make traffic appear to come from a single IP, the "SNAT pool" IP on the F5 since it is a full proxy.
Effectively, Source IP based stickiness would cause all requests to go to one container since all the source IPs would come from the same address.
So the internal load balancer remains quite "basic":
The main issue with adding "session stickyness" is that there are a hundred ways to do it.
It is also an L7 feature, whereas our load balancing operates at L3/4.
There are two high-level paths here:
Monitor events coming from the docker API to modify F5 state to route directly task slots.
Integrate with libnetwork and have the loadbalancer operate as an L7 LB would if it were running directly in the swarm.
The conclusion for now is:
If you want to handle all aspects of load balancing and not use IPVS, you can disable it by running services in DNSRR mode. You can run any load balancer inside of swarm to do load balancing, bypassing the service VIP and populate backends with the DNSRR entries.
That is why the latest release 1.12 has, with PR 827, added support for DNSRR mode and disabling ingress.