I am using Apache Camel with ActiveMQ for Routing Message to Queue. To avail high availability we can configure cluster of MQ server in case of system gets fail.
ActiveMQ also provide failover features. now i wanted to load balance two set of MQ server to single IP:port at TCP level. Can Failover feature would able to load balance two MQ servers ?
e.g.
One IP is load balanced.
192.168.0.1:61616 --> 192.168.1.1:61616,192.168.1.2:61616
Load balancing can be done through the "Network of Broker" feature of ActiveMQ, see http://activemq.apache.org/networks-of-brokers.html.
Client failover uri is for recovery, not load balancing. Load balancing messaging clients ends up being a complicated mess (since you can't predict message size).
A good approach is to use the failover:(tcp://...)?randomize=false and partition your producer traffic into groups by changing the order of the brokers in the uri.
Group1 producers: 192.168.1.1:61616,192.168.1.2:61616
Group2 producers: 192.168.1.2:61616,192.168.1.1:61616
Related
Our application is expected to receive thousands of request every second and we are considering gRPC as one of our main service is in a different language.
My queries are
Can we use something like supervisor to spawn multiple workers (one gRPC server per service) as gRPC servers listening to the same port, Or is gRPC servers limited to only one per server/port
How would i go about the performance testing to determine maximum requests per gRPC server.
Thanks in advance
While you can certainly use supervisord to spawn multiple gRPC server processes, port sharing would be a problem. However, this is a Posix limitation, not a gRPC limitation. By default, multiple processes cannot listen on the same port. (to be clear, multiple processes can bind to the same port with SO_REUSEPORT, but this would not result in the behavior you presumably want).
So you have two options in order to get traffic routed to the proper service on a single port. The first option is to run all of the gRPC services in the same process and attached to the same server.
If having only a single server process won't work for you, then you'll have to start looking at load balancing. You'd front all of your services with any HTTP/2-capable load balancer (e.g. Envoy, Nginx) and have it listen on your single desired port and route to each gRPC server process as appropriate.
This is a very broad question. The answer is "the way you'd benchmark any non-gRPC server." This site is a great resource for some principles behind benchmarking.
I have a situation where messages are being generated by an internal application but the consumers for the messages are outside our enterprise network. Will either of http(s) transport or REST connectivity work in this scenario, with HTTP reverse proxy on DMZ? If not, is it safe to have a broker on the DMZ which can act as gateway to outside consumers?
Well, the rest/http approach to connect to ActiveMQ is very limited as it does not support true messaging semantics.
Exposing an ActiveMQ broker is no less secure than any other communication software if precautions are taken (TLS, default passwords changed, high entropy passwords are used and/or mutual authentication, recent patches applied, web console/jolokia not exposed externally without precautions etc etc).
In fact - you can buy online ActiveMQ instances from Amazon - which indicates that at least they think it's not such a bad idea to put them on the Internet.
Clients are connecting to API gateway server through websocket connection. This server just orchestrates swarm of cloud functions, that are handling all of the data requesting and transforming. Server is statefull - it holds essential session data, which is defining, for example, what cloud functions are allowed to be requested by a given user.
This server doesn't use socket to broadcast data, so socket connections are not interacting between each other, and will not be doing this. So, all it needs to handle is single-client-to-server communication.
What will happen if i'll create bunch of replicas and put load balancer in front of all of them (like regular horizontal scaling)? If a user got connected to certain server instance, then his connection will stick there? or it will be switching between instances by load balancer?
There is a parameter available for load balancer that allows you to do what you are looking for: Session affinity.
"Session affinity if set attempts to send all network request from the same client to the same virtual machine instance."
Actually even if it seems to be related to load balancer you set it while creating target pools and/or backends. You should check if this solution can be applied to your particular configuration.
How does a web server handle multiple incoming requests at the same time on a single port(80)?
Example :
At the same time 300k users want to see an image from www.abcdef.com which is assigned IP 10.10.100.100 and port 80. So how can www.abcdef.com handle this incoming users' load?
Can one server (which is assigned with IP 10.10.100.100) handle this vast amount of incoming users? If not, then how can one IP address be assigned to more than one server to handle this load?
A port is just a magic number. It doesn't correspond to a piece of hardware. The server opens a socket that 'listens' at port 80 and 'accepts' new connections from that socket. Each new connection is represented by a new socket whose local port is also port 80, but whose remote IP:port is as per the client who connected. So they don't get mixed up. You therefore don't need multiple IP addresses or even multiple ports at the server end.
From tcpipguide
This identification of connections using both client and server sockets is what provides the flexibility in allowing multiple connections between devices that we take for granted on the Internet. For example, busy application server processes (such as Web servers) must be able to handle connections from more than one client, or the World Wide Web would be pretty much unusable. Since the connection is identified using the client's socket as well as the server's, this is no problem. At the same time that the Web server maintains the connection mentioned just above, it can easily have another connection to say, port 2,199 at IP address 219.31.0.44. This is represented by the connection identifier:
(41.199.222.3:80, 219.31.0.44:2199).
In fact, we can have multiple connections from the same client to the same server. Each client process will be assigned a different ephemeral port number, so even if they all try to access the same server process (such as the Web server process at 41.199.222.3:80), they will all have a different client socket and represent unique connections. This is what lets you make several simultaneous requests to the same Web site from your computer.
Again, TCP keeps track of each of these connections independently, so each connection is unaware of the others. TCP can handle hundreds or even thousands of simultaneous connections. The only limit is the capacity of the computer running TCP, and the bandwidth of the physical connections to it—the more connections running at once, the more each one has to share limited resources.
TCP Takes care of client identification
As a.m. said, TCP takes care of the client identification, and the server only sees a "socket" per client.
Say a server at 10.10.100.100 listens to port 80 for incoming TCP connections (HTTP is built over TCP). A client's browser (at 10.9.8.7) connects to the server using the client port 27143. The server sees: "the client 10.9.8.7:27143 wants to connect, do you accept?". The server app accepts, and is given a "handle" (a socket) to manage all communication with this client, and the handle will always send packets to 10.9.8.7:27143 with the proper TCP headers.
Packets are never simultaneous
Now, physically, there is generally only one (or two) connections linking the server to internet, so packets can only arrive in sequential order. The question becomes: what is the maximum throughput through the fiber, and how many responses can the server compute and send in return. Other than CPU time spent or memory bottlenecks while responding to requests, the server also has to keep some resources alive (at least 1 active socket per client) until the communication is over, and therefore consume RAM. Throughput is achieved via some optimizations (not mutually-exclusive): non-blocking sockets (to avoid pipelining/socket latencies), multi-threading (to use more CPU cores/threads).
Improving request throughput further: load balancing
And last, the server on the "front-side" of websites generally do not do all the work by themselves (especially the more complicated stuff, like database querying, calculations etc.), and defer tasks or even forward HTTP requests to distributed servers, while they keep on handling trivially (e.g. forwarding) as many requests per second as they can. Distribution of work over several servers is called load-balancing.
1) How does a web server handle multiple incoming requests at the same time on a single port(80)
==> a) one instance of the web service( example: spring boot micro service) runs/listens in the server machine at port 80.
b) This webservice(Spring boot app) needs a servlet container like mostly tomcat.
This container will have thread pool configured.
c) when ever request come from different users simultaneously, this container will
assign each thread from the pool for each of the incoming requests.
d) Since the server side web service code will have beans(in case java) mostly
singleton, each thread pert aining to each request will call the singleton API's
and if there is a need for Database access , then synchronization of these
threads is needed which is done through the #transactional annotation. This
annotation synchronizes the database operation.
2) Can one server (which is assigned with IP 10.10.100.100) handle this vast amount of incoming users?
If not, then how can one IP address be assigned to more than one server to handle this load?
==> This will taken care by loadbalancer along with routetable
answer is: virtual hosts, in HTTP Header is name of domain so the web server know which files run or send to client
We are planning a Biztalk 2009 set up in which we have 2 Biztalk Application Servers and 2 DB Servers (DB servers being in an Active/Passive Cluster). All servers are running Windows Server 2008 R2.
As part of our application, we will have incoming traffic via the MSMQ, FILE and SOAP adapters. We also have a requirement for High-availability and Load-balancing.
Let's say I create two different Biztalk Hosts and assign the FILE receive handler to the first one and the MSMQ receive handler to the second one. I now create two host instances for each of the two hosts (i.e. one for each of my two physical servers).
After reviewing the Biztalk Documentation, this is what I know so far:
For FILE (Receive), high-availablity and load-balancing will be achieved by Biztalk automatically because I set up a host instance on each of the two servers in the group.
MSMQ (Receive) requires Biztalk Host Clustering to ensure high-availability (Host Clustering however requires Windows Failover Clustering to be set up as well). No loading-balancing option is clear here.
SOAP (Receive) requires NLB to achieve Load-balancing and High-availability (if one server goes down, NLB will direct traffic to the other).
This is where I'm completely puzzled and I desperately need your help:
Is it possible to have a Windows Failover Cluster and NLB set up at the same time on the two application servers?
If yes, then please tell me how.
If no, then please explain to me how anyone is acheiving high-availability and load-balancing for MSMQ and SOAP when their underlying prerequisites are mutually exclusive!
Your help is greatly appreciated,
M
Microsoft doesn't support NLB and MSCS running on the same servers
"These two components work well together in a two or three tier application model running on separate computers. Be aware that running these two components on the same computer is unsupported and is not recommended by Microsoft due to potential hardware sharing conflicts between Cluster service and Network Load Balancing."
http://support.microsoft.com/kb/235305
If you want to provide HA for SOAP requests received in BizTalk you should configure you BizTalk servers to be in an Active/Active configuration (no MSCS) in the same BizTalk Group. Once you do this you install an configure NLB between these two. Your clients will be able to query the web services thru the NLB cluster and the NLB service will route the request to a specific server within the cluster (your asmx files should be installed and configured in both servers).
Regarding MSMQ the information you have obtained so far is right, the only way to assure HA for this adapter is clustering the BizTalk servers. If you want to implement this too then you must have a separate infrastructure for the SOAP receive hosts and the MSMQ ones.
The main reason for this scenario is that a BizTalk Isolated Host is not cluster aware so BizTalk InProcess Host could be all hung up and the Isolated Host would never know of it and would continue to receive requests.
I'm currently designing an architecture very similar so if you would like to share more comments or questions you can reach me at ignacioquijas#hotmail.com
Ignacio Quijas
Microsoft Biztalk Server Specialist