When looking at SOA (Service Oriented Architectures) the main problems seems to be distributing a services load across multiple hosts and then managing these hosts (taking hosts off line or adding new hosts)
When one service talks to another it should not need to know any host information (at the application level). Rather the SOA Environment should be able to route a service request to a specific host based on the hosts current load characteristics (so it must know all hots a service is running on and their relative load).
Are there any existing open protocols for service to report their existence and load to an SOA Environment.
SOA is a set of high-level software architecture guidelines. It is not a technical standard or recommendation and it has nothing to do with technical implementation details, like load balancing.
Load balancing is based on addressing, which is dependent on the service access technology.
Systems built in "SOA-way" may be using different service access technology, like SOAP (over HTTP, JMS, etc.), REST, asynchronous XML messages over JMS, etc.
With SOAP, the service consumer may look up a UDDI registry to locate the service provider. Some of the latest UDDI registry software provide simple (e.g. round-robin) load balancing.
Another SOAP idea is using WS-Addressing, but it is not really meant for load balancing.
I think currently the best place for load-balancing is the underlying network transport layer. With HTTP transport you can choose hardware or software (e.g. Apache HTTPD modules) load balancers that can adapt the distribution based on response times and time-outs. With JMS transport, the most popular JMS servers provide some form of load balancing. Other protocols - like CORBA or Rendezvous - usually require a custom solution.
You can also utilize an ESB software, e.g. Oracle Service Bus or TIBCO AMX Service Bus. With an ESB you can easily create a load-balancing proxy for your service instances. The proxy may be enhanced with some logic, like look-up a database table for guidance.
As you can see, there is no one-size-fits-all solution for service load balancing. The optimal solution will be based on the actual implementation architecture and vendors' recommendations.
After reading a lot the concept I was actually looking for was the Enterprise Service Bus ESP.
Though it does not explicitly define an explicit protocol it defines an architectural style that allows the solving of the problems I stated above.
Related
I have a situation where messages are being generated by an internal application but the consumers for the messages are outside our enterprise network. Will either of http(s) transport or REST connectivity work in this scenario, with HTTP reverse proxy on DMZ? If not, is it safe to have a broker on the DMZ which can act as gateway to outside consumers?
Well, the rest/http approach to connect to ActiveMQ is very limited as it does not support true messaging semantics.
Exposing an ActiveMQ broker is no less secure than any other communication software if precautions are taken (TLS, default passwords changed, high entropy passwords are used and/or mutual authentication, recent patches applied, web console/jolokia not exposed externally without precautions etc etc).
In fact - you can buy online ActiveMQ instances from Amazon - which indicates that at least they think it's not such a bad idea to put them on the Internet.
Our organization is looking into implementing new internal APIs using GRPC.
Currently, we have a microservice that is serving internal/external requests using embedded Jetty. We want to make internal communication between services to be done over GRPC.
So, we'll have 2 servers running on the same VM: jetty and GRPC. Is it a good practice, any red flags with that approach?
We do not want to split that said microservice into 2 to save costs. We should be able to run the app on the same number of VMs.
There's nothing inherently special or wrong about having Jetty and gRPC in the same JVM. The main point of potential trouble is just that you will have two ports exposed instead of one; that might matter for service discovery or firewalls.
I'm writing a website that is going to start using a load balancer and I'm trying to wrap my head around it.
Does IIS just do all the balancing for you?
Do you have a separate web layer that sits on the distributed server that does some work before sending to the sub server, like auth or other work?
It seems like a lot of the articles I keep reading don't really give me a straight answer, or I'm just not understanding them correctly, I'd like to get my head around how true load balancing works from a techincal side, and if anyone has any code to share that would also be nice.
I understand caching is gonna be a problem but that's a different topic, session as well.
IIS do not have a load balancer by default but you can use at least two Microsoft technologies:
Application Request Routing that integrates with IIS, there you should ideally have a separate web layer to do routing work,
Network Load Balancing that is integrated with Microsoft Windows Server, there you can join existing servers into NLB cluster.
Both of those technologies do not require any code per se, it a matter of the infrastructure. But you must of course remember about load balanced environment during development. For example, to make a web sites truly balanced, they should be stateless. Otherwise you will have to provide so called stickiness between client and the server, so the same client will be connecting always to the same server.
To make service stateless, do not persist any state (Session, for example, in case of ASP.NET website) on the server but on external server shared between all servers in the farm. So it is common for example to use external ASP.NET Session server (StateServer or SQLServer modes) for all sites in the cluster.
EDIT:
Just to clarify a few things, a few words about both mentioned technologies:
NLB works on network level (as a networking driver in fact), so without any knowledge about applications used. You create so called clusters consisting of a few machines/servers and expose them as a single IP address. Then another machine can use this IP as any other IP, but connections will be routed to the one of the cluster's machines automatically. A cluster is configured on each server, there is no external, additional routing machine. Depending on the clusters settings, as we have already mentioned, a stickiness can be enabled or disabled (called here a Single or None Affinity). There is also a Load weight parameter, so you can set weighed load distribution, sending more connections to the fastest machine for example. But this parameter is static, it can't be dynamically based on network, CPU or any other usage. In fact NLB does not care if target application is even running, it just route network traffic to the selected machine. But it notices servers went offline, so there will be no routing there. The advantages of NLB is that it is quite lightweight and requires no additional machines.
ARR is much more sophisticated, it is built as a module on top of IIS and is designed to make the routing decisions at application level. Network load balancing is only one of its features as it is a more complete, routing solution. It has "rule-based routing, client and host name affinity, load balancing of HTTP server requests, and distributed disk caching" as Microsoft states. You create there Server Farms with many options like load balance algorithm, load distribution and client stickiness. You can define health tests and routing rules to forward request to other servers. Disadvantage of all of it is that there should be a dedicated machine where ARR is installed, so it takes more resources (and costs).
NLB & ARR - as using a single ARR machine can be the single point of failure, Microsoft states that it is worth consideration to create a NLB cluster of ARR machines.
Does IIS just do all the balancing for you?
Yes,if you configure Application Request Routing:
Do you have a separate web layer that sits on the distributed server
Yes.
that does some work before sending to the sub server, like auth or other work?
No, ARR is pretty 'dumb':
IIS ARR doesn't provide any pre-authentication. If pre-auth is a requirement then you can look at Web Application Proxy (WAP) which is available in Windows Server 2012 R2.
It just acts as a transparent proxy that accepts and forwards requests, while adding some caching when configured.
For authentication you can look at Windows Server 2012's Web Application Proxy.
Some tips, and perhaps items to get yourself fully acquainted with:
ARR as all the above answers above state is a "proxy" that handles the traffic from your users to your servers.
You can handle State as Konrad points out, or you can have ARR do "sticky" sessions (ensure that a client always goes to "this server" - presumably the server that maintains state for that specific client). See the discussion/comments on that answer - it's great.
I haven't worn an IT/server hat for so long and frankly haven't touched clustering hands on (always "handled for me automagically" by some provider), so I did ask this question from our host, "what/how is replication among our cluster/farm" done?" - The question covers things like
I'm only working/setting things on 1 server, does that get replicated across X VMs in our cluster/farm? How long?
What about dynamically generated,code and/or user generated files (file system)? If it's on VM1's file system, and I have 10 load balanced VMs, and the client can hit any one of them at any time, then...?
What about encryption? e.g. if you use DPAPI to encrypt web.config stuff (e.g.db conn strings/sections), what is the impact of that (because it's based on machine key, and well, the obvious thing is now you have machine(s) or VM(s). RSA re-write....?
SSL: ARR can handle this for you as well, and that's great! But as with all power, comes a "con" - if you check/validate in your code, e.g. HttpRequest.IsSecureConnection, well, it'll always be false. Your servers/VMs don't have the cert, ARR does. The encrypted conn is between client and ARR. ARR to your servers/VMs isn't. As the link explains, if you prefer it the other way around (no offloading), you can...but that means all your servers/VMs should then have a cert (and how that pertains to "replication" above starts popping in your head).
Not meant to be comprehensive, just listing things out from memory...Hth
We are planning a Biztalk 2009 set up in which we have 2 Biztalk Application Servers and 2 DB Servers (DB servers being in an Active/Passive Cluster). All servers are running Windows Server 2008 R2.
As part of our application, we will have incoming traffic via the MSMQ, FILE and SOAP adapters. We also have a requirement for High-availability and Load-balancing.
Let's say I create two different Biztalk Hosts and assign the FILE receive handler to the first one and the MSMQ receive handler to the second one. I now create two host instances for each of the two hosts (i.e. one for each of my two physical servers).
After reviewing the Biztalk Documentation, this is what I know so far:
For FILE (Receive), high-availablity and load-balancing will be achieved by Biztalk automatically because I set up a host instance on each of the two servers in the group.
MSMQ (Receive) requires Biztalk Host Clustering to ensure high-availability (Host Clustering however requires Windows Failover Clustering to be set up as well). No loading-balancing option is clear here.
SOAP (Receive) requires NLB to achieve Load-balancing and High-availability (if one server goes down, NLB will direct traffic to the other).
This is where I'm completely puzzled and I desperately need your help:
Is it possible to have a Windows Failover Cluster and NLB set up at the same time on the two application servers?
If yes, then please tell me how.
If no, then please explain to me how anyone is acheiving high-availability and load-balancing for MSMQ and SOAP when their underlying prerequisites are mutually exclusive!
Your help is greatly appreciated,
M
Microsoft doesn't support NLB and MSCS running on the same servers
"These two components work well together in a two or three tier application model running on separate computers. Be aware that running these two components on the same computer is unsupported and is not recommended by Microsoft due to potential hardware sharing conflicts between Cluster service and Network Load Balancing."
http://support.microsoft.com/kb/235305
If you want to provide HA for SOAP requests received in BizTalk you should configure you BizTalk servers to be in an Active/Active configuration (no MSCS) in the same BizTalk Group. Once you do this you install an configure NLB between these two. Your clients will be able to query the web services thru the NLB cluster and the NLB service will route the request to a specific server within the cluster (your asmx files should be installed and configured in both servers).
Regarding MSMQ the information you have obtained so far is right, the only way to assure HA for this adapter is clustering the BizTalk servers. If you want to implement this too then you must have a separate infrastructure for the SOAP receive hosts and the MSMQ ones.
The main reason for this scenario is that a BizTalk Isolated Host is not cluster aware so BizTalk InProcess Host could be all hung up and the Isolated Host would never know of it and would continue to receive requests.
I'm currently designing an architecture very similar so if you would like to share more comments or questions you can reach me at ignacioquijas#hotmail.com
Ignacio Quijas
Microsoft Biztalk Server Specialist
What are some good articles/ resources to understand how load balancing is configured with Biztalk --- both in terms of inherent abilities of the product as well as employing NLB (network load balancing with Windows 2003 or later editions)?
EDIT: I am specifically interested in the impact of application protocol on load balancing? For example, how two instances of Biztalk server handle TCP/IP connections when the other party (to which Biztalk makes a connection request) doesn't allow more than one connection, etc.
The obvious resource is MSDN - There is a section entitled Planning for High Activity that covers most of the concepts and will give you the right terminology to then go looking for other resources on the web. As with a lot of Microsoft server products, MSDN also has lots of white papers covering specific BizTalk scenarios.
Most good BizTalk books also include a section on load balancing concepts (Professional BizTalk Server 2006 has an example).
Beyond that, there are several key concepts that you may find helpful, particularly around the use of terminology (some of BizTalk's usage can be misleading).
Load Balancing
BizTalk Server is, by the nature of its architecture, load balancing. What that means is that if you have more than one BizTalk Host connecting to a MessageBox database, the messages within the database will be spread evenly across the hosts participating in the BizTalk group. (with caveats around just what BizTalk processess have been configured to run in each Host).
There is also the concept of Network Load Balancing which is Microsoft Network Load Balancing Services or any equivalent service. In BizTalk this applies at the web level, for receive adapters using the HTTP protocol (e.g. the HTTP adapter, the SOAP adapter and WCF HTTP adapters). This load balancing is not actually a BizTalk service but is instead a load balancing layer provided on top of the BizTalk isolated host adapters to ensure high availability of the web resources. It is configured the same as any other NLB service.
Clustering
When clustering is mentioned in BizTalk it is used to refer to one of two things - clustering at the SQL layer to provide high availability and failover, and BizTalk Host Clustering.
SQL Clustering - this is simply (though it isn't simple to do, just say) a matter of providing a SQL server cluster that runs the BizTalk server databases, allowing for database failover. This is not a BizTalk specific technology.
BizTalk Host clustering - in this case a BizTalk Server Host is marked as being clustered when creating it inside BizTalk. This is a BizTalk specific setting that essentially states that one and only one instance of the host will be running at a time, and that by extension all the resources within this host will also only have a single instance. It is primarily intended for usage for adapters like the FTP and MSMQ adapters that behave incorrectly when more that one is allowed to run at the same time.
This edit is in response to the OP's comment asking for further details. Hopefully this make things clearer. If you have more questions about specifics I can possibly answer them but this pretty much exhausts my theory knowledge about high availability environment configuration. I'm primarily a BizTalk dev and solution designer, when it comes to network intricacies there are people where I work who fill in the nitty gritty detail and implementation of these designs.
Network Load Balancing for HTTP Based Adapters
The key point I was trying to express here was that Network Load Balancing in the context of BizTalk is no different for any other Network Load Balancing scenario.
BizTalk has two type of hosts, In Process and Isolated. In Process hosts are individual BizTalk services running on servers (with one host instance per server). Isolated hosts are actually delegates to a web server (IIS) that handle all HTTP based adapters (the HTTP adapter and the SOAP adapter plus certain configurations for the WCF adapter)
When you introduce Network Load Balancing to a BizTalk environment what you are doing is intoducing it at the web server layer, for the Isolated host hosted adapters.
Here is the MSDN page for the introduction to NLB. One of the key points about NLB is expressed in the page in the following quote:
Network Load Balancing allows all of
the computers in the cluster to be
addressed by the same set of cluster
IP addresses (but also maintains their
existing unique, dedicated IP
addresses).
By setting up NLB you allow multiple isolated host servers to handle internet traffic directed at a single dedicated IP address. The NLB configuration farms out the work.
Clustering BizTalk Adapter Handlers
In my answer above I stated that certain BizTalk adapters behave incorrectly when allowed to run within multiple BizTalk Host Instances. This is very adapter specific in terms of the why, so the best expansion on that answer I can give is the following quote from the MSDN documentation, dealing with the FTP adapter specifically.
For most of the BizTalk integrated
adapters, high availability can be
achieved by creating multiple adapter
handlers to run on BizTalk host
instances on different BizTalk servers
within a BizTalk group. FTP adapter
receive handlers should not, however,
be configured to run in multiple
BizTalk host instances simultaneously.
This recommendation is made because
the FTP receive adapter uses the FTP
protocol to retrieve files from the
target system and the FTP protocol
does not lock files to ensure that
multiple copies of the same file are
not retrieved simultaneously when
running multiple instances of the FTP
receive adapter.
As they say, the FTP adapter utilises the FTP protocol which does not lock files. Because BizTalk is natively a highly parallel system, if you allowed multiple BizTalk hosts to host an instance of the FTP adapter you would end up with multiple copies of the same FTP message received into your BizTalk system. What BizTalk clustering does is ensure that any clustered BizTalk hosts will run on 1 and only 1 host instance. By placing your FTP receive handler inside a clusterd host, you ensure that:
you will always have an FTP adapter running so long as a BizTalk host is running
you will never have more than one FTP adapter running.
Additionally you can use a BizTalk clustered host to reduce load on a system. For example, a BizTalk SQL adapter receive location that has been configured to poll, will poll on all host instances. While this would not necessarily cause multiple message instances, it could cause undue load on the SQL server you poll, or even create deadlock scenarios depending on the design of the called stored procedure, so clustering the SQL Adapter receive handler can be a good idea.