I would like to run Node-RED as a service on Kubernetes to be able to build a custom API using the HTTP IN nodes. The goal is to be able to push any number of different flows to an arbitrary container running Node-RED using the Node-RED API.
I have tried running Node-RED as a service with 5 replicas and built a flow through the UI that has an HTTP in and HTTP out node. When I try hitting the service using curl on the minikube ip (e.g. curl http://192.168.64.2:30001/test), it will only return the results if the load balancer happens to land on the container that has the flow. Otherwise, it will return an error with HTML.
Any advice on how I should go about solving this issue? Thanks!
This is working as expected. If you are interacting with the Node-RED editor via the load balancer you are only editing the flow on that instance.
If you have 5 instances of Node-RED and only one of them is running a flow with the HTTP endpoints defined then calls to that endpoint will only succeed 1 time in 5.
You need to make sure that all instances have the same endpoints defined in their flows.
There are several ways you can do this, some examples would be:
Use the Node-RED Admin API to push the flows to each of the Node-RED instances in turn. You will probably need to do this via the private IP Address of each instance to prevent the load balancer getting in the way.
Use a custom Storage plugin to store the flow in a database and have all the Node-RED instances load the same flow. You would need to restart the instances to force the flow to be reloaded should you change it.
Related
I have a gRPC client, not dockerised, and server application, which I want to dockerise.
What I don't understand is that gRPC first creates a connection with a server, which involves a handshake. So, if I want to deploy the dockerised server on ECS with multiple instances, then how will the client switch from one to the other (e.g., if one gRPC server falls over).
I know AWS loadbalancer now works with HTTP 2, but I can't find information on how to handle the fact that the server might change after the client has already opened a connection to another one.
What is involved?
You don't necessarily need an in-line load balancer for this. By using a Round Robin client-side load balancing policy along with a DNS record that points to multiple backend instances, you should be able to get some level of redundancy.
I'm currently working on a project where we are using Google Cloud. Within the Cloud we are using CloudRun to provide our services. One of these services is rather complex and has many different configuration options. To validate how these configurations affect the quality of the results and also to evaluate the quality of changes to the service, I would like to proceed as follows:
in addition to the existing service I deploy another instance of the service which contains the changes
I mirror all incoming requests and let both services process them, only the responses from the initial service are returned, but the responses from both services are stored
This allows me to create a detailed evaluation of the differences between the two services without having to provide the user with potentially worse responses.
For the implementation I have setup a NGINX which mirrors the requests. This is also deployed as a CloudRun service. This now accepts all requests and takes care of the authentication. The original service and the mirrored version have been configured in such a way that they can only be accessed internally and should therefore be accessed via a VPC network.
I have tried all possible combinations for the configuration of these parts but I always get 403 or 502 errors.
I have tried setting the NGINX service to the HTTP and HTTPS routes from the service, and I have tried all the VPC Connector settings. When I set the ingress from the service to ALL it works perfectly if I configure the service with HTTPS and port 443 in NGINX. As soon as I set the ingress to Internal I get errors with HTTPS -> 403 and with HTTP -> 502.
Does anyone have experience in this regard and can give me tips on how to solve this problem? Would be very grateful for any help.
If your Cloud Run service are internally accessible (ingress control set to internal only), you need to perform your request from your VPC.
Therefore, as you perfectly did, you plugged a serverless VPC connector on your NGINX service.
The set up is correct. Now, why it works when you route ALL the egress traffic and not only the private traffic to your VPC connector?
In fact, Cloud Run is a public resource, with a public URL, and even if you set the ingress to internal. This param say "the traffic must come to the VPC" and not say "I'm plugged to the VPC with a private IP".
So, to go to your VPC and access a public ressource (Your cloud run services), you need to route ALL the traffic to your VPC, even the public one.
I have two pods(A, B) on two separate nodes running same application. This application will receive data through tcp and save that to a remote storage. I would like to implement leader/follower pattern for these pods but i'm not sure how that can be achieved in k8s. For example, I would like pod A to receive all the traffic in the "normal situation" but when pod A fails, I want pod B to receive all the traffic. For my specific use case, the application MUST receive data in the same pod rather than load balancing the traffic.
Using Nginx Ingress you can use snippet and achieve the same.
Refer https://docs.nginx.com/nginx-ingress-controller/configuration/ingress-resources/advanced-configuration-with-snippets/
https://serverfault.com/a/480273/283777
I have a ServiceFabric with two applications. On application gets invoked from outside the ServiceFabric and then issues HTTP get requests to the other application inside the ServiceFabric.
My first attempt was to address the second application with the ServiceFabric's reverse proxy IP, the same as the first application is addressed with:
http://10.0.0.1:19081/App2/App2.Service/
This led to unreliable communication inside the ServiceFabric and the first request always failed, while the second mostly succeeded.
Then I read about internal ServiceFabric communication at https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-reverseproxy. Now I address my second application with localhost and it seems to work as expected:
http://localhost:19081/App2/App2.Service/
The only open question is: Does addressing applications inside the ServiceFabric with localhost only work because the application is also running on the same node? Or does it work because there is real reverse proxy behavior and even if the application does not run on the same node, the request gets to it regardless?
The reverse proxy runs on all nodes, so it can be reached on localhost at all times. It forwards your call to the second service, which is resolved automatically.
You could also use the built-in DNS service to resolve internal services. This way, you save some of the overhead of the reverse proxy.
Opposed to using the ip address, you don't need to know whether the service runs on localhost or on a different node. Also, you don't get into trouble if your service is moved at run-time.
I am trying to set up a HTTP load balancer for my Meteor app on google cloud. I have the application set up correctly, and I know this because I can visit the IP given in the Network Load Balancer.
However, when I try and set up a HTTP load balancer, the health checks always say that the instances are unhealthy (even though I know they are not). I tried including a route in my application that returns a status 200, and pointing the health check towards that route.
Here is exactly what I did, step by step:
Create new instance template/group for the app.
Upload image to google cloud.
Create replication controller and service for the app.
The network load balancer was created automatically. Additionally, there were two firewall rules allowing HTTP/HTTPS traffic on all IPs.
Then I try and create the HTTP load balancer. I create a backend service in the load balancer with all the VMs corresponding to the meteor app. Then I create a new global forwarding rule. No matter what, the instances are labelled "unhealthy" and the IP from the global forwarding rule returns a "Server Error".
In order to use HTTP load balancing on Google Cloud with Kubernetes, you have to take a slightly different approach than for network load balancing, due to the current lack of built-in support for HTTP balancing.
I suspect you created your service in step 3 with type: LoadBalancer. This won't work properly because of how the LoadBalancer type is implemented, which causes the service to be available only on the network forwarding rule's IP address, rather than on each host's IP address.
What will work, however, is using type: NodePort, which will cause the service to be reachable on the automatically-chosen node port on each host's external IP address. This plays more nicely with the HTTP load balancer. You can then pass this node port to the HTTP load balancer that you create. Once you open up a firewall on the node port, you should be good to go!
If you want more concrete steps, a walkthrough of how to use HTTP load balancers with Container Engine was actually recently added to GKE's documentation. The same steps should work with normal Kubernetes.
As a final note, now that version 1.0 is out the door, the team is getting back to adding some missing features, including native support for L7 load balancing. We hope to make it much easier for you soon!