I have nginx in which consul-template prescribes upstream for golang services.
When deploying, the service is started and registered with consul, after which consul-template writes it upstream. After that, nginx -t is checked and an error is generated that upstream is not found.
The problem is floating, because sometimes the deployment is successful
Can you help me understand what could be wrong?
Related
I'm a newbie to Nginx. I cannot access my Node.js application that I deploy on AWS EC2 using Nginx reverse proxy. If I do curl http://localhost:3000 I can see the application is running successfully on the server(I'm using pm2 for running node server). But when I try to access it in my browser or postman using public DNS I get the error This site can't be reached and the request gets timeout. Here's my Nginx configuration (I have followed a number of tutorials for this)
The configuration file is named nginx.conf and is in /etc/nginx/sites-enabled directory. If I do sudo nginx -t it says syntax is ok and the test is successful. Also I can see the Nginx is running using command sudo systemctl status nginx What could be the possible reason for this behaviour?
I figured it out the problem wasn't with the Nginx configuration actually I needed to allow public access for port 80 on my ec2 instance which is blocked by default. I allowed port 80 and everything is working fine. This blog helped me. Visit it for me details on how to enable port 80 for your ec2 instance.
I am using kubernetes-dashboard to view all pods, check status, login, pass commands, etc. It works good, but there is a lot of connectivity issues related to it. I am currently running it on port-8443, and forwarding the connection from 443 to 8443 via Nginx's proxy pass. But I keep getting bad gateway, and connection keeps dropping. It's not an nginx issue, since I have kubernetes error. I am using Letsencrypt certificate in nginx, What am I doing wrong?
Error log :
E0831 05:31:45.839693 11324 portforward.go:385] error copying from local connection to remote stream: read tcp4 127.0.0.1:8443->127.0.0.1:33380: read: connection reset by peer
E0831 05:33:22.971448 11324 portforward.go:340] error creating error stream for port 8443 -> 8443: Timeout occured
Theses are the 2 errors I constantly get. I am running this command as a nohup process :
nohup kubectl port-forward -n kubernetes-dashboard service/kubernetes-dashboard 8443:443 --address 0.0.0.0 &
And finally my nginx config :
default :
location / {
proxy_intercept_errors off;
proxy_pass https://localhost:8443/;
}
Thank you. :-)
Unfortunately this is an on-going issue with Kubernetes' port forwarding. You may find it not particularly reliable when used for long-running connections. If possible, try to setup a direct connection instead. A more extended discussion regarding this can be found here and here.
I can't find a way to make an nginx pod resolve another kubernetes services URLs.
I am NOT using kube-dns , we are using kube2sky solely and we are not going to implement kube-dns yet, so I need to fix in this scenario.
For example, I want nginx to resolve a service URL app.mynamespace.svc.skydns.local but if I run a ping to that URL it resolves successfully.
My nginx config part is:
location /api/v1/namespaces/mynamespace/services/app/proxy/ {
resolver 127.0.0.1
set \$endpoint \"http://app.mynamespace.svc.skydns.local/\";
proxy_pass \$endpoint;
proxy_http_version 1.1;
proxy_set_header Connection \"upgrade\";
}
I need to specify the target upstream in a variable because I want nginx to starts even if the target is not available, if I don't specify in variable nginx crashes when starting up because the upstream needs to be available and resolvable.
The problem I think is the resolver value, I've tried with 127.0.0.1, with 127.0.0.11, and with the skydns IP specified in configuration 172.40.0.2:53:
etcdctl get /skydns/config
{"dns_addr":"0.0.0.0:53","ttl":4294967290,"nameservers":["172.40.0.2:53"]}
But nginx cannot resolve the URL yet.
What IP should I specify in the resolver field in nginx config for kubernetes and skydns config?
Remember that we don't have kube-dns.
Thank you.
I don't think resolving app.mynamespace.svc.skydns.local has anything to do with configuring the upstream DNS servers. Generally, for that, you configure a well-known DNS server like 8.8.8.8 or your cloud infrastructure DNS server which would be perhaps 172.40.0.2. For example as described in the docs:
$ curl -XPUT http://127.0.0.1:4001/v2/keys/skydns/config \
-d value='{"dns_addr":"127.0.0.1:5354","ttl":3600, "nameservers": ["8.8.8.8:53","8.8.4.4:53"]}'
You might want to check the logs of your kube2sky2 pod, for any guidance and that all the config options are specified like --kube-master-url, --etcd-server. Maybe it can't talk to the Kubernetes master and receive updates of running pods so that the SRV entries get updates.
I have multiple upstream servers from an nginx load balancer:
upstream app {
# Make each client IP address stick to the same server
# See http://nginx.org/en/docs/http/load_balancing.html
ip_hash;
# Use IP addresses: see recommendation at https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/
server 1.1.1.1:6666; # app-server-a
server 2.2.2.2:6666; # app-server-a
}
Right now I ue the servers in an active/passove configuration by taking down each servers (eg systemctl myapp stop) then letting nginx detect the server is down.
However I'd like to be able to change the upstream server dyamically, without having to take either app server or nginx OSS down. I'm aware of the proprietary upstream_conf module for nginx Plus but am using nginx OSS.
How can I dynamically dynamically reconfigure the upstream server on nginx OSS?
You can use:
openresty an OSS nginx bundle with lua scripting ability
nginx with lua scripting (you can configure it by yourself using nginx OSS and luajit) to achieve this.
dynx can achieve exactly what you are looking for, it's still work in progress but the dynamic upstream functionality is there and it's configurable through a rest API.
I'm adding the details on how to deploy and configure dynx:
you need to have a docker swarm up and running (for testing purpose
you can have a 1 swarm machine), follow the docker documentation to do that.
after you need to deploy the stack, for example, with this command (you need to be on the dynx git root):
docker stack deploy -c docker-compose.yml dynx
To check if the application deployed correctly, you can use this command:
docker stack services dynx
To configure an location you can use through the api you can for instance do:
curl -v "http://localhost:8888/configure?location=/httpbin&upstream=http://www.httpbin.org/anything&ttl=10"
To test if it works:
curl -v http://localhost:8666/httpbin
Do not hesitate to contact me or open an issue on github if you are not able to get it to work
Nginx server is not responding periodically like every 30 minutes.
I have to reload it by nginx -s reload.
How to resolve the problem?
I can able to ping host. But unable to establish http request.
Problem solved after updating to latest version of nginx 1.2.3 to nginx/1.11.3