curl can't connect to only certain HTTPS hosts - http

I am trying to install Meteor.js on a VM (Ubuntu 12.04) created with Vagrant.
The install should be as simple as:
curl https://install.meteor.com | /bin/sh
However this fails with curl: (7) couldn't connect to host
I have isolated the failure to a request within that shell script to this URL:
https://warehouse.meteor.com/bootstrap/0.7.0.1/meteor-bootstrap-Linux_i686.tar.gz
When I changed it to use HTTP instead of HTTPS it works. However I am running into problems elsewhere where it needs to pull things from httpS://warehouse.meteor.com/...
I thought the problem was with https, but if I do:
curl https://google.com
I get the page no problem, so what could be the issue?

Per another Ubuntu/Meteor question, it appears that there's some kind of certificate error (Meteor's SSL CA may not be installed by default in Ubuntu?) that goes away when you:
sudo apt-get update && sudo apt-get upgrade

For me upgrade didn't solve the problem.
My solution was to download the script from install.meteor.com and replace TARBALL_URL from https to http and I ran the script manually.

Related

How do I fix 502 Bad Gateway error with GCP and NGINX

I'm trying to follow a tutorial on creating an Apache Airflow pipeline on a GCP vm instance (https://towardsdatascience.com/10-minutes-to-building-a-machine-learning-pipeline-with-apache-airflow-53cd09268977) but after building and running the docker container, I get this "502 Bad Gateway" error with Nginx 1.14 when try to access the webserver using:
http://<VM external ip>/
I'm quite new to using GCP and can't figure out how to fix this.
Some online research has suggested editing NGINX configuration files to:
keepalive_timeout 650;
keepalive_requests 10000;
But this hasn't changed anything.
The GCP instance is a N1-standard-8 with Ubuntu 18.04, and Cloud, HTTPS and HTTP access enabled.
The Nginx sites enabled are :
server {
listen 80;
location / {
proxy_pass http://0.0.0.0:8080/;
}
}
Root Cause:
The issue the you experience has nothing to do with keepalives, it is rather simpler - the docker container exits out and isn't running, so when nginx tries to proxy your request into the container, it fails and thus the error. Said failure is due to the incompatibility of airflow with current versions of sqlalchemy.
Verification:
run this command to see the logs of the failed container
sudo docker logs `sudo docker ps -a -f "ancestor=greenr-airflow" --format '{{.ID}}'`
and you will see that the python inside the container fails to import a package with the following error:
No module named 'sqlalchemy.ext.declarative.clsregistry'
Solution:
While I followed the tutorial to the letter, I'd recommend against
running commands with sudo you may want to deviate from the tutorial a
wee bit in order not to.
before running
sudo docker build -t greenr-airflow:latest .
command, edit the Dockerfile file and add the following two lines
&& pip install SQLAlchemy==1.3.23 \
&& pip install Flask-SQLAlchemy==2.4.4 \
somewhere up in the list of packages that are being installed, I've added it after
&& pip install -U pip setuptools wheel \
which is line 54 at the time of writing.
If you would like to re-use the same instance, delete and rebuild the images after making changes to the file:
sudo docker rmi greenr-airflow
sudo docker build -t greenr-airflow:latest .

Website available even with no NGINX processes running

I have pulled into my web server so it has the latest code from my repo, i try to restart nginx - this doesnt do anything.
So I try the command
sudo nginx -s stop, and get the response that its failed because there is no such file or directory "run/nginx.pid" failed.
Trying to run the command ps aux | grep nginx gives me the response: unsupported option (BSD syntax) -- it actually comes out as ps aux > grep nginx in the digital ocean console.
Basically it seems that even though there are apparently no nginx processes running (although the command to check isnt working) my website is still running and using the old code, is there a way for me to check more definitively on the running processes?
Thanks if you can help.
Try sudo netstat -plunt to check if there's any nginx process running. See if there's anything running on port 80 or 443 and then look at the corresponding program name. You might have another server running possibly apache since it ships by default with most distributions which may be why nginx failed to start.
Another reason why it won't start might be due to faulty config. Go to /etc/nginx/ and double check that it's correct. You can also run sudo nginx -t to ensure that the config syntax is correct.
Alternatively, just check your nginx access log to see if it's actually serving any requests. You can also check the error log to see why it might fail to start. These resides in /var/log/nginx by default or check your nginx.conf for any custom path to logs.

Nginx not working properly after update

Okay, so I had nginx 1.4.6 running on ubuntu 13.10 without any problems.
I tried to update nginx to 1.6.0 via this url (http://leftshift.io/upgrading-nginx-to-the-latest-version-on-ubuntu-servers)
Now nginx is not running and not willing to start (no reaction at all). nginx -v gives "nginx: command not found" as a result. So looks like nginx can't be found.
I looked around here and on other sites, but wasn't able to find the solution. So, if anyone can.. plz do..
As this was a server without any active tools or software I decided to remove and reinstall nginx.
I used this answer: How can I restore /etc/nginx?
QUOTE:
To recreate it, first uninstall using purge to remove even configuration files and records:
sudo apt-get purge nginx nginx-common nginx-full
then reinstall:
sudo apt-get install nginx
After these two commands, nginx was up and running again. I can now use my backup to upload the predefined .vhosts files to sites-enabled again.

Gitlab: Problems running Unicorn, Resque with Passenger/Nginx

I have installed a Gitlab on a brand new Ubuntu (10.04) and it is working almost correctly. Gitlab is reachable on HTTP, I can push/pull data via git to the server. There is one thing missing though, the activity feed is not updating. So I thought there is something wrong with the git hooks. I completely followed the installation process from Gitlab except I'd like to use Passenger to run Nginx in order to deploy multiple apps.
I was running the the sudo -u gitlab -H bundle exec rake gitlab:env:info RAILS_ENV=production to see if everything is set up correctly, but it said, Redis is not running. ps aux says, redis-server is up. So it is not the git hooks. Gitlab docu says, restart the gitlab service to solve that problem. In this case I get an error which I think is the problem I need to solve:
$ sudo /etc/init.d/gitlab restart
Error, unicorn not running!
My question is, how can I get around this problem? How can I run unicorn, I thought the gitlab service would start it? Am I not using Nginx? Before I start reinstalling the whole thing firstly without using Passenger, I thought I might ask the question here beforehand.
As mentioned by the OP pabera, nginx and mysql must be started, for the other components of GitLab (redis, unicorn, and now sidekiq) to run properly.
The official /etc/init.d/gitlab is here.
I have my own version of gitlabd (here), because I manage sidekiq in my own script, and I don't need to run the script as root.
You can see the run order for all the services in this script:
ssh
Apache and/or NGiNX
mysql
redis
GitLab (which will start unicorn and sidekiq)
Kind of a poke in the dark...
In the GitLab installation.md README is states:
"
Start your GitLab instance:
sudo service gitlab start
# or
sudo /etc/init.d/gitlab restart
"
I did the first AND the second and got this exact error. However, I skipped the "or" and continued to the Nginx commands and it seems to work.
Hope this helps!

simple and quick tool on linux to watch a http request coming

A HTTP request is coming your way, hitting your machine on a port( assumed that you know the request url/route and the port)
what is a simple and quick tool which helps you watch its headers, parameters being passed, using any language, or any tiny web server proxy, any tool.
Being a ruby/rails guy, I quickly created a rails app, set its routes, ran it, and simply followed its logs.
Curious to know how other web app developers deal with this.
I use WireShark and it is very easy and configurable.
If you use debian or derivates:
sudo apt-get install wireshark
else:
http://www.wireshark.org/docs/wsug_html_chunked/ChapterBuildInstall.html
I'd do this with tcpdump. See the man page. There is also a version for windows.
edit
here's the requested example: tcpdump port 80 (yes, it's that simple). Run it with sudo or as root, otherwise you'll get no suitable device found.
If you need more verbose output, add -v or -vv.
Here is how to use wireshark on a Debian-derived GNU/Linux distro:
$ sudo apt-get install wireshark wireshark-doc
$ sudo dumpcap -i eth0
Ctrl+C
$ sudo chmod 644 /tmp/wireshark_pcapng_eth0...
$ wireshark /tmp/wireshark_pcapng_eth0...

Resources