Deploying Meteor app via Meteor Up or tmux meteor - meteor

I'm a bit curious if Meteor Up (or other Meteor app deploying processes like Modulus) do anything fancy compared to copying over your Meteor application, starting a tmux session, and just running meteor to start your application on your server. Thanks ins advance!

Meteor Up and Modulus seem to just run node.js and Mongodb. They run your app after it has been packaged for production with meteor build. This will probably give your app an edge in performance.
It is possible to just run meteor in a tmux or screen session. I use meteor run --settings settings.json --production to pass settings and also use production mode which minifies the code etc. You can also use a proxy forwarder like Nginx to forward requests to port 80 (http) and 443 (https).
For reference here's my Nginx config:
server {
listen 80;
server_name example.com www.example.com;
return 301 https://example.com$request_uri;
}
server {
listen 443 ssl;
server_name www.example.com;
ssl_certificate /etc/ssl/private/example.com.unified.crt;
ssl_certificate_key /etc/ssl/private/example.com.ssl.key;
return 301 https://example.com$request_uri;
}
server {
listen 443 ssl;
server_name example.com;
ssl_certificate /etc/ssl/private/example.com.unified.crt;
ssl_certificate_key /etc/ssl/private/example.com.ssl.key;
location / {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-NginX-Proxy true;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
By using this method everything is contained within the meteor container and you have the benefit of meteor watching for changes etc. However, there may be some extra overhead on your server. I'm not sure exactly how much as I haven't tested both ways enough.
The only problem I've found by using this method is it's not easy to get everything automated on reboot, like automatically running tmux then launching meteor, as opposed to using specially designed tools like Node.js Forever or PM2 which automatically start when the server is rebooted. So you have to manually log in to the server and run meteor. If you work out an easy way to do this using tmux or screen let me know.
Edit:
I have managed to get Meteor to start on system boot with the following line in the /etc/rc.local file:
sudo -H -u ubuntu -i /usr/bin/tmux new-session -d '/home/ubuntu/Sites/meteorapp/run_meteorapp.sh'
This command runs the run_meteorapp.sh shell script inside a tmux session once the system has booted. In the run_meteorapp.sh I have:
#!/usr/bin/env bash
(cd /home/ubuntu/Sites/meteorapp && exec meteor run --settings settings.json --production)

If you look at the Meteor Up Github page: https://github.com/arunoda/meteor-up you can see what it does.
Such as:
Features
Single command server setup Single command deployment Multi server
deployment Environmental Variables management Support for
settings.json Password or Private Key(pem) based server authentication
Access, logs from the terminal (supports log tailing) Support for
multiple meteor deployments (experimental)
Server Configuration
Auto-Restart if the app crashed (using forever) Auto-Start after the
server reboot (using upstart) Stepdown User Privileges Revert to the
previous version, if the deployment failed Secured MongoDB
Installation (Optional) Pre-Installed PhantomJS (Optional)
So yes... it does a lot more...

Mupx does even more. It takes advantage of docker. It is the development version but I have found it to be more reliable than mup after updating Meteor to 1.2
More info can be found at the github repo: https://github.com/arunoda/meteor-up/tree/mupx

I have been using mupx to deploy to digital ocean. Once you set up the mup.json file you can not only deploy the app, but you can also update the code on the server easily through the CLI. There are a few other commands too that are helpful.
mupx reconfig - reconfigs app with environment variables
mupx stop - stops app duh
mupx start - ...
mupx restart - ...
mupx logs [-f --tail=100] - this gets logs which can be hugely helpful when you encounter deployment errors.
It certainly makes it easy to update your app, and I have been pretty happy with it.
Mupx does use MeteorD (Docker Runtime for Meteor Apps)
and since it uses docker it can be really useful to access the MongoDB shell via ssh with this command:
docker exec -it mongodb mongo <appName>
Give it a shot!

Related

The Streamlit app stops with "Please wait... " and then stops

Problem
The app started by running streamlit run main.py will display http://IP_ADDRESS:8501 is displayed correctly, but http://DOMAIN_NAME stops with "Please wait... " and stops.
Environment
Domain name already resolved with Route53
Deploy Streamlit App on EC2 (Amazon Linux) and run Streamlit run main.py on Tmux
Use Nginx to convert access to port80 to port8501
Changed Nginx settings
/etc/nginx/nginx.conf
server {
listen 80; #default
listen [::]:80; #default
server_name MY_DOMAIN_NAME;
location / {
proxy_pass http://MY_IP_ADDRESS:8501;
}
root /usr/share/nginx/html; #default
What I tried
I tried the following, but it did not solve the problem.
https://docs.streamlit.io/knowledge-base/deploy/remote-start#symptom-2-the-app-says-please-wait-forever
streamlit run my_app.py --server.enableCORS=false
streamlit run my_app.py --server.enableWebsocketCompression=false
Try the following conf:
server {
listen 80 default_server;
server_name MY_DOMAIN_NAME;
location / {
proxy_pass http://127.0.0.1:8501/;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
and then, use this command line:
streamlit run my_app.py --server.port 8501 --server.baseUrlPath / --server.enableCORS false --server.enableXsrfProtection false
If anyone is using Ambassador as their ingress to kubernetes you'll need to allow websockets. This is explained at https://www.getambassador.io/docs/edge-stack/latest/howtos/websockets/
But, you essentially need to add the following to your Mapping
allow_upgrade:
- websocket
In my case, the fix was downgrading streamlit from v1.14.0 to v1.11.0.
What didn't work:
--server.headless=true
--server.enableCORS=false
--server.enableWebsocketCompression=false
How was the app deployed? GCS using Cloud Run
Try to upgrade Streamlit code to version 1.11.1.
Refer to this link: https://discuss.streamlit.io/t/security-notice-immediately-upgrade-your-streamlit-code-to-version-1-11-1/28399:
On July 27th, 2022, we learned of a potential vulnerability in the
Streamlit open source library that impacts apps that use custom
components. The vulnerability does not impact apps hosted on Streamlit
Cloud.
We released a patch on the same night, July 27th, 2022 at 2:20PM PST,
and ask that you immediately upgrade your Streamlit code to version
1.11.1 or higher. You can read more in this vulnerability advisory
--server.headless=true
I think the only problem is Streamlit trying to open a browser, so simply add this parameter in your run command:
streamlit run my_app.py --server.headless=true
or try to change this in the config file which is you need to create:
Refer to this explanation.
For me the answer was to load the app in incognito mode.

Accessing GraphDB Workbench throught the internet (not localhost) in a Nginx server

I have GraphDb (standlone server version) in Ubuntu Server 16 running in localhost (with the command ./graphdb -d in /etc/graphdb/bin). But I only have ssh access to the server in the terminal, can't open Worbench in localhost:7200 locally.
I have many websites running on this machine with Ningx. If I try to access the machine's main IP with the port 7200 through the external web it doesn't work (e.g. http://193.133.16.72:7200/ = "connection timmed out").
I have tried to make a reverse proxy with Nginx with this code ("xxx" = domain):
listen 7200;
listen [::]:7200;
server_name sparql.xxx.com;
location / {
proxy_pass http://127.0.0.1:7200;
}
}
But all this fails. I checked and port 7200 is open in firewall (ufw). In the logs I get info that GraphDB is working locally in some testes. But I need Workbench access to import and create repositories (not sure how to do it or if it is possible without the Workbench GUI).
Is there a way to connect through the external web to the Workbench using a domain/IP and/or Nginx?
Read all the documentation and searched all day, but could not find a way to deal with this sorry. I only used GraphDB locally (the simple installer version), never used the standalone server in production before, sorry.
PS: two extra questions related:
a) For creating an URI endpoint it is the same procedure?
b) For GraphDB daemon to start automattically at boot time (with the command ./graphdb -d in graph/bin folder), what is the recommended way and configuration? (tryed the line "/etc/graphdb/bin ./graphdb -d" in rc.local but it didn't worked).
In case it is usefull for someone, I was able to make it work with this Nginx configuration:
server {
listen 80;
server_name sparql.xxxxxx.com;
location / {
proxy_pass http://localhost:7200;
proxy_set_header Host $host;
}
}
I think it was the "proxy_set_header Host $host;" that solved it (tried the rest without it before and didn't worked). I think GraphDB uses some headers to set configurations, and they were not passing.
I wounder if I am forgetting something else important to forward in the proxy, but in this moment the Worbench seams to work and opens in the domain used "sparql.xxxxxx.com".

dotnet core - Server hangs on Production

We are currently experiencing an issue when we run our dotnet core server setup on Production. We publish it in Bamboo and run it from an AWS linux server, and it sits behind an nginx reverse proxy.
Essentially, every few days our dotnet core server process will go mute. It silently accepts and hangs on web requests, and even silently ignores our (more polite) attempts to stop it. We have verified that it is actually the netcore process that hangs by sending curl requests directly to port 5000 from within the server. We've replicated our production deployment to the best of our ability to our test environment and have not been able to reproduce this failure mode.
We've monitored the server with NewRelic and have inspected it at times when it's gone into failure mode. We've not been able to correlate this behaviour with any significant level of traffic, RAM usage, CPU usage, or open file descriptor usage. Indeed, these measurements all seem to stay at very reasonable levels.
My team and I are a bit stuck as to what might be causing our hung server, or even what we can do next to diagnose it. What might be causing our server process to hang? What further steps can we take to diagnose the issue?
Extra Information
Our nginx conf template:
upstream wfe {
server 127.0.0.1:5000;
server 127.0.0.1:5001;
}
server {
listen 80 default_server;
location / {
proxy_set_header Host $http_host;
proxy_pass http://wfe;
proxy_read_timeout 20s;
# Attempting a fix suggested by:
# https://medium.com/#mshanak/soved-dotnet-core-too-many-open-files-in-system-when-using-postgress-with-entity-framework-c6e30eeff6d1
proxy_buffering off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_cache_bypass $http_upgrade;
fastcgi_buffers 16 16k;
fastcgi_buffer_size 32k;
}
}
Our Program.cs:
using System.Diagnostics.CodeAnalysis;
using System.IO;
using System.Net;
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Logging;
using Serilog;
namespace MyApplication.Presentation
{
[ExcludeFromCodeCoverage]
public class Program
{
public static void Main(string[] args)
{
IWebHost host = WebHost.CreateDefaultBuilder(args)
#if DEBUG
.UseKestrel(options => options.Listen(IPAddress.Any, 5000))
#endif
.UseStartup<Startup>()
.UseSerilog()
.Build();
host.Run();
}
}
}
During our CD build process, we publish our application for deployment with:
dotnet publish --self-contained -c Release -r linux-x64
We then deploy the folder bin/Release/netcoreapp2.0/linux-x64 to our server, and run publish/<our-executable-name> from within.
EDIT: dotnet --version outputs 2.1.4, both on our CI platform and on the production server.
When the outage starts, nginx logs show that server responses to requests change from 200 to 502, with a single 504 being emitted at the time of the outage.
At the same time, logs from our server process just stop. And there are warnings there, but they're all explicit warnings that we've put into our application code. None of them indicate that any exceptions have been thrown.
After a few days of investigation I've found the reason of that issue. It is being caused by glibc >= 2.27, which lead to GC hang at some conditions, so there is almost nothing to do about it. However you have a few options:
Use Alpine Linux. It doesn't rely on glibc.
Use older distro like Debian 9, Ubuntu 16.04 or any other with glibc < 2.27
Try to patch glibc by yourself at your own risk: https://sourceware.org/bugzilla/show_bug.cgi?id=25847
Or wait for the glibc patch to be reviewed by community and included in your favorite distro.
More information can be found here: https://github.com/dotnet/runtime/issues/47700

nginx + gunicorn: deploy more than one Flask application [duplicate]

This question already has answers here:
Add a prefix to all Flask routes
(15 answers)
Closed 5 years ago.
I already asked this in the Arch Linux boards but didn't get no answer. So I am trying my luck over here:
I am trying to set up nginx + gunicorn on my Arch Linux server to run multiple Flask apps. However I am seemingly failing in configuring nginx the right way to do so.
When I just got one Flask app up and running everything seems to work fine. I included the /etc/nginx/sites-available and /etc/nginx/sites-enabled in my /etc/nginx/nginx.conf.
I created a file "flask_settings" inside /etc/nginx/sites/available and linked it to /etc/nginx/sites-enabled. The file looks like this:
server {
location /{
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
I have a folder containing my Flask app (sample App, hellp.py) which i run with gunicorn in a virtual environment. I just run it using
gunicorn hello:app
If i visit my servers IP I can access the file and the different routes.
Now I tried to set up another app creating another file in /etc/nginx/sites-enabled called flask2. It looks like this:
server {
location /hello {
proxy_pass http://127.0.0.1:8001;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
I then try to run the app inside its very own virtual environment with
gunicorn --bind 127.0.0.1:8001 hello:app
When I restart nginx afterwards, I am still able to access the first app and all of its routes, but if I try to access the other one by entering my servers IP + the router (after the "/"), nginx always tells me, that the sites cannot be found. Am I missing anything here?
Any help is highly appreciated.
Thanks in advance!
You should have seperate proxy location for both apps.
i.e, have one nginx conf file but multiple locations for each route or you can have seperate conf file for each.
For example:
h1.example.com proxy to a location to the required address with the port number
h2.example.com proxy to the location of the second app.
Using the approach you have, is not the recommended way to do it. Consider making a .sock file as explained in this tutorial(Although this is for Ubuntu, It can be adapted to Arch).
Another thing, (probably more important). Your server block is missing the server name. So Nginx does not know which URL to respond to.
Unless you have the server on the same computer as your local machine, 127.0.0.1 should not be accessible from the browser. Consider passing the parameter as : app.run(host = '0.0.0.0').
When it comes to the second site file. The location of the root (e.g: https://google.com/) is not mentioned. So Nginx cannot even reach the root. Again, the server name block should be mentioned for this one too.
Hope this helps.

Can I use Clojure with nginx?

This is a follow up to my question here. I've set up a home server (just my other laptop running ubuntu and nginx) and I want to serve clojure files.
I am asking help for understanding how this process works. I am sorry at this point I am confused and I think I need to start over. I am asking a new question because I want to use nginx not lein ring server, as suggested in the answer for that question.
First I started a project guestbook with leiningen and I ran lein ring server and I see "Hello World" at localhost:3000. As far as I understand this has nothing to do with nginx!
How does nginx enter in this process? At first I was trying to create a proxy server with nginx and that worked too, but I did not know how serve clojure files with that setup.
This is what I have in my nginx.conf file adapted from this answer:
upstream ring {
server 127.0.0.1:3000 fail_timeout=0;
}
server {
root /home/a/guestbook/resources/public;
# make site accessible from http://localhost
server_name localhost;
location / {
# first attempt to serve request as file
try_files $uri $uri/ #ring;
}
location #ring {
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_pass http://ring;
}
location ~ ^(assets|images|javascript|stylesheets|system)/ {
expires max;
add_header Cache-Control public;
}
}
So I want to use my domain example.com (not localhost); how do I go about doing this?
EDIT
As per #noisesmith's comment I will opt to go with lein uberjar option. As explained here, it appears very easy to create one:
$ lein uberjar
Unpacking clojure-1.1.0-alpha-20091113.120145-2.jar
Unpacking clojure-contrib-1.0-20091114.050149-13.jar
Compiling helloworld
[jar] Building jar: helloworld.jar
$ java -jar helloworld.jar
Hello world!
Can you also direct me to the right documentation about how I can use this uberjar with nginx?
Please try Nginx-Clojure module. You can run clojure Ring handlers with Nginx without any Java Web Server, eg. Jetty.
For starters, don't use lein to run things in production. You can use lein uberjar to create a jar file with all your deps ready to run, and java -jar to run the app from the resulting jar. There is also the option of running lein ring uberwar to create a war archive to be run inside tomcat, which provides some other conveniences (like log rotation and integration with /etc/init.d as a service etc. on most Linux systems).
nginx sits in front of your app, on port 80. It will serve up the content by proxying your app. This is useful because nginx has many capabilities (especially regarding security) that you then don't need to implement in your own app, including optional integration with https and selinux integration. Using nginx in front of your app also prevents you from needing to run java as root (typically only the root user can use port 80). Furthermore you can let nginx serve static assets directly, rather than having to serve them from your app.

Resources