Is there a convention or place in the deploy manifest to specify environment variables for machines? Or would I have to write a shell script to do this? If I have to do that, do I have to set them in a pre-job hook?
When creating a release, the convention is to set and export only those environment variables that your particular job/process needs and to do so in the job's control script.
For example, the bosh release exports several environment variables like GEM_HOME and http_proxy in the control/shell script which starts the director process (ref). The GEM_HOME statically points to its own package directory, and the http_proxy is conditionally set based on operator-configured properties from the deployment manifest.
The pre-start hook is executed in its own process, so any environment variables that your pre-start script tries to export will not be propagated to other shell scripts that are later executed by bosh or monit.
Related
Documentation for Airflow https://airflow.readthedocs.io/en/1.9.0/configuration.html
talks about setting an environment variable named $AIRFLOW_HOME which is where airflow will be installed. The configuration file airflow.cfg is created by this process has an attribute called airflow_home in the [core] section at the top of the file. This makes sense.
But, the way you override airflow variables in the airflow.cfg with environment variables is with the pattern AIRFLOW__[SECTION]__VARIABLENAME. Based on that pattern, the airflow home environment variables should technically be managed by the environment variable AIRFLOW__CORE__AIRFLOW_HOME and not AIRFLOW_HOME.
Why the difference?
Are both needed?
is one of them not needed?
do they do different things?
They do different things insofar that $AIRFLOW_HOME works as intended: the value you set will be what you get, and $AIRFLOW__CORE__AIRFLOW_HOME is likely to screw things up.
The $AIRFLOW_HOME value is special in that it's a prerequisite for a handful of actions and is read without support for the $AIRFLOW__[SECTION]__VARIABLENAME interpolation.
It is interesting that we must have used Jenkins predefined build environment variables like $WORKSPACE, $BUILD_NUMBER etc in a lot of our Jenkins job.
I find it boggling to understand that, how does Jenkins set rules such that when we print $WORKSPACE, it prints the current workspace of various jobs. How does it map the variable $WORKSPACE to the corresponding Jenkins Job.
Jenkins needs to know certain things about your build environment and jobs in order to do its job properly. For instance it needs to know the current build number, the location where your project should be checked out, who started the current build, etc. These things are typically exposed to you through the web interface.
Jenkins also exposes this information to your build scripts through environment variables that are injected into your scripts by Jenkins when your it is first launched. These environment variables can then be picked up by your script to do whatever is necessary with them.
In the example you gave ($WORKSPACE) Jenkins needs to know the absolute path to this location on your build slave because if it didn't, it wouldn't be able to check out your source and build it. Since it knows this information it exposes it to you as well to make writing your scripts easier.
There's a complete list of generally available environment variables provided by Jenkins available here.
I would like to know the best practice for setting environment variables in local machine to reflect the production environment.
I want to set the private API keys in the ENV variable, rather than directly committing them in Git. In Rails, I would use plugins like figaro to put every ENV variables in a single YML file, and they will be available.
What is the common practice in Meteor?
I think I could
run SECRET_KEY=some_key OTHER_SECRET_KEY=some_other_key meteor every time I run the local server. But that's too much to remember.
set environment variables locally but I don't want them to live in the global namespace in my machine.
Any alternatives?
Found this old post while having the same problem.
Looks like meteor is offering now to start with a config file.
meteor run --settings config.json
You would exclude that (or rather gitignore it) to keep it local. More here in the docs.
What is the difference between Exported Environment Variables and Environment Variables?
I have to respond to a question:
How can we display all the environment variables that were defined in terminal?
First, I thought it was printenv but then again it said DEFINED IN TERMINAL and I thought this means Exported Environment Variables that I've read they're displayed with env.
I'm kind of confused.
export is a command that creates an environment variable. The phrase "exported environment variable" is redundant.
The shell may have some environment variables that weren't created with the export command, because every program starts out with an environment passed by the invoking program through the execve system call, so I guess you could say there are some environment variables that were never "exported".
But that's a silly distinction. The shell doesn't keep track of the historical origin of its environment variables. There's nothing you can do to make it tell you which ones were "defined in the terminal". It doesn't know. (history | grep export? Ugh...)
In response to Charles Goodwin's answer, there are no "persistent" environment variables in unix. The illusion of a persistent variable can be created by putting a definition in a shell startup file (/etc/profile, $HOME/.profile, etc.) but that definition will be an export command, indistinguishable from an export command that you run by hand.
On some systems, an /etc/environment file exists, which creates an even more powerful illusion of a set of "shared, persistent" environment variables, but actually they are neither. It doesn't contain the export keyword because it is not parsed by the shell - PAM handles it before starting the shell. It's the same principle as the /etc/profile though - the file has to be read into your initial process's environment every time you log in. You can see that the values are not shared by trying the "modify and check in another process" experiment on a variable that came from /etc/environment, or even modify the /etc/environment file and check the effect on already-existing processes - there won't be any.
Environment could more accurately be called "hereditary variables" - information flows only one way through them, from parent to child. It's a bit too late to change the terminology.
Might vary between OSes but my understanding was that exported variables are for that session only (i.e. open a terminal, export an environment variable, open another terminal and the exported env is not set on the new terminal) whereas environment variables are persistent (between sessions, reboots etc).
In terms of how the different types of env apply to applications, I was not aware of any (except obviously if you want the exported env to apply then you have to export it before you run the app).
Setting an environment variable on the localhost is done using export.
e.g. export PORT=80
My Question is how to set an environment var for the remote meteor server.
I am using Meteor's free hosting service and deploy using meteor deploy appname, and therefore have no ssh access to the remote command line.
I'd like to set DISABLE_WEBSOCKETS to true.
I've looked at the list of possible meteor commands and haven't found one which relates to setting env vars.
You do it the same way when you run your server e.g, you don't have to use export you can just put the environment variables in the line you use to start meteor.
PORT=80 node main.js
or if you use forever
PORT=80 forever start main.js
or even with meteor
DISABLE_WEBSOCKETS=TRUE meteor
I'm a bit confused about your setup, by remote meteor server you mean a production environment? You shouldn't use the meteor command in production as it is not optimized this way and performance would be very significantly affected.
Meteor gets the environment variables using process so whatever you use to start the process you can pass the environment variables to it using the typical terminal/bash/shell/ssh that you used to start the process up.