I am trying to automate my slackr posts through Github Actions and I have a question:
Was trying to run an R script that included slackr through a github aciton and I keep running into a problem where my script runs without any errors but fails to post to slack because while the requisite environment variables ('SLACK_CHANNEL', 'SLACK_TOKEN', 'SLACK_USERNAME', etc.) are present in the action environment, they're all unfortunately blank.
It was my understanding that I could set up environment variables ahead of time through Actions Secrets, call the env and specify the variables in my yaml script, and simply use slackr() along with Sys.getenv() to point the script to my env variables rather than using slackr_setup and exposing my API key.
While I'm encouraged by the fact that R seems to be seeing the env variables I set up through Action secrets, I am still unsure about why R thinks they are blank. Could it be that I am declaring the environment at the wrong point in the yaml file? Is R simply incompatible with Actions Secrets? Something else?
empty env variables
Related
I am trying to understand the use of the files .Renviron and .Rprofile. If I understand correctly, the .Rprofile is sort of a startup script, sourced as R code, that sets the options and environment variables that a user may want either all the time, or for a specific project. On the other hand .Renviron is loaded before .Rproject, and defines environment variables only.
By design I understand that R will load either the user or the project level .Renviron and .Rprofile files, but it won't load both user and project level files. Essentially, R will only load the project specific .Rprofile and .Renviron files, provided they are defined. That said, some libraries and functions would be prudent to put in the user level .Rprofile, as I need it pretty much all the time (e.g. I use dplyr syntax a lot), while at the same time I'd like to load project specific libraries and functions as well.
The purpose of the .Renviron file is more elusive to me. From what I understand, its purpose is to store environment variables, such as passwords, API keys, etc.. However, I can also set environment variables in .Rprofile using Sys.setenv(). For example, I have the environment variable set in a project's .Rprofile, to use parallelization with the package below:
Sys.setenv(OMP_NUM_THREADS=parallel::detectCores())
library(OpenMx)
Since the .Renviron doesn't use code, my assumption is I could've put this line in a .Renviron file with the following syntax:
OMP_NUM_THREADS=[number of cores]
However, I find little useful information on how to set environment variables in .Renviron, and what is advisable to put here.
My questions therefore are:
How can I load both user and project level .Renviron and .Rproject files when working in a project?
What environment variables would I typically put in .Renviron? (Any list or tutorials on how to set variables would be appreciated.)
In which cases would it be recommended to add environment variables to .Renviron over using Sys.setenv() in .Rprofile, and vice versa?
However, I can also set environment variables in .Rprofile using Sys.setenv().
"Yes but" these can under standard POSIX behaviour not alter the running process for which the variables have to be set before.
I just like you tried to get by for as long as I could with only ~/.Rprofile (or even just Rprofile.site for the whole machine) but eventually added things in .Renviron for
R_LIBS_USER to "" as I prefer not to have installations below ~
R_MAX_NUM_DLLS which has to be here
plus a few tokens for services
plus a reticulate option
plus a R CMD check option
so in some cases you do in fact have to use .Renviron (or Renvirob.site).
I'm adapting an existing program to use Autotools for its build, but the resulting process depends on an environment variable. Is there a way to permanently set this environment variable during the build or installation process?
The program is intended to be used by Unix users and I could try to concatenate an export command directly to the .bashrc file and warn the user in case it fails because most of them will actually just use Ubuntu to run it (it's a relatively simple program that targets students), but I'd like to know if there's a more portable way to do this.
That's what I wouldn't like to do:
export VAR=/my/totally/not/hardcoded/path >> $HOME/.bashrc
Sorry to come to this late, but all of the answers to date are shockingly ... incomplete.
Building and installing software are both core use cases for the Autotools, and the installation part can absolutely involve adding or modifying files that affect user environments. If the software is installed by a user with sufficient privilege, then such effects can absolutely be applied to all system users, though the details may vary a bit from system to system (and the Autotools can help with that, too!).
For example, on RedHat-family Linuxes such as RedHat Enterprise, Fedora, Oracle Linux, and various others, you can drop an appropriately named file in /etc/profile.d, and the commands in it will automatically be read and executed by every login shell. Setting environment variables for all users is one of the common uses of this feature. I'm uncertain about Debian-family Linuxes such as Ubuntu, but it is always possible to modify file /etc/profile instead to have the same effect, and you absolutely can write an Automake install hook to do that.
Or for an altogether different approach, you can always provide a wrapper script around your program that sets the needed environment variables (supposing that the point is other than to add a directory to the PATH so as to find the program in the first place). In that case, you can even install the main program in a location that is not ordinarily in the path, so that users don't accidentally run it directly. This mechanism has the advantage that the environment variables are scoped to a run of the program, not a whole login session, but the disadvantage that users cannot override them.
I guess, no.
Autotools are about building your program, not about environment setup for the program to run. That's what users/admins are supposed to do. (Well I can imagine doing this, but I really don't want to try to figure it out, because the idea itself seems broken to me)
If your program REALLY needs some environment variable during run-time, then you should patch your sources for your application to test if the variable exists, and set one to default desired value, if it doesn't. Another idea is to enforce usage of an obligatory command line switch to pass the value in.
It's not clear what this has to do with autotools (or any other build system). No build system, by itself, can arrange for an env var to be present when the program it builds is run at a later tiem.
One solution is for your program to have a hardcoded default value for the var which is used if the environment var isn't present when the program starts running. Another frequently used solution is to name your binary something like myprog.bin and install a shell script named myprog which sets up the environment before doing exec myprog.bin.
I'm adapting an existing program to use Autotools for its build, but the resulting process depends on an environment variable. Is there a way to permanently set this environment variable during the build or installation process?
You've not been very concrete about what the program is (e.g. is the program a daemon? A user program?) or the nature of the environment variable dependency (e.g. is it another program? A mount point? A URL? A DB connection string?). Being more specific might give a better answer for you.
Anyway, autotools is not likely to offer any feature to help: It's a build system. Depending on the nature of your environment variable dependency, you're likely going to need package management (if you package it) or system administration level setup.
Since you think your primary user base is on Ubuntu this help page might give you some ideas.
I'm trying to build an R package whose goal is to run a series of analyses by taking input data and writing output data to an external database (PostgreSQL).
Specifically, I need a set of operations to be scheduled to run on a daily basis. Therefore, I have written some bash scripts with R code (using the header #!/usr/bin/env Rscript) and I have saved them into the exec/ folder of the R package. The scripts make several call to the package's core functions in R/ folder.
At this point, once installed the package on a linux server, how do I set up a crontab that is able to directly access the scripts in the exec/ folder?
Is this way of proceeding correct or is there a different best practice for such operations?
We do this all the bleeping time at work. Here at home I also have a couple of recurring cronjobs, eg for CRANberries. The exec/ folder you reference works, but my preferred solution is to use, say, inst/scripts/someScript.R.
Then one initial time you need create a softlink from your package library, say, /usr/local/lib/R/site-library/myPackage/scripts/someScript.R to a directory in the $PATH, say /usr/local/bin.
The key aspect is that the softlink persists even as you update the package. So now you are golden. All you now need is your crontab entry referencing someScript.R. We use a mix of Rscript and littler scripts.
I want to pass secure parameters to shinyapps.io deployment so my application could get them via:
Sys.getenv('PASSWORD_X')
I cannot find anything for this in deployApp function in the rsconnect package.
You can use Renviron.site or .Renviron to store and access private data into your shiny application. (see here for Hadley Wickham's recommendations and instructions - ref example below).
Solution:
Storing API Authentication Keys/Tokens (Attribution: Hadley Wickham)
If your package supports an authentication workflow based on an API key or token, encourage users to store it in an environment variable. We illustrate this using the github R package, which wraps the Github v3 API. Tailor this template to your API + package and include in README.md or a vignette.
Create a personal access token in the Personal access tokens area of
your GitHub personal settings. Copy token to the clipboard.
Identify your home directory. Not sure? Enter normalizePath("~/") in the R
console.
Create a new text file. If in RStudio, do File > New File >
Text file.
Create a line like this:
GITHUB_PAT=blahblahblahblahblahblah
where the name GITHUB_PAT reminds you which API this is for and blahblahblahblahblahblah is your personal access token, pasted from the clipboard.
Make sure the last line in the file is empty (if it isn’t R will silently fail to load the file. If you’re using an editor that shows line numbers, there should be two lines, where the second one is empty.
Save in your home directory with the filename .Renviron. If questioned,
YES you do want to use a filename that begins with a dot ..
Note that by default dotfiles are usually hidden. But within RStudio, the file browser will make .Renviron visible and therefore easy to edit in the future.
Restart R. .Renviron is processed only at the start of an R session.
Use Sys.getenv() to access your token. For example, here’s how to use your GITHUB_PAT with the github package:
library(github)
ctx <- create.github.context(access_token = Sys.getenv("GITHUB_PAT"))
# ... proceed to use other package functions to open issues, etc.
FAQ: Why define this environment variable via .Renviron instead of in .bash_profile or .bashrc?
Because there are many combinations of OS and ways of running R where the .Renviron approach “just works” and the bash stuff does not. When R is a child process of, say, Emacs or RStudio, you can’t always count on environment variables being passed to R. Put them in an R-specific start-up file and save yourself some grief.
What is the difference between Exported Environment Variables and Environment Variables?
I have to respond to a question:
How can we display all the environment variables that were defined in terminal?
First, I thought it was printenv but then again it said DEFINED IN TERMINAL and I thought this means Exported Environment Variables that I've read they're displayed with env.
I'm kind of confused.
export is a command that creates an environment variable. The phrase "exported environment variable" is redundant.
The shell may have some environment variables that weren't created with the export command, because every program starts out with an environment passed by the invoking program through the execve system call, so I guess you could say there are some environment variables that were never "exported".
But that's a silly distinction. The shell doesn't keep track of the historical origin of its environment variables. There's nothing you can do to make it tell you which ones were "defined in the terminal". It doesn't know. (history | grep export? Ugh...)
In response to Charles Goodwin's answer, there are no "persistent" environment variables in unix. The illusion of a persistent variable can be created by putting a definition in a shell startup file (/etc/profile, $HOME/.profile, etc.) but that definition will be an export command, indistinguishable from an export command that you run by hand.
On some systems, an /etc/environment file exists, which creates an even more powerful illusion of a set of "shared, persistent" environment variables, but actually they are neither. It doesn't contain the export keyword because it is not parsed by the shell - PAM handles it before starting the shell. It's the same principle as the /etc/profile though - the file has to be read into your initial process's environment every time you log in. You can see that the values are not shared by trying the "modify and check in another process" experiment on a variable that came from /etc/environment, or even modify the /etc/environment file and check the effect on already-existing processes - there won't be any.
Environment could more accurately be called "hereditary variables" - information flows only one way through them, from parent to child. It's a bit too late to change the terminology.
Might vary between OSes but my understanding was that exported variables are for that session only (i.e. open a terminal, export an environment variable, open another terminal and the exported env is not set on the new terminal) whereas environment variables are persistent (between sessions, reboots etc).
In terms of how the different types of env apply to applications, I was not aware of any (except obviously if you want the exported env to apply then you have to export it before you run the app).