(GH newb attempting to cargo cult my way through some configuration ...)
I am using the GitHub actions YAML file here to test an R package. By default this workflow tests on four separate platforms, but this is overkill for my day-to-day needs. (This action does more limited testing, but I think? it also misses some stuff I wanted from the fuller workflow ...)
I would like to have the tests done on all four platforms only if the commit message includes the string "full check", otherwise testing just one platform. I know I can in principle achieve something like this if I include a conditional:
if: "contains(github.event.head_commit.message, '[full ci]')"
but I'm not sure of the precise syntax for including it in the workflow (and I'm afraid of spending many hours on trial-and-error attempts)
The chunk of the workflow that defines the set of platforms current looks like this, with three of the platforms commented out:
strategy:
fail-fast: false
matrix:
config:
- {os: ubuntu-20.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"}
## - {os: ubuntu-20.04, r: 'devel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"}
## - {os: windows-latest, r: 'release'}
## - {os: macOS-latest, r: 'release'}
I'm assuming that just sticking the if: clause below the first row won't work (because it would break up the list). I suppose something like this might work?
matrix:
config:
if: "! contains(github.event.head_commit.message, '[full ci]')"]
config:
- {os: ubuntu-20.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"}
if: "contains(...)"
config:
[list with all four configs]
(Does this syntax have an if/else construct?)
I'm looking for answers that help me achieve the narrow goal, and hopefully understand the YAML syntax a little better (I have tried to read the FM ...)
I believe it is not possible to add conditions to the matrix construct itself. You could generate the matrix data beforehand in another job and then use it in your job.
There is a simpler solution: Use two jobs, because if can be used at the job level. Copy the current job that you have and then add if-conditions to each job. Keep one job as it is for the full CI and in the other job hardcode your environment for the ordinary commits.
Something like this:
jobs:
R-CMD-check-ordinary:
if: ${{ ! contains(github.event.head_commit.message, '[full ci]') }}
# Hardcode os
runs-on: ubuntu-20.04
# Hardcode name
name: Test only on Ubuntu
env:
R_REMOTES_NO_ERRORS_FROM_WARNINGS: true
RSPM: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
# replace any occurences of matrix.config. with hardcoded values
...
# And the original job
R-CMD-check-full:
if: ${{ contains(github.event.head_commit.message, '[full ci]') }}
runs-on: ${{ matrix.config.os }}
name: ${{ matrix.config.os }} (${{ matrix.config.r }})
strategy:
fail-fast: false
matrix:
config:
- {os: ubuntu-20.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"}
- {os: ubuntu-20.04, r: 'devel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"}
- {os: windows-latest, r: 'release'}
- {os: macOS-latest, r: 'release'}
env:
R_REMOTES_NO_ERRORS_FROM_WARNINGS: true
RSPM: ${{ matrix.config.rspm }}
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
...
Regarding YAML syntax:
YAML is a data format. Don't think of YAML as a script or program. A YAML file is similar to a list of command line arguments for a program. It only defines input data for another program.
So there is no if in YAML. For YAML if: condition is like as any other key: value pair, there is no logic attached.
Related
This is cross posted on the RStudio Community site. I am using r-lib/setup-r-dependencies#v2 for a bookdown project on GitHub action. However, the job is stalling when pak tries to create a lock file. Here is an example run , and here is the workflow.
As you can see, eventually the job just times out.
The issue seems similar to this post. The solution there was to use the most current RSPM, but I believe that is already what I am using (I don't specify anything different in the workflow).
Any ideas on what might be causing this?
Edit: Here is the full yaml code for the workflow (linked above)
on:
push:
branches: [main, master]
workflow_dispatch:
name: bookdown
jobs:
bookdown:
runs-on: ubuntu-latest
# Only restrict concurrency for non-PR jobs
concurrency:
group: pkgdown-${{ github.event_name != 'pull_request' || github.run_id }}
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout#v2
- uses: r-lib/actions/setup-pandoc#v2
- uses: r-lib/actions/setup-r#v2
with:
use-public-rspm: true
- uses: r-lib/actions/setup-r-dependencies#v2
with:
cache-version: 1
- name: Cache bookdown results
uses: actions/cache#v2
with:
path: _bookdown_files
key: bookdown-${{ hashFiles('**/*Rmd') }}
restore-keys: bookdown-
- name: Build site
run: bookdown::render_book("index.Rmd", quiet = TRUE)
shell: Rscript {0}
- name: Deploy to GitHub pages 🚀
if: github.event_name != 'pull_request'
uses: JamesIves/github-pages-deploy-action#4.1.4
with:
branch: gh-pages
folder: _book
trying to implement a workflow on Github actions that will run a test script with the functions of my package and ideally print/export out the results. I am new to the YAML syntax and would appreciate any help.
This is the workflow I currently have:
on:
push:
branches:
- main
- master
pull_request:
branches:
- main
- master
name: Test-Script
jobs:
build:
runs-on: ${{ matrix.config.os }}
name: ${{ matrix.config.os }} (${{ matrix.config.r }})
strategy:
fail-fast: false
matrix:
config:
- {os: windows-latest, r: 'release'}
- {os: macOS-latest, r: 'release'}
steps:
- uses: actions/checkout#v2
- uses: r-lib/actions/setup-r#v1
with:
r-version: ${{ matrix.config.r }}
- uses: r-lib/actions/setup-pandoc#v1
- name: Test against test script
run: |
/R/test_script.R #file path in repo
shell: Rscript {0}
I get the erorr: Error: Error: unexpected '/' in "/". I have tried multiple adjustments but always get that error or another saying that the file cannot be found. I would also like to print out/export out the results but I do not know how to do so.
Would appreciate any help please. Thank you.
I am struggling to replace values in my appsettings.json file and was wondering where I'm going wrong?
I have created two secrets in GitHub entitled COSMOSDBSETTINGS_SERVICEENDPOINT and COSMOSDBSETTINGS_AUTHKEY.
- name: Publish
run: dotnet publish -c Release -o ${{env.DOTNET_ROOT}}/myapp
- name: App Settings Variable Substitution
uses: microsoft/variable-substitution#v1
with:
files: '${{env.DOTNET_ROOT}}\myapp\appsettings.json'
env:
CosmosDbSettings:ServiceEndpoint: ${{ secrets.COSMOSDBSETTINGS_SERVICEENDPOINT }}
CosmosDbSettings:AuthKey: ${{ secrets.COSMOSDBSETTINGS_AUTHKEY }}
I get the error stating
Run microsoft/variable-substitution#v1
Applying variable substitution on JSON file: C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\myapp\appsettings.json
Skipped updating file: C:\Users\runneradmin\AppData\Local\Microsoft\dotnet\myapp\appsettings.json
Error: Error: Failed to apply variable substitution
This is the appsettings locally:
"CosmosDbSettings": {
"ServiceEndpoint": "https://localhost:8081/",
"AuthKey": "KEY"
},
Use a dot instead of a colon:
env:
CosmosDbSettings.ServiceEndpoint: ${{ secrets.COSMOSDBSETTINGS_SERVICEENDPOINT }}
CosmosDbSettings.AuthKey: ${{ secrets.COSMOSDBSETTINGS_AUTHKEY }}
my purpose is simple, run cypress e2e test using Github actions upon Pull Request. I used cypress-firebase for testing and all my test should run with Firebase Emulator. And I also used cypress-io/github-action for CI.
My problem is, when using cypress-io/github-action, I need to pass some environment variables for my react app to work with Firebase emulator, and all environment variables can not be recognized by the whole entire app. See my workflow file to understand.
Here are my related part of my Github action workflow file:
- name: Cypress run
uses: cypress-io/github-action#v2
with:
browser: chrome
headless: true
record: true
start: yarn run ci:start:emulator
wait-on: "http://localhost:3000"
wait-on-timeout: 300 # wait for 5 minutes
env:
# Add debugger
# https://github.com/cypress-io/github-action#debugging
DEBUG: "#cypress/github-action"
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
PROJECT_ID: ${{ secrets.REACT_APP_PROJECT_ID }}
FIREBASE_CONFIG: ${{ secrets.FIREBASE_CI_CLOUD_FUNCTIONS_CONFIG }}
# Branch settings
GITHUB_HEAD_REF: ${{ github.head_ref }}
GITHUB_REF: ${{ github.ref }}
# pass the Cypress Dashboard variables
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
CYPRESS_PROJECT_ID: ${{ secrets.CYPRESS_PROJECT_ID }}
# UID of user to login during test
CYPRESS_BASE_URL: http://localhost:3000
CYPRESS_TEST_UID: ${{ secrets.CYPRESS_TEST_UID }}
# Service Account (used for creating custom auth tokens)
SERVICE_ACCOUNT: ${{ secrets.CYPRESS_SERVICE_ACCOUNT }}
# Environment variables
REACT_APP_ANY_KEY: ${{ secrets.REACT_APP_ANY_KEY }}
And here is my ci:start:emulator command in package.json file:
"ci:start:emulator": "firebase emulators:exec 'yarn start'"
The thing is yarn start command does not recognize my above REACT_APP_ANY_KEY environment variable. It seems cypress-io/github-action did not pass them to my yarn start command.
What I tried is changing above command into
"ci:start:emulator": "cross-env REACT_APP_ANY_KEY=<SOME_HARD_CODE_VALUE> firebase emulators:exec 'yarn start'"
It works perfectly! But of course, we don't want to pass tons of environment variables via this command like this.
Any help will be highly appreciated!
According to the docs You can define environment variables for a step, job, or entire workflow, so here you defined those env variables only for this step Cypress run and not for the entire job, to solve this you should define env variables using this, an example:
jobs:
job1:
env:
# Add debugger
# https://github.com/cypress-io/github-action#debugging
DEBUG: "#cypress/github-action"
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
PROJECT_ID: ${{ secrets.REACT_APP_PROJECT_ID }}
FIREBASE_CONFIG: ${{ secrets.FIREBASE_CI_CLOUD_FUNCTIONS_CONFIG }}
# Branch settings
GITHUB_HEAD_REF: ${{ github.head_ref }}
GITHUB_REF: ${{ github.ref }}
# pass the Cypress Dashboard variables
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
CYPRESS_PROJECT_ID: ${{ secrets.CYPRESS_PROJECT_ID }}
# UID of user to login during test
CYPRESS_BASE_URL: http://localhost:3000
CYPRESS_TEST_UID: ${{ secrets.CYPRESS_TEST_UID }}
# Service Account (used for creating custom auth tokens)
SERVICE_ACCOUNT: ${{ secrets.CYPRESS_SERVICE_ACCOUNT }}
# Environment variables
REACT_APP_ANY_KEY: ${{ secrets.REACT_APP_ANY_KEY }}
env variables should be accessible across this Job's steps, if you wanted to set global environment variables, You should define them before the jobs.
In a cloudbuild template, I have a step to deploy a Cloud function.
I try to deploy the function passing clear environment variables but also secretEnv at the same step.
I have tried several things without success and the documentation is clear about the fact we cannot use --set-env-vars and --env-vars-file nor --update-env-vars in the same command.
Does anyone succeeded in sending both variables types: clear ones from a file and secret ones with secretEnv ?
The following definition successfully create variables from .env.prod.yaml but USER and PASSWORD secrets are not created into the Cloud function.
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['functions',
'deploy', 'my-function',
'--runtime', 'go111',
'--entry-point', 'MyFunction',
'--env-vars-file', '.env.prod.yaml',
'--trigger-topic', 'my-topic']
secretEnv: ['USER', 'PASSWORD']
secrets:
- kmsKeyName: projects/my-project/locations/global/keyRings/my-keyring/cryptoKeys/my-key
secretEnv:
USER: XXXXXXXXXXXXXXXXXXXXXXXXXXX
PASSWORD: XXXXXXXXXXXXXXXXXXXXXXXXXXX
Any ideas, best practice or nice workaround ?
Plaintext environment variables should be set on the build step itself in the env field:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
env:
- USER=my-username
secretEnv: ['PASSWORD']
secretEnv:
- kmsKeyName: projects/...
secretEnv:
PASSWORD: ajklddafjkalda....
Just deploy the function twice. First to deploy the changes and set the env variables from the file, and then again using the --update-env-vars to add encrypted variables.
As per the Updating environment variables documentation, the --update-env-vars will not remove all existing environment variables before adding new ones.
An example of the cloudbuild.yaml could be:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['functions', 'deploy', 'my-function’,
'--runtime', 'go111',
'--entry-point', 'MyFunction',
'--env-vars-file', 'env.prod.yaml',
'--trigger-topic', 'my-topic']
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args: ['-c', 'gcloud functions deploy my-function --update-env-vars USER=$$USER,PASSWORD=$$PASSWORD']
secretEnv: ['USER', 'PASSWORD']
secrets:
- kmsKeyName: projects/[PROJECT-ID]/locations/global/keyRings/[KEYRING-NAME]/cryptoKeys/[KEY-NAME]
secretEnv:
USER: XXXXXXXXXXXXXXXXXXXXXXXXX
PASSWORD: XXXXXXXXXXXXXXXXXXXXX