CI testing on github actions - hasura

Good day everyone, i have this issue that i have been struggling with since Friday. i'm trying to write a CI test using github actions, but hasura migrate apply keeps throwing this error - level=fatal msg="database not found: error determining database kind for default, check if database exists on hasura". I have tried everything, anything suggestions will be deeply appreciated. i don't even know what else to pass as the anymore as the --database-name
name: Hasura Tests
on:
pull_request:
branches:
- develop
jobs:
build:
runs-on: ubuntu-latest
services:
postgres:
image: kartoza/postgis:12.4
env:
POSTGRES_USER: postgres
POSTGRES_PASS: postgrespassword
POSTGRES_DBNAME: postgres
options: --health-cmd pg_isready --health-interval 30s --health-retries 12
ports:
- 5432:5432
hasura:
image: hasura/graphql-engine:v2.6.2
env:
HASURA_GRAPHQL_METADATA_DATABASE_URL: postgres://postgres:postgrespassword#postgres:5432/postgres
HASURA_GRAPHQL_DATABASE_URL: postgres://postgres:postgrespassword#postgres:5432/postgres
PG_DATABASE_URL: postgres://postgres:postgrespassword#postgres:5432/postgres
HASURA_GRAPHQL_ENABLE_CONSOLE: "true"
HASURA_GRAPHQL_DEV_MODE: "true"
HASURA_GRAPHQL_LOG_LEVEL: debug
HASURA_GRAPHQL_ENABLED_APIS: "metadata,graphql"
HASURA_GRAPHQL_ENABLED_LOG_TYPES: startup, http-log, webhook-log, websocket-log, query-log
HASURA_GRAPHQL_ADMIN_SECRET: super-secret
ports:
- 4000:8080
steps:
- uses: actions/checkout#v2
- uses: c-hive/gha-yarn-cache#v2
- name:
run: |
docker container ls
- name: Hasura CI/CD
uses: browniefed/hasura-runner#master
with:
args: metadata apply
env:
PATH_TO_HASURA_PROJECT_ROOT: ./
HASURA_CLI_VERSION: v2.6.2
HASURA_ENDPOINT: http://hasura:8080
HASURA_ADMIN_SECRET: super-secret
- name: Hasura CI/CD
uses: browniefed/hasura-runner#master
with:
args: migrate apply --database-name default
env:
PATH_TO_HASURA_PROJECT_ROOT: ./
HASURA_CLI_VERSION: v2.6.2
HASURA_ENDPOINT: http://hasura:8080
HASURA_ADMIN_SECRET: super-secret
- name: Hasura CI/CD
uses: browniefed/hasura-runner#master
with:
args: metadata reload
env:
PATH_TO_HASURA_PROJECT_ROOT: ./
HASURA_CLI_VERSION: v2.6.2
HASURA_ENDPOINT: http://hasura:8080
HASURA_ADMIN_SECRET: super-secret
- name: Hasura CI/CD
uses: browniefed/hasura-runner#master
with:
args: seeds apply --database-name default
env:
PATH_TO_HASURA_PROJECT_ROOT: ./
HASURA_CLI_VERSION: v2.6.2
HASURA_ENDPOINT: http://hasura:8080
HASURA_ADMIN_SECRET: super-secret
- name: Run unit tests
run: |
npm install
npm run test:ci
working-directory: ./

Related

AWS CodeDeploy No space left on device With PNPM Project

Recently, I start NestJS project with PNPM.
every steps are very okay but when I deploy project to EC2 I got this AWS CodeDeploy Error
as you can see my EC2 storage got full.
this problem happened every CodeDeploy install event hook...
this is a CI/CD strategy.
CI with GitAction -> build -> compress tar.gz -> upload S3 -> call CodeDeploy -> PM2 Deploy
this image is CodeDeployAgent auto decompression tar file form deployment-root
when code deploy execute install event hook the node_modules file size are way bigger than before like this..
inside of node_modules every dependency reinstall they are dependencies.
ex) node_modules 1.5G
- .pnpm 1.5G
- lib 1
- node_modules 500M ..
- lib 2
- node_modules 500M ..
- lib 3
- node_modules 500M ..
...
this is part of GitAction workflow
...
build:
name: 🪜 Build & S3 Upload
needs: dependency_install
runs-on: ubuntu-latest
strategy:
matrix:
pnpm-version: [ 7.x ]
node-version: [ 18.12.x ]
steps:
- uses: actions/checkout#v3
- name: Setup Environments
uses: ./.github/actions/setup
- name: Use pnpm ${{ matrix.pnpm-version }}
uses: pnpm/action-setup#v2
with:
version: ${{ matrix.pnpm-version }}
- name: Use Node ${{ matrix.node-version }}
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: pnpm
- name: Check dependency cache
uses: actions/cache#v3
with:
path: |
${{ github.workspace }}/node_modules
${{ github.workspace }}/common/*/node_modules
${{ github.workspace }}/apps/${{env.SERVICE}}/node_modules
key: ${{ needs.dependency_install.outputs.dependency_cache_key }}
- name: Build
run: pnpm build:${{ env.SERVICE }}
- name: Remove DevDependencies
run: pnpm install --production
- name: AWS access registration
uses: aws-actions/configure-aws-credentials#v1-node16
with:
aws-access-key-id: ${{ secrets.AWS_IAM_MANAGE_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_IAM_MANAGE_SECRET_ACCESS_KEY }}
aws-region: ap-northeast-2
- name: add .env
run: aws s3 cp s3://${{env.BUCKET}}/env/.env.${{ env.TARGET_ENVIRONMENT }} ./
- name: tar compress
run: tar -cpzf ./build.tgz --files-from='tar_include.txt' apps/${{ env.SERVICE }}/ecosystem.config.js apps/${{ env.SERVICE }}/dist apps/${{ env.SERVICE }}/node_modules apps/${{ env.SERVICE }}/package.json .env.*
shell: bash
- name: S3 Upload
run: aws s3 cp --region ap-northeast-2 ./build.tgz s3://${{env.BUCKET}}/${{env.TARGET_ENVIRONMENT}}/build.tgz
deploy:
name: 📤 Deploy
needs: [test, build]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Setup Environments
uses: ./.github/actions/setup
- name: AWS access registration
uses: aws-actions/configure-aws-credentials#v1-node16
with:
aws-access-key-id: ${{ secrets.AWS_IAM_MANAGE_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_IAM_MANAGE_SECRET_ACCESS_KEY }}
aws-region: ap-northeast-2
- name: Code Deploy
run: aws deploy create-deployment --application-name $CODE_DEPLOY_APP_NAME --file-exists-behavior OVERWRITE --deployment-config-name CodeDeployDefault.OneAtATime --deployment-group-name ${{env.BUCKET}}-${{env.TARGET_ENVIRONMENT}} --s3-location bucket=$BUCKET,bundleType=tgz,key=${{env.TARGET_ENVIRONMENT}}/build.tgz
...
appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/app/my-project
overwrite: yes
permissions:
- object: /home/ubuntu/app/my-project
owner: ubuntu
group: ubuntu
mode: 755
hooks:
BeforeInstall:
- location: scripts/beforeInstall.sh
timeout: 120
runas: ubuntu
AfterInstall:
- location: scripts/deploy.sh
timeout: 180
runas: ubuntu
beforeInstall.sh
#!/bin/bash
REPOSITORY=/home/ubuntu/app/my-project
if [ -d $REPOSITORY ]; then
rm -rf $REPOSITORY
fi
mkdir -vp $REPOSITORY
how come node_modules storage size bigger and is there any solution to deploy project?
thank you!

Why separate jobs in Github Actions instead of adding steps?

In this Github Actions deployment script example, the author separate the build and deployment into distinct jobs.
Given that:
these jobs run sequentially (needs: build)
and on the same runner (runs-on: ubuntu-latest)
What is the advantage of separating into two jobs instead of simply inserting a build step inside the deploy job?
Here is the example:
name: Build and Deploy
on:
push:
branches:
- master
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Install Dependencies
run: npm install
- name: Build
run: npm run build-prod
- name: Archive Production Artifact
uses: actions/upload-artifact#master
with:
name: dist
path: dist
deploy:
name: Deploy
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Download Artifact
uses: actions/download-artifact#master
with:
name: dist
path: dist
- name: Deploy to Firebase
uses: w9jds/firebase-action#master
with:
args: deploy --only hosting
env:
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
For me the most important reson id CD part. When you want to deploy to enviroments you may sometine want to have control over it. And thus may vary depdends on the env. Using mutliple jobs you can have one job = envionment which is not possible if you put everytning into one job, as then you have Dev or Test env but not both.
Please check what environments gives you here.
Another useful thing is retry. You may want to retry failing phase and if this is deployment there is no point of retrying the whole build process.

Firebase deployment on a subfolder in GitHub Workflows not triggering

I'm trying to setup workflow for my github repo and it seems that it is not triggering.
GitHub repo structure -
I want to only trigger the workflow on any commit to master if there is a change done in client folder. Below is the yml of the workflow. Please tell me where should I look into to fix this.
name: Firebase Deploy
on:
push:
branches:
- master
paths: -"client/**"
defaults:
run:
working-directory: client
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: "12"
- run: npm ci
- run: CI=false npm run build
- name: GitHub Action for Firebase
uses: w9jds/firebase-action#v1.5.0
with:
args: deploy --only hosting
env:
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
Note - I've setup the FIREBASE_TOKEN in repo also.

Need help for CI and CD in Github Workflow for Firebase Web App using HUGO Framework

I'm using HUGO to build a website and connected it in Forestry.io for headless cms. Everything is in my GitHub repository.
I have this 2 files in workflows folder under .github folder: deploy.yml and integrate.yml
I always get this error:
13 Insufficient number of arguments or no entry found.
14 Alternatively, run 'webpack(-cli) --help' for usage info.
------deploy.yml and integrate.yml files------
name: Firebase Continuous Deployment
on:
push:
branches: [master]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
- uses: actions/setup-node#master
with:
node-version: 10.16.0
- run: npm ci
- run: npm run build
- uses: w9jds/firebase-action#master
with:
args: deploy --only hosting
env:
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
name: Node Continuous Integration
on:
pull_request:
branches: [master]
jobs:
test_pull_request:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v1
with:
node-version: 12
- run: npm ci
- run: npm test
- run: npm run build
Double-check with the w9jds/firebase-action GitHub Action instructions:
the build job is suppose to execute the npm commands, not the deploy job
the deploy job is supposed to call Firebase (w9jds/firebase-action#master) with paths parameters
That is:
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Download Artifact
uses: actions/download-artifact#master
with:
name: dist
path: dist
Try with this:
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Install Dependencies
run: npm ci
- name: Build
run: npm run build
- name: Archive Production Artifact
uses: actions/upload-artifact#master
with:
name: dist
path: dist
deploy:
name: Deploy
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Download Artifact
uses: actions/download-artifact#master
with:
name: dist
path: dist
- name: Deploy to Firebase
uses: w9jds/firebase-action#master
with:
args: deploy --only hosting
env:
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}

Deploy environment variables to Firebase hosting

I use Firebase in React and when initializing Firebase, I use environment variables fetched from my .env file with dotenv. I want to build and deploy my React app to Firebase hosting, I use GitHub Actions with the following .yml workflow file:
name: Deploy
on:
push:
branches:
- master
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Install Dependencies
run: npm install
- name: Build
run: npm run build
- name: Archive Production Artifact
uses: actions/upload-artifact#master
with:
name: public
path: public
deploy:
name: Deploy
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Download Artifact
uses: actions/download-artifact#master
with:
name: public
- name: Deploy
uses: w9jds/firebase-action#master
with:
args: deploy --only hosting
env:
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
However, my build step fails (i.e. npm run build) because the environment variables are not found. If I hardcode the values instead of using my environment variables, the workflow is successful.
Do I need to do add my environment variables to GitHub secrets similar to how I added FIREBASE_TOKEN in my deploy step and add them to the workflow to be something like:
name: Deploy
on:
push:
branches:
- master
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Install Dependencies
run: npm install
- name: Build
run: npm run build
env:
FIREBASE_API_KEY: ${{ secrets.FIREBASE_API_KEY }}
FIREBASE_AUTH_DOMAIN: ${{ secrets.FIREBASE_AUTH_DOMAIN }}
FIREBASE_DATABASE_URL: ${{ secrets.FIREBASE_DATABASE_URL }}
FIREBASE_PROJECT_ID: ${{ secrets.FIREBASE_PROJECT_ID }}
FIREBASE_STORAGE_BUCKET: ${{ secrets.FIREBASE_STORAGE_BUCKET }}
FIREBASE_MESSAGING_SENDER_ID: ${{ secrets.FIREBASE_MESSAGING_SENDER_ID }}
FIREBASE_APP_ID: ${{ secrets.FIREBASE_APP_ID }}
- name: Archive Production Artifact
uses: actions/upload-artifact#master
with:
name: public
path: public
deploy:
name: Deploy
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Download Artifact
uses: actions/download-artifact#master
with:
name: public
- name: Deploy
uses: w9jds/firebase-action#master
with:
args: deploy --only hosting
env:
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
I receive the same error, saying that it can't find the API token.
When I browse workflow files in other firebase-based React apps, I can't see anyone adding these environment variables like I have suggested. What is the proper way of handling environment variables in apps deployed to Firebase? Do they belong in the build or deploy step?
Your custom env variables must start with REACT_APP_. Any other variables except NODE_ENV will be ignored to avoid accidentally exposing a private key on the machine that could have the same name.
Please take a look at this article Adding Custom Environment Variables.
As an additional note you can also run the npm run build along with your env variables passed as parameters like this:
REACT_APP_FIREBASE_APIKEY=${{ secrets.REACT_APP_FIREBASE_APIKEY }} REACT_APP_FIREBASE_AUTHDOMAIN=${{ secrets.REACT_APP_FIREBASE_AUTHDOMAIN }} REACT_APP_FIREBASE_DBURL=${{ secrets.REACT_APP_FIREBASE_DBURL }} npm run build
Instead of adding a .env file you can easily add the secrets to github secrets.
Once the secrets are added to github secrets you can add them to env at the build process as follow in your workflow.
name: Firebase Deployment
on:
push:
branches:
- master
- feature/secrets_fix
env:
API_KEY: ${{ secrets.FIREBASE_API_KEY }}
APP_ID: ${{ secrets.FIREBASE_APP_ID }}
AUTH_DOMAIN: ${{ secrets.FIREBASE_AUTH_DOMAIN }}
MESSAGE_SENDER_ID: ${{ secrets.FIREBASE_MESSAGE_SENDER_ID }}
PROJECT_ID: ${{ secrets.FIREBASE_PROJECT_ID }}
STORAGE_BUCKET: ${{ secrets.FIREBASE_STORAGE_BUCKET }}
TOKEN: ${{ secrets.FIREBASE_TOKEN }}
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#v2.3.2
- name: Install Dependencies
run: npm install
- name: Build
env:
CI: false
run: npm run build
- name: Archive Production Artifact
uses: actions/upload-artifact#v2
with:
name: build
path: build
deploy:
name: Deploy
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#v2.3.2
- name: Download Artifact
uses: actions/download-artifact#v2
with:
name: build
path: build
- name: Deploy to Firebase
uses: w9jds/firebase-action#master
with:
args: deploy --only hosting
env:
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
secrets.${YOUR_GITHUB_SECRET_NAME}
Once they are mapped as above under env, you can easily retrieve them from react application using
process.env.YOUR_ENV_VARIABLE_NAME_HERE
Following is a example on firebaseConfig
const firebaseConfig = {
apiKey: process.env.API_KEY,
authDomain: process.env.AUTH_DOMAIN,
projectId: process.env.PROJECT_ID,
storageBucket: process.env.STORAGE_BUCKET,
messagingSenderId: process.env.MESSAGE_SENDER_ID,
appId: process.env.APP_ID
};
Hope this will help someone.

Resources