I have a concourse pipeline which takes git source code, build then deploy it to pcf.
Now I have to make two deployments after the build, pcf-dev and pcf-qa with dependency of qa over dev. It means if dev deployment is successful then do the qa deployment.
groups: []
resources:
- name: pcf-dev
type: cf
- name: pcf-qa
type: cf
- name: source-code
type: git
resource_types: []
jobs:
- name: build-deploy
public: true
plan:
- get: source-code
- task: build
privileged: true
config:
platform: linux
image_resource:
type: docker-image
source:
repository: java
tag: openjdk-8-alpine
run:
path: sh
args:
- -exc
- |
set -e -u -x
cd source-code/api/
./mvnw package
cp target/*.jar ../../build-output/api.jar
cd /tmp
find .
inputs:
- name: source-code
outputs:
- name: build-output
- put: pcf-dev
params:
path: build-output/api.jar
- put: pcf-qa
params:
path: build-output/api.jar
I don't know how to use "passed" condition for such case. I know I can use it with "get" but don't know how to use it with "put" for my case.
Can anyone please help?
It should work as is. If pcf-dev fails then the job will fail and stop, and pcf-qa won't run. If pcf-dev passes then pcf-qa will run. Tasks only run at the same time if they are in an aggregate block.
Related
I am working on the .NET Core solution below:
and I am using the Gitlab CI definition below:
.gitlab-ci.yml:
image: docker:stable
services:
- docker:dind
stages:
- test
- build
- deploy-db
- deploy
variables:
DOCKER_REGISTRY: "our.registry.eu"
IMAGE_NAME: "${DOCKER_REGISTRY}/membercreditor"
# secrets are on GitLab environment variables (settings > ci/cd)
RANCHER_ENV: 1a10
RANCHER_STACK: api
RANCHER_SVC: membercreditor
RANCHER_DEV_KEY: a dev key
RANCHER_DEV_URL: http://our.dev.rancher.io:8080
RANCHER_PREPROD_KEY: a preprod key
RANCHER_PREPROD_URL: http://our.preprod.rancher.io:8080
RANCHER_PROD_KEY: a prod key
RANCHER_PROD_URL: http://our.prod.rancher.io:8080
.test:
stage: test
image: mcr.microsoft.com/dotnet/core/sdk:3.1-alpine
script:
- dotnet test ./Rm.MemberCreditor.Api.Tests/ -l:junit /p:CollectCoverage=true /p:Include="[Rm.*]*" /p:CoverletOutput=./TestResults/
- dotnet test ./Rm.MemberCreditor.Domain.Tests/ -l:junit /p:CollectCoverage=true /p:Include="[Rm.*]*" /p:CoverletOutput=./TestResults/ /p:MergeWith=../Rm.MemberCreditor.Api.Tests/TestResults/coverage.json
- dotnet test ./Rm.MemberCreditor.IntegrationTests/ -l:junit /p:CollectCoverage=true /p:Include="[Rm.*]*" /p:CoverletOutput=./TestResults/ /p:MergeWith=../Rm.MemberCreditor.Domain.Tests/TestResults/coverage.json
coverage: '/Total[ |]+(\d+[,.]\d+)%/'
artifacts:
reports:
junit: ./*.Tests/TestResults/*.xml
.build:
stage: build
tags:
- dind
script:
- docker build -t $IMAGE_NAME:$APP_VERSION -f Dockerfile .
- docker login $DOCKER_REGISTRY -u svc_finance_revmgt -p $SVC_FINANCE_REVMGT_TOKEN
- docker push $IMAGE_NAME:$APP_VERSION
.deploy-db:
stage: deploy-db
image: mcr.microsoft.com/dotnet/core/sdk:3.1-alpine
script:
- export CONNECTIONSTRINGS__MEMBERCREDITOR=$CONNECTIONSTRINGS__MEMBERCREDITOR
- export SECRETS__CS_MEMBERCREDITOR=$SECRETS__CS_MEMBERCREDITOR
- dotnet tool restore
- dotnet ef database update -p ./Rm.MemberCreditor.Data.Migrations -s ./Rm.MemberCreditor.Api -v
.deploy:
stage: deploy
image: cdrx/rancher-gitlab-deploy
script:
- upgrade --rancher-key $RANCHER_KEY
--rancher-secret $RANCHER_SECRET
--rancher-url $RANCHER_URL
--environment $RANCHER_ENV
--stack $RANCHER_STACK
--service $RANCHER_SVC
--new-image $IMAGE_NAME:$APP_VERSION
###########################
### Merge Requests
###########################
test-merge-request:
extends: .test
only:
refs:
- merge_requests
###########################
### Integration
###########################
test-integration:
extends: .test
environment:
name: integration
only:
- develop
build-integration:
extends: .build
environment:
name: integration
dependencies:
- test-integration
before_script:
- APP_VERSION=latest-integration
only:
- develop
deploy-db-integration:
extends: .deploy-db
environment:
name: integration
dependencies:
- build-integration
before_script:
- export ASPNETCORE_ENVIRONMENT=Integration
only:
- develop
deploy-integration:
extends: .deploy
environment:
name: integration
dependencies:
- deploy-db-integration
before_script:
- APP_VERSION=latest-integration
- RANCHER_KEY=$RANCHER_DEV_KEY
- RANCHER_SECRET=$RANCHER_DEV_SECRET
- RANCHER_URL=$RANCHER_DEV_URL
only:
- develop
###########################
### UAT
###########################
test-uat:
extends: .test
environment:
name: uat
only:
- master
build-uat:
extends: .build
environment:
name: uat
dependencies:
- test-uat
before_script:
- APP_VERSION=latest-uat
only:
- master
deploy-db-uat:
extends: .deploy-db
environment:
name: uat
dependencies:
- build-uat
before_script:
- export ASPNETCORE_ENVIRONMENT=Uat
only:
- master
deploy-uat:
extends: .deploy
environment:
name: uat
dependencies:
- deploy-db-uat
before_script:
- APP_VERSION=latest-uat
- RANCHER_KEY=$RANCHER_PREPROD_KEY
- RANCHER_SECRET=$RANCHER_PREPROD_SECRET
- RANCHER_URL=$RANCHER_PREPROD_URL
only:
- master
###########################
### Production
###########################
build-prod:
extends: .build
environment:
name: production
before_script:
- APP_VERSION=$CI_COMMIT_TAG
only:
- /^\d+.\d+.\d+$/
except:
- /^(?:[^m]|m[^a]|ma[^s]|mas[^t]|mast[^e]|maste[^r]).*#/
deploy-db-prod:
extends: .deploy-db
environment:
name: production
dependencies:
- build-prod
before_script:
- export ASPNETCORE_ENVIRONMENT=Production
only:
- /^\d+.\d+.\d+$/
except:
- /^(?:[^m]|m[^a]|ma[^s]|mas[^t]|mast[^e]|maste[^r]).*#/
deploy-prod:
extends: .deploy
environment:
name: production
when: manual
dependencies:
- deploy-db-prod
before_script:
- APP_VERSION=$CI_COMMIT_TAG
- RANCHER_KEY=$RANCHER_PROD_KEY
- RANCHER_SECRET=$RANCHER_PROD_SECRET
- RANCHER_URL=$RANCHER_PROD_URL
only:
- /^\d+.\d+.\d+$/
except:
- /^(?:[^m]|m[^a]|ma[^s]|mas[^t]|mast[^e]|maste[^r]).*#/
When the Gitlab is running the test-merge-request, I am getting the error below:
Fetching changes with git depth set to 50...
00:02
Initialized empty Git repository in /builds/finance/products/revenuemgmt/modules/membercreditor/.git/
Created fresh repository.
From https://our.org/finance/products/revenuemgmt/modules/membercreditor
* [new ref] refs/merge-requests/38/head -> refs/merge-requests/38/head
* [new ref] refs/pipelines/355543 -> refs/pipelines/355543
Checking out 603e9a2c as refs/merge-requests/38/head...
Skipping Git submodules setup
$ dotnet test ./Rm.MemberCreditor.Api.Tests/ -l:junit /p:CollectCoverage=true /p:Include="[Rm.*]*" /p:CoverletOutput=./TestResults/
00:12
FSC : error FS0078: Unable to find the file 'Legacy/Queries/GetOrderRelatedInfoForAccountingEntryCreation.sql' in any of /builds/finance/products/revenuemgmt/modules/membercreditor/Rm.MemberCreditor.Data [/builds/finance/products/revenuemgmt/modules/membercreditor/Rm.MemberCreditor.Data/Rm.MemberCreditor.Data.fsproj]
Uploading artifacts...
00:02
WARNING: ./*.Tests/TestResults/*.xml: no matching files
ERROR: No files to upload
ERROR: Job failed: exit code 1
GetOrderRelatedInfoForAccountingEntryCreation.sql is defined as an embedded resource, I tried to force to copy the file to the output but I still am getting the same very same error.
I am not sure why this post has been down-voted, but fun fact, seems there is a bug in Rider 2019.3.1.
The issue arose because even though the file was showing up as GetOrderRelatedInfoForAccountingEntryCreation.sql in Rider aka pascal case it was actually written as a camel case (i.e. getOrderRelatedInfoForAccountingEntryCreation.sql) on the disk, go figure.
Once the name adjusted to pascal for the actual file on the disk, everything worked fine.
I want to automatically discover new stemcell versions on pivnet, download them from pivotal network and upload them to a local artifactory.
However, the upload (to artifactory) task fails with the following error:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 410M 0 0 100 410M 0 439M --:--:-- --:--:-- --:--:-- 440M
curl: (18) transfer closed with outstanding read data remaining
I get this error on uploading any kind of release and stemcell.
My pipeline configuration looks like this:
---
resource_types:
- name: artifactory
type: docker-image
source:
repository: pivotalservices/artifactory-resource
- name: pivnet
type: docker-image
source:
repository: pivotalcf/pivnet-resource
tag: latest-final
resources:
- name: git-repository
type: git
source:
uri: ssh://<git-repository>
private_key: ((ssh_key))
- name: stemcell
type: pivnet
check_every: 1m
source:
api_token: ((pivnet-api-token))
product_slug: stemcells
- name: artifactory
type: artifactory
source:
endpoint: https://((artifactory_domain)):443/artifactory
repository: "/<path>/stemcells/bosh-vsphere-esxi-ubuntu-trusty-go_agent"
regex: "bosh-vsphere-esxi-ubuntu-trusty-go_agent-(?<version>.*).tgz"
username: ((artifactory_username))
password: ((artifactory_password))
jobs:
- name: download-and-upload
plan:
- get: <git-repository>
- get: stemcell
trigger: true
version: every
- task: rename-files
file: <git-repository>/tasks/rename-stemcell/task.yml
- put: artifactory
params: { file: renamed-stemcell/stemcell/bosh-vsphere-esxi-ubuntu-trusty-go_agent*.tgz }
...
I use the concourse version v3.9.1 and the stemcell bosh-vsphere-esxi-ubuntu-trusty-go_agent/3468.21. Concourse is deployed as a BOSH release.
Any hints what could be the root cause of this error?
I tried to manually issue the curl command which resulted in the same error.
Then, I tried to upload the stemcell manually.
By this, it came out that there were missing deploy permissions on artifactory.
My tests work fine on circle 1.0 config. But when i try to migration to the new 2.0 config, it gives me this error -
[Error: Selenium server did not start.Another Selenium process may already be running or your java version may be out of date.]
I've already tried added jre/jdk installation. I logged in with ssh and checked - there is nothing else running on 4444 port and java is installed.. So I'm not sure whats wrong.
Here is my 1.0 config
machine:
node:
version: 4.6.2
dependencies:
pre:
- echo $METEOR_SETTINGS > settings.json
- echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
cache_directories:
- "~/.npm"
- "~/.meteor"
- "node_modules"
- "./.meteor/local/build"
- "./.meteor/local/bundler-cache"
- "./.meteor/local/isopacks"
- "./.meteor/local/plugin-cache"
- "/opt/circleci/nodejs/v4.6.2/bin"
- "/opt/circleci/nodejs/v4.6.2/lib/node_modules"
override:
- ./.testing/upgrade_chrome_version.sh
- ./.testing/cache_meteor.sh
- ./.testing/cache_npm_dependencies.sh
- ./.testing/cache_build_and_dependencies.sh
checkout:
post:
- git submodule update --init
test:
override:
- case $CIRCLE_NODE_INDEX in 0) meteor npm test ;; 1) ./tests/acceptance_run ;; esac:
parallel: true
Here is my 2.0 config
version: 2
jobs:
build:
working_directory: ~/newkeyz
docker:
- image: circleci/node:4.8.3-browsers
environment:
_JAVA_OPTIONS: "-Xms512m -Xmx1024m"
- image: selenium/standalone-chrome-debug
- image: mongo:3.4.4
steps:
- checkout
- restore_cache:
name: Restore Meteor Cache
key: meteor-cache-{{ checksum ".meteor/release" }}
- restore_cache:
name : Restore NPM Cache
key: npm-cache-{{ checksum "package.json" }}
- restore_cache:
name: Restore Meteor Package Cache
key: packages-cache-{{ checksum ".meteor/versions" }}
- run: sudo sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && sudo dpkg-reconfigure --frontend=noninteractive locales
- run: sudo apt-get install default-jdk
- run:
name: Create Settings File
command: echo $METEOR_SETTINGS > settings.json
- run: ./.testing/cache_meteor.sh
- save_cache:
name: Save Meteor Cache
key: meteor-cache-{{ checksum ".meteor/release" }}
paths:
- '~/.meteor'
- run: meteor npm install
- run: ./.testing/cache_npm_dependencies.sh
- save_cache:
name: Save NPM Cache
key: npm-cache-{{ checksum "package.json" }}
paths:
- '~/.npm'
- 'node_modules'
- run:
name: Run Test
command: ./tests/acceptance_run
- save_cache:
key: packages-cache-{{ checksum ".meteor/versions" }}
paths:
- './.meteor/local/build'
- './.meteor/local/bundler-cache'
- './.meteor/local/isopacks'
- './.meteor/local/plugin-cache'
I would like to override default tmp.conf at /usr/lib/tmpfiles.d/ with /etc/tmpfiles.d/tmp.conf and run the cron job at midnight on everyday. The service need to run as systemd-tmpfiles --clean. How can I run the service at midnight, Somebody help me please?
Sample code:
tmp.conf:
file.managed:
- name: /etc/tmpfiles.d/tmp.conf
- source: salt://tmp/files/tmp.conf
- user: root
- mode: 644
- require:
- user: root
run_systemd-tmpfiles:
cron.present:
- user: root
- minute: 0
- hour: 0
- require:
- file: tmp.conf
enable_tmp_service:
service.running:
- name: systemd-tmpfiles --clean
- enable: True
- require:
- cron: run_systemd-tmpfiles
If you just want the command to run as part of a cron, you would need to have that cron.present setup to run the command.
cron_systemd-tmpfiles:
cron.present:
- name: systemd-tmpfiles --clean
- user: root
- minute: 0
- hour: 0
- require:
- file: tmp.conf
If you then want to run it in this state, you can't use the tmpfile.service, you would just run the command through a cmd.run, or if you only want it run when the file.managed changes, you would use cmd.wait
run tmpfiles:
cmd.wait:
- name: systemd-tmpfiles --clean
- listen:
- file: tmp.conf
But systemd-tmpfiles.service is already run on boot if you are using systemd, so there is no reason to enable it again. And when it runs during the beginning of the boot process, it will run the same way tmpfile --clean runs.
I have the following states:
copy_over_systemd_service_files:
file.managed:
- name: /etc/systemd/system/consul-template.service
- source: salt://mesos/files/consul-template.service
- owner: consul
start_up_consul-template_service:
service.running:
- name: consul-template
- enable: True
- restart: True
- require:
- file: copy_over_systemd_service_files
- watch:
- /etc/systemd/system/consul-template.service
when I run my state file I get the following error:
ID: start_up_consul-template_service
Function: service.running
Name: consul-template
Result: False
Comment: Service consul-template is already enabled, and is dead
Started: 17:27:38.346659
Duration: 2835.888 ms
Changes:
I'm not sure what this means. All I want to do is restart the service once it's been copied over and I've done this before without issue. Looking back through the stack trace just shows that Salt ran systemctl is-enabled consult-template
I think I was over complicating things. Instead I'm doing this:
consul-template:
service.running:
- require:
- file: copy_over_systemd_service_files
- watch:
- /etc/systemd/system/consul-template.service