After a long search I did not find the answer
There is a playbook Ansible.
- name: myscript
hosts: myhost
tasks:
- name: myscript
docker_container:
name: myscript
image: myimage
detach: false
working_dir: "/opt/R/project"
command: Rscript $(find ./*_Modules -iname *_Script.R)
This command works: Rscript ./01_Modules/02_Script.R
This command NOT works: Rscript $(find ./*_Modules -iname *_Script.R) - Treats $(find not as a command, but as a path.
At the same time, in linux, this line is successfully run and finds the script.
How do I pass full-fledged linux commands with && and similar features to command?
Here is a simplified version of your problem
- name: Create a test container
docker_container:
name: test
image: busybox
command: ls |grep var && echo 'it doesn\'t work!'
Output :
ls: |grep: No such file or directory
ls: &&: No such file or directory
ls: echo: No such file or directory
ls: it fails: No such file or directory
var:
spool
www
If I wrap it in quote and use
/bin/sh -c
- name: Create a test container
docker_container:
name: test
image: busybox
command: /bin/sh -c "ls |grep var && echo 'it works!'"
Output :
var
it works!
Related
I'm trying to encrypt a file with sops with github actions, my workflow code is
name: Encrypt application secrets
on:
workflow_dispatch:
jobs:
encrypt:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v2
with:
fetch-depth: 1
- name: sops install
run: |
curl -O -L -C - https://github.com/mozilla/sops/releases/download/v3.7.1/sops-v3.7.1.darwin
sudo mv sops-v3.7.1.darwin /usr/bin/sops
sudo chmod +x /usr/bin/sops
- name: upload keystore
run: gpg --import .github/.gpg
- name: encrypt file
run: |
sudo chmod +x /usr/bin/sops
sudo sops --encrypt --in-place .github/application.secrets.yaml
But I get this error
Run sudo chmod +x /usr/bin/sops
sudo chmod +x /usr/bin/sops
sudo sops --encrypt --in-place .github/application.secrets.yaml
shell: /usr/bin/bash -e {0}
/usr/bin/sops: 1: ����
�: not found
/usr/bin/sops: 8: Syntax error: word unexpected (expecting ")")
Is there someone who can help please ?
Following worked for my github pipline (though for decryption purposes):
# main.yaml
...
jobs:
build-publish-deploy:
name: Build, Publish and Deploy
runs-on: ubuntu-latest
steps:
...
- name: Decrypt secret
run: |-
curl -O -L -C - https://github.com/mozilla/sops/releases/download/v3.7.3/sops-v3.7.3.linux
sudo mv sops-v3.7.3.linux /usr/bin/sops
sudo chmod +x /usr/bin/sops
export SOPS_AGE_KEY=${{ secrets.GKE_DWK_SOPS_AGE_KEY }}
sops --decrypt manifests/secret.enc.yaml > manifests/secret.yaml
...
Darwin files are usually for MacOS and you are requesting to run on ubuntu-latest.
Apologies in advance as I'm not very confident writing GitLab pipelines. I have a pair of public and private keys encrypted, committed to the GitLab repo. I have introduced a new stage into my pipeline in order to decrypt the keys and deploy.
decryption:
stage: decryption
allow_failure: false
before_script:
- mkdir -p ~/.ssh
- eval $(ssh-agent -s)
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- chmod 660 ./keys/vault_password.txt
- echo $ANSIBLE_VAULT_PASSWORD > ./keys/vault_password.txt
- chmod 660 ./keys/private.key
- chmod 660 ./keys/public.key
- ansible-vault decrypt --vault-password-file ./keys/vault_password.txt ./keys/private.key
- ansible-vault decrypt --vault-password-file ./keys/vault_password.txt ./keys/public.key
- echo "$(cat ./keys/private.key)"
- echo "$(cat ./keys/public.key)"
artifacts:
untracked: true
My next stage is build.
build:
stage: build
allow_failure: false
dependencies:
- decryption
script:
- rm -rf vendor/drupal/coder
- composer install
- ./vendor/bin/robo ci:build
- ls -la vendor/drupal/coder
- echo "$(cat ./keys/private.key)"
- echo "$(cat ./keys/public.key)"
artifacts:
name: "mycompany_build_{$CI_COMMIT_SHA}"
expire_in: '1 week'
paths:
- ./build
When I try to echo the keys in the decryption stage I can see the decrypted keys. But, when I try to access the keys like this in the build stage like below, it shows me the encrypted files. I'm just trying to see if I can access the decrypted files at the build stage and then I can pass these keys to be deployed. So clearly something is not correct with the pipeline.
- echo "$(cat ./keys/private.key)"
- echo "$(cat ./keys/public.key)"
Maybe the way I have written my pipeline needs to be changed in order to pass the changed untracked public.key and private.key into the build stage and possibly to the deploy stage as well.
Could someone please point me in the correct direction on this?. Do I have to change something in the artifacts ?. How can I do that?. Thanks in advance.
I don't know too much about GitLab-ci but I think you are not referencing properly the decrypted file, on the decrypting step you should save the decrypted value to a variable and then call it on the build step, the way you are doing now is referencing the file itself in the build step the file is not decrypted, you decrypt on the decrypt step and save the decrypted value to use late.
I'm not sure if this will work, but maybe you can get the idea:
Decrypt:
decryption:
stage: decryption
allow_failure: false
before_script:
- mkdir -p ~/.ssh
- eval $(ssh-agent -s)
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- chmod 660 ./keys/vault_password.txt
- echo $ANSIBLE_VAULT_PASSWORD > ./keys/vault_password.txt
- chmod 660 ./keys/private.key
- chmod 660 ./keys/public.key
- ansible-vault decrypt --vault-password-file ./keys/vault_password.txt ./keys/private.key
- ansible-vault decrypt --vault-password-file ./keys/vault_password.txt ./keys/public.key
- echo "private_key_value=$(cat ./keys/private.key)"
- echo "public_key_value=$(cat ./keys/public.key)"
artifacts:
untracked: true
And then the build step:
```yml
uild:
stage: build
allow_failure: false
dependencies:
- decryption
script:
- rm -rf vendor/drupal/coder
- composer install
- ./vendor/bin/robo ci:build
- ls -la vendor/drupal/coder
- echo $private_key_value
- echo $public_key_value
artifacts:
name: "mycompany_build_{$CI_COMMIT_SHA}"
expire_in: '1 week'
paths:
- ./build
All Events Succeeded but flask app not starting
appspec.yml
version: 0.0
os: linux
files:
- source: /testServerRegadv.py
destination: /path to folder in server/python
hooks:
ApplicationStart:
- location: scripts/start_server.sh
timeout: 300
runas: root
ApplicationStop:
- location: scripts/stop_server.sh
timeout: 300
runas: root
start_server.sh
#!/bin/bash
echo "In start server" >>results.txt 2> errors.log &
python /path in server/python/testServerRegadv.py > results.txt 2> errors.log &
stop_server.sh
isExistApp = `lsof -t -i:1515`
if [[ -n $isExistApp ]]; then
kill -9 $(lsof -t -i:1515)
fi
Also I am using code commit for storing the code and before pushing to aws code-commit I am executing chmod +x scripts/ * to make the scripts executable.
For the https://github.com/ellakcy/piwik-with-wordpress I am making a restore bash script in order to restore the backup generated from the https://github.com/ellakcy/piwik-with-wordpress/blob/master/scripts/pre-backup script
The main idea is to set a path with a tarball containing the backup and recreating the folders that volumes are mounted.
The script is the following:
#!/bin/bash
# Printing functions
black='\E[30;40m'
red='\E[31;40m'
green='\E[32;40m'
yellow='\E[33;40m'
blue='\E[34;40m'
magenta='\E[35;40m'
cyan='\E[36;40m'
white='\E[37;40m'
#Echo a string with color
cecho () # Color-echo.
# Argument $1 = message
# Argument $2 = color
{
local default_msg="No message passed."
# Doesn't really need to be a local variable.
message=${1:-$default_msg} # Defaults to default message.
color=${2:-$black} # Defaults to black, if not specified.
echo -e "$color"
echo "$message"
tput sgr0 # Reset to normal.
return
}
#Echo a string as error with color
cecho_err () # Color-echo.
# Argument $1 = message
# Argument $2 = color
{
local default_msg="No message passed."
# Doesn't really need to be a local variable.
message=${1:-$default_msg} # Defaults to default message.
color=${2:-$red} # Defaults to black, if not specified.
echo >&2 -e "$color"
echo >&2 "$message"
tput sgr0 # Reset to normal.
return
}
backup_file=${1}
cecho "Creating the correct folders" $cyan
cecho "Deleting data folder in order to recreate it" $red
sudo rm -rf ./data
mkdir ./data/
sudo chown root:root ./data/
sudo chmod 755 ./data/
if [ ! -f restore ]; then
mkdir ./restore/
fi
tar -xf ${backup_file} -C ./restore/
cecho "Restoring backup data for wordpress" $cyan
sudo mkdir ./data/wordpress
sudo chown root:root ./data/wordpress
sudo chmod 755 ./data/wordpress
sudo mv ./restore/wordpress/data/www ./data/wordpress/
sudo chown www-data:www-data ./data/wordpress/www
cecho "Restoring environment" $cyan
wordpress_env=$(tr '\n' ' ' <./restore/wordpress/env.txt)
echo ${wordpress_env}
cecho "Restoring database" $cyan
sudo mkdir ./data/wordpress/db
echo "sudo env ${wordpress_env} docker run --volume \"./data/wordpress/db\":/var/lib/mysql --volume ./restore/wordpress/db:/docker-entrypoint-initdb.d -e MYSQL_ROOT_PASSWORD=\$WORDPRESS_MYSQL_ROOT_PASSWORD -e MYSQL_DATABASE=\"wordpress\" -e MYSQL_USER=\$WORDPRESS_MYSQL_USER -e MYSQL_PASSWORD=\$WORDPRESS_MYSQL_PASSWORD mariadb" > ./restore_db.sh
chmod +x ./restore_db.sh
./restore_db.sh
# rm -rf ./restore_db.sh
rm -rf ./restore
And I get this error when I try to restore the database:
docker: Error response from daemon: create ./data/wordpress/db: "./data/wordpress/db" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed.
See 'docker run --help'.
As you can see it generates a temporary scripts (that later will be deleted) one example of generated script is:
sudo env WORDPRESS_MYSQL_ROOT_PASSWORD=passwd WORDPRESS_MYSQL_USER=wordpress WORDPRESS_MYSQL_PASSWORD=wordpress WORDPRESS_ADMIN_USER=admin WORDPRESS_ADMIN_PASSWORD=admin WORDPRESS_URL=http://0.0.0.0:8080 docker run --volume "./data/wordpress/db":/var/lib/mysql --volume ./restore/wordpress/db:/docker-entrypoint-initdb.d -e MYSQL_ROOT_PASSWORD=$WORDPRESS_MYSQL_ROOT_PASSWORD -e MYSQL_DATABASE="wordpress" -e MYSQL_USER=$WORDPRESS_MYSQL_USER -e MYSQL_PASSWORD=$WORDPRESS_MYSQL_PASSWORD mariadb
What is the best option in order to generate the correct volume data in ./data/wordpress/db that mounts on a container's /var/lib/mysql?
When we specify --volume <host_dir>:<container_dir>, host_dir must be an absolute path. If it is not an absolute path, then it considered to be the volume's name. Hence the message invalid characters for a local volume name. Try providing absolute path for the host directory.
I tried to add:
mypack:
pkg:
- installed
- pkgs:
- mercurial
- git
cmd.run:
- name: 'mkdir -p /opt/mypack'
cmd.run: 'hg pull -u -R /opt/mypack || hg clone -R /opt https://...'
cmd.run: 'ln -s /opt/mypack/etc/init.d/xxx /etc/init.d/xxx'
But for some reason this the state seems to execute/install but the commands are not executed, or at least not all of them.
I need a solution to run multiple commands and to fail the deployment if any of these fails.
I know that I could write a bash script and include this bash script, but I was looking for a solution that would work with only the YAML file.
You want this:
cmd-test:
cmd.run:
- name: |
mkdir /tmp/foo
chown dan /tmp/foo
chgrp www-data /tmp/foo
chmod 2751 /tmp/foo
touch /tmp/foo/bar
Or this, which I would prefer, where the script is downloaded from the master:
cmd-test:
cmd.script:
- source: salt://foo/bar.sh
- cwd: /where/to/run
- user: fred
In addition to the above (better) suggestions, you can do this:
cmd-test:
cmd.run:
- names:
- mkdir -p /opt/mypack
- hg pull -u -R /opt/mypack || hg clone -R /opt https://...
- ln -s /opt/mypack/etc/init.d/xxx /etc/init.d/xxx
For reasons I don't understand yet (I'm a Salt novice), the names are iterated in reverse order, so the commands are executed backwards.
You can do as Dan pointed out, using the pipe or a cmd.script state. But it should be noted that you have some syntax problems in your original post. Each new state needs a name arg, you can't just put the command after the colon:
mypack:
pkg:
- installed
- pkgs:
- mercurial
- git
cmd.run:
- name: 'my first command'
cmd.run:
- name: 'my second command'
However, that actually may fail as well, because I don't think you can put multiple of the same state underneath a single ID. So you may have to split them out like this:
first:
cmd.run:
- name: 'my first command'
second:
cmd.run:
- name: 'my second command'
As one of the users pointed out above, this works in proper order (salt 3000.2)
install_borg:
cmd.run:
- names:
- cd /tmp
- wget https://github.com/borgbackup/borg/releases/download/1.1.15/borg-linux64
- mv borg-linux64 /usr/local/bin/borg
- chmod u+x /usr/local/bin/borg
- chown root:root /usr/local/bin/borg
- ln -s /usr/local/bin/borg /usr/bin/borg
- unless: test -f /usr/bin/borg