Saltstack pillar - using sub-directories - salt-stack

I need to organize my pillars into sub-directories.
/srv/pillar/
├── app1
│   └── env1
│   └── conf.sls
├── data.sls
└── top.sls
I've put in top.sls:
base:
'*':
- data
- app1/env1/conf
When I request data.sls for variable info, it works :
salt '*' pillar.get info
local:
some data
But when I request conf.sls for variable info, nothing works:
salt '*' pillar.get app1.env1.info
shows nothing !
I already executed:
saltutil.refresh_pillar
and restarted salt process.
What should I do to make salt pillars recognize sub directories ?

There are several misconceptions in your example. Organizing your pillar files in subdirectories does not translate into a namespace on the resulting pillar variable. The variables in the subdirectory-nested-pillar file will still be at the root of the pillar dict.
To include subdirectories in your pillar top.sls file, you use dot notation:
Create pillar so/test/example.sls
cd /srv/pillar
mkdir -p so/test
echo 'foo: bar' > so/test/example.sls
Edit top.sls
base:
'*':
- users
lead:
- so.test.example
Refresh pillar on minion 'lead'
$ sudo salt lead saltutil.refresh_pillar
lead:
None
Extract value of foo from pillar
$ sudo salt lead pillar.get foo
lead:
bar
If you want to namespace the variable in the pillar dict, express that in so/test/example.sls:
$ cat /srv/pillar/so/test/examples.sls
so:
test:
foo: bar
$ sudo salt lead pillar.get so --out=json
{
"lead": {
"test": {
"foo": "bar"
}
}
}
$ sudo salt lead pillar.get so:test:foo --out=json
{
"lead": "bar"
}

Related

Pass directory into statle.sls

I am configuring a local Salt setup and I have hit a bit of a wall.
My setup is:
CentOS: Red Hat Enterprise Linux Server release 7.7 (Maipo)
Salt: salt 3000.1
I have a very basic configuration with nothing changed from default in the Master or Minion config.
My directory structure is as follows:
/srv/salt/apache/init.sls
/srv/salt/uptodate/common.sls
If I run the following:
salt '*' state.sls apache Test=true
It correctly applies the sls files inside the apache folder.
If I run:
salt '*' state.sls uptodate Test=true
It returns:
minion:
Data failed to compile:
----------
No matching sls found for 'uptodate' in env 'base'
I have no top.sls files configured and if I move common.sls into the apache directory it also does not get applied.
Does anyone have any idea what is going wrong here?
The init.sls can be compared to an index.html file on a webserver.
If you want to apply a statefile other than a init.sls you need to add the name of the state file.
This should work for you:
salt '*' state.sls uptodate.common test=True

How to apply a top file when using salt-ssh and roster file

I'm new to salt, and I'm trying to use salt-ssh to manage hosts. I have the following roster file
~/salt/roster
pi:
host: raspberypi1.local
tty: True
sudo: True
I have salt states
~/salt/states/docker.sls
I am able to apply the salt states by calling the state explicitly
sudo salt-ssh '*' -c . state.apply docker
How can I make it so that I don't have to call the state directly? I want the raspberypi1.local node to always run the docker state.
Things I've tried
Make ~/salt/top.sls
base:
'pi*':
- docker
However the top.sls appears to be ignored by salt-ssh
I've tried editing ~/salt/Saltfile to point at a specific file_roots
salt-ssh:
roster_file: /Users/foobar/salt/roster
config_dir: /Users/foobar/salt
log_file: /Users/foobar/salt/log.txt
ssh_log_file: /Users/foobar/salt/ssh-log.txt
file_roots:
base:
- /Users/foobar/salt/top.sls
Here file_roots also appears to be ignored.
Whats the proper way to tie states to nodes when using salt-ssh?
I moved ~/salt/top.sls to ~/salt/states/top.sls, and removed file_roots: entirely from the Saltfile (it belongs in the master file). And now I am able to apply states like so:
sudo salt-ssh '*' -c . state.apply

Rsync - Files with same file ending

I have these JSON files in a large directory structure. Some are just "abc.json" and some the added ".finished". I want to rsync only the files without ".finished".
$ find
.
./a
./a/abc.json.finished
./a/abc.json <-- this file
./a/index.html
./a/somefile.css
./b
./b/abc.json.finished
./b/abc.json <-- this file
Sample rsync command that copies all the "abc.json" AND the "abc.json.finished". I just want the "abc.json".
$ rsync --exclude="finished" --include="*c.json" --recursive \
--verbose --dry-run . server:/tmp/rsync
sending incremental file list
created directory /tmp/rsync
./
a/
a/abc.json
a/abc.json.finished
a/index.html
a/somefile.css
b/
b/abc.json
b/abc.json.finished
sent 212 bytes received 72 bytes 113.60 bytes/sec
total size is 0 speedup is 0.00 (DRY RUN)
Update: Added more files to the folders. HTML files, CSS and other files are present in my scenario. Only files ending in "c.json" should be transferred.
Scenario can be recreated with the following commands:
mkdir a
touch a/abc.json.finished
touch a/abc.json
touch a/index.html
touch a/somefile.css
mkdir b
touch b/abc.json.finished
touch b/abc.json
Try the following command. It assumes that you also want to replicate the source directory tree, (for any directories containing files which end with c.json), in the destination location:
$ rsync --include="*c.json" --exclude="*.*" --recursive \
--verbose --dry-run . server:/tmp/rsync
Explanation of command:
--include="*c.json" includes only assets whose name ends with c.json
--exclude="*.*" excludes all other assets (i.e. assets whose name includes a dot .)
--recursive recurse into directories.
--verbose log the results to the console.
--dry-run shows what would have been copied, without actually copying the files. This option/flag should be omitted to actually perform the copy task.
. the path to the source directory.
server:/tmp/rsync the path to the destination directory.
EDIT: Unfortunately, the command provided above also copies files whose filename does not include a dot character. To avoid this consider utlizing both rsync and find as follows:
$ rsync --dry-run --verbose --files-from=<(find ./ -name "*c.json") \
./ server:/tmp/rsync
This utilizes process substitution, i.e. <(list), to pass the output from the find command to the --files-from= option/flag of the rsync command.
source tree
.
├── a
│   ├── abc.json
│   ├── abc.json.finished.json
│   ├── index.html
│   └── somefile.css
└── b
├── abc.json
└── abc.json.finished.json
resultant destination tree
server
└── tmp
└── rsync
├── a
│ └── abc.json
└── b
└── abc.json
A hacky solution is use grep and create a file containing all file names we want to transfer.
find |grep "c.json$" > rsync-files
rsync --files-from=rsync-files --verbose --recursive --compress --dry-run \
./ \
server:/tmp/rsync
rm rsync-files
Content of 'rsync-files':
./a/abc.json
./b/abc.json
Output when running rsync command:
sending incremental file list
created directory /tmp/rsync
./
a/
a/abc.json
b/
b/abc.json

How to create a directory using Ansible

How do you create a directory www at /srv on a Debian-based system using an Ansible playbook?
You want the file module. To create a directory, you need to specify the option state: directory :
- name: Creates directory
file:
path: /src/www
state: directory
You can see other options at https://docs.ansible.com/ansible/latest/collections/ansible/builtin/file_module.html
You can even extend the file module and even set the owner,group & permission through it. (Ref: Ansible file documentation)
- name: Creates directory
file:
path: /src/www
state: directory
owner: www-data
group: www-data
mode: 0775
Even, you can create the directories recursively:
- name: Creates directory
file:
path: /src/www
state: directory
owner: www-data
group: www-data
mode: 0775
recurse: yes
This way, it will create both directories, if they didn't exist.
Additional for all answers here, there is lot of situations when you need to create more then one directory so it is a good idea to use loops instead creating separate task for each directory.
- name: creates multiple directories in one task
file:
path: "{{ item }}"
state: directory
loop:
- /srv/www
- /dir/foo
- /dir/bar
you can create using:
Latest version 2<
- name: Create Folder
file:
path: /srv/www/
owner: user
group: user
mode: 0755
state: directory
Older version
- name: Create Folder
file:
path=/srv/www/
owner=user
group=user
mode=0755
state=directory
Refer - http://docs.ansible.com/ansible/file_module.html
Directory can be created using file module only, as directory is nothing but a file.
# create a directory if it doesn't exist
- file:
path: /etc/some_directory
state: directory
mode: 0755
owner: foo
group: foo
- name: Create a directory
ansible.builtin.file:
path: /etc/some_directory
state: directory
mode: '0755'
- file:
path: /etc/some_directory
state: directory
mode: 0755
owner: someone
group: somegroup
That's the way you can actually also set the permissions, the owner and the group. The last three parameters are not obligatory.
You can create a directory. using
# create a directory if it doesn't exist
- file: path=/src/www state=directory mode=0755
You can also consult
http://docs.ansible.com/ansible/file_module.html
for further details regaridng directory and file system.
Just need to put condition to execute task for specific distribution
- name: Creates directory
file: path=/src/www state=directory
when: ansible_distribution == 'Debian'
You can use the statement
- name: webfolder - Creates web folder
file: path=/srv/www state=directory owner=www-data group=www-data mode=0775`
enter code here
- name: creating directory in ansible
file:
path: /src/www
state: directory
owner: foo
you can refer to ansible documentation
If you want to create a directory in windows:
- name: create folder in Windows
win_file:
path: C:\Temp\folder\subfolder
state: directory
See the win_file module for more information.
to create directory
ansible host_name -m file -a "dest=/home/ansible/vndir state=directory"
We have modules available to create directory , file in ansible
Example
- name: Creates directory
file:
path: /src/www
state: directory
you can use the "file" module in this case, there are so many arguments that you can pass for a newly created directory like the owner, group, location, mode and so on.....
please refer to this document for the detailed explanation on the file module...
https://docs.ansible.com/ansible/latest/modules/file_module.html#file-module
remember this module is not just for creating the directory !!!
To check if directory exists and then run some task (e.g. create directory) use the following
- name: Check if output directory exists
stat:
path: /path/to/output
register: output_folder
- name: Create output directory if not exists
file:
path: /path/to/output
state: directory
owner: user
group: user
mode: 0775
when: output_folder.stat.exists == false
You can do it as one of the following ways:
Example 1: If Parent Directory already exists:
- name: Create a new directory www at given path
ansible.builtin.file:
path: /srv/www/
state: directory
mode: '0755'
Example 2: If Parent Directory does not exist:
- name: Create a new directory www at given path recursively
ansible.builtin.file:
path: /srv/www/
state: directory
mode: '0755'
recurse: yes
Here in Example 2, it will recursively create both directories if they are not present.
You can see the Official Documentation for further info on file_module
You can directly run the command and create directly using ansible
ansible -v targethostname -m shell -a "mkdir /srv/www" -u targetuser
OR
ansible -v targethostname -m file -a "path=/srv/www state=directory" -u targetuser
---
- hosts: all
connection: local
tasks:
- name: Creates directory
file: path=/src/www state=directory
Above playbook will create www directory in /src path.
Before running above playbook. Please make sure your ansible host connection should be set,
"localhost ansible_connection=local"
should be present in /etc/ansible/hosts
for more information please let me know.
Use file module to create a directory and get the details about file module using command "ansible-doc file"
Here is an option "state" that explains:
If directory, all immediate subdirectories will be created if they do not exist, since 1.7 they will be created with the supplied permissions.
If file, the file will NOT be created if it does not exist, see the [copy] or [template] module if you want that behavior.
If link, the symbolic link will be created or changed. Use hard for hardlinks.
If absent, directories will be recursively deleted, and files or symlinks will be unlinked.
Note that file will not fail if the path does not exist as the state did not change.
If touch (new in 1.4), an empty file will be created if the path does not
exist, while an existing file or directory will receive updated file
access and modification times (similar to the way touch works from
the command line).
Easiest way to make a directory in Ansible.
name: Create your_directory if it doesn't exist.
file:
path: /etc/your_directory
OR
You want to give sudo privileges to that directory.
name: Create your_directory if it doesn't exist.
file:
path: /etc/your_directory
mode: '777'
Hello good afternoon team.
I share the following with you.
- name: Validar Directorio
stat:
path: /tmp/Sabana
register: sabana_directorio
- debug:
msg: "Existe"
when: sabana_directorio.stat.isdir == sabana_directorio.stat.isdir
- name: Crear el directorio si no existe.
file:
path: /tmp/Sabana
state: directory
when: sabana_directorio.stat.exists == false
With which you can validate if the directory exists before creating it
I see lots of Playbooks examples and I would like to mention the Adhoc commands example.
$ansible -i inventory -m file -a "path=/tmp/direcory state=directory ( instead of directory we can mention touch to create files)
You need to use file module for this case. Below playbook you can use for your reference.
---
- hosts: <Your target host group>
name: play1
tasks:
- name: Create Directory
files:
path=/srv/www/
owner=<Intended User>
mode=<Intended permission, e.g.: 0750>
state=directory
here is easier way.
- name: create dir
command: mkdir -p dir dir/a dir/b

Saltstack for "configure make install"

I'm getting my feet wet with SaltStack. I've made my first state (a Vim installer with a static configuration) and I'm working on my second one.
Unfortunately, there isn't an Ubuntu package for the application I'd like my state to install. I will have to build the application myself. Is there a "best practice" for doing "configure-make-install" type installations with Salt? Or should I just use cmd?
In particular, if I was doing it by hand, I would do something along the lines of:
wget -c http://example.com/foo-3.4.3.tar.gz
tar xzf foo-3.4.3.tar.gz
cd foo-3.4.3
./configure --prefix=$PREFIX && make && make install
There are state modules to abstract the first two lines, if you wish.
file.managed: http://docs.saltstack.com/ref/states/all/salt.states.file.html
archive.extracted: http://docs.saltstack.com/ref/states/all/salt.states.archive.html
But you could also just run the commands on the target minion(s).
install-foo:
cmd.run:
- name: |
cd /tmp
wget -c http://example.com/foo-3.4.3.tar.gz
tar xzf foo-3.4.3.tar.gz
cd foo-3.4.3
./configure --prefix=/usr/local
make
make install
- cwd: /tmp
- shell: /bin/bash
- timeout: 300
- unless: test -x /usr/local/bin/foo
Just make sure to include an unless argument to make the script idempotent.
Alternatively, distribute a bash script to the minion and execute. See:
How can I execute multiple commands using Salt Stack?
As for best practice? I would recommend using fpm to create a .deb or .rpm package and install that. At the very least, copy that tarball to the salt master and don't rely on external resources to be there three years from now.
Let's assume foo-3.4.3.tar.gz is checked into GitHub. Here is an approach that you might pursue in your state file:
git:
pkg.installed
https://github.com/nomen/foo.git:
git.latest:
- rev: master
- target: /tmp/foo
- user: nomen
- require:
- pkg: git
foo_deployed:
cmd.run:
- cwd: /tmp/foo
- user: nomen
- name: |
./configure --prefix=/usr/local
make
make install
- require:
- git: https://github.com/nomen/foo.git
Your configuration prefix location could be passed as a salt pillar. If the build process is more complicated, you may consider writing a custom state.

Resources