I should exclude some sls from run
I use reactor to apply highstate when minions starts up, like this:
/etc/salt/master.d/reactor/start.sls:
reactor: # Master config section "reactor"
- 'salt/minion/*/start': # Match tag "salt/minion/*/start"
- /srv/salt/reactor/start.sls # Things to do when a minion starts
/srv/salt/reactor/start.sls:
highstate_run:
local.state.apply:
- tgt: {{ data['id'] }}
It works. But how to exclude some hosts e.g by name ?
I tried to use compounds in sub sls files. But unfortunately excluding in sls files does not work.
I would use [jinja][1] for this.
For a simple example, if you want to exclude minion names that start with 'region1', you can do
{% if not data['id'].startswith('region1') %}
highstate_run:
local.state.apply:
- tgt: {{ data['id'] }}
{% endif %}```
[1]: https://docs.saltstack.com/en/latest/topics/jinja/index.html
Related
Goal
How can I install files with file.recurse on the minion which are specific to minions?
Current Strategy
This would works:
files_per_minion:
file.recurse:
- source: salt://monitoring/files/per_minion/{{ grains.id }}
- name: /
- template: jinja
... but it fails for minions which don't have this directory on the master. I don't want to create a directory for every minion on my master.
I search for a way for an optional include. Here is a condition in pseudo code:
{% if magic_way_to_check_if_source_exists salt://monitoring/files/per_minion/{{ grains.id }} %}
files_per_minion:
file.recurse:
- source: salt://monitoring/files/per_minion/{{ grains.id }}
- name: /
- template: jinja
{% endif %}
Question
How to write the condition magic_way_to_check_if_source_exists ?
Other solutions welcome
The condition magic_way_to_check_if_source_exists is just one strategy to get to the goal. Other solutions are welcome.
Use Case
Imagine I want cron_tab_file_for_xhost to get installed, but only on the host called xhost. I could solve this by creating a directory tree and a file for this host like this:
monitoring/files/per_minion/xhost/etc/cron.d/cron_tab_file_for_xhost
There are different ways depending on your use cases and state tree.
The easiest one is to create a separate state and attach this using top.sls.
If you work on formulas, which are configured through pillar. I would write this information somewhere in my pillar. The states then decide based on the pillar data.
pillar.example:
yourformula:
getspecificfile: true
somestate.sls:
{% if salt['pillar.get']("yourformula:getspecificfile") %}
files_per_minion:
file.recurse:
- source: salt://monitoring/files/per_minion/{{ grains.id }}
- name: /
- template: jinja
{% endif %}
UPDATE:
i just had a look in the docs of the file.managed state
The source parameter can be specified as a list. If this is done, then the first file to be matched will be the one that is used. This allows you to have a default file on which to fall back if the desired file does not exist on the salt fileserver. Here's an example:
/etc/foo.conf:
file.managed:
- source:
- salt://foo.conf.{{ grains['fqdn'] }}
- salt://foo.conf.fallback
- user: foo
- group: users
- mode: 644
- backup: minion
This seems to be another option - if you don't care to roll out empty files which are not necessary on your minions.
I created a complex state for API service, it involves git checkouts, python venv, uwsgi, nginx, etc etc. It works fine.
Now I would like to turn it into a template and execute it several times per minion, with variables supplied from pillar - i.e something like.
{% for apiserver in pillar.apiservers %}
include apiserver_template.sls, locals: apiserver.config
{% endfor %}
where apiserver_template will work with context supplied to it, with apiserver.config having all config data for each API instance. I know syntax is wrong but hopefully I am communicating the idea - ideally, something like executing ruby partials with supplying local variables.
How is it done properly in saltland?
It sounds to me like Jinja Macro is something you want to use for this. You can find more information about usage here: https://docs.saltstack.com/en/2015.8/topics/development/conventions/formulas.html#jinja-macros
In short what you will have in your case may look like:
{% macro api_server(git_repo, python_venv_path, python_venv_requirements) %}
{{python_venv_path}}:
virtualenv.managed:
- system_site_packages: False
- requirements: salt://{{python_venv_requirements}}
{{git_repo}}:
git.latest:
- name: {{git_repo}}
{% endmacro %}
Assuming you have a pillar apiservers where each api server has git_repo, python_venv_path and python_venv_requirements values, you can use the macro like this:
{% for server in salt.pillar.get('apiservers', []) %}
{{ api_server(server['git_repo'], server['python_venv_path'], server['python_venv_requirements']) }}
{% endfor %}
If you want - you can also put a macro in a separate state file and then import a marco as a regular salt resource.
Please also not that instead of pillar.apiservers I used salt.pillar.get('apiservers', []). This is a safer way to get data from pillar. If for some reason a pillar is unavailable - the later code will result in empty dict instead of failure in first case.
I'm having trouble using the salt mine in a pillar to dynamically create a list of hosts based on a grain value match. I don't get any error, I get no output for all hosts. Actually, I can't get any output for mine from the pillar at all even when using the example from the salt docs. I know it isn't an issue with my top file, because I can access all of the other pillar values. My test minion's mine.interval is set to 5. I've refreshed pillar data, and ran mine.update.
Here's an example of my pillar:
mine_functions:
network.ip_addrs: []
grains.item:
- host
- role
My template file that access the mine functions:
#I know this is writing the same list for each match, I'm just doing this for testing and I'll concat the results into a string when I know it works:
{% for host in salt['mine.get']('roles:web', 'grains.items:host', expr_form='grain') | dictsort() %}
serverList= {{ host }}
{% endfor %}
Output from CLI:
salt "server.domain.com" mine.get "*" "*"
server.domain.com:
----------
How do I get this to work? I get no errors, no output, it just runs smoothly, but nothing is written in the file and I get nothing from the command line. My goal here is to be able to dynamically build a list of servers that match a specific grain to set a configuration value in a config template. Am I down the wrong path here, is there a better way?
I'd recommend using mine.get directly in your sls file to get that list of hosts. I don't think there's any need to pass that through pillar data.
#Utah_Dave, thanks so much for the help both here and in IRC.
Posting this as an answer so anyone else searching for this gets a good example...
pillar:
mine_functions:
grains.items: []
template file:
{% set ft_hosts = [] %}
{% for grain_vals in salt['mine.get']('role:ps:ft:True', 'grains.items', expr_form='grain').items() %}
{% do ft_hosts.append(grain_vals[1]['host']) %}
{% endfor %}
ft.ps.server.hosts={{ ft_hosts|join('|') }}
I'm trying to have salt automatically deploy files to users' home directory (after creating them). The directory structure (under file_roots) is as follows:
users/
init.sls
user_list.jinja
files/
userX/
some_dir_I_want_deployed_to_userX_homedir/
some_script_I_want_deployed.sh
etc
user_list.jinja has a list of users. The idea is that if there's a directory for userX under files/, that directory subtree should be deployed to userX's home. However, I'd rather not have to create empty directories for userY etc if they don't have anything to deploy, so I'm trying to test existence of the directory to avoid an error.
Here's the relevant excerpt from users/init.sls:
{% from "users/user_list.jinja" import users with context %}
{% for name, user in users.items() %}
{% set files_path = '{0}/files/{1}'.format(salt['file.dirname'](tplpath), name) %}
{% if salt['file.directory_exists'](files_path) %}
{{ user.home }}:
file.recurse:
- source: salt://users/files/{{ name }}
- user: {{ name }}
- group: {{ name }}
{% endif %}
{% endfor %}
With a fair bit of debugging (which was rather necessary to figure out the above), I've worked out that this is a chicken-and-egg situation, namely:
The file.directory_exists test is run on the minion (that's fair)
The salt file-server seems to have an optimization whereby it only deploys to minion (to their local cache under /var/cache/salt/minion/file) items which are referenced in states (more likely, the minions only request stuff which they see referenced).
So unless the directory subtree for users/files/userX already exists on the minion, file.directory_exists returns False, which means the entire portion which starts with {{ user.home }} is suppressed during Jinja rendering; hence it's not referenced and the copy (to the minion's local cache) never occurs.
If I manually create an empty directory structure for users/files/userX on the minion, everything starts to work. This tells me my theory is at least partially correct.
I can "feel" I'm doing smth wrong here (the whole thing feels too procedural). What's the best approach to achieve this? The requirement itself doesn't seem too far-fetched.
The more salt-ish way to do this is to have some data in pillar data and check for the existence of that key. Something like user.enabled. But, that would require you to keep settings in 2 places, in pillar and in the file_roots.
You don't want to check for the existence of the directory on the minion server, you want to check for the existence of the file in your file roots.
Unfortunately, I don't think it's possible to check for the existence of a file under salt:// scheme. If I'm wrong, then all you have to do is replace your check for directory existence with the syntax to check for file_root file existence.
The more salt-ish approach is to define which users are enabled/disabled on each machine in pillar data, and use the user module to add them to the system.
https://github.com/saltstack-formulas/users-formula
You can add to the standard pillar that's given with the standard users formula and put a key that says to sync files
#pillar
users:
ausername:
fullname: A User
syncfiles: True
busername:
fullname: B User
syncfiles: False
#state
{% for name, user in pillar.get('users', {}).items() if user.absent is not defined or not user.absent %}
{% if user.syncfiles %}
/home/{{ user.username }}:
file.recurse:
- source: salt://users/files/{{ user.username }}
- user: {{ user.username }}
{% if user.prime_group %}
- group: {{ user.prime_group.name }}
{% endif %}
{% endif %}
{% endfor %}
Actually, the standard users-formula already handles pre-populating with files. I would just use that formula. I know it's keeping track of data in 2 places, but you get the bonus of leveraging an already built state file.
I am doing something like following in TOP file:
'roles:*database*':
- match: grain
- {{ salt['pillar.get']("server:database:states") }}
And in pillar the states are defined based on role:
server:
database:
states:
- module1.applySecPatch
- module2.firewallRules
I don't get an error and there is not helpful logging too at TRACE level. Is this the right way to populate the state file list dynamically?
This answer is actually answered by Seth House on Salt-users group, just posting here for benefit of others
The syntax is not quite right.
'roles:*database*':
- match: grain
- {{ salt['pillar.get']("server:database:states") }}
Will produce something incorrect like:
'roles:*database*':
- match: grain
- ['module1.applySecPatch', 'module2.firewallRules']
You need a loop. Something more like this:
'roles:*database*':
- match: grain
{% for file in salt['pillar.get']("server:database:states") %}
- {{ file }}
{% endfor %}