Hello helpful friends,
We have quite a setup here of 100+ servers being managed by Salt states. With different roles in the organization executed by different people, I'd really like to have a possibility to "aggregate" some states. In this case: updating (yum) packages.
I would really like to have our sysadmins safely being able to execute a command like this on the master:
salt '*' state.apply update.packages
while maybe our developers would be able to execute:
salt 'dev-*' state.apply update.application
Of course we have a large set of sls files and the key to this issue is that I don't want all those states executed, but just a selected bunch of them.
To achieve this, I've tried to create an update/packages.sls state, containing:
update-packages:
test.nop
And then added to, for example the following existing state:
nagios-plugins-all:
pkg.latest:
- require:
- pkg: corepackages
a watch_in as follows:
nagios-plugins-all:
pkg.latest:
- require:
- pkg: corepackages
- watch_in:
- test: update-packages
Unfortunately, this is clearly not the way to go, as executing salt 'testserver001' state.apply update.packages now only returns:
testserver001:
----------
test_|-update-packages_|-update-packages_|-nop:
----------
__id__:
update-packages
__run_num__:
0
changes:
----------
comment:
Success!
duration:
0.946
name:
update-packages
result:
True
start_time:
12:10:46.035686
while I know for sure that updated packages are available. I can't include all the existing state files into the update/packages.sls file, as that would cause all states to be executed in those files and that's not what I want either. It would also become a very messy file.
I also don't want to just execute salt '*' pkg.upgrade as I have states depending on updates; i.e. if the package nagios is updated, the states concerning the up-to-date config files should be run and consequently a restart of the nagios service should be executed. All of that is configured in salt using watch and require arguments, so I'd like to use that also when updating my packages. Also, I want to be in control of which packages can be updated.
I don't know if I'm on the right path, or whether this is possible with Salt at all, but maybe someone here has a brilliant idea on how to achieve this behavior. I would be very thankful!
You might want to look at External Auth System of salt.
This way you can limit users and group to specific minions and commands, and even restrict the parameters.
Related
I am trying to craft a salt state file to simply ensure-enabled and re-run my one-shot service. I thought it would be nice to re-run if any of the dependent files changed, but honestly this is simple enough and the short-lived service is almost never going to be running when I want to update.
Current attempt:
myown-systemd-service-unit-file:
...
myown-systemd-service-executable-file:
...
myown-service:
systemd.force_reload:
- name: myown
- enable: True
- watch:
- myown-systemd-service-unit-file
- myown-systemd-service-executable-file
is failing at with errror:
----------
ID: myown-service
Function: systemd.force_reload
Name: myown
Result: False
Comment: State 'systemd.force_reload' was not found in SLS 'something.myown'
Reason: 'systemd.force_reload' is not available.
Changes:
By enable, I mean to have the equivalent of this CLI call be applied:
sudo systemctl enable myown.service
Relevant docs: https://docs.saltproject.io/en/latest/ref/modules/all/salt.modules.systemd_service.html#module-salt.modules.systemd_service
The systemd_service module is an execution module, and the syntax to use such modules is slightly different. The state declaration you are using is for state modules. Also, the example from the documentation points to use of service.force_reload rather than systemd.force_reload.
salt '*' service.force_reload <service name>
Considering all this, the below example restarts and enables myown service when the service unit file changes.
myown-service:
module.run:
- service.restart:
- name: myown
onchanges:
- file: myown-systemd-service-unit-file
- service.enable:
- name: myown
Note that I've used restart instead of force_reload to bounce the service. Also I'm using onchanges for file module as you haven't shown how you manage the two files. You can use the appropriate module and state IDs.
I'm just started with saltstack, so could anyone help with this question: How to fail the whole state.sls if one statement inside fails? Are requisites
require/watch
suitable for this?
You can use requisites and all depending states will fail as well as the required one.
An alternative is to abort the whole execution on a single state failure:
abort_on_failure_state_example:
test.fail_without_changes:
- failhard: True
No further state will be executed, even those from other included sls files.
I use that to ensure that some required grains are set before applying states, without having to check/require them on every state.
This is documented in https://docs.saltstack.com/en/latest/ref/states/failhard.html
Well. You can specify that a certain state will only be applied if another state is successfully applied as well. Like this:
vim:
pkg.installed: []
/etc/vimrc:
file.managed:
- source: salt://edit/vimrc
- require:
- pkg: vim
The vimrc file will only be managed if the package vim is successfully is installed. It doesn't mater if vim was already installed or that it is a new installed package after this state. If the pkg: vim is successfully installed it will run the second state.
This is useful if you have a particularly state that might fail. You can add the require to all other states to make sure they will only be applied if this particularly state is successful.
To answer your question:
It's not possible to make all states inside the state.sls fail if one of them fails. You can work around this issue by running your state.sls with salt '*' state.apply state test=True to see what would happen. If one of the states would fail you can decide to not actually apply the state.
I hope this answers your question. If it's still unclear you might want to come up with an example :-)
I think I am missing something really fundamental here but I can't seem to it figure out.
I am deploying a mesosphere environment using Salt, and what I want to do is run state files depending on the minion's role.
I have seen an example here where they're targeting using the top.sls file, but there are very few examples I can find doing the same thing.
So if my file-structure is thus:
mesos
|_ init.sls
|_ mesos-master.sls
|_ mesos-slave
and I only want to run the mesos-slave.sls on a minion with the slave role, what is the best way to do this.
In my infinite wisdom I thought doing the following would work (see fundamental misunderstanding opening paragraph)
init.sls
add_mesosphere_apt_repo:
pkgrepo.managed:
- name: deb http://repos.mesosphere.io/ubuntu {{ UBUNTU_VER }} main
- dist: {{ UBUNTU_VER }}
- file: /etc/apt/sources.list.d/mesosphere.list
- keyid: E56151BF
- keyserver: keyserver.ubuntu.com
{% if salt[grains.get]('role') == 'master' %}
include:
- .mesos-master
{% endif %}
but all I get here are errors of duplicate IDs.
I'm sure the answer is very simple, I just can't seem to find anything conclusive using Google.
Matching using grains
You can use grain data when targeting minions:
salt -G 'role:mesos-slave' test.ping
Matching using grains in the topfile
Matching using grains in the top.sls can be very efficient:
'role:mesos-slave':
- match: grain
- mesos.mesos-slave
Manually syncing grains
Grains are automatically synced when state.highstate is called. It's however possible to sync and reload them manually:
salt '*' saltutil.sync_grains
salt '*' saltutil.sync_all
Is targeting using grains secure?
Grains can be set by users that have access to the minion configuration files on the local system, therefore grains are considers less secure than other identifiers in Salt!
Note: it's best practice to not use grains for matching in your pillar top file for any sensitive pillars!
Duplicate ID's
... but all I get here are errors of duplicate IDs.
Salt currently checks for duplicate IDs before execution. The ID must be unique across the entire state tree. All subsequent ID declarations with the same name will be ignored.
A simple solution for this problem might to ensure each ID is unique. You could for example include the SLS file name in the ID declaration:
For the mesos.mesos_master you could use:
mesos_master:
file.managed:
- name: ...
- ...
For the mesos.mesos_slave you could use:
mesos_slave:
file.managed:
- name: ...
- ...
This ways you won't receive the 'duplicate ID' errors when including and excluding other SLS files.
I have decided to go down the targeting via top.sls like so:
'roles:ms':
- match:grain
- mesos.mesos-slave
I am writing a build process for a WordPress installation using Ansible. It doesn't have a application-level build system at the moment, and I've chosen Ansible so that it can cleanly integrate with server build scripts, so I can bring up a working server at the touch of a button.
Most of my WordPress plugins are being installed with the unarchive feature, pointing to versioned plugin builds on the official wordpress.org installation server. I've encountered a problem with just one of these, which is that it is always being marked as "changed" even though the files are exactly the same.
Having examined the state of ls -Rl before and after, I noticed that this plugin (WordPress HTTPS) is the only one to use internal sub-directories, and upon each decompression, the modification time of folders is getting bumped.
It may be useful to know that this is a project build script, with a connection of local. I guess therefore that means that SSH is not being used.
Here is a snippet of my playbook:
- name: Install the W3 Total Cache plugin
unarchive: >
src=https://downloads.wordpress.org/plugin/w3-total-cache.0.9.4.1.zip
dest=wp-content/plugins
copy=no
- name: Install the WP DB Manager plugin
unarchive: >
src=https://downloads.wordpress.org/plugin/wp-dbmanager.2.78.1.zip
dest=wp-content/plugins
copy=no
# #todo Since this has internal sub-folders, need to work out
# how to preserve timestamps of the original folders rather than
# re-writing them, which forces Ansible to record a change of
# server state.
- name: Install the WordPress HTTPS plugin
unarchive: >
src=https://downloads.wordpress.org/plugin/wordpress-https.3.3.6.zip
dest=wp-content/plugins
copy=no
One hacky way of fixing this is to use ls -R before and after, using options to include file sizes but not timestamps, and then md5sum that output. I could then mark it as changed if there is a change in checksum. It'd work but it's not very elegant (and I'd want to do that for all plugins, for consistency).
Another approach is to abandon the task if a plugin file already exists, but that would cause problems when I bump the plugin version number to the latest copy.
Thus, ideally, I am looking for a switch to present to unarchive to say that I want the folder modification times from the zip file, not from playbook runtime. Is it possible?
Update: a commenter asked if the file contents could have changed in any way. To determine whether they have, I wrote this script, which creates a checksum for (1) all file contents and (2) all file/directory timestamps:
#!/bin/bash
# Save pwd and then change dir to root location
STARTDIR=`pwd`
cd `dirname $0`/../..
# Clear collation file
echo > /tmp/wp-checksum
# List all files recursively
find wp-content/plugins/wordpress-https/ -type f | while read file
do
#echo $file
cat $file >> /tmp/wp-checksum
done
# Get checksum of file contents
sha1sum /tmp/wp-checksum
# Get checksum of file sizes
ls -Rl wp-content/plugins/wordpress-https/ | sha1sum
# Go back to original dir
cd $STARTDIR
I ran this as part of my playbook (running it in isolation using tags) and received this:
PLAY [Set this playbook to run locally] ****************************************
TASK [setup] *******************************************************************
ok: [localhost]
TASK [jonblog : Run checksum command] ******************************************
changed: [localhost]
TASK [jonblog : debug] *********************************************************
ok: [localhost] => {
"checksum_before.stdout_lines": [
"374fadc4df1578f78fd60b1be6758477c2c533fa /tmp/wp-checksum",
"10d66f7bdbbdd3af531d1b11a3db3059a5868838 -"
]
}
TASK [jonblog : Install the WordPress HTTPS plugin] ***************
changed: [localhost]
TASK [jonblog : Run checksum command] ******************************************
changed: [localhost]
TASK [jonblog : debug] *********************************************************
ok: [localhost] => {
"checksum_after.stdout_lines": [
"374fadc4df1578f78fd60b1be6758477c2c533fa /tmp/wp-checksum",
"719c9da94b525e723b1abe188ee9f5bbaf121f3f -"
]
}
PLAY RECAP *********************************************************************
localhost : ok=6 changed=3 unreachable=0 failed=0
The debug lines reflect the checksum hash of the contents of the files (this is identical) and then the checksum hash of ls -Rl of the file structure (this has changed). This is in keeping with my prior manual finding that directory checksums are changing.
So, what can I do next to track down why folder modification times are incorrectly flagging this operation as changed?
Rather than overwriting all files each time and find a way to keep the same modification datetime, you may want to use the creates option of the unarchive module.
As you maybe already know, this tells Ansible that a specific file/folder will be created as a result of the task. Thus, next time the task will not be run again if that file/folder already exists.
See http://docs.ansible.com/ansible/unarchive_module.html#options
My solution is to modify the checksum script and to make that a permanent feature of the Ansible process. It feels a bit hacky to do my own checksumming, when Ansible should do it for me, but it works.
New answers that explain that I am doing something wrong, or that a new version of Ansible fixes the problem, would be most welcome.
If I get a moment, I will raise this as a possible bug with the Ansible team. However I do sometimes wonder about the effort/reward ratio when raising bugs on a busy tracker - I already have one item outstanding, it has been waiting a while, and I've chosen to work around that too.
Update (18 months later)
This Ansible build system never made it into live. It felt like I was always working around something. Recently, when I decided I needed to move my blog to another server, I finally Dockerised it. This took several weeks (since there is a surprising amount of things to think about in a real WordPress installation) but in general I found the process much nicer than using orchestration tools.
New to salt,and i add first server(wx-1),it works ,but when i add a differnt server, test.ping is ok,but when execute salt 'qing' state.highstate, it fails,the error info is:
No Top file or external nodes data matches found
Here is my top.sls:
base:
'wx-1':
- bin.nginx
- git
- web
- mongo
- redis
'qing':
- bin.nginx
qing is a new server and it's config is different to wx-1,don't know if this is ok,thanks for your help:)
If you make changes to your sls files. Make sure that you restart the master in order for it to update. This solved my problem when receiving the same error...
You didn't give much information. But here are a few things to check:
test if salt qing state.sls bin.nginx works, if not continue reading
make sure file_roots:base in master config points to /srv/salt
use salt-master/minion --version to check salt versions, make sure they are the same. Because different versions might diff
Give further info if you tried all the above.