SaltStack extending another pillar - salt-stack

I'm using SaltStack, and I'm trying to re-use the values of a nested dictionary from one Pillar config in another one. Here's a simple example of what I'm trying to do:
Say I have pillar/app/common.sls which has the following items:
app:
lookup:
custom1: 'change the default'
custom2: 'change the default'
service1:
value1: 'foo'
value2: 'bar'
list1:
- apple
- banana
value3: 'xen'
What I'm aiming for is to have a new service (and the YAML key would be service2) on the same machine, but set up in such a way so that I'm not writing values twice (keep it DRY!). I would also like to override the value of one of the parameters. Essentially, I'm trying to extend one pillar into another.
The end result would be:
app:
lookup:
custom1: 'change the default'
custom2: 'change the default'
service1:
value1: 'foo'
value2: 'bar'
list1:
- apple
- banana
value3: 'xen'
service2:
value1: 'foo'
value2: 'bar'
list1:
- apple
- banana
value3: 'future'
I've tried the following with a pillar/app/someserver.sls:
{% import_yaml "app/common.sls" as common %}
app:
service2:
{{ common.app.service1 }}
value3: 'future'
What I've found is {{ common.app.service1 }} successfully renders, but it doesn't like the additional value3: 'future', which is my attempt to override value3 which comes from common.sls.
I'm using Salt 2016.11.4 on Ubuntu Xenial 16.04.2. Any ideas would be helpful. Thank you!

Since your common.app.service1 variable is nothing more than a common Python dict, you should be able to modify it using update. After that, you can use Jinja's yaml filter to render it into the new pillar:
{% import_yaml "app/common.sls" as common %}
{% set service2 = common.app.service1.copy() %}
{% do service2.update({value3: 'future'}) %}
app:
service2: {{ service2 | yaml }}

Related

Apply map filter to value of dictionary with ansible/jinja2

I'm trying to write an ansible playbook that outputs some details about a system, in a nicely formatted way. In particular, disk sizes.
Input variable looks something like:
- friendly_name: 'disk1 name'
size: 123456
- friendly_name: 'disk2 name'
size: 654321
{{ dict(ansible_facts.disks | json_query('[].[friendly_name, size]')) }}
I'm struggling to come up with a way to apply a function to the 'value' of the dictionary (or the second value of the nested list, prior to converting it to a dict) - I'd like to apply human_readable(unit='G') or similar, without resorting to set_fact or FilterPlugins
So ideally I'd have an output variable of the form:
{'disk1 name': '1024G', 'disk2 name': '8192G'}
You could split the dictionary ansible_facts.disks into two lists, one containing the size and the other one the friendly name, then apply the human_readable filter to the list containing the size with the map filter, then zip the two lists back together.
Given the task:
- debug:
msg: "{{ dict(
ansible_facts.disks | map(attribute='friendly_name') |
zip(ansible_facts.disks | map(attribute='size') | map('human_readable','unit','G'))
) }}"
vars:
ansible_facts:
disks:
- friendly_name: 'disk1 name'
size: 1099511627776
- friendly_name: 'disk2 name'
size: 8796093022208
This yields:
TASK [debug] ********************************************************************
ok: [localhost] => {
"msg": {
"disk1 name": "1024.00 Gb",
"disk2 name": "8192.00 Gb"
}
}
Without the formatting you could simply use items2dict
- debug:
msg: "{{ ansible_facts.disks|items2dict(key_name='friendly_name',
value_name='size') }}"
gives
msg:
disk1 name: 1099511627776
disk2 name: 8796093022208
Use Jinja to change the format, e.g.
- debug:
msg: "{{ _disks|from_yaml }}"
vars:
_disks: |
{% for i in ansible_facts_disks %}
{{ i.friendly_name }}: {{ i.size|human_readable(unit='G') }}
{% endfor %}
gives
msg:
disk1 name: 1024.00 GB
disk2 name: 8192.00 GB

Dictionary value access using ansible

I have a situation where we have 2 dictionary defined in ansible role default and the selection of dictionary is based of an input variable. I want to set the fact with one of the dict's specific key value.
Below is the example code:
test.yml paybook content:
- hosts: localhost
gather_facts: true
roles:
- role1
tags: ['role1']
roles/role1/tasks/main.yml content:
- name: set fact
set_fact:
node_vip: "{% if node_vip_run == 'no' %}node_vip_no{% elif node_vip_run == 'yes' %}node_vip_yes{% endif %}"
- debug:
var: node_vip
verbosity: 1
- debug:
var: "{{ node_vip }}.ece_endpoint"
verbosity: 1
- name: set fact
set_fact:
ece_endpoint_fact: "{{ node_vip[ece_endpoint] }}"
- debug:
var: ece_endpoint
verbosity: 1
roles/role1/defaults/main.yml content:
node_vip_yes:
ece_endpoint: "https://1.1.1.1:8080"
cac_endpoint: "https:2.2.2.2:8080"
node_vip_no:
ece_endpoint: "http://3.3.3.3:8080"
cac_endpoint: "http:4.4.4.4:8080"
Run playbook:
ansible-playbook test.yaml --extra-vars 'node_vip_run=no' -v
The set fact of variable "ece_endpoint_fact" should have value "https://1.1.1.1:8080 OR http://3.3.3.3:8080" depending on the parameter input in ansible command. But I keep on getting below error:
TASK [role1 : set fact] *******************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'unicode object' has no attribute u'http://3.3.3.3:8080'\n\nThe error appears to be in '/root/roles/role1/tasks/main.yml': line 46, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: set fact\n ^ here\n"}
Please suggest what needs to be done to resolve this.
Thanks
Right now, you set node_vip to either the literal string "node_vip_no" or "node_vip_yes". But if you change it to do {{ node_vip_no }} / {{ node_vip_yes }}, then node_vip will have the value of the variable node_vip_no / node_vip_yes instead of being a literal string.
- name: set fact
set_fact:
node_vip: "{% if node_vip_run == 'no' %}{{ node_vip_no }}{% elif node_vip_run == 'yes' %}{{ node_vip_yes }}{% endif %}"
This will have node_vip's value be something like:
TASK [debug] ***************************************************************
ok: [localhost] => {
"node_vip": {
"cac_endpoint": "https:2.2.2.2:8080",
"ece_endpoint": "https://1.1.1.1:8080"
}
}
Then in your other set_fact, it should work if you put quotes around the property name:
- name: set fact
set_fact:
ece_endpoint_fact: "{{ node_vip['ece_endpoint'] }}"
# Added quotes ^ ^

If statements with nested grain returns nothing

I am trying to write an if statement based on a nested grain. I have tried this statement in multiple different ways:
System Services Needed:
module.run:
- name: service.systemctl_reload
- onchanges:
- file: /lib/systemd/system/salt-minion.service
{% if salt['grains.get']('Project:DeviceTypeID') == '2' %}
- file: /etc/rc.local
- file: /opt/interfaces_init.sh
{% endif %}
Returns:
Rendering SLS 'Development:System' failed: Jinja variable 'dict object' has no attribute 'Project:DeviceTypeID'
System Services Needed:
module.run:
- name: service.systemctl_reload
- onchanges:
- file: /lib/systemd/system/salt-minion.service
{% if grains['Project']['DeviceTypeID'] == '2' %}
- file: /etc/rc.local
- file: /opt/interfaces_init.sh
{% endif %}
System Services Needed:
module.run:
- name: service.systemctl_reload
- onchanges:
- file: /lib/systemd/system/salt-minion.service
{% if grains['Project:DeviceTypeID'] == '2' %}
- file: /etc/rc.local
- file: /opt/interfaces_init.sh
{% endif %}
As you can tell from the example their are multiple device type IDs. In this example DeviceTypeID = 2 I need to worry about rc.local and a shell script. I can not seem to get this work for the life of me. I know the grain exists as I can run the following:
sudo salt 'Dev-Box' grains.get Project
and I will get:
Dev-Box:
DeviceTypeID:
1
IsActive:
True
SoftwareEnvironmentName:
Production
SoftwareVersion:
Foo
This is either a bug or I am missing something (significantly more likely I am missing something). Any help would be much appreciated.
Edit 1:
Added ['grains.get']('Project:DeviceTypeID') example
in salt grains.get return a dictionary in the following format:
{'minion-id': value}
I believe if you change your code into something like bellow, it should works.
{% if salt['grains.get']('Project:DeviceTypeID')[minion-id] == '2' %}
If you can't do:
salt 'Dev-Box' grains.get 'Project:DeviceTypeID'
Then you don't actually have the proper grain set.
Try the following:
salt 'Dev-Box' grains.setval Project '{"DeviceTypeID": 2, "IsActive": True, "SoftwareEnvironmentName": "Production", "SoftwareVersion": "Foo"}'
Then the following state:
Do the {{ salt['grains.get']('Project:DeviceTypeID') }} things:
test.succeed_with_changes:
- some: thing
You should get:
ID: Do the 2 things
Function: test.succeed_with_changes
Result: True
Comment: Success!
Started: 17:10:42.739240
Duration: 0.491 ms
Changes:
----------
testing:
----------
new:
Something pretended to change
old:
Unchanged
Given what you wrote elsewhere
salt Dev-Box grains.setval BETTI "{'DeviceTypeID': 2, 'IsActive': True SoftwareEnvironmentName': 'Production', 'SoftwareVersion': 'Foo'}"
Your problem is that you have ' and " confused.
Wrapping the value with " makes it a string. Wrapping it with ' and providing valid JSON makes it a dictionary value.

Salt stack grains data from sqlite3 external pillar

I am trying to pull results from a sqlite3 database (set up as an external salt pillar) and use jinja templating to set grains data.
Here is the relevant section of my salt master file:
sqlite3:
database: '/var/lib/salt/stations.db'
timeout: 5.0
ext_pillar:
- sqlite3:
fromdb:
query: 'SELECT * FROM table;'
And here is the relevant part of the init.sls file I am using to create the grains file:
{% set station_id = salt['grains.filter_by']({
{% for row in query_result %}
{% hostname = station_id %}
}, default="UNKNOWN", grain="host") %}
I confirmed that the external pillar produces results by running
salt '*' sqlite3.fetch /var/lib/salt/stations.db 'SELECT * FROM test;'
But I can't figure out how to get results into the jinja file.
I want something like
'SELECT * FROM table WHERE hostname=station_id LIMIT 1;'
and use the result to set the grain environmental variable called 'hostname'.
But am not sure how to get there from here.
Any help is greatly appreciated.
Thanks to the good folks in Saltstack IRC this problem is solved.
Master:
- sqlite3:
station_map:
query: 'SELECT hostname, id
FROM stations
WHERE hostname like ?'
init.sls:
{% set station_id = salt['grains.filter_by']({
{% for row in station_map %}
{{ hostname }} : {{ station_id }}
}, default="UNKNOWN", grain="host") %}

Run salt states twice with different pillar data?

I have set up pillar data for websites, e.g. web_root, virtualhost and mysql:
web_root:
config_file: salt://some/path.conf
key: some data
directory_name: directoryA
virtualhost:
config_file: salt://some/path.conf
name: websiteA
mysql:
database:
- websiteA_db
These map to states for web_root, virtualhost and mysql (using formula).
I'd like to use have a minion run these states multiple times, using separate pillar data, e.g.
include:
- apache
- php
{% for instance in [instanceA, instanceB] -%}
{% load pillar data /pillar/{{ instance }} -%}
- web_root #run the state
- virtualhost #run the state
- mysql #run the state
{% endfor -%}
Is this possible? I know I can set up pillar data like so:
web_root:
instanceA:
config_file: salt://some/pathA.conf
key: some data
directory_name: directoryA
instanceB:
config_file: salt://some/pathB.conf
key: some data
directory_name: directoryB
virtualhost:
instanceA:
config_file: salt://some/pathA.conf
name: websiteA
instanceB:
config_file: salt://some/pathB.conf
name: websiteB
mysql:
database:
- websiteA_db
- websiteB_db
But it means I have to add loops to each state file, making it less readable as well as use different syntax, e.g. for mysql which is a formula with set syntax requirements.
You'll want to do something like this:
Pillar Data
web_root:
instances:
A:
- name: A
- key: key_A_data
B:
- name: B
- key: key_B_data
State file
{% set names = salt['pillar.get']('web_root:instances') %}
apache:
pkg.installed: []
{% for name in names %}
instance{{ name }}:
- config_file: salt://some/path{{ name }}.conf
- key: {{ key }}
- directory_name: directory{{ name }}
{% endfor %}
Then just do the same thing for the rest of your objects. This way you don't have to change your state file when you add objects to the pillar.

Resources