Salt changing /etc/hosts, but still caching old one? - salt-stack

Is salt caching /etc/hosts?
I'm in a situation where I change /etc/hosts such that the FQDN points to the external IP address instead of 127.0.0.1
The problem is that in the first run, the fqdn_ipv4 stays 127.0.0.1 and I need to rerun salt '*' state.highstate to get the right values. This leads to problems like this, which cost me a lot of time.
Is salt rendering everything before execution (or caches DNS)? How do I address this problem?
The state file looks like this:
127.0.0.1:
host.absent:
- name: {{ nodename }}
- ip: 127.0.0.1
127.0.1.1:
host.absent:
- name: {{ nodename }}
- ip: 127.0.1.1
{% for minion, items in salt['mine.get']('environment:' + environment, 'grains.item', expr_form='grain')|dictsort %}
{{ minion }}:
host.present:
- ip: {{ items['ip_addr'] }}
- names:
- {{ minion }}
- {{ minion.split('.')[0] }}
{% endfor %}
And the code that uses the IP looks like this:
{% set ipv4 = salt['config.get']('fqdn_ip4') -%}
# IP Address that Agent should listen on
listening_ip={{ ipv4[0] }}

Salt is caching the values of grains. Therfore the salt['config.get']('fqdn_ip4') will retrieve the value from the beginning of the script.
Use the following in your state file to refresh the grain information:
refreshgrains:
module.run:
- name: saltutil.sync_grains
Salt will render the state before executing it, so you might not be able to use any new grain information inside the state file itself.
But you will be able to use the new grain values in Jinja templates for files. I assume the second code snippet is from a template that is used by Salt's file.managed, so you should be safe here.

Related

SaltStack - mine.get is able to grab mine_function data from master, but not in .sls or jinja variable

I hope you can help me with a rather frustrating issue I have been having. I have been trying to remove static config from some config files and move this to Pillar/Mine data using Salt-Stack.
Everything is going well, with the exception of 1 specific task.
This is grabbing data (custom grain) from 3 specific minions to make 3 different variables in an .sls (context) or a jinja file (direct variable) on other minions, but I cannot seem to get it to work.
(My scenario is flexible as I can call this in either a state file or jinja variable in a config file.)
This is on AWS EC2 instances, but can be replicated away from AWS in my lab. The grain I need is: "public_ipv4" and the reason I cannot use the network.util in salt runner is because this is NAT'd and the box doesn't have a 2nd interface with the public IP assigned to it. (This cannot be changed)
Pillar data works and I have a init.sls for the mine function:
mine_functions:
grains.item:
- location
- environment
- roles
- srvtype
- instance
- az
- public_ipv4
- fqdn
- ipv4
- ipv6
(Also the custom grain: "public_ipv4" works being called by the minion so I know it is the not the grains themselves being incorrect.)
When targeting via the master using the below it brings back the requested information:
my-minion:
----------
minion-with-data-i-want-1:
----------
az:
c
environment:
dev
fqdn:
correct_fqdn
instance:
3
ipv4:
- Correct_local_ip
- 127.0.0.1
ipv6:
- ::1
- Correct_ip
location:
correct_location
public_ipv4:
Correct_public_ip
roles:
Correct_role
srvtype:
None
It is key to note here that the above comes from:
salt '*globbed_target*' mine.get '*minions-with-data-i-need-glob*' grains.item
This is from the master, but I cannot single out a specific grain by using indexing or any args/kwargs etc.
So I put some syntax into a state file and some jinja templates and I cannot get it to work. Here are a few I have tried so far:
Jinja:
{% set ip1 = salt['mine.get']('*minion-with-data-i-need-glob*', 'grains.item')[7] %}
Above returns nothing.
State file:
- context:
- ip1: {{ salt['mine.get']('*minions-with-data-i-need-glob*', 'grains.item') }}
The above returns a dict error:
Context must be formed as a dict
Running latest salt-minion/master from apt.
Steps I have taken:
Running: salt '*' mine.update after every change and checking with: salt '*' mine.valid after every change and they show.
Any help is appreciated.
This looks like you are running into a classic problem. Not knowing what you are getting as the return value.
first your {# set ip1 = salt['mine.get']('*minion-with-data-i-need-glob*', 'grains.item')[7] #} returns nothing because it is a jinja comment. {% set ip1 = salt['mine.get']('*minion-with-data-i-need-glob*', 'grains.item') %}
the next problem you have is that you are passing a list to context. when it is supposed to take a dict. the error isn't even related to mine.
try this instead
- context:
ip1: {{ salt['mine.get']('*minions-with-data-i-need-glob*', 'grains.item') | json}}
next learn to use slsutil.renderer to look at how things are rendered. such as salt minion slsutil.renderer salt://thing/init.sls default_renderer=jinja

Not able to compile data with Salstack state file

I am trying to using Salt state files to configure network devices. I will briefly describe my current setup:
I have pillar ntp.sls file saved as /etc/salt/pillar/ntp.sls and it looks like this:
ntp.servers:
- 11.1.1.1
- 2.2.2.2
Then I have Jinja template saved as /etc/salt/states/ntp/templates/ntp.jinja and looks like this:
{%- for server in servers %}
ntp {{ server }}
{%- endfor %}
Finally I have state file saved as /etc/salt/states/ntp/init.sls as this:
ntp_example:
netconfig.managed:
- template_name: salt://ntp/templates/ntp.jinja
- debug: true
- servers: {{ salt.pillar.get('ntp.servers') }}
I am getting the following error while trying to run the command: sudo salt sw state.sls ntp, where sw is the proxy minion, so here is the error:
sw:
Data failed to compile:
ID ntp.servers in SLS ntp is not a dictionary
Command to get data from pillar is working, command: sudo salt sw pillar.get ntp.servers
Output:
sw:
- 11.1.1.1
- 2.2.2.2
Any suggetions what could be wrong and how to fix it?
Thanks
I think you should declare in /etc/salt/pillar/ntp.sls something like:
ntp-servers:
- 11.1.1.1
- 2.2.2.2
and than load these values with:
- servers: {{ salt.pillar.get('ntp-servers') }}
The . is a directory separator in SaltStack.

Consul Template - "If Service Exists" condition?

I've just inherited an Nginx proxy/app server setup that makes use of Consul and Consul Template for service discovery and registration. The Nginx proxy has a config file with an entry like this to register the downstream app servers:
<snip>
upstream appservers {
{{ range service "my-app-servers" }}
server {{ .Address }}.{{ .Port }};
{{ end }}
}
<snip>
I have consul-template running in the background to catch any updates to my-app-servers, update the nginx.conf file appropriately, and then reload the nginx config. This all works great, and we're able to add and remove app servers from the mix as needed. That said, if there is a scenario where we have no app servers available, we end up with an empty upstream block and that causes nginx to fail the reload.
Is there a way in consul-template to have "if service my-app-servers exists, then..." and "if not, then..." logic? I'd like to be able to have my nginx.conf file have one configuration for scenarios where upstream servers exist, and another contingency setup that displays error pages when the upstream servers do not exist. I'm still getting up to speed on consul-template and haven't seen any examples that show the syntax for such logic. Any help?
You can achieve this by storing the result of the service lookup in a variable, then using a conditional that only outputs the upstream block if the variable is not empty.
{{- $upstream_services := service "my-app-servers" -}}
{{- if $upstream_services -}}
upstream appservers {
{{- range $upstream_services }}
server {{ .Address }}.{{ .Port }};
{{- end }}
}
{{- end }}

Is Jinja in pillars rendered before or after it's sent to the minion?

Suppose I have different credentials in two different environments, but that's the only thing that differs between them, and I don't want to make extra pillar files for a single item.
Suppose I attack the problem like this:
{%- set deployment = grains.get('deployment') %}
{%- load_yaml as credentials %}
prod: prodpassword
test: testpassword
dev: devpassword
{%- endload %}
some_app:
user: someuser
password: {{ credentials[deployment] }}
...more configuration here...
This works as expected. But can a minion in test theoretically get the password for prod? That depends on whether the dict lookup happens before or after data is sent to the client, I think, which in turn depends on when the jinja is rendered. Does the master render it first and then send the resulting data, or does the minion receive the pillar file as-is, then render it itself?
Pillar data is always rendered on the master, never the minion. The master does have access to the minion's grains, however, which is why your example works.
Given a Pillar SLS file with the following contents:
test: {{ grains['id'] }}
The following pillar data will result:
# salt testminion pillar.item test
testminion:
----------
test:
testminion
Source: I'm a SaltStack core developer.

saltstack template with grains - bad replacement

i have created a saltstack template file where i want to replace a variable with the servers ipv4 address:
in my test.template file i have:
Address {{ grains['fqdn_ip4'] }}
in my init.sls i have:
/opt/test/test.conf:
file.managed:
- source: salt://test/test.template
- template: jinja
- user: root
- group: root
- mode: 0644
so the problem is, that the variable is replaced with the correct ip address but not in the correct format:
Address ['111.111.111.111']
instead of:
Address 111.111.111.111
so my question: how can i remove the [' and '] form the replacement?
Solution:
Address {{ grains['fqdn_ip4'][0] }}
Result:
Address 111.111.111.111

Resources