I want to read the data from one pillar file into another. for example:
top.sls
- one
- two
one.sls
env: test
two.sls
dir: {{salt'pillar.get'}}_value
But it is not getting the value of pillar 'env' from one.sls in two.sls. Any idea how we can do that?
Thanks,
Sanjiv
This currently not available out of the box. There's an issue, but at this rate it probably wont make it in any of the 2016.x releases.
The cleanest workaround I've found is import:
one.sls:
{% set foo = 'bar' %}
baz: {{ foo }}
two.sls:
{% from 'one.sls' import foo %}
baz2: {{ foo }}
Related
I install a configuration for rails that looks like this (simplified):
production:
secret_key_base: 800afb35d5086b2c60ebd35c01b2bd2b522c2492
db_username: ...
db_password: ...
and so it gets installed from a template file
{{ role }}:
secret_key_base: {{ secret_key }}
db_username: {{ db_user }}
db_password: {{ db_pass }}
And the role and db user/pass get pulled from pillar and installed in that file. The secret_key it would make sense to generate randomly, e.g., {{ salt['random.get_str'](length=80) }}. But I want to generate it once, not every time the template is rendered. (Changing the secret key invalidates cookies, not something to do on each salt run.)
The only solution I've found is two-phase: I have a template.in file
{{ role }}:
secret_key_base: ||secret_key_base||
db_username: {{ db_user }}
db_password: {{ db_pass }}
that I sed into my template file on any given minion:
/srv/salt/rails/secrets.yml:
cmd.run:
# Fill in the secret key base (used for cookies). We can't use
# jinja2 for this, since jinja would complain about the other
# variables that it doesn't know how to replace. We want our
# output to be a jinja template.
- name: |
cat /srv/salt/rails/secrets.yml.in | \
sed -e 's/||secret_key_base||/{{ salt['random.get_str'](length=80) }}/;' | \
cat > /srv/salt/rails/secrets.yml
chmod 400 /srv/salt/rails/secrets.yml
- creates: /srv/salt/rails/secrets.yml
- runas: root
/var/railroad/{{host_role}}/shared/config/secrets.yml:
file.managed:
- source: salt://rails/secrets.yml
- mode: 400
- user: railroad-{{host_role}}
- group: railroad-{{host_role}}
- template: jinja
- defaults:
role: host_role
db_username: m_u
db_password: m_p
This works but has the disadvantage that a change to secrets.yml.in would not be propagated on to secrets.yml. (Suppose we add another key to the secrets file.) It also feels clunkier than necessary.
Is there a better way?
A better way
As noted in the comments, a better way is to just generate the secret by hand (after all, it's only done at host setup) and store it in pillar, where we anyway have to say a few things about each host.
Here is what the working code eventually looked like, unsimplified for those who might want to see something more complex. Much of the complexity is my host_credentials pillar data, which tries to characterise all that we need to know about each host.
{% set fqdn = grains.get('fqdn', 'unknown-host-fqdn') %}
{% set host_role = pillar['host_credentials']
[grains.get('fqdn')]
['role'] %}
{% set examplecom_web_app = pillar['host_credentials']
[grains.get('fqdn')]
['examplecom-web-app'] %}
{% set mysql_server_host = examplecom_web_app.mysql.host %}
{% set mysql_server_database = examplecom_web_app.mysql.database %}
{% set mysql_server_role = examplecom_web_app.mysql.role %}
{% set mysql_server_spec = pillar['host_credentials']
[mysql_server_host]
['mysql'] %}
{% set mongodb_server_host = examplecom_web_app.mongodb.host %}
{% set mongodb_server_spec = pillar['host_credentials']
[mongodb_server_host]
['mongodb'] %}
/var/examplecom/railroad/{{host_role}}/shared/config/secrets.yml:
file.managed:
- source: salt://rails/secrets.yml
- mode: 400
- user: railroad-{{host_role}}
- group: railroad-{{host_role}}
- template: jinja
- defaults:
role: {{ host_role }}
secret_key_base: {{ examplecom_web_app.secret_key_base }}
mysql_hostname: {{ mysql_server_host }}
mysql_username: {{ mysql_server_spec[mysql_server_database]
[mysql_server_role]
['username'] }}
mysql_password: {{ mysql_server_spec[mysql_server_database]
[mysql_server_role]
['password'] }}
mongodb_hostname: {{ mongodb_server_host }}
mongodb_username: {{ mongodb_server_spec.username }}
mongodb_password: {{ mongodb_server_spec.password }}
As an aside, I was happy to discover that jinja2 is white-space agnostic, which helps enormously with readability on such lookups.
I recommend putting the secret in a pillar and generate the value once (manually) on the master. That way you can avoid doing the stateful on-the-fly-magic in your SLS file.
jma updated his question to include an example solution.
Do I understand you correctly:
- this is salt-masterless run?
- you generate some template file on masterless node (to have some static parts generated once)
- you apply file.managed state to previously generated file?
Assuming my understandings:
First of all it will be propagated to another states if you set proper watch/require statements but in your case this would be harder to achieve as you've used cmd.run for template parsing (you must add stateful argument to express that there is some potential state changes underlying)
Remarks:
- did you see file.blockreplace? it seems that you can use this to replace first cmd.run and get file change detection "for free"
- as for one time password generation, there is nice trick to do this simply by using grains.get_or_set_hash
As the generated password is not changing for given minion (you master-less node in your case) the file.blockreplace won't report any changes unless you add changes to the template
Having said this, I think we can go further and your state actually can be as simple as this (template changes will be propagated always, key will be generated once):
/var/railroad/{{host_role}}/shared/config/secrets.yml:
file.managed:
- source: salt://rails/secrets.yml
- mode: 400
- user: railroad-{{host_role}}
- group: railroad-{{host_role}}
- template: jinja
- defaults:
role: host_role
db_username: m_u
db_password: {{ salt['grains.get_or_set_hash']('some:place:secret_key_base') }}
I have several states that are almost the same. All of them deploy project, create virtualenv and configure supervisor. Difference is only in repo, project name and some additional actions.
A lot of code is duplicated. Is it possible to put the same parts into file and include it with additional variables?
In Ansible it can be done this way:
tasks:
- include: wordpress.yml
vars:
wp_user: timmy
ssh_keys:
- keys/one.txt
- keys/two.txt
This question looks similar to this one
If I understood your question correctly - I believe the best way to achieve what you want is to use Salt Macros.
With this most of your state will go to macros with placeholders being parameters like:
# lib.sls
{% macro create_user(user, password) %}
{{user}}:
user.present:
- home: /home/{{user}}
- password: {{password}}
{% endmacro %}
Then your state will look like:
# john.sls
{% from 'lib.sls' import create_user with context %}
{{ create_user('john', '<password hash>') }}
and:
# jane.sls
{% from 'lib.sls' import create_user with context %}
{{ create_user('john', '<password hash>') }}
As I found out there is another way to archive it without messing with templates (more Ansible way). Create an abstract state "python-project". Create concrete roles then and provide different pillars to these roles:
salt/top.sls:
base:
'roles:python-project-1':
- match: grain
- python-project
'roles:python-project-2':
- match: grain
- python-project
pillar/top.sls:
base:
'roles:python-project-1':
- match: grain
- common-pillars
- pillars-for-the-first
'roles:python-project-2':
- match: grain
- common-pillars
- pillars-for-the-second
Structure:
pillar/top.sls
pillar/common-pillars/init.sls
pillar/pillars-for-the-first/init.sls
pillar/pillars-for-the-second/init.sls
salt/top.sls
salt/python-project/init.sls
You can use Jinja imports to do that:
top.sls
{% set user = 'john' %}
{% include 'config.sls' %}
config.sls
file.managed:
- name: '/Users/{{ user }}/.config
- user: {{ user }}
Is it possible to translate the value of twig variables in a template with the 'trans' tag?
Say for instance I am passing a product to my template. This product has a definition with a trans tag e.g {{ product.definition|trans }}. This definition could either be in EN or DE or some other language. How could I translate the definition.
What are you trying to do is not a good way, It would look like this:
messages.en.yml
product:
definition:
some_value1: Some value 1
some_value2: Some value 2
and in template, you would do something like this:
{% set definition_value = product.definition %}
{% set trans_definition = 'product.definition.' ~ definition_value %}
{{ trans_definition|trans }}
it'll work, if it finds the key. What if it cant find it?
That's why you should use DoctrineBehaviors from KnpLabs, which handles all the dynamic translations for you..
If {{ product.definition }} equals 'cellphone' the following should work.
message.language.yml:
'cellphone': This will work!
However if you want to map it with the 'product' key in your message file like this:
product:
'cellphone': This also works
add the key to the twig template like so:
{{('product.'~product.definition)|trans }}
How is it possible to get something like the following running:
{% if not exist('/tmp/dummy/') then %}
dummy:
file.touch:
- name: /tmp/dummy/tmp.txt
...
{% endif %}
I need it for installing software from a ZIP-file. I want to unzip on the minions, but there I don't want to have any remnants of licence files, which I only need for installation, left.
You can use unless for this.
dummy:
file.touch:
- name: /tmp/dummy/tmp.txt
- unless: test -d /tmp/dummy/
{% if 1 == salt['cmd.retcode']('test -f /tmp/woo.test') %}
ack:
file.touch:
- name: /tmp/woo.test
{% endif %}
You could use the pure python renderer for salt. Here is how it would look like.
#!py
import os
def run():
# defines the hash config
config = {}
if (not os.path.isfile("/tmp/dummy")):
config["dummy"] = {
"file.touch": [
{'name': '/tmp/dummy'},
],
}
return config
How can one get a twig global variable to maintain modification after changing it with includes? My desired output is "set # deeper" though I get "original setting".
app/config/config.yml
twig:
globals:
testvar: "original setting"
root.html.twig
{% include "MyBundle::levelone.html.twig" %}
{{ testvar }}
levelone.html.twig
{% set testvar = "set # levelone" %}
{% include "MyBundle::deeper.html.twig" %}
deeper.html.twig
{% set testvar = "set # deeper" %}
From a container aware object :
$this->get('twig')->addGlobal('ThomasDecauxIsAwsome', 4);
Interesting question and I don't know an answer but no amount fooling around with blocks and stuff will work. I looked in the generated php cache template files. When you echo a variable it looks like:
// {{ testvar }}
echo twig_escape_filter($this->env, $this->getContext($context, "testvar"), "html", null, true);
So basically it first looks for testvar in your local context. If not found then it looks in the global context.
When you set the value of test var you get:
// {% set testvar = 'level one' }}
$context["testvar"] = "level one";
So only the local context gets updated. The changed value goes away when the included template returns.
So by default at least it looks like global variables are really read only.
It might be possible to do this through an extension. I don't know enough about the internals.
Have you tried to define a global variable through Twig? You can do it in your config.yml file as follows:
twig:
globals:
my_global_var : myvalue
So {{my_global_var}} in your template will print myvalue
For more info read the official docs.
It's possible by defining a Twig Extension via Services, check here