I created a complex state for API service, it involves git checkouts, python venv, uwsgi, nginx, etc etc. It works fine.
Now I would like to turn it into a template and execute it several times per minion, with variables supplied from pillar - i.e something like.
{% for apiserver in pillar.apiservers %}
include apiserver_template.sls, locals: apiserver.config
{% endfor %}
where apiserver_template will work with context supplied to it, with apiserver.config having all config data for each API instance. I know syntax is wrong but hopefully I am communicating the idea - ideally, something like executing ruby partials with supplying local variables.
How is it done properly in saltland?
It sounds to me like Jinja Macro is something you want to use for this. You can find more information about usage here: https://docs.saltstack.com/en/2015.8/topics/development/conventions/formulas.html#jinja-macros
In short what you will have in your case may look like:
{% macro api_server(git_repo, python_venv_path, python_venv_requirements) %}
{{python_venv_path}}:
virtualenv.managed:
- system_site_packages: False
- requirements: salt://{{python_venv_requirements}}
{{git_repo}}:
git.latest:
- name: {{git_repo}}
{% endmacro %}
Assuming you have a pillar apiservers where each api server has git_repo, python_venv_path and python_venv_requirements values, you can use the macro like this:
{% for server in salt.pillar.get('apiservers', []) %}
{{ api_server(server['git_repo'], server['python_venv_path'], server['python_venv_requirements']) }}
{% endfor %}
If you want - you can also put a macro in a separate state file and then import a marco as a regular salt resource.
Please also not that instead of pillar.apiservers I used salt.pillar.get('apiservers', []). This is a safer way to get data from pillar. If for some reason a pillar is unavailable - the later code will result in empty dict instead of failure in first case.
Related
I have the locale set to one language. Let's say german "de" and I want to have a part of the whole text translated into different languages (Ex. This happened 100 years ago -> This happened vor 100 Jahre). For this I am trying the following code:
//this returns 'site.dates.years'
{% set yearsText = yearsCount|displayTimeDivisions() %}
( {{ yearsCount }} {% trans with {'%content%': yearsText } from "messages" into "en" %}%content%{% endtrans %}
after using the trans method i get the string from yearsText ('site.dates.years') and not the translated content. Is this even possible to translate or I should drop it?
This works as expected, what you are trying to translate is %content%, not site.dates.years. Try this:
//this returns 'site.dates.years'
{% set yearsText = yearsCount|displayTimeDivisions() %}
( {{ yearsCount }} {% trans from "messages" into "en" %}{{ yearsText }}{% endtrans %}
Edit
The previous suggestion doesn't work as using trans in that way only works with simple text, not with variables.
This works for me:
{{ yearsText | trans({}, 'messages', 'en') }}
EDIT: clear cache
Sometimes Symfony has a too strong caching mechanism. If you work without locales first, and then modify controllers and twigs to be multi-locale and you also add a new messages file, Symfony loads all new controller/twigs but somehow misses that there are new message files. A simple cache clear, done ONLY ONCE (!) right after you add the message files, solves this problem. And thenceforth you don't need to clear the cache every time you change a translation.
Try this as a last resort. If this doesn't help I'd recommend to read carefully the documentation (link in the last line of this answer):
app/console cache:clear --env=dev
You may try one or all (!) of these, it's impossible to tell what is going wrong.
{{ yearsCount|trans }}
{{ 'site.dates.years'|trans }}
See if these translate. If the second does, you're setting the variable wrong, somehow.
If not, pass the _locale along in the route:
defaults: { _controller: Bundle:Controller:action, _locale: de }
Or have it as a route parameter:
path: /a/path/{_locale}/whatever
defaults: { _controller: Bundle:Controller:action }
requirements:
_locale: en|de
Print the locale to be sure it's properly set in the twig:
{{ app.request.locale }}
{{ app.request.getLocale() }}
If the locale prints out fine but the translation still does not work, it's because Symfony does not find the identifier. Go to the translations file and check that it's written correctly:
site:
dates:
years: Jahre
If you find it in messages.de.yml then maybe it's the wrong placement of the file. The loader searches for these files:
Symfony looks for message files (i.e. translations) in the following default locations:
the app/Resources/translations directory;
the app/Resources/MyAppBundle/translations directory;
the Resources/translations/ directory inside of any bundle.
If fiddling around with these files doesn't yield results, it's a configuration error in config.yml, or many overlapping errors (all of the above?) that work against you.
Here's the source of all pain/info about translations, but you've probably memorized every paragraph, by now: Symfony translations.
I'm having trouble using the salt mine in a pillar to dynamically create a list of hosts based on a grain value match. I don't get any error, I get no output for all hosts. Actually, I can't get any output for mine from the pillar at all even when using the example from the salt docs. I know it isn't an issue with my top file, because I can access all of the other pillar values. My test minion's mine.interval is set to 5. I've refreshed pillar data, and ran mine.update.
Here's an example of my pillar:
mine_functions:
network.ip_addrs: []
grains.item:
- host
- role
My template file that access the mine functions:
#I know this is writing the same list for each match, I'm just doing this for testing and I'll concat the results into a string when I know it works:
{% for host in salt['mine.get']('roles:web', 'grains.items:host', expr_form='grain') | dictsort() %}
serverList= {{ host }}
{% endfor %}
Output from CLI:
salt "server.domain.com" mine.get "*" "*"
server.domain.com:
----------
How do I get this to work? I get no errors, no output, it just runs smoothly, but nothing is written in the file and I get nothing from the command line. My goal here is to be able to dynamically build a list of servers that match a specific grain to set a configuration value in a config template. Am I down the wrong path here, is there a better way?
I'd recommend using mine.get directly in your sls file to get that list of hosts. I don't think there's any need to pass that through pillar data.
#Utah_Dave, thanks so much for the help both here and in IRC.
Posting this as an answer so anyone else searching for this gets a good example...
pillar:
mine_functions:
grains.items: []
template file:
{% set ft_hosts = [] %}
{% for grain_vals in salt['mine.get']('role:ps:ft:True', 'grains.items', expr_form='grain').items() %}
{% do ft_hosts.append(grain_vals[1]['host']) %}
{% endfor %}
ft.ps.server.hosts={{ ft_hosts|join('|') }}
I'm trying to have salt automatically deploy files to users' home directory (after creating them). The directory structure (under file_roots) is as follows:
users/
init.sls
user_list.jinja
files/
userX/
some_dir_I_want_deployed_to_userX_homedir/
some_script_I_want_deployed.sh
etc
user_list.jinja has a list of users. The idea is that if there's a directory for userX under files/, that directory subtree should be deployed to userX's home. However, I'd rather not have to create empty directories for userY etc if they don't have anything to deploy, so I'm trying to test existence of the directory to avoid an error.
Here's the relevant excerpt from users/init.sls:
{% from "users/user_list.jinja" import users with context %}
{% for name, user in users.items() %}
{% set files_path = '{0}/files/{1}'.format(salt['file.dirname'](tplpath), name) %}
{% if salt['file.directory_exists'](files_path) %}
{{ user.home }}:
file.recurse:
- source: salt://users/files/{{ name }}
- user: {{ name }}
- group: {{ name }}
{% endif %}
{% endfor %}
With a fair bit of debugging (which was rather necessary to figure out the above), I've worked out that this is a chicken-and-egg situation, namely:
The file.directory_exists test is run on the minion (that's fair)
The salt file-server seems to have an optimization whereby it only deploys to minion (to their local cache under /var/cache/salt/minion/file) items which are referenced in states (more likely, the minions only request stuff which they see referenced).
So unless the directory subtree for users/files/userX already exists on the minion, file.directory_exists returns False, which means the entire portion which starts with {{ user.home }} is suppressed during Jinja rendering; hence it's not referenced and the copy (to the minion's local cache) never occurs.
If I manually create an empty directory structure for users/files/userX on the minion, everything starts to work. This tells me my theory is at least partially correct.
I can "feel" I'm doing smth wrong here (the whole thing feels too procedural). What's the best approach to achieve this? The requirement itself doesn't seem too far-fetched.
The more salt-ish way to do this is to have some data in pillar data and check for the existence of that key. Something like user.enabled. But, that would require you to keep settings in 2 places, in pillar and in the file_roots.
You don't want to check for the existence of the directory on the minion server, you want to check for the existence of the file in your file roots.
Unfortunately, I don't think it's possible to check for the existence of a file under salt:// scheme. If I'm wrong, then all you have to do is replace your check for directory existence with the syntax to check for file_root file existence.
The more salt-ish approach is to define which users are enabled/disabled on each machine in pillar data, and use the user module to add them to the system.
https://github.com/saltstack-formulas/users-formula
You can add to the standard pillar that's given with the standard users formula and put a key that says to sync files
#pillar
users:
ausername:
fullname: A User
syncfiles: True
busername:
fullname: B User
syncfiles: False
#state
{% for name, user in pillar.get('users', {}).items() if user.absent is not defined or not user.absent %}
{% if user.syncfiles %}
/home/{{ user.username }}:
file.recurse:
- source: salt://users/files/{{ user.username }}
- user: {{ user.username }}
{% if user.prime_group %}
- group: {{ user.prime_group.name }}
{% endif %}
{% endif %}
{% endfor %}
Actually, the standard users-formula already handles pre-populating with files. I would just use that formula. I know it's keeping track of data in 2 places, but you get the bonus of leveraging an already built state file.
In Saltstack, I have the following use case:
There is a state redis.sls which can be included by other states. The result of redis.sls should be configured differently, depending on the state which included redis.sls.
For example:
redis.sls:
--------
{% if x==1 %}
#do something
{% else %}
#do something else
{% endif %}
state_a.sls
-----------
{% set x=1 %}
include:
- redis
state_b.sls
-----------
{% set x=2 %}
include:
- redis
But x is not recognized in *state_a* and *state_b*
I also tried setting a pillar value with something like this:
{{salt['pillar.set']('x', 1)}}
but that didn't work either.
Any other ideas?
I'd like to hear what the experts say but I have a similar use case. What I did was use the jinja template to extend a base template then my child templates populated the variables.
{% extends "base.template.sls" %}
{% block x %}1{% endblock %}
Only problem might be that you now have to call state_a and state_b separately but you can always put them in a comma separated list if you want both called.
Make your redis state a jinja macro.
redis.sls:
--------
{% macro redis(x) %}
{% if x==1 %}
#do something
{% else %}
#do something else
{% endif %}
{% endmacro %}
state_a.sls
-----------
{% from 'redis.sls' import redis %}
{{ redis(1) }}
state_b.sls
-----------
{% from 'redis.sls' import redis %}
{{ redis(2) }}
For clarity redis.sls should be renamed to redis.jinja here.
This, and many other ways of managing state customization is best explained in the Salt Formulas conventions guide. Specifically the part about Jinja macros
Note that your if x==1 logic can be probably avoided altogether, take a look at the 'better' version of haproxy example in the guide.
It looks like you want to parameterize a state based on either what depends on it, or where it is used. That sounds like whatever is setting the parameter(s) on which the redis.sls state is supposed to mutate, depends on a specific configuration of redis.
To me, that seems like there are more than one distinct states in which redis could be, and that some of your states depend on one state of redis, and others of your states depend on other states of redis.
So, give the installation of redis one state, and the specific configurations of redis would each get their own state. Your state_a could depend on redis_state_1 and your state_b would in turn depend on redis_state_2. Both redis_state_1 and redis_state_2 would depend on redis. It seems to me that the parameter passing you're asking about would be less explicit.
SALT.STATES.ENVIRON might work for you:
set_secret_key:
environ.setenv:
- name: SECRET_KEY
- value: ABC123!##abc
- update_minion: True
[..]
settings_secret_key:
file.replace:
- name: {{ salt['pillar.get']('data:source_folder') }}superlists/settings.py
- pattern: "SECRET_KEY =.+$"
- repl: 'SECRET_KEY = os.environ["SECRET_KEY"]'
I use the Symfony2.1 and have the default config.yml
Documentation said:
{# but static strings are never escaped #}
{{ '<h3>foo</h3>'|trans }}
But if I copy and paste it into the my empty template (without any additional autoescapes or another) I got the escaped string <h3>foo</h3>. What I do wrong?
Try it with the twig raw filter:
{{ '<h3>foo</h3>' | trans | raw }}
However, do not use the raw filter if you are processing any user input! It allows for cross-site-scripting attacks, according to the creators of Symfony. See this similar question for a secure but more tedious alternative.
Holding HTML stuff in translations is wrong, because translators usually break it. But if you really need it:
{% trans %}<h3>foo</h3>{% endtrans %}
https://github.com/symfony/symfony/issues/2713#issuecomment-12510417