I have two dictionaries as below
"fictional_characters": {
"male": [
"Donkey",
"Humpty",
"Piranha"
],
"female": [
"Fiona",
"Kitty_Softpaws",
"Diane_Foxington"
]
}
And
"movie_names": {
"Donkey": "Shrek",
"Humpty": "Puss_in_Boots",
"Piranha": "The_Bad_Guys",
"Fiona": "Shrek",
"Kitty_Softpaws": "Puss_in_Boots",
"Diane_Foxingtin": "The_Bad_Guys"
}
I would like to change the dictionary values in "fictional_characters" to the values in "movie_names", e.g.
"fictional_characters": {
"male": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys"
],
"female": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys"
]
}
I started by converting the "fictional_characters" dictionary into a list
- name: Convert fictional_characters to a list
set_fact:
fictional_characters_list: "{{fictional_characters | dict2items }}"
That gave me
"fictional_characters_list": [
{
"key": "male",
"value": [
"Donkey",
"Humpty",
"Piranha"
],
},
"key": "female",
"value": [
"Fiona",
"Kitty_Softpaws",
"Diane_Foxington"
]
}
]
Next, some Jinja
- name: Using Jinja to swap dict values
set_fact:
fict_char_movies: |
{% for e in fictional_characters_list %}
{{ e.key }}:
{% for char in e.value %}
{% if char in movie_names %}
- {{ movie_names[char]|split(',') %}
{% endif %}
{% endfor %}
{% endfor %}
- name: Print the result
debug:
msg: "{{ fict_char_movies | from_yaml }}"
the above returns the following result
"male": [
[
"Shrek"
],
[
"Puss_in_Boots"
],
[
"The_Bad_Guys"
],
"female": [
[
"Shrek"
],
[
"Puss_in_Boots"
],
[
"The_Bad_Guys"
]
]
How do I get rid of this nested list so that I get the below structure ?
"fictional_characters": {
"male": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys"
],
"female": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys"
]
}
The declarations below
fict_char_movies_str: |
{% for k,v in fictional_characters.items() %}
{{ k }}:
{% for char in v %}
{% if char in movie_names %}
- {{ movie_names[char] }}
{% endif %}
{% endfor %}
{% endfor %}
fict_char_movies: "{{ fict_char_movies_str|from_yaml }}"
The template can be simplified
fict_char_movies_str: |
{% for k,v in fictional_characters.items() %}
{{ k }}:
{{ v|map('extract', movie_names) }}
{% endfor %}
give what you want
fict_char_movies:
female:
- Shrek
- Puss_in_Boots
- The_Bad_Guys
male:
- Shrek
- Puss_in_Boots
- The_Bad_Guys
If you want to iterate set_fact the task below gives the same result
- set_fact:
fict_char_movies: "{{ fict_char_movies|d({})|combine(_dict|from_yaml) }}"
loop: "{{ fictional_characters|dict2items }}"
vars:
_dict: "{ {{ item.key }}: {{ item.value|map('extract', movie_names) }} }"
Example of a complete playbook for testing
- hosts: localhost
vars:
fictional_characters:
female:
- Fiona
- Kitty_Softpaws
- Diane_Foxington
male:
- Donkey
- Humpty
- Piranha
movie_names:
Diane_Foxington: The_Bad_Guys
Donkey: Shrek
Fiona: Shrek
Humpty: Puss_in_Boots
Kitty_Softpaws: Puss_in_Boots
Piranha: The_Bad_Guys
fict_char_movies_str: |
{% for k,v in fictional_characters.items() %}
{{ k }}:
{{ v|map('extract', movie_names) }}
{% endfor %}
fict_char_movies: "{{ fict_char_movies_str|from_yaml }}"
tasks:
- debug:
var: fict_char_movies
Vladimir's solution is probably cleaner, but it's also possible to get to the same place without using a Jinja template. For example, we can write:
- hosts: localhost
gather_facts: false
vars:
fictional_characters:
male:
- Donkey
- Humpty
- Piranha
female:
- Fiona
- Kitty_Softpaws
- Diane_Foxington
- Meilin_Lee
movie_names:
Donkey: Shrek
Humpty: Puss_in_Boots
Piranha: The_Bad_Guys
Fiona: Shrek
Kitty_Softpaws: Puss_in_Boots
Diane_Foxington: The_Bad_Guys
Meilin_Lee: Turning_Red
tasks:
- set_fact:
new_fictional_characters: >-
{{
new_fictional_characters|combine(
{
'male': item.0|ternary(new_fictional_characters.male + [movie_names[item.0]], new_fictional_characters.male),
'female': item.1|ternary(new_fictional_characters.female + [movie_names[item.1]], new_fictional_characters.female),
}
)
}}
loop: >-
{{ fictional_characters.male|zip_longest(fictional_characters.female) }}
vars:
new_fictional_characters:
male: []
female: []
- set_fact:
fictional_characters: "{{ new_fictional_characters }}"
- debug:
var: fictional_characters
Note that I've added an additional character here to demonstrate that this works when the lists are of different lengths. Running this produces:
PLAY [localhost] ***************************************************************
TASK [set_fact] ****************************************************************
ok: [localhost] => (item=['Donkey', 'Fiona'])
ok: [localhost] => (item=['Humpty', 'Kitty_Softpaws'])
ok: [localhost] => (item=['Piranha', 'Diane_Foxington'])
ok: [localhost] => (item=[None, 'Meilin_Lee'])
TASK [set_fact] ****************************************************************
ok: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => {
"fictional_characters": {
"female": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys",
"Turning_Red"
],
"male": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys"
]
}
}
PLAY RECAP *********************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
An even simpler solution is to drop the following into filter_plugins/swap_names.py:
def filter_swap_names(val, name_map):
return {k: [name_map[name] for name in v] for k, v in val.items()}
class FilterModule:
def filters(self):
return dict(swap_names=filter_swap_names)
And then write the playbook like this:
- hosts: localhost
gather_facts: false
vars:
fictional_characters:
male:
- Donkey
- Humpty
- Piranha
female:
- Fiona
- Kitty_Softpaws
- Diane_Foxington
- Meilin_Lee
movie_names:
Donkey: Shrek
Humpty: Puss_in_Boots
Piranha: The_Bad_Guys
Fiona: Shrek
Kitty_Softpaws: Puss_in_Boots
Diane_Foxington: The_Bad_Guys
Meilin_Lee: Turning_Red
tasks:
- set_fact:
fictional_characters: "{{ fictional_characters|swap_names(movie_names) }}"
- debug:
var: fictional_characters
I'm trying to install nginx deployment and store all the nginx configuration via configmap.
All the nginx conf file are in a separate folder(outside of the template) in a new folder called "nginx.conf"
Once I'm running the installation for the chart, I'm getting error:
rromano-ltm1:eks-devops-nginx rromano$ helm install nginx-int nginx-int-chart
Error: INSTALLATION FAILED: Deployment.apps "nginx-int-nginx-int-chart" is invalid: spec.template.spec.containers[0].volumeMounts[0].name: Not found: "config"
those are the configmap, deploymant yaml files:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "nginx-int-chart.name" . }}-config
data:
{{- (.Files.Glob "nginx.conf/*").AsConfig | nindent 2 }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "nginx-int-chart.fullname" . }}
namespace: {{ .Values.namespace }}
labels:
{{- include "nginx-int-chart.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
{{- include "nginx-int-chart.selectorLabels" . | nindent 6 }}
strategy:
rollingUpdate:
maxSurge: {{ .Values.rollingUpdate.maxSurge }}
maxUnavailable: {{ .Values.rollingUpdate.maxUnavailable }}
type: RollingUpdate
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "nginx-int-chart.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "nginx-int-chart.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: 80
protocol: TCP
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
volumeMounts:
- mountPath: /etc/nginx/conf.d
name: config
subPath: conf.d
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- with .Values.nodeSelector }}
volumes:
- name: config
configMap:
name: { { template "nginx-int-chart.name" . } }-config
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
What am I missing?
it looks like you insert the volume section inside the node selector with statement, you need to change the location of the {{- with .Values.nodeSelector }} line, to be just above the nodeSelector: line
I have snippet like this in the init.sls:
{% for server, args in pillar.get('servers', {}).items() %}
software-server#{{ server }}
service.running:
- enable: true
- require:
- pkg: software_pkgs
- watch:
- file: software_config
/etc/software/{{server}}.json:
file.managed:
- source: salt://software/files/config.json.j2
- template: jinja
{% endfor %}
config.json.j2:
{
listen: {{server}}:{{listen_addr}}
}
and in the pillar:
software.servers:
server1:
listen_addr:10.0.0.1
server2:
listen_addr:127.0.01
in each of the {{server}}.json the listen_addr is different. I don't know if saltstack has something like a scope for current loop, or is there a workaround for this.
You probably need to use context or defaults options in file.managed:
file.managed
In your example it would like like :
/etc/software/{{server}}.json:
file.managed:
- source: salt://software/files/config.json.j2
- template: jinja
- context:
server: {{ server }}
listen_addr: {{ server['listen_addr'] }}
I use saltstack to deploy my application on AWS. The formulas fetch the jar from artifactory and run the application as a service.
It works fine for production(release version ex: 1.1.3) but it fails on dev environment with snapshot version (ex: 1.1.4-SNAPSHOT).
My formula :
artifactory.downloaded:
- artifact:
artifactory_url: {{ artifactory_url }}
repository: {{ repository }}
artifact_id: {{ artifact_id }}
group_id: {{ group_id }}
packaging: {{ packaging }}
classifier: {{ classifier }}
version: '{{ version }}'
- target_dir: {{ folder }}
The error: 'NoneType' object is not iterable
I think I figure it out.
The state artifactory.downloaded use the module artifactory.get_snapshot for snapshot and artifactory.get_release for release.
The get_snapshot module needs a snapshot_version properties and version properties (I think it's an issue) but you can't pass snapshot_version properties from artifactory.downloaded state.
So to resolve this issue, I don't longer use artifactory.downloaded state but artifactory.get_snapshot / artifactory.get_release module :
artifact_download:
module.run:
- name: artifactory.get_snapshot
- artifactory_url: {{ artifactory_url }}
- repository: {{ repository }}
- artifact_id: {{artifact_id }}
- group_id: {{ group_id }}
- packaging: {{ packaging }}
- classifier: {{ classifier }}
- version: '{{ version }}'
- snapshot_version: '{{ version }}'
- target_dir: {{ folder }}
⚠️ - snapshot_version and version properties are both required.
I am trying to automate the creation of my users with Saltstack.
I created a pillar conf:
users:
homer:
fullname: Homer Simpson
uid: 1007
gid: 1007
groups:
- sudo
- adm
crypt: $6H7kNJefhBeY
pub_ssh_keys:
- ssh-rsa ...
And in my state I use the following:
{% for username, details in pillar.get('users', {}).items() %}
{{ username }}:
group:
- present
- name: {{ username }}
- gid: {{ details.get('gid', '') }}
user:
- present
- fullname: {{ details.get('fullname','') }}
- name: {{ username }}
- shell: /bin/bash
- home: /home/{{ username }}
- uid: {{ details.get('uid', '') }}
- gid: {{ details.get('gid', '') }}
- password: {{ details.get('crypt','') }}
{% if 'groups' in details %}
- groups:
{% for group in details.get('groups', []) %}
- {{ group }}
{% endfor %}
{% endif %}
{% if 'pub_ssh_keys' in details %}
ssh_auth:
- present
- user: {{ username }}
- names:
{% for pub_ssh_key in details.get('pub_ssh_keys', []) %}
- {{ pub_ssh_key }}
{% endfor %}
- require:
- user: {{ username }}
{% endif %}
{% endfor %}
The user creation is okay, ssh-rsa keys are added properly but my main isssue is with the password: I tried the following:
crypt: password
crypt: some-hash
But when I connect to my server, I have a wrong password issue for this user.
Can you tell me how can I generate a good password compliant with the format salt is expecting? Is there a special command to use to generate it ?
Thank you.
To create hashed user passwords in Debian/Ubuntu, usable in salt, I do the following:
apt-get install makepasswd
echo '<password>' | makepasswd --clearfrom=- --crypt-md5 | awk '{ print $2 }'
This gives e.g.: $id$salt$encrypted
The id in "$id$salt$encrypted" should be 1, meaning it's an md5 hash.
Copy/paste this hash into your pillar.
Hope this works for you as well.
I wouldn't use md5, which is denoted by $1
If you look in your /etc/shadow file and see other passwords are $6 then it is using sha-512.
Don't use makepassword, use "mkpasswd"
mkpasswd -m sha-512
Password: [enter password]
$6$fYewyeO5lMP/$CLbYqRdUootlGA3hJzXye84k0Of9VX4z39TOnsDxfIaFcL4uGznfJsGEJMiEaHKHZDSIUK7o4r22krvhezpZq1
Thanks die the makepasswd example. It whould be great, if it works. But in my case it doesn't.
The hashed password seems not to be correct. Maybe another encryption should be used?
I have used the standard user formula from the salt stack GitHub repositories,