Ansible - synchronize module - pattern for directory in source path - directory

I am trying to use synchronize module to take a backup of files. Currently, I have hardcoded the directory for testing, but I need that to be pattern.
Ansible version 2.9
I need to make it generic for version* instead of version-2 that I currently have.
I had tried to use it in source but then I want the same pattern in dest and when I tried the version* in src, I was getting error in when using it in dest.
Here is my playbook.
---
- name: Backup configurations from all confluent components
hosts: all
tasks:
- set_fact:
inv_host_path: "/tmp/synctest/properties-backup/{{ inventory_hostname }}/"
systemd_path: "/etc/systemd/system/"
zookeeper_data_path: "/var/lib/zookeeper/version-2/"
- name: Cretate directories.
file:
path: "{{ item }}"
state: directory
delegate_to: localhost
loop:
- "{{ inv_host_path }}{{ systemd_path }}"
- "{{ inv_host_path }}{{ zookeeper_data_path }}"
- name: synchronise files in one loop
synchronize:
src: "{{ item.src }}{{ item.syncpattern }}"
dest: "{{ inv_host_path }}{{ item.dest }}"
rsync_path: "sudo rsync"
mode: pull
delegate_to: localhost
register: sync_output
loop:
- { src: "{{ systemd_path }}", dest: "{{ systemd_path }}", syncpattern: 'confluent*' }
- { src: '/etc/', dest: '/etc/', syncpattern: 'confluent*' }
- { src: '/etc/', dest: '/etc/', syncpattern: 'kafka' }
# - { src: "{{ zookeeper_data_path }}", dest: "{{ inv_host_path }}{{ zookeeper_data_path }}", syncpattern: 'snapshot*.*' }
# check if directory exists
- name: check if directory exists
stat:
path: "{{ zookeeper_data_path }}"
register: zookeeper_data_dir_stat
- name: synchronise files in second loop
synchronize:
src: "{{ item.src }}{{ item.syncpattern }}"
dest: "{{ item.dest }}"
rsync_path: "sudo rsync"
mode: pull
delegate_to: localhost
register: sync_output
loop:
- { src: "{{ zookeeper_data_path }}", dest: "{{ inv_host_path }}{{ zookeeper_data_path }}", syncpattern: 'snapshot*.*' }
when: zookeeper_data_dir_stat.stat.exists

Related

Ansible - Replace values in a dictionary by looping through another dictionary

I have two dictionaries as below
"fictional_characters": {
"male": [
"Donkey",
"Humpty",
"Piranha"
],
"female": [
"Fiona",
"Kitty_Softpaws",
"Diane_Foxington"
]
}
And
"movie_names": {
"Donkey": "Shrek",
"Humpty": "Puss_in_Boots",
"Piranha": "The_Bad_Guys",
"Fiona": "Shrek",
"Kitty_Softpaws": "Puss_in_Boots",
"Diane_Foxingtin": "The_Bad_Guys"
}
I would like to change the dictionary values in "fictional_characters" to the values in "movie_names", e.g.
"fictional_characters": {
"male": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys"
],
"female": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys"
]
}
I started by converting the "fictional_characters" dictionary into a list
- name: Convert fictional_characters to a list
set_fact:
fictional_characters_list: "{{fictional_characters | dict2items }}"
That gave me
"fictional_characters_list": [
{
"key": "male",
"value": [
"Donkey",
"Humpty",
"Piranha"
],
},
"key": "female",
"value": [
"Fiona",
"Kitty_Softpaws",
"Diane_Foxington"
]
}
]
Next, some Jinja
- name: Using Jinja to swap dict values
set_fact:
fict_char_movies: |
{% for e in fictional_characters_list %}
{{ e.key }}:
{% for char in e.value %}
{% if char in movie_names %}
- {{ movie_names[char]|split(',') %}
{% endif %}
{% endfor %}
{% endfor %}
- name: Print the result
debug:
msg: "{{ fict_char_movies | from_yaml }}"
the above returns the following result
"male": [
[
"Shrek"
],
[
"Puss_in_Boots"
],
[
"The_Bad_Guys"
],
"female": [
[
"Shrek"
],
[
"Puss_in_Boots"
],
[
"The_Bad_Guys"
]
]
How do I get rid of this nested list so that I get the below structure ?
"fictional_characters": {
"male": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys"
],
"female": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys"
]
}
The declarations below
fict_char_movies_str: |
{% for k,v in fictional_characters.items() %}
{{ k }}:
{% for char in v %}
{% if char in movie_names %}
- {{ movie_names[char] }}
{% endif %}
{% endfor %}
{% endfor %}
fict_char_movies: "{{ fict_char_movies_str|from_yaml }}"
The template can be simplified
fict_char_movies_str: |
{% for k,v in fictional_characters.items() %}
{{ k }}:
{{ v|map('extract', movie_names) }}
{% endfor %}
give what you want
fict_char_movies:
female:
- Shrek
- Puss_in_Boots
- The_Bad_Guys
male:
- Shrek
- Puss_in_Boots
- The_Bad_Guys
If you want to iterate set_fact the task below gives the same result
- set_fact:
fict_char_movies: "{{ fict_char_movies|d({})|combine(_dict|from_yaml) }}"
loop: "{{ fictional_characters|dict2items }}"
vars:
_dict: "{ {{ item.key }}: {{ item.value|map('extract', movie_names) }} }"
Example of a complete playbook for testing
- hosts: localhost
vars:
fictional_characters:
female:
- Fiona
- Kitty_Softpaws
- Diane_Foxington
male:
- Donkey
- Humpty
- Piranha
movie_names:
Diane_Foxington: The_Bad_Guys
Donkey: Shrek
Fiona: Shrek
Humpty: Puss_in_Boots
Kitty_Softpaws: Puss_in_Boots
Piranha: The_Bad_Guys
fict_char_movies_str: |
{% for k,v in fictional_characters.items() %}
{{ k }}:
{{ v|map('extract', movie_names) }}
{% endfor %}
fict_char_movies: "{{ fict_char_movies_str|from_yaml }}"
tasks:
- debug:
var: fict_char_movies
Vladimir's solution is probably cleaner, but it's also possible to get to the same place without using a Jinja template. For example, we can write:
- hosts: localhost
gather_facts: false
vars:
fictional_characters:
male:
- Donkey
- Humpty
- Piranha
female:
- Fiona
- Kitty_Softpaws
- Diane_Foxington
- Meilin_Lee
movie_names:
Donkey: Shrek
Humpty: Puss_in_Boots
Piranha: The_Bad_Guys
Fiona: Shrek
Kitty_Softpaws: Puss_in_Boots
Diane_Foxington: The_Bad_Guys
Meilin_Lee: Turning_Red
tasks:
- set_fact:
new_fictional_characters: >-
{{
new_fictional_characters|combine(
{
'male': item.0|ternary(new_fictional_characters.male + [movie_names[item.0]], new_fictional_characters.male),
'female': item.1|ternary(new_fictional_characters.female + [movie_names[item.1]], new_fictional_characters.female),
}
)
}}
loop: >-
{{ fictional_characters.male|zip_longest(fictional_characters.female) }}
vars:
new_fictional_characters:
male: []
female: []
- set_fact:
fictional_characters: "{{ new_fictional_characters }}"
- debug:
var: fictional_characters
Note that I've added an additional character here to demonstrate that this works when the lists are of different lengths. Running this produces:
PLAY [localhost] ***************************************************************
TASK [set_fact] ****************************************************************
ok: [localhost] => (item=['Donkey', 'Fiona'])
ok: [localhost] => (item=['Humpty', 'Kitty_Softpaws'])
ok: [localhost] => (item=['Piranha', 'Diane_Foxington'])
ok: [localhost] => (item=[None, 'Meilin_Lee'])
TASK [set_fact] ****************************************************************
ok: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => {
"fictional_characters": {
"female": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys",
"Turning_Red"
],
"male": [
"Shrek",
"Puss_in_Boots",
"The_Bad_Guys"
]
}
}
PLAY RECAP *********************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
An even simpler solution is to drop the following into filter_plugins/swap_names.py:
def filter_swap_names(val, name_map):
return {k: [name_map[name] for name in v] for k, v in val.items()}
class FilterModule:
def filters(self):
return dict(swap_names=filter_swap_names)
And then write the playbook like this:
- hosts: localhost
gather_facts: false
vars:
fictional_characters:
male:
- Donkey
- Humpty
- Piranha
female:
- Fiona
- Kitty_Softpaws
- Diane_Foxington
- Meilin_Lee
movie_names:
Donkey: Shrek
Humpty: Puss_in_Boots
Piranha: The_Bad_Guys
Fiona: Shrek
Kitty_Softpaws: Puss_in_Boots
Diane_Foxington: The_Bad_Guys
Meilin_Lee: Turning_Red
tasks:
- set_fact:
fictional_characters: "{{ fictional_characters|swap_names(movie_names) }}"
- debug:
var: fictional_characters

Deployment failed due config map is not valid

I'm trying to install nginx deployment and store all the nginx configuration via configmap.
All the nginx conf file are in a separate folder(outside of the template) in a new folder called "nginx.conf"
Once I'm running the installation for the chart, I'm getting error:
rromano-ltm1:eks-devops-nginx rromano$ helm install nginx-int nginx-int-chart
Error: INSTALLATION FAILED: Deployment.apps "nginx-int-nginx-int-chart" is invalid: spec.template.spec.containers[0].volumeMounts[0].name: Not found: "config"
those are the configmap, deploymant yaml files:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "nginx-int-chart.name" . }}-config
data:
{{- (.Files.Glob "nginx.conf/*").AsConfig | nindent 2 }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "nginx-int-chart.fullname" . }}
namespace: {{ .Values.namespace }}
labels:
{{- include "nginx-int-chart.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
{{- include "nginx-int-chart.selectorLabels" . | nindent 6 }}
strategy:
rollingUpdate:
maxSurge: {{ .Values.rollingUpdate.maxSurge }}
maxUnavailable: {{ .Values.rollingUpdate.maxUnavailable }}
type: RollingUpdate
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "nginx-int-chart.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "nginx-int-chart.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: 80
protocol: TCP
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
volumeMounts:
- mountPath: /etc/nginx/conf.d
name: config
subPath: conf.d
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- with .Values.nodeSelector }}
volumes:
- name: config
configMap:
name: { { template "nginx-int-chart.name" . } }-config
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
What am I missing?
it looks like you insert the volume section inside the node selector with statement, you need to change the location of the {{- with .Values.nodeSelector }} line, to be just above the nodeSelector: line

Ansible: print elements from joined list of dictionaries

I am trying to access elements from both lists: lista1 and lista2.
First join them into one dictionary and then to be able to do something like this:
Vegetable {{ lista1.fruits.name }} has taste {{ lista1.fruits.taste }}
This is what I got so far, but it's really bad,and I would like to have dictionary like access instead of regex_search.
---
- name: demo how register works
hosts: localhost
vars:
lista1:
fruits:
- name: tomato
taste: tomato-like
- name: lemon
taste: sour
lista2:
vegetables:
- name: carrot
taste: sweet
tasks:
- name: debug
debug:
var: item
loop:
- lista1.fruits
- lista2.vegetables
register: echo
- name: show register results
debug:
msg: "Food named: {{ item| map(attribute='name')|list|join(', ') |regex_search('tomato') }} tastes: {{ item| map(attribute='taste')|list|join(', ')|regex_search('tomato-like') }}"
loop: "{{ lista1.fruits|zip(lista2.vegetables)|list }} "
A debug task to show the "food" with its "taste" can be displayed by accessing item.name and item.taste by looping through lista1.fruits and lista2.vegetables.
Example:
- name: show register results
debug:
msg: "Food named: {{ item.name }} tastes: {{ item.taste }}"
with_items:
- "{{ lista1.fruits }}"
- "{{ lista2.vegetables }}"
Update
Instead of with_items, loop can be used as:
- name: show register results
debug:
msg: "Food named: {{ item.name }} tastes: {{ item.taste }}"
loop: "{{ lista1.fruits + lista2.vegetables }}"

accessinig current pillar items in a loop from another template?

I have snippet like this in the init.sls:
{% for server, args in pillar.get('servers', {}).items() %}
software-server#{{ server }}
service.running:
- enable: true
- require:
- pkg: software_pkgs
- watch:
- file: software_config
/etc/software/{{server}}.json:
file.managed:
- source: salt://software/files/config.json.j2
- template: jinja
{% endfor %}
config.json.j2:
{
listen: {{server}}:{{listen_addr}}
}
and in the pillar:
software.servers:
server1:
listen_addr:10.0.0.1
server2:
listen_addr:127.0.01
in each of the {{server}}.json the listen_addr is different. I don't know if saltstack has something like a scope for current loop, or is there a workaround for this.
You probably need to use context or defaults options in file.managed:
file.managed
In your example it would like like :
/etc/software/{{server}}.json:
file.managed:
- source: salt://software/files/config.json.j2
- template: jinja
- context:
server: {{ server }}
listen_addr: {{ server['listen_addr'] }}

Custom content type in Symfony2 Assetic (phpless)

I'm using Phpless(0.3.0) to compile less css in Symfony(2.4.2) automatically
My files:
config_dev.yml
assetic:
debug: "%kernel.debug%"
use_controller: true
filters:
cssrewrite: ~
lessphp:
apply_to: "\.less$"
formatter: "compressed"
preserve_comments: false
view.twig:
{% stylesheets 'application/assets/less/bootstrap.less' %}
<link rel="stylesheet" type="text/css" href="{{ asset_url }}">
{% endstylesheets %}
So, in the response, everything is compiled.. but the content type of the generated css (that comes from symfony controller (use_controller: true)) is "text/html"
4798dcc_bootstrap_1.css GET 304 text/html login:20 0 B 320 ms
There are any setting of workaround to return this content type as text/css?
Taking into account that is the development environment, and I want avoid to generate manually the css with:
php app/console assetic:dump
Thanks!
The problem might be with fos_rest catching all requests
fos_rest:
format_listener:
rules:
- { path: ^/, priorities: [html], fallback_format: html, prefer_extension: false }
Add rule before that to exclude your assets

Resources