I need to convert a dictionary with list values into a list of dictionaries.
Given:
my_dict:
key1: ["111", "222"]
key2: ["444", "555"]
Desired output:
my_list:
- key1: "111"
key2: "444"
- key1: "222"
key2: "555"
What I've tried:
- set_fact:
my_list: "{{ my_list | default([]) + [{item.0.key: item.1}] }}"
loop: "{{ my_dict | dict2items | subelements('value') }}"
And what I've got:
[
{
"key1": "111"
},
{
"key1": "222"
},
{
"key2": "444"
},
{
"key2": "555"
}
]
Thankful for any help and suggestions!
Get the keys and values of the dictionary first
keys: "{{ my_dict.keys()|list }}"
vals: "{{ my_dict.values()|list }}"
gives
keys: [key1, key2]
vals:
- ['111', '222']
- ['444', '555']
Transpose the values
- set_fact:
tvals: "{{ tvals|d(vals.0)|zip(item)|map('flatten') }}"
loop: "{{ vals[1:] }}"
gives
tvals:
- ['111', '444']
- ['222', '555']
Create the list of the dictionaries
my_list: "{{ tvals|map('zip', keys)|
map('map', 'reverse')|
map('community.general.dict')|
list }}"
gives
my_list:
- key1: '111'
key2: '444'
- key1: '222'
key2: '555'
Notes
Example of a complete playbook
- hosts: localhost
vars:
my_dict:
key1: ["111", "222"]
key2: ["444", "555"]
keys: "{{ my_dict.keys()|list }}"
vals: "{{ my_dict.values()|list }}"
my_list: "{{ tvals|map('zip', keys)|
map('map', 'reverse')|
map('community.general.dict')|
list }}"
tasks:
- set_fact:
tvals: "{{ tvals|d(vals.0)|zip(item)|map('flatten') }}"
loop: "{{ vals[1:] }}"
- debug:
var: my_list
You can use a custom filer to transpose the matrix. For example,
shell> cat filter_plugins/numpy.py
# All rights reserved (c) 2022, Vladimir Botka <vbotka#gmail.com>
# Simplified BSD License, https://opensource.org/licenses/BSD-2-Clause
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
from ansible.errors import AnsibleFilterError
from ansible.module_utils.common._collections_compat import Sequence
import json
import numpy
def numpy_transpose(arr):
if not isinstance(arr, Sequence):
raise AnsibleFilterError('First argument for numpy_transpose must be list. %s is %s' %
(arr, type(arr)))
arr1 = numpy.array(arr)
arr2 = arr1.transpose()
return json.dumps(arr2.tolist())
class FilterModule(object):
''' Ansible wrappers for Python NumPy methods '''
def filters(self):
return {
'numpy_transpose': numpy_transpose,
}
Then you can avoid iteration. For example, the playbook below gives the same result
- hosts: localhost
vars:
my_dict:
key1: ["111", "222"]
key2: ["444", "555"]
keys: "{{ my_dict.keys()|list }}"
vals: "{{ my_dict.values()|list }}"
tvals: "{{ vals|numpy_transpose()|from_yaml }}"
my_list: "{{ tvals|map('zip', keys)|
map('map', 'reverse')|
map('community.general.dict')|
list }}"
tasks:
- debug:
var: my_list
Transposing explained
Let's start with the matrix 2x2
vals:
- ['111', '222']
- ['444', '555']
The task below
- set_fact:
tvals: "{{ tvals|d(vals.0)|zip(item) }}"
loop: "{{ vals[1:] }}"
gives step by step:
a) Before the iteration starts the variable tvals is assigned the default value vals.0
vals.0: ['111', '222']
b) The task iterates the list vals[1:]. These are all lines in the array except the first one
vals[1:]:
- ['444', '555']
c) The first, and the only one, iteration zip the first and the second line. This is the result
vals.0|zip(vals.1):
- ['111', '444']
- ['222', '555']
Let's proceed with matrix 3x3
vals:
- ['111', '222', '333']
- ['444', '555', '666']
- ['777', '888', '999']
The task below
- set_fact:
tvals: "{{ tvals|d(vals.0)|zip(item)|map('flatten') }}"
loop: "{{ vals[1:] }}"
gives step by step:
a) Before the iteration starts the variable tvals is assigned the default value vals.0
vals.0: ['111', '222', '333']
b) The task iterates the list vals[1:]
vals[1:]:
- ['444', '555', '666']
- ['777', '888', '999']
c) The first iteration zip the first and the second line, and assigns it to tvals. The filer flatten has no effect on the lines
vals.0|zip(vals.1)|map('flatten'):
- ['111', '444']
- ['222', '555']
- ['333', '666']
d) The next iteration zip tvals and the third line
tvals|zip(vals.2):
- - ['111', '444']
- '777'
- - ['222', '555']
- '888'
- - ['333', '666']
- '999
e) The lines must be flattened. This is the result
tvals|zip(vals.2)|map('flatten'):
- ['111', '444', '777']
- ['222', '555', '888']
- ['333', '666', '999']
---
- hosts: localhost
vars:
mydict:
key1: val1
key2: val2
key3:
subkey1: subval1
subkey2: subval2
tasks:
- debug:
msg: "{{ TODO }}"
How would I make the above debug message print out all key/value pairs from the nested dictionary? Assume the depth is unknown. I would expect the output to be something like:
{
"key1": "val1",
"key2": "val2",
"subkey1": "subval1"
"subkey2": "subval2"
}
Write a filter plugin and use pandas.json_normalize, e.g.
shell> cat filter_plugins/dict_normalize.py
from pandas.io.json import json_normalize
def dict_normalize(d):
df = json_normalize(d)
l = [df.columns.values.tolist()] + df.values.tolist()
return(l)
class FilterModule(object):
def filters(self):
return {
'dict_normalize': dict_normalize,
}
The filter returns lists of keys and values
- set_fact:
mlist: "{{ mydict|dict_normalize }}"
gives
mlist:
- - key1
- key2
- key3.subkey1
- key3.subkey2
- - val1
- val2
- subval1
- subval2
Create a dictionary, e.g.
- debug:
msg: "{{ dict(mlist.0|zip(mlist.1)) }}"
gives
msg:
key1: val1
key2: val2
key3.subkey1: subval1
key3.subkey2: subval2
If the subkeys are unique remove the path
- debug:
msg: "{{ dict(_keys|zip(mlist.1)) }}"
vars:
_regex: '^(.*)\.(.*)$'
_replace: '\2'
_keys: "{{ mlist.0|map('regex_replace', _regex, _replace)|list }}"
gives
msg:
key1: val1
key2: val2
subkey1: subval1
subkey2: subval2
Notes
Install the package, e.g. python3-pandas
The filter might be extended to support all parameters of json_normalize
The greedy regex works also in nested dictionaries
Q: "Turn the key3.subkey1 to get the original dictionary."
A: Use json_query. For example, given the dictionary created in the first step
- set_fact:
mydict_flat: "{{ dict(mlist.0|zip(mlist.1)) }}"
mydict_flat:
key1: val1
key2: val2
key3.subkey1: subval1
key3.subkey2: subval2
iterate the keys and retrieve the values from mydict
- debug:
msg: "{{ mydict|json_query(item) }}"
loop: "{{ mydict_flat|list }}"
gives
msg: val1
msg: val2
msg: subval1
msg: subval2
Trying to substract a number for a variable, which is an int in Ansible.
var:
number: 30
tasks:
- set_fact: me={{ number -1 }}
- debug: var=me
Expectation: me = 29
Result:
fatal: [node1]: FAILED! => {"failed": true, "msg": "Unexpected templating type error occurred on ({{ number - 1 }}): unsupported operand type(s) for -: 'AnsibleUnicode' and 'int'"}
It is a known issue with Ansible/Jinja that you can't preserve numeric type after templating.
Use int filter inside {{..}} expression:
- set_fact: me={{ number | int - 1 }}
How to convert a datetime HH:MM:SS:mmm in SS.mmm with a filter in Twig please ( `` )?
exemple :
00:01:30.600 => 90.600
I tried that {{ Object.time | date("s") }} but it doesn't work ...
thanks !
As #john Smith told in his comment (unfortunately I can't +1 but he deserves it), you can use:
{{ Object.time | date("s.u") }}
To change microseconds to miliseconds, you have 2 choices:
Round to 3 digits:
{{ Object.time | date("s.u") | round(3) }}
Slice the 3 last chars:
{{ Object.time | date("s.u") | slice(0, -3) }}
Anyone knows how to addition two variable in twig I want to do something like:
{{ var1 }} + {{ var2 }}
Just do it inside the {{ }}
{{ var1 + var2 }}
If you want to assign it to some other variable:
{% set foo = var1 + var2 %}
{{ foo }}
Declaration :
{% set counter = 0 %}
Doing the addition :
{% set counter = counter + 1 %}
For displaying in twig template :
{{ counter }}