Ansible filter to extract specific keys from a dict into another dict - dictionary

Using the nios lookup modules, I can get a list of dicts of records
- set_fact:
records: "{{ lookup('community.general.nios', 'record:a', filter={'name~': 'abc.com'}) }}"
This returns something like
- ref: record:a/someBase64:name/view
name: abc.com
ipv4addr: 1.2.3.4
view: default
- ref: record:a/someBase64:name/view
name: def.abc.com
ipv4addr: 1.2.3.5
view: default
- ref: record:a/someBase64:name/view
name: ghi.abc.com
ipv4addr: 1.2.3.6
view: default
I want to convert this into a dict of dicts of {name}: a: {ipv4addr}
abc.com:
a: 1.2.3.4
def.abc.com:
a: 1.2.3.5
ghi.abc.com:
a: 1.2.3.6
So that I can then run a similar lookup to get other record types (e.g. cname) and combine them into the same dict. The items2dict filter seems halfway there, but I want the added a: key underneath.

If you just wanted a dictionary that maps name to an ipv4 address, like:
{
"abc.com": "1.2.3.4",
...
}
You could use a simple json_query expression. Take a look at the
set_fact task in the following example:
- hosts: localhost
gather_facts: false
vars:
data:
- ref: record:a/someBase64:name/view
name: abc.com
ipv4addr: 1.2.3.4
view: default
- ref: record:a/someBase64:name/view
name: def.abc.com
ipv4addr: 1.2.3.5
view: default
- ref: record:a/someBase64:name/view
name: ghi.abc.com
ipv4addr: 1.2.3.6
view: default
tasks:
- set_fact:
name_map: "{{ dict(data|json_query('[].[name, ipv4addr]')) }}"
- debug:
var: name_map
Running that playbook will output:
PLAY [localhost] ***************************************************************
TASK [set_fact] ****************************************************************
ok: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => {
"name_map": {
"abc.com": "1.2.3.4",
"def.abc.com": "1.2.3.5",
"ghi.abc.com": "1.2.3.6"
}
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
You could use a similar structure to extract other data (e.g. cname records). This would get you dictionary per type of data, rather than merging everything together in a single dictionary as you've requested, but this might end up being easier to work with.
To get exactly the structure you want, you can use set_fact in a loop, like this:
- hosts: localhost
vars:
data:
- ref: record:a/someBase64:name/view
name: abc.com
ipv4addr: 1.2.3.4
view: default
- ref: record:a/someBase64:name/view
name: def.abc.com
ipv4addr: 1.2.3.5
view: default
- ref: record:a/someBase64:name/view
name: ghi.abc.com
ipv4addr: 1.2.3.6
view: default
gather_facts: false
tasks:
- set_fact:
name_map: "{{ name_map|combine({item.name: {'a': item.ipv4addr}}) }}"
loop: "{{ data }}"
vars:
name_map: {}
- debug:
var: name_map
This will produce:
PLAY [localhost] ***************************************************************
TASK [set_fact] ****************************************************************
ok: [localhost] => (item={'ref': 'record:a/someBase64:name/view', 'name': 'abc.com', 'ipv4addr': '1.2.3.4', 'view': 'default'})
ok: [localhost] => (item={'ref': 'record:a/someBase64:name/view', 'name': 'def.abc.com', 'ipv4addr': '1.2.3.5', 'view': 'default'})
ok: [localhost] => (item={'ref': 'record:a/someBase64:name/view', 'name': 'ghi.abc.com', 'ipv4addr': '1.2.3.6', 'view': 'default'})
TASK [debug] *******************************************************************
ok: [localhost] => {
"name_map": {
"abc.com": {
"a": "1.2.3.4"
},
"def.abc.com": {
"a": "1.2.3.5"
},
"ghi.abc.com": {
"a": "1.2.3.6"
}
}
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Related

When I am running playbook its showing Variable not defined

- hosts: switch
connection: network_cli
become_method: enable
gather_facts: no
vars_prompt:
- name: vlan_id
prompt: enter the vlan_id
private: no
vars:
cli:
username: admin
password: int123$%^
vlans:
100: "CORE"
200: "MONITORING"
300: "ACCESS"
400: "GUEST_WIFI"
ansible_buffer_read_timeout: 2
tasks:
- name: "creating the vlans"
ios_vlans:
config:
- vlan_id: "{{ vlan_id }}"
mtu: 700
state: active
shutdown: disabled
register: show_vlan
- debug:
var: show_vlan.stdout_lines
Output:
enter the vlan_id: 11
PLAY [switch] ****************************************************************************************************************************************************
TASK [creating the vlans] **************************************************************************************************************************************** changed: [172.16.1.252]
TASK [debug]
ok: [172.16.1.252] => show_vlan.stdout_lines: VARIABLE IS NOT DEFINED!
PLAY RECAP 172.16.1.252 : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
ios_vlans module does not have stdout_lines key in it's return values. Please check the documention here
so debug show_lan
- debug:
var: show_vlan

Get a YAML file with HTTP and use it as a variable in an Ansible playbook

Background
I have a YAML file like this on a web server. I am trying to read it and make user accounts in the file with an Ansible playbook.
users:
- number: 20210001
name: Aoki Alice
id: alice
- number: 20210002
name: Bob Bryant
id: bob
- number: 20210003
name: Charlie Cox
id: charlie
What I tried
To confirm how to read a downloaded YAML file dynamically with include_vars, I had written a playbook like this:
- name: Add users from list
hosts: workstation
tasks:
- name: Download yaml
get_url:
url: http://fqdn.of.webserver/path/to/yaml.yml
dest: "/tmp/tmp.yml"
notify:
- Read yaml
- List usernames
handlers:
- name: Read yaml
include_vars:
file: /tmp/tmp.yml
name: userlist
- name: List usernames
debug:
var: "{{ item }}"
loop: "{{ userlist.users }}"
Problem
In the handler Read yaml, I got the following error message. On the target machine (workstation.example.com), /tmp/tmp.yml is downloaded correctly.
RUNNING HANDLER [Read yaml] *****
fatal: [workstation.example.com]: FAILED! => {"ansible facts": {"userlist": []},
"ansible included var files": [], "changed": false, "message": "Could not find o
r access '/tmp/tmp. yml' on the Ansible Controller.\nIf you are using a module a
nd expect the file to exist on the remote, see the remote src option"}
Question
How can I get a YAML file with HTTP and use it as a variable with include_vars?
Another option would be to use the uri module to retrieve the value into an Ansible variable, then the from_yaml filter to parse it.
Something like:
- name: Add users from list
hosts: workstation
tasks:
- name: Download YAML userlist
uri:
url: http://fqdn.of.webserver/path/to/yaml.yml
return_content: yes
register: downloaded_yaml
- name: Decode YAML userlist
set_fact:
userlist: "{{ downloaded_yaml.content | from_yaml }}"
Note that uri works on the Ansible Controller, while get_url works on the target host (or on the host specified in delegate_to); depending on your network configuration, you may need to use different proxy settings or firewall rules to permit the download.
The include_vars task looks for files on the local (control) host, but you've downloaded the file to /tmp/tmp.yml on the remote host. There are a number of ways of getting this to work.
Perhaps the easiest is just running the download task on the control machine instead (note the use of delegate_to):
tasks:
- name: Download yaml
delegate_to: localhost
get_url:
url: http://fqdn.of.webserver/path/to/yaml.yml
dest: "/tmp/tmp.yml"
notify:
- Read yaml
- List usernames
This will download the file to /tmp/tmp.yml on the local system, where it will be available to include_vars. For example, if I run this playbook (which grabs YAML content from an example gist I just created)...
- hosts: target
gather_facts: false
tasks:
- name: Download yaml
delegate_to: localhost
get_url:
url: https://gist.githubusercontent.com/larsks/70d8ac27399cb51fde150902482acf2e/raw/676a1d17bcfc01b1a947f7f87e807125df5910c1/example.yaml
dest: "/tmp/tmp.yml"
notify:
- Read yaml
- List usernames
handlers:
- name: Read yaml
include_vars:
file: /tmp/tmp.yml
name: userlist
- name: List usernames
debug:
var: item
loop: "{{ userlist.users }}"
...it produces the following output:
RUNNING HANDLER [Read yaml] ******************************************************************
ok: [target]
RUNNING HANDLER [List usernames] *************************************************************
ok: [target] => (item=bob) => {
"ansible_loop_var": "item",
"item": "bob"
}
ok: [target] => (item=alice) => {
"ansible_loop_var": "item",
"item": "alice"
}
ok: [target] => (item=mallory) => {
"ansible_loop_var": "item",
"item": "mallory"
}
Side note: based on what I see in your playbook, I'm not sure you want
to be using notify and handlers here. If you run your playbook a
second time, nothing will happen because the file /tmp/tmp.yml
already exists, so the handlers won't get called.
With #Larsks 's answer, I made this playbook that works correctly in my environment:
- name: Download users list
hosts: 127.0.0.1
connection: local
become: no
tasks:
- name: Download yaml
get_url:
url: http://fqdn.of.webserver/path/to/yaml/users.yml
dest: ./users.yml
- name: Add users from list
hosts: workstation
tasks:
- name: Read yaml
include_vars:
file: users.yml
- name: List usernames
debug:
msg: "{{ item.id }}"
loop: "{{ users }}"
Point
Run get_url on the control host
As #Larsks said, you have to run the get_url module on the control host rather than the target host.
Add become: no to the task run on the control host
Without "become: no", you will get the following error message:
TASK [Gathering Facts] ******************************************************
fatal: [127.0.0.1]: FAILED! => {"ansible_facts": {}, "changed": false, "msg":
"The following modules failed to execute: setup\n setup: MODULE FAILURE\nSee
stdout/stderr for the exact error\n"}
Use connection: local rather than local_action
If you use local_action rather than connection: local like this:
- name: test get_url
hosts: workstation
tasks:
- name: Download yaml
local_action:
module: get_url
url: http://fqdn.of.webserver/path/to/yaml/users.yml
dest: ./users.yml
- name: Read yaml
include_vars:
file: users.yml
- name: output remote yaml
debug:
msg: "{{ item.id }}"
loop: "{{ users }}"
You will get the following error message:
TASK [Download yaml] ********************************************************
fatal: [workstation.example.com]: FAILED! => {"changed": false, "module_stde
rr": "sudo: a password is required\n", "module_stdout":"", "msg":"MODULE FAIL
URE\nSee stdout/stderr for the exact error", "rc": 1}
get_url stores a file on the control host
In this situation, the get_url module stores users.yml on the control host (in the current directory). So you have to delete the users.yml if you don't want to leave it.

Ansible datetime timezone conversion

Is there a way to convert ansible date into a different timezone in a "debug" statement in my playbook ? I dont want a global timezone setting at the playbook level. I have this :
debug:
msg: "{{ '%Y-%m-%d %H:%M:%S' | strftime(ansible_date_time.epoch) }}"
This works fine but displays time in UTC. I need the time to be displayed in EDT without setting timezone at the global playbook level. How do I accomplish this ?
If you use a command task to run date rather than relying on the ansible_date_time variable, you can set the timezone via an environment variable. E.g. the following playbook:
- hosts: localhost
vars:
ansible_python_interpreter: /usr/bin/python
tasks:
- command: "date '+%Y-%m-%d %H:%M:%S'"
register: date_utc
environment:
TZ: UTC
- command: "date '+%Y-%m-%d %H:%M:%S'"
register: date_us_eastern
environment:
TZ: US/Eastern
- debug:
msg:
- "{{ date_utc.stdout }}"
- "{{ date_us_eastern.stdout }}"
Results in this output:
PLAY [localhost] *****************************************************************************
TASK [Gathering Facts] ***********************************************************************
ok: [localhost]
TASK [command] *******************************************************************************
changed: [localhost]
TASK [command] *******************************************************************************
changed: [localhost]
TASK [debug] *********************************************************************************
ok: [localhost] => {
"msg": [
"2020-05-12 15:21:05",
"2020-05-12 11:21:06"
]
}
PLAY RECAP ***********************************************************************************
localhost : ok=4 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Ansible - How to loop with a dict and a module's values

I'm trying to create a task with Ansible (=> 2.5) that will configure network interfaces such as that:
- name: Set up network interfaces addr
interfaces_file:
dest: "/etc/network/interfaces.d/{{ item.device }}"
iface: "{{ item.device }}"
state: present
option: address
value: "{{ item.addr }}"
with_items:
- "{{ network }}"
when: item.addr is defined
notify: Restart interface
- name: Set up network interfaces netmask
interfaces_file:
dest: "/etc/network/interfaces.d/{{ item.device }}"
iface: "{{ item.device }}"
state: present
option: netmask
value: "{{ item.netmask }}"
with_items:
- "{{ network }}"
when: item.netmask is defined
notify: Restart interface
- name: Set up network interfaces dns
interfaces_file:
dest: "/etc/network/interfaces.d/{{ item.device }}"
iface: "{{ item.device }}"
state: present
option: dns-nameservers
value: "{{ item.dns }}"
with_items:
- "{{ network }}"
when: item.dns is defined
notify: Restart interface
- name: Set up network interfaces dns-search
interfaces_file:
dest: "/etc/network/interfaces.d/{{ item.device }}"
iface: "{{ item.device }}"
state: present
option: dns-search
value: "{{ item.dns_search }}"
with_items:
- "{{ network }}"
when: item.dns_search is defined
notify: Restart interface
This works.
But from my point of view, that's not so clean ..
So I'm trying to use 2 loops ... Which is not working obviously.
- name: Set up network interfaces
interfaces_file:
dest: "/etc/network/interfaces.d/{{ item.iDunnoWhatToPutHere }}"
iface: "{{ item.iDunnoWhatToPutHere }}"
state: present
option: {{ item.option }}
value: "{{ item.value }}"
with_together:
- "{{ network }}"
- { option: address, value: item.0.addr }
- { option: netmask, value: item.0.netmask }
- { option: dns-nameservers, value: item.0.dns }
when: item.dns_search is defined
notify: Restart interface
[...]
Edit: This is good but it's strict. I should loop on vars which should loop on each option and its value for any options. Because I also have options for bridge such as "vlan_raw_device, bridge_ports, bridge_stp ...". So it should just loop blindly on a dict of options and values.
Edit2: With variable network
network:
- name: admin
device: admin
method: static
address: X.X.X.X/X
netmask: X.X.X.X
up:
net: X.X.X.X/X
gateway: X.X.X.X/X
down:
net: X.X.X.X/X
gateway: X.X.X.X/X
Why I'm trying all this ?
Because I need to change all the values if it has to be changed.
Because I want to restart (ifup, ifdown) only the interface that
Because I'm surprised that I have to use multiple times the same module.
Can you guys help me find out how to use that ?
Maybe it's not possible ?
Thanks folks !
here is a task that will hopefully meet your needs. i have replaced the interfaces_file with debug module, just to print the variables you need to actually use in the interfaces_file module. for the sake of the demo, i added a second interface in the network variable:
playbook with the variable and the task:
---
- hosts: localhost
connection: local
gather_facts: false
vars:
network:
- name: admin
device: admin
method: static
address: 10.10.10.22
netmask: 255.255.255.0
up:
net: X.X.X.X/X
gateway: X.X.X.X/X
down:
net: X.X.X.X/X
gateway: X.X.X.X/X
- name: admin22
device: admin22
method: static
address: 20.20.20.22
netmask: 255.255.255.192
up:
net: X.X.X.X/X
gateway: X.X.X.X/X
down:
net: X.X.X.X/X
gateway: X.X.X.X/X
tasks:
- name: process network config
debug:
msg: "dest: {{ item[0].name }}, option: {{ item[1].option }}, value: {{ item[0][item[1].value] }}"
with_nested:
- "{{ network }}"
- [{ option: address, value: address }, { option: netmask, value: netmask }]
result:
TASK [process network config] ******************************************************************************************************************************************************************************************
ok: [localhost] => (item=None) => {
"msg": "dest: admin, option: address, value: 10.10.10.22"
}
ok: [localhost] => (item=None) => {
"msg": "dest: admin, option: netmask, value: 255.255.255.0"
}
ok: [localhost] => (item=None) => {
"msg": "dest: admin22, option: address, value: 20.20.20.22"
}
ok: [localhost] => (item=None) => {
"msg": "dest: admin22, option: netmask, value: 255.255.255.192"
}
hope it helps

How to define an Ansible playbook environment with passed in environment dictionary

My problem is that I'm unable to set the environment for the entire playbook by passing in a dict to be set as the environment. Is that possible?
For example, here is my sample ansible playbook:
- hosts: localhost
vars:
env_vars: "{{ PLAY_ENVS }}"
environment: "{{ env_vars }}"
tasks:
- name: Here is what you passed in
debug: msg="env_vars == {{ env_vars }}"
- name: What is FAKE_ENV
debug: msg="FAKE_ENV == {{ lookup('env', 'FAKE_ENV') }}"
And I'm passing the command:
/bin/ansible-playbook sample_playbook.yml --extra-vars '{PLAY_ENVS: {"FAKE_ENV":"/path/to/fake/destination"}}'
The response I'm getting is the following:
PLAY [localhost] ***************************************************************
TASK [setup] *******************************************************************
ok: [localhost]
TASK [Here is what you passed in] **********************************************
ok: [localhost] => {
"msg": "env_vars == {u'FAKE_ENV': u'/path/to/fake/destination'}"
}
TASK [What is FAKE_ENV] ********************************************************
ok: [localhost] => {
"msg": "FAKE_ENV == "
}
PLAY RECAP *********************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0
As you can see 'FAKE_ENV' is not being set in the environment. What am I doing wrong?
Lookups in Ansible are executed in a context of parent ansible process.
You should check your environment with a spawned process, like this:
- hosts: localhost
vars:
env_vars:
FAKE_ENV: foobar
environment: "{{ env_vars }}"
tasks:
- name: Test with spawned process
shell: echo $FAKE_ENV
And get expected result: "stdout": "foobar",

Resources