I have three tasks in an ansible playbook that are unique enough that I can't wrap them in a loop. For simplicity, let's assume they are creating three different VM instances in a cloud provider and that each task has a specific configuration
- name: Create VM set A
cloud_provider.vm:
{stuff here}
- name: Create VM set B
cloud_provider.vm:
{stuff here}
- name: Create VM set C
cloud_provider.vm:
{stuff here}
Now this works, and creates the three sets of VM, but I'd like to run them in parallel so I am not waiting as long (the VMs run through several startup processes and it takes a while)
I've looked at
https://docs.ansible.com/ansible/latest/collections/ansible/builtin/async_status_module.html#async-status-module
and
https://docs.ansible.com/ansible/latest/user_guide/playbooks_async.html
but they seem to use loops which I can't do here because the different sets are pretty different.
How/Can I async these tasks?
Q: "They seem to use loops which I can't do here because the different sets are pretty different. How/Can I async these tasks?"
A: You can use a loop to wait for the tasks. For example
- hosts: localhost
tasks:
- shell: 'sleep 5 && echo VMA created'
async: 1000
poll: 0
register: cmd_result_A
- shell: 'sleep 9 && echo VMB created'
async: 1000
poll: 0
register: cmd_result_B
- shell: 'sleep 7 && echo VMC created'
async: 1000
poll: 0
register: cmd_result_C
- async_status:
jid: '{{ item }}'
register: result
until: result.finished
retries: 20
delay: 1
loop:
- "{{ cmd_result_A.ansible_job_id }}"
- "{{ cmd_result_B.ansible_job_id }}"
- "{{ cmd_result_C.ansible_job_id }}"
- debug:
msg: >-
{{ item.start.split(' ')|last }} -
{{ item.end.split(' ')|last }} |
stdout: {{ item.stdout }}
loop: "{{ result.results }}"
gives (abridged)
msg: '13:31:13.477444 - 13:31:18.483527 | stdout: VMA created'
msg: '13:31:13.722892 - 13:31:22.728505 | stdout: VMB created'
msg: '13:31:14.001067 - 13:31:21.008131 | stdout: VMC created'
To simplify the code you might want to try and loop the tasks too. For example
- shell: "{{ item }}"
async: 1000
poll: 0
register: cmd_result
loop:
- 'sleep 5 && echo VMA created'
- 'sleep 9 && echo VMB created'
- 'sleep 7 && echo VMC created'
- async_status:
jid: '{{ item }}'
register: result
until: result.finished
retries: 20
delay: 1
loop: "{{ cmd_result.results|map(attribute='ansible_job_id')|list }}"
Related
I am trying to use ansible to manage VM extensions on our Azure VMs across multiple subscriptions. The implementation of inner and outer loops to correctly access nested lists of dictionaries has me struggling to understand how I can run this async without polling so this doesn't take days to execute (doubt anybody has a maintenance window that big). The biggest issue I am running into is I cannot change extensions on VMs that are not running, so I need to power them on, install or remove the extension, then power those previously powered on devices back off.
Because I have to use separate files to iterate (nested lists of dictionaries are fun), I don't know how to use async to run these deployments concurrently.
This is what I am currently using:
main.yml
---
- name: Get subscription
azure_rm_subscription_info:
register: sub_result
- name: get all vms
azure_rm_virtualmachine_info:
subscription_id: "{{ item.subscription_id }}"
loop: "{{ sub_result.subscriptions }}"
register: all_vms
- include_tasks: loopvms.yml
loop: "{{ all_vms.results }}"
loop_control:
loop_var: vmlist
loopvms.yml
---
# set the subscription ID for current loop
- set_fact:
sub_id: "{{ vmlist.item.subscription_id }}"
- include_tasks: extensions.yml
loop: "{{ vmlist.vms }}"
loop_control:
loop_var: vm
when: vmlist.vms is defined
ignore_errors: true
extensions.yml
---
- debug:
msg: "{{ vm.name }} in subscription {{ sub_id }}"
- name: power on vm if off
azure_rm_virtualmachine:
name: '{{ vm.name }}'
resource_group: '{{ vm.resource_group }}'
subscription_id: "{{ sub_id }}"
started: true
register: vmstate
ignore_errors: true
- name: Install Extension - Windows
azure_rm_virtualmachineextension:
subscription_id: "{{ sub_id }}"
resource_group: "{{ vm.resource_group }}"
virtual_machine_name: "{{ vm.name }}"
name: "{{ extension_name }}"
publisher: "{{ publisher }}"
type_handler_version: 1.0
auto_upgrade_minor_version: true
virtual_machine_extension_type: "{{ extension_type }}"
state: "{{ 'present' if (vm.os_type == 'Windows' and target_os == 'Windows') else 'absent' }}"
ignore_errors: true
- name: Install Extension - Linux
azure_rm_virtualmachineextension:
subscription_id: "{{ sub_id }}"
resource_group: "{{ vm.resource_group }}"
virtual_machine_name: "{{ vm.name }}"
name: "{{ extension_name }}"
publisher: "{{ publisher }}"
type_handler_version: 1.0
auto_upgrade_minor_version: true
virtual_machine_extension_type: "{{ extension_type }}"
state: "{{ 'present' if (vm.os_type == 'Linux' and target_os == 'Linux') else 'absent' }}"
ignore_errors: true
- name: power off vms that were turned on
azure_rm_virtualmachine:
name: '{{ vm.name }}'
resource_group: '{{ vm.resource_group }}'
subscription_id: '{{ sub_id }}'
started: false
when: vmstate.powerstate_change == 'poweron'
ignore_errors: true
I am currently ignoring errors to avoid a deployment failing on one VM preventing the rest of the VMs from getting the deployment.
Since I have to use include_tasks twice to iterate, I am not sure how I can run these in parallel. The way it currently executes the 'extensions.yml' task file is only working with one VM at a time. I need to have the tasks grouped like this though, since I need to keep track of the power state per VM. Is there maybe some other way I should be going about this?
'data_list' consists of the values in the csv file. I want to use the values in 'data_list' to loop through the parameters in the 'Create user' section of the playbook, but I am getting this error after running my playbook:
TASK [Create Multiple Users : Create multiple users] ***************************
fatal: [10.16.220.30]: FAILED! => {"reason": "Vars in a Task must be specified as a dictionary, or a list of dictionaries\n\nThe error appears to be in '/runner/project/Windows AD/roles/Create Multiple Users/tasks/Create_multiple_users.yml': line 14, column 9, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n - \"{{ item.groups }}\"\n vars: data_list\n ^ here\n"}
This is my playbook:
---
- name: Read Users
hosts: localhost
vars:
data_list: []
tasks:
- read_csv:
path: user.csv
key: name
fieldnames: name,firstname,surname,displayName,groups
delimiter: ','
register: userdata
- name: Extract the list
set_fact:
data_list: "{{ data_list + [{ 'name': item.value.name, 'firstname': item.value.firstname, 'surname': item.value.surname, 'displayName': item.value.displayName, 'groups': item.value.groups }] }}"
loop: "{{ userdata.dict|dict2items }}"
- name: Create user accounts
hosts: "{{ hostname }}"
gather_facts: false
any_errors_fatal: false
become: yes
become_method: runas
become_user: admin
roles:
- { role: Create Multiple Users }
- name: Create users
community.windows.win_domain_user:
name: "{{ item.name }}"
firstname: "{{ item.firstname }}"
surname: "{{ item.surname }}"
attributes:
displayName: "{{ item.firstname + ' ' + item.surname }}"
groups:
- "{{ item.groups }}"
vars: data_list
with_items:
- "{{ data_list }}"
What is the correct vars that I should write?
This is the line causing the error in your task
vars: data_list
As mentioned in your error message, the vars section should look like:
vars:
var1: value1
var2: value2
But this is not the only problem in you above script. You are gathering your csv data in a separate play on localhost and setting that info as a fact in variable data_list. When your first play is over, that var will only be known from the localhost target. If you want to reuse it in a second play targeting other hosts, you'll have to get that var from the hostvars magic variable
{{ hostvars.localhost.data_list }}
This is not the best approach here as you can easily shorten your playbook to a single play. The trick here is to delegate your csv gathering task to localhost and set run_once: true so that the registered var is calculated only once and distributed to all hosts with the same value. You can also drop the set fact which basically copies the same key: value to a new var.
Here is an (untested) example playbook to show you the way:
---
- name: Create multiple Windows AD user accounts from CSV
hosts: "{{ hostname }}"
gather_facts: false
tasks:
- name: read csv from localhost (single run same registered var for all hosts)
read_csv:
path: user.csv
key: name
fieldnames: name,firstname,surname,displayName,groups
delimiter: ','
register: userdata
run_once: true
delegate_to: localhost
- name: Create users
community.windows.win_domain_user:
name: "{{ item.name }}"
firstname: "{{ item.firstname }}"
surname: "{{ item.surname }}"
attributes:
displayName: "{{ item.firstname + ' ' + item.surname }}"
groups:
- "{{ item.groups }}"
# This will work on any number of hosts as `userdata`
# now has the same value for each hosts inside this play.
# we just have to extract the values from each keys from
# `userdata` and loop on that list
loop: "{{ userdata.dict | dict2items | map(attribute='value') }}"
I want to extract a list of users created with the following method:
group_vars/users.yml
add_users: "{{ users_all }}"
users_all:
- name: bob
create_home: yes
shell: /bin/bash
password: "{{ bob_user_pass | password_hash('sha512', bob_user_pass_salt) }}"
state: present
- name: jane
create_home: yes
shell: "/bin/bash"
password: "{{ jane_user_pass | password_hash('sha512', jane_user_pass_salt) }}"
state: present
play_create_users.yml
- name: Create Users
ansible.builtin.user:
name: "{{ item.name }}"
create_home: "{{ item.create_home }}"
shell: "{{ item.shell }}"
password: "{{ item.password }}"
state: "{{ item.state }}"
loop: "{{ add_users }}"
register: user_config
user_config
{
- user_config: {
- msg: All items completed
- results: [
- {
- name: bob
- state: present
- move_home: False
- password: NOT_LOGGING_PASSWORD
- changed: False
- home: /home/bob
- shell: /bin/bash
- failed: False
- item: {
- name: bob
- create_home: True
- shell: /bin/bash
- password: $6$xxxxxxxxxxxxxxx
- state: present
}
- ansible_loop_var: item
}
- {
- name: jane
- state: present
- move_home: False
- password: NOT_LOGGING_PASSWORD
- changed: False
- home: /home/jane
- shell: /bin/bash
- failed: False
- item: {
- name: jane
- create_home: True
- shell: /bin/bash
- password: $6$xxxxxxxxxx
- state: present
}
- ansible_loop_var: item
}
]
- skipped: False
- changed: False
}
}
I can get a single result with:
- debug:
var: user_config['results'][0]['name']
{
- user_config['results'][0]['name']: bob
}
Could anyone assist with a method of getting the list of created usernames? I have tried a few methods of iteration over the "user_config" variable without success.
The idea is to get a list that can be used as a loop value in subsequent tasks.
Cheers.
This answer was provided in a comment by #β.εηοιτ.βε.
Create a new fact containing the list of users.
- name: Create User List
set_fact:
user_list: "{{ user_config.results | map(attribute='name') }}"
Use the new fact in a play.
- name: Print User List
debug:
msg: "{{ user_list }}"
Gives the following result from the example above.
{
- msg: [
- bob
- jane
]
}
I need to sum an integer number to a registered value with ansible.. I tried with this but it doesn't works -->
---
- hosts: localhost
gather_facts: false
tasks:
- name: Ricavo ultima linea /etc/passwd
shell: tail -1 /etc/passwd
register: lastuser
ignore_errors: no
- set_fact:
uid: "{{ lastuser.stdout.split(':')[2] }}"
gid: "{{ lastuser.stdout.split(':')[3] }}"
newuid: "{{ uid|int + 5 }}"
newgid: "{{ gid|int + 5 }}"
- debug:
msg: "UID is {{ uid }} GID is {{ gid }}"
- debug:
msg: "NEW UID is {{ newuid }} GID is {{ newgid }}"
but i get -->
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'uid' is undefined\n\nThe error appears to have been in '/tmp/test_ansible/get_lastuser.yaml': line 10, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - set_fact:\n ^ here\n"}
Thanks in advance!
I think the issue is that the uid and gid variables are not available till the completion of set_fact task. It should work if you split the set_fact tasks. Something like below:
- set_fact:
uid: "{{ lastuser.stdout.split(':')[2] }}"
gid: "{{ lastuser.stdout.split(':')[3] }}"
- set_fact:
newuid: "{{ uid|int + 5 }}"
newgid: "{{ gid|int + 5 }}"
- debug:
msg:
- "UID is {{ uid }} GID is {{ gid }}"
- "NEW UID is {{ newuid }} GID is {{ newgid }}"
In Ansible I've used register to save the results of a task in the variable services.
It has this structure:
"stdout_lines": [
"arp-ethers.service \u001b[1;31mdisabled\u001b[0m",
"auditd.service \u001b[1;32menabled \u001b[0m",
"autovt#.service \u001b[1;31mdisabled\u001b[0m",
"blk-availability.service \u001b[1;31mdisabled\u001b[0m"]
and I would like to receive this:
{
"arp-ethers.service": "disabled",
"auditd.service": "enabled",
"autovt#.service": "disabled",
"blk-availability.service":"disabled"
}
I'd like to use a subsequent set_fact task to generate a new variable with a dictionary, but I'm going round in circles with no luck so far.
- name: Collect all services for SYSTEMD
raw: systemctl list-unit-files --type=service --no-pager -l --no-legend`
register: services
changed_when: false
- debug:
var: services
- debug:
msg: "{{ item.split()[0]|to_json }} : {{ item.split()[1]|to_json }}"
with_items:
- "{{ services.stdout_lines }}"
- name: Populate fact list_services for SYSTEMD
set_fact:
cacheable: yes
list_services: "{{ list_services|default({}) | combine ( {item.split()[0]|to_json: item.split()[1]|to_json} ) }}"
with_items: "{{ services.stdout_lines }}"
This return :
FAILED! => {"msg": "|combine expects dictionaries, got u'arp-ethers.service \\x1b[1;31mdisabled\\x1b[0m\\r\\nauditd.service \\x1b[1;32menabled \\x1b[0m\\r\\nautovt#.service \\x1b[1;31mdisabled\\x1b[0m\\r\\nblk-availability.service \\x1b[1;31mdisabled\\x1b[0m\\r\\n'"}
What you want is to switch list-unit-files into json output using --output=json (yes, that's a link to the journalctl man page, because the systemctl one links there)
roughly like this, although I didn't test it:
- name: Collect all services for SYSTEMD
raw: systemctl --output=json list-unit-files --type=service
register: services_json
changed_when: false
- set_fact:
services: '{{ services_json.stdout | from_json }}'
Use service_facts. For example
- service_facts:
- set_fact:
dict_services: "{{ dict(ansible_facts.services|
dict2items|
json_query('[].[key, value.status]')) }}"