In ansible you can register a variable to capture the output of running a command, specify a changed_when argument, and use that variable to determine if the state changed:
- name: Combine multiple conditions to override 'changed' result
ansible.builtin.command: /bin/fake_command
register: result
ignore_errors: True
changed_when:
- '"ERROR" in result.stderr'
- result.rc == 2
In salt I found that there is a stateful argument to cmd.run but the output appears that it needs to be in a specific json or key=value pair format to do the equivalent. In salt is there a way to do the equivalent of the ansible example to search for the existence of a specific string in the output to determine if the state changed?
You'd need to write a wrapper that returns the stateful-compatible response.
do the thing:
cmd.run:
- name: '/bin/fake_command 2>&1 | grep "ERROR" && echo "changed=true" || true'
- stateful: true
Though outputting "ERROR" seems an odd way for this command to indicate successful changes.
Alternatively, you can write an entirely custom state for more complex logic.
Related
I have input files:
Bob_1.fastq.gz
Bob_2.fastq.gz
Bob_3.fastq.gz
Bob_4.fastq.gz
Ron_1.fastq.gz
Ron_2.fastq.gz
Ron_3.fastq.gz
Ron_4.fastq.gz
I am running demultiplexing and trimming steps in one snakefile, like this:
workdir: "/path/to/dir/"
(SAMPLES,) =glob_wildcards('/path/to/dir/raw/{sample}.fastq.gz')
rule all:
input:
expand("demulptiplex/{sample}.fastq.gz", sample=SAMPLES),
expand("trimmed/{sample}.trimmed.fastq.gz", sample=SAMPLES)
rule sabre:
input:
infile="/path/to/dir/raw/{sample}.fastq.gz",
barcodefile= "files/{sample}.txt"
output:
unknownfile=temp("demulptiplex/unknown_barcode_{sample}.fastq.gz"),
shell:
"""
/Tools/sabre-master2/sabre se -f {input.infile} -b {input.barcodefile} -u {output.unknownfile}
"""
rule trimmomatic_se:
input:
r="{sample}.fastq.gz"
output:
r="trimmed/{sample}.trimmed.fastq.gz"
threads: 10
shell:
"""java -jar /Tools/Trimmomatic-0.36/trimmomatic-0.36.jar SE -threads {threads} {input.r} {output.r} ILLUMINACLIP:/Tools/Trimmomatic-0.36/adapters/TruSeq3-SE.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15 MINLEN:36"""
The demultiplex output files are like this:
Bob_1_CL1.fastq.gz.... Bob_1_CL345.fastq.gz
Bob_2_CL1.fastq.gz.... Bob_1_CL248.fastq.gz
Ron_1_dad1.fastq.gz... Ron_1_dad67.fastq.gz
and so on
So,if I do not specify the demultiplex output file the program would create it by itself. My problem is how to specify/introduce a new wildcard from the output of the previous rule in the next trimming step, as the wildcards are different from initial sample now.
Wildcards just need to be consistent in a rule, not across the workflow. The issue here is that you have a rule generating 'unknown' outputs that you need to process further. For that you need to use checkpoints.
Read through the second block of code about aggregating. Your checkpoint will be demultiplexing and if you don't have any other steps, all will be your aggregate step that calls checkpoints.demultiplex.get. If you search for checkpoint on stackoverflow you will find lots of examples; it's a hard feature to use at first!
Using salt i want to find the attribute(key) and replace it with value based on specific stanza. The attribute(key) is present in multiple times in a file under different stanzas. I want to find my attribute under specific stanza and replace with value.
Example:
output.kafka:
# Boolean flag to enable or disable the output module.
enabled:
I need to find enabled: under output.kafka: and replace it with value. The enabled: attribute present multiple times in my file.
Thanks
Bala.
Salt has a few commands like file.line, file.replace and file.blockreplace that can modify an existing file, but I highly recommend managing the whole file using file.managed. It makes for a less brittle experience.
Here's an example based off your question:
Pillar top file:
cat /srv/pillar/top.sls
base:
'*':
- common
'minion01':
- minion01kafkasettings
Set our pillar data:
cat /srv/pillar/minion01kafkasettings.sls
kafka_output: True
Here's our filebeat template:
cat /srv/salt/filebeat.tmpl
output.kafka:
# Boolean flag to enable or disable the output module.
enabled: {{ pillar.get('kafka_output', True) }}
Here's the filebeat Salt sls file:
cat /srv/salt/filebeat.sls
the_filebeat_file:
file.managed:
- name: /etc/filebeat/filebeat.yml
- template: jinja
- user: root
- group: root
Then we can run the following:
Refresh our pillar data
salt 'minion01' saltutil.refresh_pillar
Then apply the sls file:
salt 'minion01' state.sls filebeat
I have another theory using file.seralize that might work but not in its current state, Maybe Dave could help.
{% set json_data = salt.cp.get_file_str('/etc/filebeat/filebeat.yml') | load_yaml %}
{% do json_data.update({'enabled': pillar.get('kafka_output', True) }) %}
update_config:
file.serialize:
- name: /etc/filebeat/filebeat.yml
- user: root
- group: root
- mode: 644
- formatter: yaml
- dataset: |
{{ json_data | yaml(False)| indent(8) }}
This state should load the whole configuration file then you can modify any of its values based on your pillar setting using the do statement in your case it could be
{% do json_data.update({'enabled': pillar.get('kafka_output', True) }) %}
The config file is populated but not as exepcted as the result will be as following:
'enabled: true
status: active
'
Note there are quotes and the yaml is not intended correctly, is there another way to make it work ? I will update this answer if I found any new results
I'm trying to set up a dictionary as a variable (so I can use it as a Resource and access its values from another file) and there is something that is driving me crazy.
Here is the code I have (just for testing purposes):
*** Settings ***
Documentation Suite description
Library Collections
*** Variables ***
&{SOME DICT} key1=value1 key2=value2
*** Test Cases ***
Dict Test # why $ instead of &?
${RANDOM VAR}= Get From Dictionary ${SOME DICT} key1
Log ${RANDOM VAR} WARN
If I run that, I got the expected result ([ WARN ] value1) BUT the IDE (PyCharm) is complaining about that ${SOME DICT} variable is not defined, and the dictionary declaration is not highlighted the same as variable or a list.
If I change that to &{SOME DICT} the IDE won't complain anymore, but the test fails with the following output:
Dict Test | FAIL |
Keyword 'Collections.Get From Dictionary' got positional argument after named arguments.
That is puzzling me to no end: why I have to use a $ instead of a & if it's a dictionary to make it work? Is there something I am doing wrong and it is just running by luck?
Thanks for any advice or guidance you may have!
Have a look into "Get from Dictionary" libdoc,looks like example is showing the same as your working snippet:
Name: Get From Dictionary
Source: Library (Collections)
Arguments: [dictionary, key]
Returns a value from the given ``dictionary`` based on the given ``key``.
If the given ``key`` cannot be found from the ``dictionary``, this
keyword fails.
The given dictionary is never altered by this keyword.
Example:
| ${value} = | Get From Dictionary | ${D3} | b |
=>
| ${value} = 2
Keyword implementation details are as follows:
try:
return dictionary[key]
except KeyError:
raise RuntimeError("Dictionary does not contain key '%s'." % key)
So indeed, Robot sends representation of dict content and not dict name thus value for key can be returned.
This is the same as direct call in python:
a = {u'key1': u'value1', u'key2': u'value2'}
print(a['key1'])
In the end, libdoc for that KW is not straightforward but your PyCharm plugin for Robot does not work properly in this case.
In RED Robot Editor (Eclipse based), proper case does not rise any warnings in editor, wrong-case provides error marker about arguments (better but still not clear what is exactly wrong. Blame minimalistic libdoc info).
ps. I am lead of RED project to be clear.
Simple Example to Use Key Value Variable in robot framework
Set value to dictionary
Get value from dictionary
&{initValues} Create Dictionary key1=value1 key2=value2
Set To Dictionary ${initValues} key1=newvalue1
Set To Dictionary ${initValues} key2=newvalue2
Set To Dictionary ${initValues} key3=newvalue3
${value} Get From Dictionary ${intialValues} key1
I am running the following test inside Robot Framework:
*** Settings ***
Documentation This initializes testrun.robot
Library Process
Library OperatingSystem
Library XML
Library Collections
Output Is A Valid XML File Against JATS format
Start Process xmllint --dtdvalid http://jats.nlm.nih.gov/archiving/1.1d3/JATS-archivearticle1.dtd ./output/nlm/out.xml shell=True
${result}= Wait For Process timeout=45 secs
Log ${result.stdout}
Log ${result.stderr}
Run Keyword Unless '${result.stderr} == ${EMPTY}' Should Contain ${result.stderr} element xref: validity error : IDREFS attribute rid references an unknown
The variable ${result.stderr} is a string that contains the substring 'element xref: validity error : IDREFS attribute rid references an unknown'. As far as I know, I'm not dealing with any whitespace errors or quotation problems, although I could be wrong. (I've been fiddling around with that for a while now.)
Thanks for the help!
Edit: The log that Robot Framework generates tells me that the process has finished (that's how I know what result.stderr contains.)
Consider this statement:
Run Keyword Unless '${result.stderr} == ${EMPTY}' ...
The keyword will never run because you aren't comparing two variables, you are simply checking whether the string literal string '${result.stderr} == ${EMPTY}' is not empty (and it's not, because it has 28 characters).
It is the same as if you did this in python:
if len('${result.stderr} == ${EMPTY}') > 0:
...
You need to put the single quotes around each variable separately so that you are passing a valid expression to the if statement, rather than a string that looks like a valid expression:
Run Keyword Unless '${result.stderr}' == '${EMPTY}' ...
I have quite extensive salt config and I want to be able to see what has changed. If I just run salt '*' state.highstate I got the whole list with things that were present and not changed - like 3 to 4 screens of log. But I'd really like to see only things that changed in the last job.
It doesn't have to work for the salt call, it can also employ salt-run jobs.lookup_jid.
You can set state_verbose: False in /etc/salt/master or /etc/salt/minion.
If you want to shorten the output to one line per state, set state_output: terse.
You can also pass these filters on command line:
salt --state-output=terse '*' state.highstate
If you only want to see changes, you can use state-output=changes or state-output=mixed. The latter one will show more information on a failure.
See the following answers fore more detail: basepi, psarossy
We've also added state_output: mixed which will give you the same output as terse, except if there's a failure, in which case it will give you the more verbose output.
To actually answer the question, yes, there is an output filter for changes only:
salt '*' state.highstate --state-output=changes
This will display one liners for things that are in the right state and the proper output for the changes. ie:
<...>
Name: /etc/sudoers - Function: file.managed - Result: Clean
Name: /etc/timezone - Function: file.managed - Result: Clean
Name: /etc/pki/tls/certs/logstash-forwarder.crt - Function: file.managed - Result: Clean
Name: /etc/init.d/logstash-forwarder - Function: file.managed - Result: Clean
----------
ID: /etc/logstash-forwarder
Function: file.managed
Result: True
Comment: File /etc/logstash-forwarder updated
Started: 14:14:28.580950
Duration: 65.664 ms
Changes:
----------
diff:
---
+++
## -1,6 +1,6 ##
{
"network": {
- "servers": [ "10.0.0.104:5000" ],
+ "servers": [ "10.0.0.72:5000" ],
"timeout": 15,
"ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"
},
Name: deb http://packages.elasticsearch.org/logstashforwarder/debian stable main - Function: pkgrepo.managed - Result: Clean
Name: logstash-forwarder - Function: pkg.installed - Result: Clean
<...>
There are 2 options, first is to change the state_output in master's configuration file, like mentioned in the accepted answer, and it also possible to override the state output in command line, like:
salt --state-output=mixed \* test.version
As of the following PR that was merged into Salt 2015.8.0 (https://github.com/saltstack/salt/pull/26962) it is now possible to toggle the state_verbose flag from command line when running highstate. This overrides the config you can set in /etc/salt/master that was mentioned in previous answers.
The following command should now display only the changes and errors from a highstate run salt '*' state.highstate --state-verbose=False
You can use the below to shorten the output in one line and then filter that output to show only the changes:
salt -v 'minion' state.highstate test=True --state-output=terse --state-verbose=False