I am trying to retrieve a file from a remote host. However, after looking at some examples on stackoverflow, the bottom two methods result in the follow errors:
- shell: ls -f ubuntu_s*
register: file_name
- fetch: src=/home/ubuntu/{{file_name.stdout_lines}} dest=/home/user
- shell: ls -f ubuntu_s*
register: file_name
- fetch: src={{item}} dest=/home/user
with_items: "{{file_name.stdout_lines}}"
Error:
ERROR! this task 'fetch' has extra params, which is only allowed in the following modules: command, shell, script, include, include_vars, add_host, group_by, set_fact, raw, meta
The error appears to have been in '/home/user/BuildPkg.yml': line 49, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
register: file_name
- fetch: src=/home/ubuntu/{{file_name.stdout_lines}} dest=/home/user
^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
The error appears to have been in '/home/user/BuildPkg.yml': line 49, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
register: file_name
- fetch: src=/home/ubuntu/{{file_name.stdout_lines}} dest=/home/user
^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
Both approaches give the same error. What seems to be wrong?
Try to avoid shell where possible. Modules – this is the Ansible way.
If you need to get a list of files from remote host and fetch them:
- find:
pattern: ubuntu_s*
path: /home/ubuntu/
register: myfiles
- fetch:
src: "{{ item.path }}"
dest: /home/user
flat: yes
with_items: "{{ myfiles.files }}"
The correct way is looping over file globs, like:
- fetch: src={{ item }} dest=/home/user
with_fileglob:
- ubuntu_s*
Note: know what you may face whenever trying to parse ls output
Related
ln -fs /opt/app/$(ls -rt file-*.jar | tail -n1) /opt/app/file.jar
works in bash very well
dir contains
file-1.jar
file-2.jar
file-3.jar
How can I do this in a salt stack state sls formula ?
To achieve this in Saltstack we need roughly two steps:
Get the latest file
Link the file
For the first part, we can use some salt module such as file.find, but I feel the existing logic of using ls -rt is simpler.
So we can use this command to get the latest JAR file into a variable. Then use a Salt state to link the file.
Example:
{% set latest_jar = salt.cmd.run('ls -rt /opt/app/file-*.jar | tail -n1') %}
link-latest-jar:
file.symlink:
- name: /opt/app/file.jar
- target: {{ latest_jar }}
Update:
With newer version of Saltstack we need to use salt.cmd.shell to set the latest_jar variable.
{% set latest_jar = salt['cmd.shell']('ls -rt /opt/app/file-*.jar | tail -n1') %}
link-latest-jar:
file.symlink:
- name: /opt/app/file.jar
- target: {{ latest_jar }}
works for me... thx for help
I actually use jq (1.5) with Windows 10 to Format different json files. I tried today to move the filters to a filter file to cut the length of my cmd commands.
I copied the filter directly from the command with all Quotations but i received an Syntax error. I tried to remove the qotations or Change them to ' but i still receive the Syntax error:
jq: error: syntax error, unexpected IDENT, expecting $end (Windows cmd shell quoting issues?) at <top-level>, line 1:
[.cruises[] | { nid: .cruise_nid, shipcategory: .ship_category, ship: .ship_title, company: .company_title, includeflight: .includes_flight, nights, waypoints: .waypoint_cities, title: .route_title}] C:\import\dreamlines_cruises.json > C:\Import\import_cruises.json
Any tips?
Regards Timo
Your jq filter as given (i.e. without quotation marks) looks fine, so let's assume you have successfully placed the text (hopefully formatted for readability :-) in a file, say format.jq
Then you would run something like this:
jq -f format.jq dreamlines_cruises.json
I use UNIX fairly infrequently so I apologize if this seems like an easy question. I am trying to loop through subdirectories and files, then generate an output from the specific files that the loop grabs, then pipe an output to a file in another directory whos name will be identifiable from the input file. SO far I have:
for file in /home/sub_directory1/samples/SSTC*/
do
samtools depth -r chr9:218026635-21994999 < $file > /home/sub_directory_2/level_2/${file}_out
done
I was hoping to generate an output from file_1_novoalign.bam in sub_directory1/samples/SSTC*/ and to send that output to /home/sub_directory_2/level_2/ as an output file called file_1_novoalign_out.bam however it doesn't work - it says 'bash: /home/sub_directory_2/level_2/file_1_novoalign.bam.out: No such file or directory'.
I would ideally like to be able to strip off the '_novoalign.bam' part of the outfile and replace with '_out.txt'. I'm sure this will be easy for a regular unix user but I have searched and can't find a quick answer and don't really have time to spend ages searching. Thanks in advance for any suggestions building on the code I have so far or any alternate suggestions are welcome.
p.s. I don't have permission to write files to the directory containing the input folders
Beneath an explanation for filenames without spaces, keeping it simple.
When you want files, not directories, you should end your for-loop with * and not */.
When you only want to process files ending with _novoalign.bam, you should tell this to unix.
The easiest way is using sed for replacing a part of the string with sed.
A dollar-sign is for the end of the string. The total script will be
OUTDIR=/home/sub_directory_2/level_2
for file in /home/sub_directory1/samples/SSTC/*_novoalign.bam; do
echo Debug: Inputfile including path: ${file}
OUTPUTFILE=$(basename $file | sed -e 's/_novoalign.bam$/_out.txt/')
echo Debug: Outputfile without path: ${OUTPUTFILE}
samtools depth -r chr9:218026635-21994999 < ${file} > ${OUTDIR}/${OUTPUTFILE}
done
Note 1:
You can use parameter expansion like file=${fullfile##*/} to get the filename without path, but you will forget the syntax in one hour.
Easier to remember are basename and dirname, but you still have to do some processing.
Note 2:
When your script first changes the directory to /home/sub_directory_2/level_2 you can skip the basename call.
When all the files in the dir are to be processed, you can use the asterisk.
When all files have at most one underscore, you can use cut.
You might want to add some error handling. When you want the STDERR from samtools in your outputfile, add 2>&1.
These will turn your script into
OUTDIR=/home/sub_directory_2/level_2
cd /home/sub_directory1/samples/SSTC
for file in *; do
echo Debug: Inputfile: ${file}
OUTPUTFILE="$(basename $file | cut -d_ -f1)_out.txt"
echo Debug: Outputfile: ${OUTPUTFILE}
samtools depth -r chr9:218026635-21994999 < ${file} > ${OUTDIR}/${OUTPUTFILE} 2>&1
done
I want to do something like that:
if file A exists or there is no symlink B, I want to create a symlink B -> A.
For now I have:
B:
file:
- symlink:
- target: A
- exists:
- name: A
But this is bad it checks not the thing I want. How can I achive this simple thing in salt ?
We can use file.directory_exists
{% if not salt['file.directory_exists' ]('/symlink/path/A') %}
symlink:
file.symlink:
- name: /path/to/A
- target: /symlink/path/A
{% endif %}
You should use Dan Garthwaite's excellent answer here as a basis for how to check for the existence of a file. I have modified his solution to answer your question.
{% if 1 == salt['cmd.retcode']('test -f /path/to/A') %}
/path/to/A:
file.symlink:
- target: /symlink/path/A
{% endif %}
/path/to/symlink/B:
file.symlink:
- target: /path/to/target/A
- onlyif:
- test -f /path/to/target/A # check that the target exists
- test ! -L /path/to/symlink/B # check that B is not a symlink
This will require both conditions to be True for the symlink to be created. Note that -L will also return 1 (False) if the file exists but is not a symlink.
From the docs:
The onlyif requisite specifies that if each command listed in onlyif returns True, then the state is run. If any of the specified commands return False, the state will not run.
NOTE: Under the hood onlyif calls cmd.retcode with python_shell=True. This means the commands referenced by onlyif will be parsed by a shell, so beware of side-effects as this shell will be run with the same privileges as the salt-minion. Also be aware that the boolean value is determined by the shell's concept of True and False, rather than Python's concept of True and False.
I'm trying to automate a check for missing routes a Play! web application.
The routing table is in a file in the following format:
GET /home Home.index
GET /shop Shop.index
I've already managed to use my command line-fu to crawl through my code and make a list of all the actions that should be present in the file. This list is in the following format:
Home.index
Shop.index
Contact.index
About.index
Now I'd like to pipe the output of this text into another command that checks if each line is present in the route file. I'm not sure how to proceed though.
The result should be something like this:
Contact.index
About.index
Does someone have a helpful suggestion on how I can accomplish this?
try this line:
awk 'NR==FNR{a[$NF];next}!($0 in a)' routes.txt list.txt
EDIT
if you want the above line to accept list from stdin:
cat list.txt|awk 'NR==FNR{a[$NF];next}!($0 in a)' routes.txt -
replace cat list.txt with your magic command