I reduced my problem to this minimal example:
# This is script myscript
echo $ZSH_VERSION
zparseopts -D vxa:=opt_a vx:=opt_b
echo $1
I'm calling the script with
zsh myscript -vxa xx -vx zz a
I would expect that after processing the options, $1 would output a, but I get
5.8
xx
being printed. Why is this the case?
I am aware about a problem with my script, which is that the option vx is a prefix of vxa, and when I modify the script to
zparseopts -D vxa:=opt_a vxb:=opt_b
and call it accordingly, I indeed get the expected result (a) in the output. So, I can fix my script by just renaming the options so that neither one is a prefix of a different option.
However, I also would like to understand, why I see the xx with my original code. There is no error in the invocation (if I replace the -D by -F, I don't get an error message). I could understand it, if my original script would simply fill the wrong variable (taking vx as an abbreviation for vxa), but I don't understand, why the script silently stops parsing after consuming the -vxa option, but not even picking up the mandatory argument xx. What is going on here, and why?
Yes it's due to the overlapping names + ambiguity regarding how -vxa should be parsed vs -vx a (the issue being that both are being taken as a -vx option with option-arg a - its mandatory arg has been consumed, and the unhyphenated xx looks like a non-option so parsing stops without any error).
man zshmodules has some useful info (bold emphasis added):
In all cases, option-arguments must appear either immediately following the option in the same positional parameter or in the next one. ...
When the names of two options that take no arguments overlap, the longest one wins, so that parsing for the specs -foo -foobar (for example) is unambiguous. However, due to the aforementioned handling of option-arguments, ambiguities may arise when at least one overlapping spec takes an argument, as in -foo: -foobar. In that case, the last matching spec wins.
If the specs are swapped around so that the shorter of the overlapping names are placed before longer ones (i.e. zparseopts -D vx:=opt_b vxa:=opt_a) it should work as you expect. For example:
#!/bin/zsh -
echo $ZSH_VERSION
zparseopts -D vx:=opt_b vxa:=opt_a
print -r -- arg:$^# opt_a:$^opt_a opt_b:$^opt_b
$ zsh thatScript -vxa xx -vx zz a
5.9
arg:a opt_a:-vxa opt_a:xx opt_b:-vx opt_b:zz
Related
I have a Unix ksh script that has been in daily use for years (kicked off at night by the crontab). Recently one function in the script is behaving erratically as never happened before. I tried various ways to find out why, but have no success.
The function validates an input string, which is supposed to be a string of 10 numeric characters. The function checks if the string length is 10, and whether it contains any non-numeric characters:
#! /bin/ksh
# The function:
is_valid_id () {
# Takes one argument, which is the ID being tested.
if [[ $(print ${#1}) -ne 10 ]] || print "$1" | /usr/xpg4/bin/grep -q [^0-9] ; then
return 1
else
return 0
fi
}
cat $input_file | while read line ; do
id=$(print $line | awk -F: '{print $5}')
# Calling the function:
is_valid_id $id
stat=$?
if [[ $stat -eq 1 ]] ; then
print "The ID $id is invalid. Request rejected.\n" >> $ERRLOG
continue
else
...
fi
done
The problem with the function is that, every night, out of scores or hundreds of requests, it finds the IDs in several requests as invalid. I visually inspected the input data and found that all the "invalid" IDs are actually strings of 10 numeric characters as should be. This error seems to be random, because it happens with only some of the requests. However, while the rejected requests persistently come back, it is consistently the same IDs that are picked out as invalid day after day.
I did the following:
The Unix machine has been running for almost a year, therefore might need to be refreshed. The system admin to reboot the machine at my request. But the problem persists after the reboot.
I manually ran exactly the same two tests in the function, at command prompt, and the IDs that have been found invalid at night are all valid.
I know the same commands may behave differently invoked manually or in a script. To see how the function behaves in script, the above code excerpt is the small script I ran to reproduce the problem. And indeed, some (though not all) of the IDs found to be invalid at night are also found invalid by the small trouble-shooting script.
I then modified that troubleshooting script to run the two tests one at a time, and found it is the /usr/xpg4/bin/grep -q [^0-9] test that erroneously finds some of the ID as containing non-numeric character(s). Well, the IDs are all numeric characters, at least visually.
I checked if there is any problem with the xpg4 grep command file (ls -l /usr/xpg4/bin/grep), to see if it is put there recently. But its timestamp is year 2005 (this machine runs Solaris 10).
Knowing that the data comes from a central ERP system, to which data entry is performed from different locations using all kinds of various terminal machines running all kinds of possible operating systems that support various character sets and encodings. The ERP system simply allows them. But can characters from other encodings visually appear as numeric characters but the encoded values are not as the /usr/xpg4/bin/grep command expects to be on our Unix machine? I tried the od (octal dump) command but it does not help me much as I am not familiar with it. Maybe I need to know more about od for solving this problem.
My temporary work-around is omitting the /usr/xpg4/bin/grep -q [^0-9] test. But the problem has not been solved. What can I try next?
Your validity test function happens to be more complicated than it should be. E.g. why do you use a command substitution with print for ${#1}? Why don't you use ${#1} directly? Next, forking grep to test for a non-number is a slow and expensive operation. What about this equivalent function, 100% POSIX and blazingly fast:
is_valid_id () {
# Takes one argument, which is the ID being tested.
if test ${#1} -ne 10; then
return 1 # ID length not exactly 10.
fi
case $1 in
(*[!0-9]*) return 1;; # ID contains a non-digit.
(*) return 0;; # ID is exactly 10 digits.
esac
}
Or even more simple, if you don't mind repeating yourself:
is_valid_id () {
# Takes one argument, which is the ID being tested.
case $1 in
([0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]) # 10 digits.
return 0;;
(*)
return 1;;
esac
}
This also avoids your unquoted use of a grep pattern, which is error-prone in the presence of one-character file names. Does this work better?
I am attempting to bend zsh, my shell of choice, to my will, and am completely at a loss on the syntax and operation of completions.
My use case is this: I wish to have completions for 'ansible-playbook' under the '-e' option support three variations:
Normal file completion: ansible-playbook -e vars/file_name.yml
Prepended file completion: ansible-playbook -e #vars/file_name.yml
Arbitrary strings: ansible-playbook -e key=value
I started out with https://github.com/zsh-users/zsh-completions/blob/master/src/_ansible-playbook which worked decently, but required modifications to support the prefixed file pathing. To achieve this I altered the following lines (the -e line):
...
"(-D --diff)"{-D,--diff}"[when changing (small files and templates, show the diff in those. Works great with --check)]"\
"(-e --extra-vars)"{-e,--extra-vars}"[EXTRA_VARS set additional variables as key=value or YAML/JSON]:extra vars:(EXTRA_VARS)"\
'--flush-cache[clear the fact cache]'\
to this:
...
"(-D --diff)"{-D,--diff}"[when changing (small files and templates, show the diff in those. Works great with --check)]"\
"(-e --extra-vars)"{-e,--extra-vars}"[EXTRA_VARS set additional variables as key=value or YAML/JSON]:extra vars:__at_files"\
'--flush-cache[clear the fact cache]'\
and added the '__at_files' function:
__at_files () {
compset -P #; _files
}
This may be very noobish, but for someone that has never encountered this before, I was pleased that this solved my problem, or so I thought.
This fails me if I have multiple '-e' parameters, which is totally a supported model (similar to how docker allows multiple -v or -p arguments). What this means is that the first '-e' parameter will have my prefixed completion work, but any '-e' parameters after that point become 'dumb' and only allow for normal '_files' completion from what I can tell. So the following will not complete properly:
ansible-playbook -e key=value -e #vars/file
but this would complete for the file itself:
ansible-playbook -e key=value -e vars/file
Did I mess up? I see the same type of behavior for this particular completion plugin's '-M' option (it also becomes 'dumb' and does basic file completion). I may have simply not searched for the correct terminology or combination of terms, or perhaps in the rather complicated documentation missed what covers this, but again, with only a few days experience digging into this, I'm lost.
If multiple -e options are valid, the _arguments specification should start with * so instead of:
"(-e --extra-vars)"{-e,--extra-vars}"[EXTR ....
use:
\*{-e,--extra-vars}"[EXTR ...
The (-e --extra-vars) part indicates a list of options that can not follow the one being specified. So that isn't needed anymore because it is presumably valid to do, e.g.:
ansible-playbook -e key-value --extra-vars #vars/file
zsh is great but its completion system is very diverse. And the documentation lacks good examples. Is there a template for completing for a specific application. The completion would get its match data from a file, separated by newlines?
I tried modifying an older example of mine that takes match data "live":
~ % cat .zsh/completers/_jazzup
#compdef jazz_up
_arguments "2: :(`mpc lsplaylists|sed -e 's# #\\\\ #g'`)"
I could supply cat my_file there instead of mpc invocation and so on but would there be a more elegant way to do this simple task? And that completion there is placement-specific: can you provide an example where zsh would attempt to complete at any point after the program name is recognized?
The match data will have whitespaces and so on, the completion should escape the WS. Example of that:
Foo bar
Barbaric
Get it (42)
Now if that completion would be configured for a command Say, we should get this kind of behaviour out of zsh:
$ Say Fo<TAB>
$ Say Foo\ bar
$ Say Ge<TAB>
$ Say Get\ it\ \(42\)
Simple completion needs are better addressed with _describe, it pairs an array holding completion options and a description for them (you can use multiple array/description pairs, check the manual).
(_arguments is great but too complex.)
[...]
First create a file
echo "foo\nbar\nbaz\nwith spac e s\noh:noes\noh\:yes" >! ~/simple-complete
Then create a file _simple somewhere in your $fpath:
#compdef simple
# you may wish to modify the expansion options here
# PS: 'f' is the flag making one entry per line
cmds=( ${(uf)"$(< ~/simple-complete)"} )
# main advantage here is that it is easy to understand, see alternative below
_describe 'a description of the completion options' cmds
# this is the equivalent _arguments command... too complex for what it does
## _arguments '*:foo:(${cmds})'
then
function simple() { echo $* }
autoload _simple # do not forget BEFORE the next cmd!
compdef _simple simple # binds the completion function to a command
simple [TAB]
it works. Just make sure the completion file _simple is placed somewhere in your fpath.
Notice that : in the option list is supposed to be used for separating an option from their (individual) description (oh:noes). So that won't work with _describe unless you quote it (oh\:yes). The commented out _arguments example will not use the : as a separator.
Without changing anything further in .zshrc (I already have autoload -Uz compinit
compinit) I added the following as /usr/local/share/zsh/site-functions/_drush
#compdef drush
_arguments "1: :($(/usr/local/bin/aliases-drush.php))"
Where /usr/local/bin/aliases-drush.php just prints a list of strings, each string being a potential first argument for the command drush. You could use ($(< filename)) to complete from filename.
I based this on https://unix.stackexchange.com/a/458850/9452 -- it's surprising how simple this is at the end of the day.
I'm starting to think I have broken GNU make. Its behavior seems to have become unpredictable. Do I get a prize for that?
In my commands section, I have been using the '#' character to suppress unwanted output. In one makefile, I removed the '#'s for one run, to verify it was doing what I wanted. Since I put the '#'s back in, I'm getting shell errors:
#mkdir: not found
What in the world is that about? What could make the '#' stop being recognized as a standard command modifier?
Grrrr! I hates that potto! (from the cover of Managing projects with GNU make)
The # character is only valid at the beginning of a line executed by make. If you have continuation characters such as \ ending the previous line, this won't be executed by make, it will be executed by the shell. The shell won't know what to do with the # sign, hence your error.
Additionally if you use .ONESHELL you should note that command prefix will only be valid on the very first line of the recipe, after that it will throw an error since all the lines are passed to one shell for execution.
The same rules apply to the other prefix operator (-) as well.
I am using the maven plugin appassembler to generate a unix script. In its tag, I put sth like:
<commandLineArguments>
<commandLineArgument>$1</commandLineArgument>
<commandLineArgument>$2</commandLineArgument>
<commandLineArgument>$3</commandLineArgument>
</commandLineArguments>
The resultant script, however, shows
$1 $2 $3 "$#"
I don't know where the last one came from, it therefore repeat the first 3 arguments.
Mojo's AppAssembler Maven Plugin generates a script that always appends all the command line arguments provided to the script onto the JVM's launch command. Thus if you did nothing, the "$#" will be the last thing on the JVM command used to start the program.
The <commandLineArguments> tag is used to inject additional command line arguments before the ARGLIST matcher.
It seems (to me) that you think you needed to add the positional markers in order to get the parameters passed through, hence the snippet you were adding. That is both:
Unnecessary, as by default the plugin generates a script that passes all required parameters.
Actually a potential bug, as what you have configured does not handle argument quoting and escaping correctly.
With regard to the second point consider the case where the second parameter is the name of a file that contains a space charater. If I launch the script for you program like so
$ bin/foo.sh Document.txt Document\ 2.txt "Copy of Document 3.txt" Doc4.txt
you will actually see the following being passed through to your Java program with the configuration you provided:
Document.txt (all of $1)
Document ($2 is expanded, but not quoted so now gets re-evaluated)
2.txt
Copy ($3 is expanded, but not quoted, so also gets re-evaluated, spaces seen as argument separator again)
of
Document
3.txt
Document.txt (now the ARGLIST matcher provides everything correctly)
Document 2.txt
Copy of Document 3.txt
Doc4.txt
The solution is simple. Stop trying to configure something you don't need to configure!