I would like to add some additional syntax to zsh. I would like //some/path/to/file to be relative to the root of my source code tree (/src/repositories/projects). So running ls //some/path/to/directory should list files in /src/repositories/projects/some/path/to/directory.
The best way I've found to do this is with a ZLE accept-line widget which rewrites //... paths with the full UNIX path /src/repositories/projects/... and then invokes the command. This works quite well, but it expands the //... syntax inline and stores the expansion in history, which can make searches a bit more complicated. I would like to store the original //... syntax in the history while keep the /src/repositories/projects/... syntax out.
I am able to use print -s to add the original //... syntax to history, but I am having trouble keeping the expanded syntax out of history. Currently my code looks like this:
function accept-line-override() {
# Add untranslated command to history
print -s "$BUFFER"
# Translate //... paths to /src/repositories/projects/...
BUFFER=$(fix-paths "$BUFFER")
# Invoke command with new paths
zle .accept-line
}
zle -N accept-line accept-line-override
This gives me two lines of history for every invoked command:
$ echo //test
/src/repositories/projects/test
$ history | tail -n 2
10020 echo //test
10021 echo /src/repositories/projects/test
Is there a way I can prevent zle .accept-line from adding its command to history? I've seen methods involving history -d but those only seem to work for bash. I also saw somewhere to do:
function accept-line-override() {
# Add untranslated command to history
print -s "$BUFFER"
# Translate //... paths to /src/repositories/projects/...
BUFFER=$(fix-paths "$BUFFER")
# Manually invoke line so it is not added to history
echo
eval $BUFFER
echo
# Reset prompt
BUFFER=""
zle .reset-prompt
}
But this leads to a lot of undesired side-effects, from Emacs refusing to open to auto-completions not clearing before the command is executed. I'd very much like to avoid that complexity if at all possible. Any idea on getting around this?
I have an R script which I'm running in the terminal by firstly generating a .ksh file called myscript.ksh with the following information:
#!/bin/ksh
Rscript myscript.R 'Input1'
and then run the function with
./mycode.ksh
which sends the script to a node on the cluster in our department (the processes that we send to the cluster must be as a .ksh file).
'Input1' is an input argument that is used by the R script to some analysis.
The issue that I now have is that I need to run this script a number of times with different input arguments to the function. One solution is to generate a few .ksh files, such as:
#!/bin/ksh
Rscript myscript.R 'Input2'
and
#!/bin/ksh
Rscript myscript.R 'Input3'
and then execute them seperately, but I was hoping to find a better solution.
Note that I have to do this for 100 different input arguments so it is not realistic to write 100 of these files. Is there a way of generating another file with the information needed to be supplied to the function e.g. 'Input1' 'Input2' 'Input3' and then run myscript.ksh for these individually.
For example, I could have a variable defining the name of the input arguments and then have a loop which would pass it to myscript.ksh. Is that possible?
The reason for running these in this manner is so that each iteration will hopefully be send to a different node on the cluster, thus analysing the data at a much faster rate.
You need to do two things:
Create an array of all your input variables
Loop through the array and initiate all your calls
The following illustrates the concept:
#!/bin/ksh
#Create array of inputs - space separator
inputs=(Input1 Input2 Input3 Input4)
# Loop through all the array items {0 ... n-1}
for i in {0..3}
do
echo ${inputs[i]}
done
This will output all the values in the inputs array.
You just need to replace the contents of the do-loop with:
Rscript myscript.R ${inputs[i]}
Also, you may need to add a ` &' at the end of the Rscript command line to spawn off each Rscript command as a separate thread -- otherwise, the shell would wait for a return from each Rscript command before going onto the next.
EDIT:
Based on your comments, you need to actually generate .ksh scripts to submit to qsub. For this you just need to expand the do loop.
For example:
#!/bin/ksh
#Create array of inputs - space separator
inputs=(Input1 Input2 Input3 Input4)
# Loop through all the array items {0 ... n-1}
for i in {0..3}
do
cat > submission.ksh << EOF
#!/bin/ksh
Rscript myscript.R ${inputs[i]}
EOF
chmod u+x submission.ksh
qsub submission.ksh
done
The EOF defines the beginning and end of what will be taken as input (STDIN) and the output (STDOUT) will written to submission.ksh.
Then submission.ksh is made executable with the chmod command.
And then the script is submitted via qsub. I'll let you fill in any other arguments you need for qsub.
When your script doesn't know all parameters when it starts, you can make a .ksh file called mycode.ksh with the following information:
#!/bin/ksh
if [ $# -ne 1 ]; then
echo "Usage: $0 input"
exit 1
fi
# Or start at the background with nohup .... &, other question
Rscript myscript.R "$1"
and then run the function with
./mycode.ksh inputX
When your application knows all arguments, you can use a loop:
#!/bin/ksh
if [ $# -eq 0 ]; then
echo "Usage: $0 input(s)"
exit 1
fi
for input in $*; do
Rscript myscript.R "${input}"
done
and then run the function with
./mycode.ksh input1 input2 "input with space in double quotes" input4
I realize, when I'm in my terminal, I would expect to press Enter on empty input to make a ls or a git status when I'm on a git repos.
How can I achieve that? I mean, have a custom behavior on Empty input -> Enter in zsh?
EDIT: Thanks for the help. Here's my take with preexec...
precmd() {
echo $0;
if ["${0}" -eq ""]; then
if [ -d .git ]; then
git status
else
ls
fi;
else
$1
fi;
}
On Enter zsh calls the accept-line widget, which causes the buffer to be executed as command.
You can write your own widget in order to implement the behaviour you want and rebind Enter:
my-accept-line () {
# check if the buffer does not contain any words
if [ ${#${(z)BUFFER}} -eq 0 ]; then
# put newline so that the output does not start next
# to the prompt
echo
# check if inside git repository
if git rev-parse --git-dir > /dev/null 2>&1 ; then
# if so, execute `git status'
git status
else
# else run `ls'
ls
fi
fi
# in any case run the `accept-line' widget
zle accept-line
}
# create a widget from `my-accept-line' with the same name
zle -N my-accept-line
# rebind Enter, usually this is `^M'
bindkey '^M' my-accept-line
While it would be sufficient to run zle accept-line only in cases where there actually was a command, zsh would not put a new prompt after the output. And while it is possible to redraw the prompt with zle redisplay, this will probably overwrite the last line(s) of the output if you are using multi-line prompts. (Of course there are workarounds for that, too, but nothing as simple as just using zle accept-line.
Warning: This redfines an (the most?) essential part of your shell. While there is nothing wrong with that per se (else I would not have posted it here), it has the very real chance to make your shell unusable if my-accept-line does not run flawlessly. For example, if zle accept-line were to be missing, you could not use Enter to confirm any command (e.g. to redefine my-accept-line or to start an editor). So please, test it before putting it into your ~/.zshrc.
Also, by default accept-line is bound to Ctrl+J, too. I would recommend to leave it that way, to have an easy way to run the default accept-line.
In my .zshrc I use a combination of precmd and preexec found here:
http://zsh.sourceforge.net/Doc/Release/Functions.html#Hook-Functions
I also find that the git-prompt is super useful:
https://github.com/olivierverdier/zsh-git-prompt
I want to use preexec() to modify certain commands before they are run but I need to be able to evaluate the current entered command. Is there a variable that contains the entire command before it is executed? I know !! is the last command but I need the current line before it's saved to history.
An example of what I want to do would probably help
ls -l /root please
And then I want preexec to see I wrote "please" at the end and replace it with
sudo ls -l /root
I think something like
preexec() {
if [[ $CURRENT_LINE =~ please$ ]]; then
$CURRENT_LINE="sudo ${CURRENT_LINE% please}"
fi
Would work but I can't find a variable in zsh that gives me the correct $CURRENT_LINE
For bonus points I also want to be able to enter please on a line by itself and have it run sudo !! but I could probably do that with some form of alias.
I think it might be better to make a please function that I can pipe a command to but I don't think that'll work as well because the command will run and fail (before piping) before it is run again with sudo.
As far as I know that the preexec is not for the right place to modify the command to be executed though. We can not change the commands to be executed from inside of the preexec function…
Although the actual command to be executed are passed as $1, $2 and $3.
preexec
Executed just after a command has been read and is about to be executed. If the history mechanism is active (and the line was not discarded from the history buffer), the string that the user typed is passed as the first argument, otherwise it is an empty string. The actual command that will be executed (including expanded aliases) is passed in two different forms: the second argument is a single-line, size-limited version of the command (with things like function bodies elided); the third argument contains the full text that is being executed.
-- zshmisc(1) 9.3.1 Hook Functions
For example:
alias ls='ls -sF --color=auto'
preexec () {
print ">>>preexec<<<"
print -l ${(qqq)#}
}
If I have above in ~/.zshrc then I will get follows:
% echo test preexec<Esc-Return>
ls<Return>
;# outputs below
>>>preexec<<<
"echo test preexec
ls"
"echo test preexec; ls -sF --color=auto"
"echo test preexec
ls -sF --color=auto"
test preexec
total 1692
...
You could add your own zle widget functions to the zsh line editor for manipulating the line editor buffer. (zshzle(1))
You could add the zle widget function to change the behavior for hitting Enter.
my-accept-line () {
if [[ "$BUFFER" == *" please" ]]; then
BUFFER="sudo ${BUFFER% please}"
fi
zle .accept-line
}
zle -N accept-line my-accept-line
The above snippets changes the functionality for accept-line from the built-in behavior to my-accept-line defined here.
Adding the abbreviations also could help which is described below:
Cloning vim's abbreviation feature
-- “examples:zleiab [ZshWiki]” - http://zshwiki.org/home/examples/zleiab
Is there an alternative to tee which captures standard output and standard error of the command being executed and exits with the same exit status as the processed command?
Something like the following:
eet -a some.log -- mycommand --foo --bar
Where "eet" is an imaginary alternative to "tee" :) (-a means append and -- separates the captured command). It shouldn't be hard to hack such a command, but maybe it already exists and I'm not aware of it?
This works with Bash:
(
set -o pipefail
mycommand --foo --bar | tee some.log
)
The parentheses are there to limit the effect of pipefail to just the one command.
From the bash(1) man page:
The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully.
I stumbled upon a couple of interesting solutions at Capture Exit Code Using Pipe & Tee.
There is the $PIPESTATUS variable available in Bash:
false | tee /dev/null
[ $PIPESTATUS -eq 0 ] || exit $PIPESTATUS
And the simplest prototype of "eet" in Perl may look as follows:
open MAKE, "command 2>&1 |" or die;
open (LOGFILE, ">>some.log") or die;
while (<MAKE>) {
print LOGFILE $_;
print
}
close MAKE; # To get $?
my $exit = $? >> 8;
close LOGFILE;
Here's an eet. Works with every Bash I can get my hands on, from 2.05b to 4.0.
#!/bin/bash
tee_args=()
while [[ $# > 0 && $1 != -- ]]; do
tee_args=("${tee_args[#]}" "$1")
shift
done
shift
# now ${tee_args[*]} has the arguments before --,
# and $* has the arguments after --
# redirect standard out through a pipe to tee
exec | tee "${tee_args[#]}"
# do the *real* exec of the desired program
exec "$#"
(pipefail and $PIPESTATUS are nice, but I recall them being introduced in 3.1 or thereabouts.)
This is what I consider to be the best pure-Bourne-shell solution to use as the base upon which you could build your "eet":
# You want to pipe command1 through command2:
exec 4>&1
exitstatus=`{ { command1; echo $? 1>&3; } | command2 1>&4; } 3>&1`
# $exitstatus now has command1's exit status.
I think this is best explained from the inside out – command1 will execute and print its regular output on stdout (file descriptor 1), then once it's done, echo will execute and print command1's exit code on its stdout, but that stdout is redirected to file descriptor three.
While command1 is running, its stdout is being piped to command2 (echo's output never makes it to command2 because we send it to file descriptor 3 instead of 1, which is what the pipe reads). Then we redirect command2's output to file descriptor 4, so that it also stays out of file descriptor one – because we want file descriptor one clear for when we bring the echo output on file descriptor three back down into file descriptor one so that the command substitution (the backticks) can capture it.
The final bit of magic is that first exec 4>&1 we did as a separate command – it opens file descriptor four as a copy of the external shell's stdout. Command substitution will capture whatever is written on standard out from the perspective of the commands inside it – but, since command2's output is going to file descriptor four as far as the command substitution is concerned, the command substitution doesn't capture it – however, once it gets "out" of the command substitution, it is effectively still going to the script's overall file descriptor one.
(The exec 4>&1 has to be a separate command to work with many common shells. In some shells it works if you just put it on the same line as the variable assignment, after the closing backtick of the substitution.)
(I use compound commands ({ ... }) in my example, but subshells (( ... )) would also work. The subshell will just cause a redundant forking and awaiting of a child process, since each side of a pipe and the inside of a command substitution already normally implies a fork and await of a child process, and I don't know of any shell being coded to recognize that it can skip one of those forks because it's already done or is about to do the other.)
You can look at it in a less technical and more playful way, as if the outputs of the commands are leapfrogging each other: command1 pipes to command2, then the echo's output jumps over command2 so that command2 doesn't catch it, and then command2's output jumps over and out of the command substitution just as echo lands just in time to get captured by the substitution so that it ends up in the variable, and command2's output goes on its way to the standard output, just as in a normal pipe.
Also, as I understand it, at the end of this command, $? will still contain the return code of the second command in the pipe, because variable assignments, command substitutions, and compound commands are all effectively transparent to the return code of the command inside them, so the return status of command2 should get propagated out.
A caveat is that it is possible that command1 will at some point end up using file descriptors three or four, or that command2 or any of the later commands will use file descriptor four, so to be more hygienic, we would do:
exec 4>&1
exitstatus=`{ { command1 3>&-; echo $? 1>&3; } 4>&- | command2 1>&4; } 3>&1`
exec 4>&-
Commands inherit file descriptors from the process that launches them, so the entire second line will inherit file descriptor four, and the compound command followed by 3>&1 will inherit the file descriptor three. So the 4>&- makes sure that the inner compound command will not inherit file descriptor four, and the 3>&- makes sure that command1 will not inherit file descriptor three, so command1 gets a 'cleaner', more standard environment. You could also move the inner 4>&- next to the 3>&-, but I figure why not just limit its scope as much as possible.
Almost no programs uses pre-opened file descriptor three and four directly, so you almost never have to worry about it, but the latter is probably best to keep in mind and use for general-purpose cases.
{ mycommand --foo --bar 2>&1; ret=$?; } | tee -a some.log; (exit $ret)
KornShell, all in one line:
foo; RET_VAL=$?; if test ${RET_VAL} != 0;then echo $RET_VAL; echo Error occurred!>/tmp/out.err;exit 2;fi |tee >>/tmp/out.err ; if test ${RET_VAL} != 0;then exit $RET_VAL;fi
#!/bin/sh
logfile="$1"
shift
exec 2>&1
exec "$#" | tee "$logfile"
Hopefully this works for you.
Assuming Bash or Z shell (zsh),
my_command >>my_log 2>&1
N.B. The sequence of redirection and duplication of standard error onto standard output is significant!
I didn't realise you wanted to see the output on screen as well. This will of course direct all output to the file my_log.