Syntax to pass multiple commands (consisting of bash and python ) to the command parameter in DockerOperator
For example if I have mix of command like -> python test.py , mkdir -p /test to be executed inside the docker container
We achieved to run multiple commands with DockerOperator and retrieving the logs by filling the command argument like "bash -c 'command1 && command2 && command3'".
Pass multiple commands to the command parameter separated by &&, like this:
command='(command1) && (command2) && (command3)'
So the full code looks like:
t_docker = DockerOperator(
task_id='docker_command',
image='bde2020/spark-master:latest',
api_version='auto',
auto_remove=True,
command='(python a.py) && (echo "done") && (mkdir /mydirectory) && (echo "directory created"))',
network_mode='bridge'
)
Related
I know that is possible to use && (and) statement to go running multiple commands for a same alias. However for long combinations it loses in readability. For example:
save = !git status && git add -A && git commit -m \"$1\" && git push --force && git log && :
Is there a multi-line way to write it?
Maybe wrapping it with {} for example?
You can use a line escape (\) to break lines like this:
[alias]
save = !git status \
&& git add -A \
&& git commit -m \"$1\" \
&& git push -f \
&& git log -1 \
&& : # Used to distinguish last command from arguments
You can also put multiple statements inside a function like this:
[alias]
save = "!f() { \
git status; \
git add -A; \
git commit -m "$1"; \
git push -f; \
git log -1; \
}; \
f; \
unset f"
See Also: Git Alias - Multiple Commands and Parameters
I'd refrain from writing such extensive aliases in the config file. You can also add new commands by adding an executable file named git-newcommand to your PATH. This could be a Bash script, Python script or even a binary as long as it's executable and named with the prefix "git-".
In case of scripts you've to add the proper Hashbang:
#!/usr/bin/env python
Export the PATH, for example in your home:
export PATH="${PATH}:${HOME}/bin"
This is more modular, portable and easier debuggable.
I have a R-script within which i call a shell script using system command and use the paste command to pass arguments to the shell script ( on unix machine) and i now would like execute the same R-script on a windows machine and am struggling to get it working
Here are the steps i followed
R code
source('C:\\Users\\xxxx\\Documents\\R\\R-3.5.2\\ms\\ms\\MS_Config.R')
if(is.null(git_version) | git_version == "" | length(git_version) == 0){
print('ERROR: EXECUTION STOPPED !!!')
print('PLEASE SPECIFY GITHUB TAG_ID')
stop()
}
print("test4444")
print(enable_data_pull)
print (getwd())
system(paste('C:\\Users\\xxxx\\Documents\\R\\R- 3.5.2\\ms\\ms\\MS_ALLM_Parallel_Runner.sh -c ', num_cores,
'-s ', snapshot_dt,
'-p ' , local_storage_path,
'-t ', tag,
'-g ', git_version,
'-y ', enable_data_pull
))
print ("after shell script execution")
I tried the following, but did not succeed
Installed cygwin and called the rscript from the cygwin terminal(PATH variable is updated to include R and its binaries)
rscript "C:\Users\xxxx\Documents\R\R-3.5.2\ms\ms\MS_Model_Kickoff.R"
Below is the error message that i see after the r-script attempts to run the shell script
'CreateProcess' failed to run 'C:\Users\xxxx\DOCUME~1\R\R-35~1.2\ms\ms\CONRM_~1.SH -c 25 -s 201811 -p C:\Users\xxxx\Documents\Test -t Analytical -g verModelRefit2.2.2 -y N'
what does the above error mean and how do i fix this and execute the shell script within from the R-script on windows machine?r
So I'm not exactly sure whether this is something wrong with optparse-applicative's script or if I'm using it wrong.
In the optparse-applicative readme, it states that programs are made available with automatic completion scripts, with options for zsh. For my program setup:
$> setup --zsh-completion-script `which setup`
Outputs:
#compdef setup
local request
local completions
local word
local index=$((CURRENT - 1))
request=(--bash-completion-enriched --bash-completion-index $index)
for arg in ${words[#]}; do
request=(${request[#]} --bash-completion-word $arg)
done
IFS=$'\n' completions=($( /Users/anrothan/.local/bin/setup "${request[#]}" ))
for word in $completions; do
local -a parts
# Split the line at a tab if there is one.
IFS=$'\t' parts=($( echo $word ))
if [[ -n $parts[2] ]]; then
if [[ $word[1] == "-" ]]; then
local desc=("$parts[1] ($parts[2])")
compadd -d desc -- $parts[1]
else
local desc=($(print -f "%-019s -- %s" $parts[1] $parts[2]))
compadd -l -d desc -- $parts[1]
fi
else
compadd -f -- $word
fi
done
I'm running the following in my zshrc (I use oh-my-zsh, but I removed it and this still happens in a bare-minimum config with only a small PATH addition to get the setup script).
autoload -U +X compinit && compinit
autoload -U +X bashcompinit && bashcompinit
source <(setup --zsh-completion-script `which setup`)
I get the following error several times:
/dev/fd/11:compadd:24: can only be called from completion function
I've run compinit, and the completion script seems to look right to me, and I've looked around but I can't seem to figure out why this error is happening...
You don't need to source zsh-completion scripts, they just need to be added to your fpath parameter.
So just place the output of setup --zsh-completion-script $(which setup) in a file call _setup in $HOME/.config/zsh/completions.
fpath=($HOME/.config/zsh/completions $fpath)
autoload -U compinit && compinit
I'm trying to create an alias to run tests in my Vim. But I get this error
Not an editor command vagrant ssh -c 'cd /srv/www/wp-content/themes/project/ && phpunit'
Here's my .vimrc
command! Runtest execute "cd ~/ToMyProject | vagrant ssh -c 'cd /srv/www/wp-content/themes/project/ && phpunit'"
:execute is for evaluation of Vimscript; it appears you want to run external commands. To do so while observing the commands' output, use :!:
command! Runtest !cd ~/ToMyProject && vagrant ssh -c 'cd /srv/www/wp-content/themes/project/ && phpunit'
Alternatively, you could :call system('cd ~/ToMyProject | ...'); this would still block, but capture and return the output and exit status (which you could discard or inspect in Vimscript).
Is it possible to check if a directory exists and delete if it does,in Unix using a single command? I have situation where I use ANT 'sshexec' task where I can run only a single command in the remote machine. And I need to check if directory exists and delete it...
Why not just use rm -rf /some/dir? That will remove the directory if it's present, otherwise do nothing. Unlike rm -r /some/dir this flavor of the command won't crash if the folder doesn't exist.
Assuming $WORKING_DIR is set to the directory... this one-liner should do it:
if [ -d "$WORKING_DIR" ]; then rm -Rf $WORKING_DIR; fi
(otherwise just replace with your directory)
Try:
bash -c '[ -d my_mystery_dirname ] && run_this_command'
This will work if you can run bash on the remote machine....
In bash, [ -d something ] checks if there is directory called 'something', returning a success code if it exists and is a directory. Chaining commands with && runs the second command only if the first one succeeded. So [ -d somedir ] && command runs the command only if the directory exists.
Here is another one liner:
[[ -d /tmp/test ]] && rm -r /tmp/test
&& means execute the statement which follows only if the preceding
statement executed successfully (returned exit code zero)
I recommend opening documentation of rm command.
If open then you will see that there is a
-f flag does what you want. Example: rm -f -R ./my_folder