How to run command in a different folder using EcsOperator Airflow - airflow

The command I want to run in the container is this:
/bin/bash -c "cd /opt/project/A && ls"
Basically I'm using this python code to override the command in the container:
return ECSOperator(
task_id=task_id,
cluster=runOncluster,
task_definition=task_definition,
overrides={
'containerOverrides': [
{
'name': task_definition,
'command': ['/bin/bash', '-c', '"cd', '/opt/project/A', '&&', 'ls"']
}
]
},
region_name=region,
launch_type="EC2",
group=task_definition
)
However I have an error saying:
/bin/bash: -": invalid option
Also I tried this other command, simpler, but I nothing is executed:
['cd', '/opt/project/A', '&&', 'ls']

The one that works is this:
["/bin/bash", "-c", "cd /opt/project/A && ls"]

Related

Starship cross-shell prompt is not working on Mac zsh

I'm using a Mac and followed the instructions to install with homebrew and added eval "$(starship init zsh)" to ~/.zshrc.
However, when I restarted my terminal it still shows the default one.
Here's the content of my .zshrc (added the line at the bottom)
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/opt/anaconda3/bin/conda' 'shell.zsh' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/opt/anaconda3/etc/profile.d/conda.sh" ]; then
. "/opt/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/opt/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
export PATH="$HOME/.rbenv/bin:$PATH"
eval "$(rbenv init -)"
export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$PATH
export GOBIN=$GOPATH/bin
eval "$(starship init zsh)"

lint-staged custom config > error Command failed with exit code 1

I'm following the first example (copy/paste) of custom config file (lint-staged.config.js) for lint-staged packaged from its github README without success. I get error Command failed with exit code 1. always.
I tried this...
I have tried three things, for each case I had my lint-staged.config.js in the root directory.
package.json: result is error Command failed with exit code 1.
"lint-staged": {
"packages/**/*.{ts,tsx}": [
"yarn lint-staged --config ./lint-staged.config.js"
]
},
husky/pre-commit: result is error Command failed with exit code 1.
npx lint-staged --config ../lint-staged.config.js
cmd line: result is error Command failed with exit code 1.
yarn lint-staged --config lint-staged.config.js
The problem
Im just looking for run a custom config file.
The problem is that the execution fails, the error message its related to the command but the command itself its correct as lint-staged [options] (yarn/npx lint-staged -h) then to provide a custom config file it would as lint-staged [--config [path]] but it fails (I even provide all kind of quotes for path).
The issue is that when the module doesn't provide an explicit positive answer to the validation it will always return error Command failed with exit code 1 meaning that the validation has fail.
To properly work as expected it should, in my case:
first, the module had to have a return.
Secondly it should be in form of string array.
Third, the first string of the array had to be terminal-like response as '0' or 'true' of 'false'.
Then, the next following strings could be a message or messages like 'error some A' and 'error some B' and 'error some C' and so on...
For example: ['0', 'error some A', 'error some B', 'error some C']
const path = require("path");
module.exports = (absolutePaths) => {
const cwd = process.cwd();
const relativePaths = absolutePaths.map((file) => path.relative(cwd, file));
console.log("query", relativePaths)
return ['0', 'error some A', 'error some B', 'error some C']
};
This runs ok, but as Andrey Mikhaylov said in this post to run something like
"lint-staged": {
"packages/**/*.{ts,tsx}": [
"yarn lint-staged --config ./lint-staged.config.js"
]
},
If the lint returns an error, It will blow away the staged files causing a regression that will drop the commit completely, which means that all the work will be lost.
I fix this not intended/desired behaviour running the same command yarn lint-staged --config ./lint-staged.config.js but from husky at the pre-commit file as
#!/bin/sh
. "$(dirname "$0")/_/husky.sh"
yarn lint-staged --config ./lint-staged.config.js`

Confusion about the composition of the `cmd` parameter used in Deno.run()

I tried to use Deno as a replacement for shell script, but got stuck.
I attempted to use Deno/Typescript to carry out the equivalent job as this:
docker run \
-d \
-v pgdata:/var/lib/postgresql/data \
--name pg \
-e POSTGRES_PASSWORD=123456 \
--rm \
-p 5432:5432 \
postgres
ts code looks like this:
function runCmd(s: string[]): Deno.Process {
return Deno.run({ cmd: s, stdout: "piped", stderr: "piped" })
}
function runPg() {
const cmd = [
"docker",
`run -d -v ${VOLUME}:/var/lib/postgresql/data --name pg -e POSTGRES_PASSWORD=${PASSWORD} --rm -p 5432:5432 postgres`
];
return runCmd(cmd);
}
add execution bit to this ts file and run it in terminal:
after this, I tried
function runPg() {
const cmd = [
"docker",
"run",
`-d -v ${VOLUME}:/var/lib/postgresql/data --name pg -e POSTGRES_PASSWORD=${PASSWORD} --rm -p 5432:5432 postgres`
];
return runCmd(cmd);
}
move out subcommand run from command options.
I got this:
I guess that Deno.run doesn't simply concatenate the passed-in string of command particles, but I cannot find enough information on this subject in order to fix the issue.
I haven't gone through the rust source code on this API, but I thought it's better to ask for help before trying the hard way.
You need to specify each part of the command as a separate string in the cmd array:
function runPg() {
const cmd = [
"docker",
"run",
"-d",
"-v",
`${VOLUME}:/var/lib/postgresql/data`,
"--name",
"pg",
"-e",
`POSTGRES_PASSWORD=${PASSWORD}`,
"--rm",
"-p",
"5432:5432",
"postgres"
];
return runCmd(cmd);
}
This will send run as the first argument to docker instead of sending run -d … as the first argument.
You can also build your command as a single string and then use split(" ") as long as no arguments contain spaces.
a follow up on my trial-n-error journey on this top.
While reading a book about unix shell programming, it points out a way to help the shell differentiate the space in identifier and the space as delimiter. When one tries to cat a file named a b (there is a space in between), the command should be cat a\ b or, using quotes, cat 'a b'.
This gives me an idea why my command does not work in Deno. See, each item in the cmd string list is an identifier, when I mix up delimiter-space with identifiers, the command is interpreted in the wrong way.
For example.
If I'd like to cat a file named a b. In Deno.run, I need to use { cmd: ["cat", "a b"] }.
If I'd like to cat two files named a and b. In Deno.run, I need to use { cmd: ["cat", "a", "b"] }.
Just remember that space in a command particle counts as a part of that term.

zsh: print time next to command on execute

I want to configure zsh to append the time each command started next to the line command was executed on. For example:
# before I press ENTER
$ ./script
# after I press enter
$ ./script [15:55:58]
Running script...
I came up with the following config (which also colors the timestamp yellow):
preexec () {
TIME=`date +"[%H:%M:%S] "`
echo -e "$1 %{$fg[yellow]%}\033[1A\033[1C${TIME}$reset_color"
}
But it breaks and prints { and % characters on basic commands such as cat and echo. It also breaks on password prompts (macOS terminal). For example with echo:
$ echo "hello" [15:55:58]
hello"hello" %{%}
How can I fix this config?
Thank you.
You inspired me and based on your script I wrote mine. I have tested this on zsh 5.4.smth.
preexec () {
local TIME=`date +"[%H:%M:%S] "`
local zero='%([BSUbfksu]|([FK]|){*})'
local PROMPTLEN=${#${(S%%)PROMPT//$~zero/}}
echo "\033[1A\033[$(($(echo -n $1 | wc -m)+$PROMPTLEN))C $fg[blue]${TIME}$reset_color"
}
In your ~/.zshrc file, put:
function preexec() {
timer=${timer:-$SECONDS}
}
function precmd() {
if [ $timer ]; then
timer_show=$(($SECONDS - $timer))
export RPROMPT="%F{cyan}${timer_show}s %F{$black%}"
unset timer
fi
}
And that should give you something like this:

Unix troubleshooting, missing /etc/init.d file

I am working through this tutorial on daemonizing php scripts. When I run the following Unix command:
. /etc/init.d/functions
#startup values
log=/var/log/Daemon.log
#verify that the executable exists
test -x /home/godlikemouse/Daemon.php || exit 0RETVAL=0
prog="Daemon"
proc=/var/lock/subsys/Daemon
bin=/home/godlikemouse/Daemon.php
start() {
# Check if Daemon is already running
if [ ! -f $proc ]; then
echo -n $"Starting $prog: "
daemon $bin --log=$log
RETVAL=$?
[ $RETVAL -eq 0 ] && touch $proc
echo
fi
return $RETVAL
}
I get the following output:
./Daemon: line 12: /etc/init.d/functions: No such file or directory
Starting Daemon: daemon: unrecognized option `--log=/var/log/Daemon.log'
I looked at my file system and there was no /etc/init.d file. Can anyone tell me what this is and where to obtain it? Also is the absence of that file what's causing the other error?
Separate your args within their own " " double-quotes:
args="--node $prog"
daemon "nohup ${exe}" "$args &" </dev/null 2>/dev/null
daemon "exe" "args"

Resources