I have a yaml config, and I want to replace one of the fields with a new value?
Here is an example of the yaml config:
application: "app_name"
pipeline: "app-pipeline"
parameters:
spinnaker_environment: "production"
perform_cleanup: false
build_image_tag: "foobar"
I want to replace "foobar" under the build_image_tag field and replace it with a different value like "latest". How do I do this?
I tried the following:
sed '/^parameters:/{n;s/build_image_tag:.*/build_image_tag: "latest"/;}' exec.yml
But that didn't seem to work. The yaml config should look like this:
application: "app_name"
pipeline: "app-pipeline"
parameters:
spinnaker_environment: "production"
perform_cleanup: false
build_image_tag: "latest"
With go yq or python yq, the proper tools:
$ yq -i '.parameters.build_image_tag="latest"' file.yaml
application: "app_name"
pipeline: "app-pipeline"
parameters:
spinnaker_environment: "production"
perform_cleanup: false
build_image_tag: "latest"
yq use the same syntax than jq for JSON
the -i is the same as sed -i: replace on the fly
There is an 'S' command in sed for this. It works like so:
# sed 's/pattern/replacement/[g]' - g is an optional flag to replace all instances
# examples:
echo "a b b c" | sed 's/b/d/'
a d b c
echo "a b b c" | sed 's/b/d/g'
a d d c
For your example it would be like this:
sed 's/foobar/latest/' exec.yml
I'd recommend to create a template file like this to simplify things:
cat template.yml
application: "_APP_"
pipeline: "_PIP_"
parameters:
spinnaker_environment: "_ENV_"
perform_cleanup: _CLN_
build_image_tag: "_IMG_"
and change it with sed like so:
sed '
s/_APP_/app_name/;
s/_PIP_/app-pipeline/;
s/_ENV_/production/;
s/_CLN_/false/;
s/_IMG_/latest/;
' template.yml > new.yml
Using vars:
app=app_name
pip=app-pipeline
env=production
cln=false
img=latest
sed "
s/_APP_/$app/;
s/_PIP_/$pip/;
s/_ENV_/$env/;
s/_CLN_/$cln/;
s/_IMG_/$img/;
" template.yml > new.yml
Related
I have a file that has some data I want to get automatically using a script
the file looks like this
var1 = value1
var2 = value2
I am trying using sed
sed -n -e 's/^.*var2\(.*\)/\1/p'
but I cannot manage to remove the spaces and the = sign
In general I do not know how many spaces there are
the expected result would be:
value2
instead I get
= value2
cat file | tr -d ' ' | cut -d'=' -f2
I suppose this:
sed -n -e 's/^.*var2 [[:space:]]*=[[:space:]]*\(.*\)/\1/p' testfile.txt
works
I have a text file that looks like this:
>long_name
AAC-TGA
>long_name2
CCTGGAA
And a list of column numbers: 2, 4, 7. Of course I can have these as a variable like:
cols="2 4 7"
I need to replace every column of the rows that don't start with > with a single character, e.g an N, to result in:
>long_name
ANCNTGN
>long_name2
CNTNGAN
Additional details - the file has ~200K lines. All lines that don't start with > are the same length. Line indices will never exceed the length of the non > lines.
It seems to me that some combination of sed and awk must be able to do this quickly, but I cannot for the life of me figure out how to link it all together.
E.g. I can use sed to work on all lines that don't start with a > like this (in this case replacing all spaces with N's):
sed -i.bak '/^[^>]/s/ /N/g' input.txt
And I can use AWK to replace specific columns of lines as I want to like this (I think...):
awk '$2=N'
But I am struggling to stitch this together
With GNU awk, set i/o field separators to empty string so that each character becomes a field, and you can easily update them.
awk -v cols='2 4 7' '
BEGIN {
split(cols,f)
FS=OFS=""
}
!/^>/ {
for (i in f)
$(f[i])="N"
}
1' file
Also see Save modifications in place with awk.
You can generate a list of replacement commands first and then pass them to sed
$ printf '2 4 7' | sed -E 's|[0-9]+|/^>/! s/./N/&\n|g'
/^>/! s/./N/2
/^>/! s/./N/4
/^>/! s/./N/7
$ printf '2, 4, 7' | sed -E 's|[^0-9]*([0-9]+)[^0-9]*|/^>/! s/./N/\1\n|g'
/^>/! s/./N/2
/^>/! s/./N/4
/^>/! s/./N/7
$ sed -f <(printf '2 4 7' | sed -E 's|[0-9]+|/^>/! s/./N/&\n|g') ip.txt
>long_name
ANCNTGN
>long_name2
CNTNGAN
Can also use {} grouping
$ printf '2 4 7' | sed -E 's|^|/^>/!{|; s|[0-9]+|s/./N/&; |g; s|$|}|'
/^>/!{s/./N/2; s/./N/4; s/./N/7; }
Using any awk in any shell on every UNIX box:
$ awk -v cols='2 4 7' '
BEGIN { split(cols,c) }
!/^>/ { for (i in c) $0=substr($0,1,c[i]-1) "N" substr($0,c[i]+1) }
1' file
>long_name
ANCNTGN
>long_name2
CNTNGAN
I have a line like this:
"LExx 01236723 LE12"
I want to find LE12 and replace LExx with LE12.
Is that possible with sed?
you can try this:
$ echo "LExx 01236723 LE12" | sed 's/LExx\(.*LE\(..\)$\)/LE\2\1/'
LE12 01236723 LE12
explanation
s # substitute
/LExx # find LExx
\(.* # save rest in arg1 (\1)
LE\(..\)$ # find LE and save next 2 char in arg2 (\2)
\) # end arg1
/
LE\2\1 # print LE + arg2 + arg1
/
Assuming both search strings are constants, then using GNU sed (running under bash) with the regexp extension \b (for word boundary):
sed '/LE12/s/\bLExx\b/LE12/' <<< "LExx 01236723 LE12"
Output:
LE12 01236723 LE12
I have a file with some lines:
a
b
c
d
I would like to cat this file into a awk command to produce something like this:
letter is a
letter is b
letter is c
letter is d
using something like this:
cat file.txt | awk 'letter is $1'
But it's not printing out as expected:
$ cat raw.txt | awk 'this is $1'
a
b
c
d
At the moment, you have no { action } block, so your condition evaluates the two empty variables this and is, concatenating them with the first field $1, and checks whether the result is true (a non-empty string). It is, so the default action prints each line.
It sounds like you want to do this instead:
awk '{ print "letter is", $1 }' raw.txt
Although in this case, you might as well just use sed:
sed 's/^/letter is /' raw.txt
This command matches the start of each line and adds the string.
Note that I'm passing the file as an argument, rather than using cat with a pipe.
Not sure if you wanted sed or awk but this is in awk:
$ awk '{print "letter is " $1}' file
letter is a
letter is b
letter is c
letter is d
I want to compare each cell size/length and change its content depending on its length.
The current table is of format
AB
CD
AB
AB
CD
155668/01
AB
1233/10
I want to replace the cells which have length more than "2" to DE.
Output
AB
CD
AB
AB
CD
DE
AB
DE
I tried
awk -F "," '{ if($(#1) > "2") {print "DE"} else {print $1 }}'
It says syntax error.
If I use wc -m in place if $(# the output is same is the input.
The easiest way is to use sed:
sed '/^..$/!s/.*/DE/' file
In awk, you could say:
awk '!/^..$/ { $0 = "DE" } 1' file
In both cases, the idea is the same: if the line does not consist of exactly two characters, replace the whole line with DE. In the case of sed, the whole line is .*, in the case of awk, it is $0.
Try this -
$ awk '{print (length($1)>2?"DE":$1)}' f
AB
CD
AB
AB
CD
DE
AB
DE
The idiomatic way would be:
awk 'length($1) > 2 { $1 = "DE" } 1'