sample entry:
post_content: " some <strong >blablablabla</strong> text in <html>"
post_title: Kontakt
post_password:
post_name: kontakt
question:
i have an yaml-file with entries like above and i like to parse the content of post_content with cat & grep and pipe it in to different file.
$ cat posts.yaml | grep post_content >> different-file.yaml
This works. very well :) but in this way i only excude all post_content from the *posts.yaml
on top of it i like to separate each post_content in to separate filea named like post_name.yaml - i think its possible to do with some sed-foo merge this in one line of shell comand. but atm i have no idea to do so.
Try:
awk '/post_content:/{content=$0} /post_name:/{print content>$2".yaml"; close($2".yaml")}' posts.yaml
Example
Consider this test file:
$ cat posts.yaml
post_content: " some <strong >blablablabla</strong> text in <html>"
post_title: Kontakt
post_password:
post_name: kontakt
post_content: " some other text in <html>"
post_title: Kontakt
post_password:
post_name: contact
We then run:
awk '/post_content:/{content=$0} /post_name:/{print content>$2".yaml"; close($2".yaml")}' posts.yaml
After this command is run, there will, in addition to posts.yaml, be two new files in the current directory:
$ ls
contact.yaml kontakt.yaml posts.yaml
The contents of the new files are:
$ cat kontakt.yaml
post_content: " some <strong >blablablabla</strong> text in <html>"
$ cat contact.yaml
post_content: " some other text in <html>"
How it works
/post_content:/{content=$0}
Every time that we reach a line that contains post_content:, we save the line in variable content.
/post_name:/{print content>$2".yaml"; close($2".yaml")}
Every time that we reach a line that contains post_name:, we print the variable content to a file whose name is given by the second field on the line followed by.yaml`.
Related
I am getting input data in file with "|" delimeter but some of records breaking and coming as two records . below is the example ,
for most of description fields data is coming like this only, actually these are entering through web when user press enter while adding comment it going to newline , can you please let me know how to handle this.
I want this record in single line,
"2016-03-03 22:26:20|0|I|NOT SET ||||||||||||||||2015-02-12-04.34.38.734657|2015-02-13| |0|METER FAILURE >30 DAYS"
" |259000-056608 |TRBLRPRT|BDMTRRPL| || |0||| "
echo "$yourdata" | tr -d '\r\n'
Or
cat filename.txt | tr -d '\r\n'
To update file:
datastr=$(cat filename.txt | tr -d '\r\n')
then
echo -n "$datastr" > filename.txt
Hope it helps
This substitutes a space for carriage returns or line feeds
cat test.txt | tr ["\n","\r"] ' '
I am trying to search for a particular string in a Unix file from each and every line and error out those records. Can someone let me how can I improve my code which is as below. Also please share your thoughts if you have a better solution.
v_filename=$1;
v_new_file="new_file";
v_error_file="error_file";
echo "The input file name is $var1"
while read line
do
echo "Testing $line"
v_cnt_check=`grep ',' $line | wc -l`
echo "Testing $v_cnt_check"
# if [ $v_cnt_check > 2 ]; then
# echo $line >> $v_error_file
# else
# echo $line >> $v_new_file
# fi
done < $v_filename
Input:
1,2,3
1,2,3,4
1,2,3
Output:
(New file)
1,2,3
1,2,3
(Error file)
1,2,3,4
awk -F ',' -v new_file="$v_new_file" -v err_file="$v_error_file" \
'BEGIN { OFS="," }
NF == 3 { print >new_file }
NF != 3 { print >err_file }' $v_filename
The first line sets the file name variables and sets the field separator to comma. The second line sets the output field separator to comma too. The third line prints lines with 3 fields to the new file; the fourth line prints lines with other than 3 fields to the error file.
Note that your code would be excruciatingly slow on big files because it executes two processes per line. This code has only one process operating on the whole file — which will be really important if the input grow to thousand or millions or more lines.
From the grep manpage:
General Output Control
-c, --count
Suppress normal output; instead print a count of matching lines for each input file. With the -v, --invert-match option (see below), count non-
matching lines. (-c is specified by POSIX.)
You could do something like:
grep --count "your pattern" v_filename
to get the number of occurrences. If you just want the number of lines with your pattern, replace the grep shown above with:
grep "your pattern" v_filename | wc -l
I have two files System_Names & system_appendix_names . I want to Concatenate each & every line of one file to other file`s line and save output to other file.
root#bt:~/kevin/new# cat system_appendix_names
adm
-adm
_adm
root#bt:~/kevin/new#cat System_Names
help
not
now
give
you
haha
what
where
if
I made following script.
#!/usr/bin/env bash
cat System_Names | while read line1
do
cat system_appendix_names | while read line2
do
out="${line1}${line2}"
echo "$out" >> 1.txt
done
done
Output of script:
root#bt:~/kevin/new# cat 1.txt
helpadm
help-adm
help_adm
notadm
not-adm
not_adm
nowadm
now-adm
now_adm
giveadm
give-adm
give_adm
youadm
you-adm
you_adm
hahaadm
haha-adm
haha_adm
whatadm
what-adm
what_adm
whereadm
where-adm
where_adm
ifadm
if-adm
if_adm
Above script work for small amount of lines in file.Actually i have files which have more lines. SO i tried it, but can not Concatenate string line by line.
i want to set the value of inputLineNumber to 20. I tried checking if no value is given by user by [[-z "$inputLineNumber"]] and then setting the value by inputLineNumber=20. The code gives this message ./t.sh: [-z: not found as message on the console. How to resolve this? Here's my full script as well.
#!/bin/sh
cat /dev/null>copy.txt
echo "Please enter the sentence you want to search:"
read "inputVar"
echo "Please enter the name of the file in which you want to search:"
read "inputFileName"
echo "Please enter the number of lines you want to copy:"
read "inputLineNumber"
[[-z "$inputLineNumber"]] || inputLineNumber=20
for N in `grep -n $inputVar $inputFileName | cut -d ":" -f1`
do
LIMIT=`expr $N + $inputLineNumber`
sed -n $N,${LIMIT}p $inputFileName >> copy.txt
echo "-----------------------" >> copy.txt
done
cat copy.txt
Changed the script after suggestion from #Kevin. Now the error message ./t.sh: syntax error at line 11: `$' unexpected
#!/bin/sh
truncate copy.txt
echo "Please enter the sentence you want to search:"
read inputVar
echo "Please enter the name of the file in which you want to search:"
read inputFileName
echo Please enter the number of lines you want to copy:
read inputLineNumber
[ -z "$inputLineNumber" ] || inputLineNumber=20
for N in $(grep -n $inputVar $inputFileName | cut -d ":" -f1)
do
LIMIT=$((N+inputLineNumber))
sed -n $N,${LIMIT}p $inputFileName >> copy.txt
echo "-----------------------" >> copy.txt
done
cat copy.txt
Try changing this line from:
[[-z "$inputLineNumber"]] || inputLineNumber=20
To this:
if [[ -z "$inputLineNumber" ]]; then
inputLineNumber=20
fi
Hope this helps.
Where to start...
You are running as /bin/sh but trying to use [[. [[ is a bash command that sh does not recognize. Either change the shebang to /bin/bash (preferred) or use [ instead.
You do not have a space between [[-z. That causes bash to read it as a command named [[-z, which clearly doesn't exist. You need [[ -z $inputLineNumber ]] (note the space at the end too). Quoting within [[ doesn't matter, but if you change to [ (see above), you will need to keep the quotes.
Your code says [[-z but your error says [-z. Pick one.
Use $(...) instead of `...`. The backticks are deprecated, and $() handles quoting appropriately.
You don't need to cat /dev/null >copy.txt, certainly not twice without writing to it in-between. Use truncate copy.txt or just plain >copy.txt.
You seem to have inconsistent quoting. Quote or escape (\x) anything with special characters (~, `, !, #, $, &, *, ^, (), [], \, <, >, ?, ', ", ;) or whitespace and any variable that could have whitespace. You don't need to quote string literals with no special characters (e.g. ":").
Instead of LIMIT=`expr...`, use limit=$((N+inputLineNumber)).
I grep a pattern from a directory and the 4 lines before that pattern, I need to further grep the top line from each result , but not getting how to do .
Please suggest regarding this.
The problem explained with example :
in a directory 'direktory'
there are multiple files with different name like 20130611 and 2013400 etc..
the data wrote in the files, which I am interested in is like this :
[
My name is
.....
......
......
Name has been written above
]
now in every instance "Name has been written above" is written in the unit of lines but the value keep on changing in place of "My name is" so I want to grep this particular line from every occurrence .
Please suggest some method to get the result.
Thanks in advance.
a#x:/tmp$ cat namefile
[
My name is
.....
......
......
Name has been written above
]
a#x:/tmp$ cat namefile | grep -B 4 "Name has been written above" | head -1
My name is
Where "4" can be replaced by N i.e. number of lines the target data lies above the grepped line
Try something like
for file in $(ls <wherever>)
do
# Tell the user which file we're looking at
echo ""
echo $file
echo ""
# Output the first line of the file
head -1 $file
# Output the line continaing <pattern> and the four
# preceding lines
<your grep command here>
done