I am tracking some keywords on Twitter using the command below. I want to print just the "screen_name" property of the tweet author. I could get the command below working but want to remove "quotes" from the author screen_name. How could I do this?
curl -N -d #tracking http://stream.twitter.com/1/statuses/filter.json \
-umyuser:mypass | \
sed -e 's/[{}]/''/g' | \
awk -v RS=',"' -F: '/^screen_name/ {print $2}'
Why use sed?
| tr -d '"'
Right tool for the right job.
You can do:
...existing_commands | sed 's/"//g'
A little late to the party, this this utility sounds like it can be useful for parsing the json twitter returns: http://stedolan.github.io/jq/
cat a.txt | tr -d "\042"
It is better because it works in Windows too (using gnuwin32)
Related
So I am new to SED and Unix and I would like to replace the following file:
1500:../someFile.C:111 error
1869:../anotherFile.C:222 error
1869:../anotherFile2.Cxx:333 error
//thousands of more lines with same structure
With the followig file
someFile.c
anotherFile.c
anotherFile2.Cxx
Basically, I just want to extract the filename from every line.
So far, I have read the documentation on sed and the second answer here. My best attempt was to use a regex as follows:
sed "s/.\*\/.:.*//g" myFile.txt
Lots of ways to do this.
Sure, you could use sed:
sed 's/^[^:]*://;s/:.*//;s#\.\./##' input.txt
sed 's%.*:\.\./\([^:]*\):.*%\1%' input.txt
Or you could use a series of grep -o instances in a pipe:
grep -o ':[^:]*:' input.txt | grep -o '[^:]\{1,\}' | grep -o '/.*' | grep -o '[^/]\{1,\}'
You could even use awk:
awk -F: '{sub(/\.\.\//,"",$2); print $2}' input.txt
But the simplest way would probably be to use cut:
cut -d: -f2 input.txt | cut -d/ -f2
You can capture the substring between last / and following : and replace the whole string with the captured string(\1).
sed 's#.*/\([^:]\+\).*#\1#g' myFile.txt
someFile.C
anotherFile.C
anotherFile2.Cxx
OR , with little less escaping, sed with -r flag.
sed -r 's#.*/([^:]+).*#\1#g' myFile.txt
Or if you want to use grep,this will only work if your grep supports -P flag which will enable PCRE:
grep -oP '.*/\K[^:]+' myFile.txt
someFile.C
anotherFile.C
anotherFile2.Cxx
Suppose I have string as follwing:
Rajat;Harshit Srivastava;Mayank 123;5
Now i want result as following using cut command
Rajat
Harshit Srivastava
Mayank 123
5
I have tried but cut is not working on string containing spaces.
man cut would tell you:
-d, --delimiter=DELIM
use DELIM instead of TAB for field delimiter
--output-delimiter=STRING
use STRING as the output delimiter the default is to use the
input delimiter
If you insist on using cut for changing the delimiters:
$ echo "Rajat;Harshit Srivastava;Mayank 123;5" | cut -d \; --output-delimiter=\ -f 1-
Rajat Harshit Srivastava Mayank 123 5
but instead you should use sed or tr or awk for it. Try man tr, for example.
Try this
echo "Rajat;Harshit Srivastava;Mayank 123;5" | sed 's/;/ /g'
I have a particular sentence like this -
|/billing/gcdr/ftpdir|fw43/collectors/ANHMCA04ANT|
I would required to extract the following portion from this particular sentence -
/billing/gcdr/ftpdir
Is there any possiblity that i can do it with sed ? If yes , please help me.
along with sed and awk you can also use cut. using cut:
echo "|/billing/gcdr/ftpdir|fw43/collectors/ANHMCA04ANT|" | cut -d'|' -f2
This is not a sed solution, but if you can live with awk:
echo "|/billing/gcdr/ftpdir|fw43/collectors/ANHMCA04ANT|"| awk -F"|" '{print $2}'
will yield:
/billing/gcdr/ftpdir
Explanation of awk command:
awk -F"|" '{print $2}'
-F="|" specifies the "field separator" - it separates/groups the input line into different fields using the '|' character (rather than using whitepace by default) and then prints the second field of that line.
sed can easily do that:
sed 's/^|//;s/|.*//;'
The first sed command (s/^|//) will remove first | symbol, and the second one (s/|.*//) will remove next | and all symbols after that.
To test it run in console:
echo "|/billing/gcdr/ftpdir|fw43/collectors/ANHMCA04ANT|" | sed 's/^|//;s/|.*//;'
This might work for you:
echo '|/billing/gcdr/ftpdir|fw43/collectors/ANHMCA04ANT|' |
sed 's/^|\([^|]*\).*/\1/'
/billing/gcdr/ftpdir
This question already has answers here:
How to pass command output as multiple arguments to another command
(5 answers)
Closed 5 years ago.
I have this for:
for i in `ls -1 access.log*`; do tail $i |awk {'print $4'} |cut -d: -f 1 |grep - $i > $i.output; done
ls will give access.log, access.log.1, access.log.2 etc.
tail will give me the last line of each file, which looks like: 192.168.1.23 - - [08/Oct/2010:14:05:04 +0300] etc. etc. etc
awk+cut will extract the date (08/Oct/2010 - but different in each access.log), which will allow me to grep for it and redirect the output to a separate file.
But I cannot seem to pass the output of awk+cut to grep.
The reason for all this is that those access logs include lines with more than one date (06/Oct, 07/Oct, 08/Oct) and I just need the lines with the most recent date.
How can I achieve this?
Thank you.
As a sidenote, tail displays the last 10 lines.
A possible solution would be to grepthis way:
for i in `ls -lf access.log*`; do grep $(tail $i |awk {'print $4'} |cut -d: -f 1| sed 's/\[/\\[/') $i > $i.output; done
why don't you break it up into steps??
for file in *access.log
do
what=$(tail "$i" |awk {'print $4'} |cut -d: -f 1)
grep "$what" "$file" >> output
done
You shouldn't use ls that way. Also, ls -l gives you information you don't need. The -f option to grep will allow you to pipe the pattern to grep. Always quote variables that contain filenames.
for i in access.log*; do awk 'END {sub(":.*","",$4); print substr($4,2)}' "$i" | grep -f - $i > "$i.output"; done
I also eliminated tail and cut since AWK can do their jobs.
Umm...
Use xargs or backticks.
man xargs
or
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_03_04.html , section 3.4.5. Command substitution
you can try:
grep "$(stuff to get piped over to be grep-ed)" file
I haven't tried this, but my answer applied here would look like this:
grep "$(for i in `ls -1 access.log*`; do tail $i |awk {'print $4'} |cut -d: -f 1 |grep - $i > $i.output; done)" $i
The following code is working as expected. But I can not format the output.
It will print something like this:
mysql
test
someDB
I want the output on a single line
mysql test someDB
I tried using sed in the script but it did not work.
#!/bin/sh
for dbName in `mysqlshow -uroot -pPassWord | awk '{print $2}'`
do
echo "$dbName" | egrep -v 'Databases|information_schema';
done
whenever you want to combine all lines of output into one you can also use xargs:
e.g.
find
.
./zxcv
./fdsa
./treww
./asdf
./ewr
becomes:
find |xargs echo
. ./zxcv ./fdsa ./treww ./asdf ./ewr
you can use tr to get your output to one line
<output from somewhere> | tr "\n" " "
To do a variation combining naumcho's and rsp's answers that will work for small numbers of results:
echo $(mysqlshow -uroot -pPassWord | awk '{print $2}' | egrep -v 'Databases|information_schema')
The newline is generated by the echo command most likely, the following should do the same without the newlines (not tested)
mysqlshow -uroot -pPassWord | awk '{print $2}' | egrep -v 'Databases|information_schema'
and has the added bonus of spawning just 1 grep instead of 3 grep processes.