I have a .nc file that I convert to text using the following command :
ncdump in.nc > out.txt
However, this particular file has thousands of "-999" values (for nodata). Is there a way to stop ncdump from adding these to the output?
Use sed, e.g.,
ncdump in.nc | sed -e 's/ -999,//g' > out.txt
ncks in.nc | sed -e 's/_, //g' -e 's/ -999,//g' > out.txt
Add -e commands until all unwanted patterns disappear.
Related
I am using the following sed script to perform some find and replace:
parallel --pipepart --block 1000M -a input.txt sed -ise 's/cat/dog/g; s/abc/xyz/g; s/def/22/g' > output.txt
But I am getting the following error:
sed: no input files
I tried removing the -i option but the outcome is still the same.
input.txt file contains a combination of alphabets and numbers and contains around 30 million lines
You need -q:
parallel -q --pipepart --block 1000M -a input.txt sed -se 's/cat/dog/g; s/abc/xyz/g; s/def/22/g' > output.txt
To understand why:
https://www.gnu.org/software/parallel/man.html#QUOTING
https://www.gnu.org/software/parallel/parallel_design.html#Always-running-commands-in-a-shell
Also --block -1 --lb may be more efficient than --block 1000M.
How do I calculate the number of lines starting with "N" in a fastq file in UNIX? I have tried sed but I am not getting an expected output
Try:
sed -n '/^N/p' file.txt | wc -l
I don't know the layout of a fastq file, but can you use grep?
grep -c "^N" file.txt
So I am new to SED and Unix and I would like to replace the following file:
1500:../someFile.C:111 error
1869:../anotherFile.C:222 error
1869:../anotherFile2.Cxx:333 error
//thousands of more lines with same structure
With the followig file
someFile.c
anotherFile.c
anotherFile2.Cxx
Basically, I just want to extract the filename from every line.
So far, I have read the documentation on sed and the second answer here. My best attempt was to use a regex as follows:
sed "s/.\*\/.:.*//g" myFile.txt
Lots of ways to do this.
Sure, you could use sed:
sed 's/^[^:]*://;s/:.*//;s#\.\./##' input.txt
sed 's%.*:\.\./\([^:]*\):.*%\1%' input.txt
Or you could use a series of grep -o instances in a pipe:
grep -o ':[^:]*:' input.txt | grep -o '[^:]\{1,\}' | grep -o '/.*' | grep -o '[^/]\{1,\}'
You could even use awk:
awk -F: '{sub(/\.\.\//,"",$2); print $2}' input.txt
But the simplest way would probably be to use cut:
cut -d: -f2 input.txt | cut -d/ -f2
You can capture the substring between last / and following : and replace the whole string with the captured string(\1).
sed 's#.*/\([^:]\+\).*#\1#g' myFile.txt
someFile.C
anotherFile.C
anotherFile2.Cxx
OR , with little less escaping, sed with -r flag.
sed -r 's#.*/([^:]+).*#\1#g' myFile.txt
Or if you want to use grep,this will only work if your grep supports -P flag which will enable PCRE:
grep -oP '.*/\K[^:]+' myFile.txt
someFile.C
anotherFile.C
anotherFile2.Cxx
I am trying to extract the variable "flash_lon" from a file and output to a text file in plain text - using ncks.
When I use the following command, it displays the variables I need on screen and outputs to a file.
ncks -v flash_lon -x file.nc output.txt
However, the file is not in readable text. In the documentation for ncks, it says that "ncks will print netCDF data in ASCII format ".
What do I need to do in order to simply extract the variable to text? It is just text. I have attached an image below showing the data in the command line working, surely there must be a way to get it to output. I am on Windows 10.
If you have ncdump and sed you can output just the data only like this
ncdump -v flash_lon file.nc | sed -e '1,/data:/d' -e '$d' > output.txt
A solution I use frequently and found here:
https://www.unidata.ucar.edu/mailing_lists/archives/netcdfgroup/2011/msg00317.html
If you don't want even the first lines with the variable name, you can cut those with tail:
ncdump -v flash_lon file.nc | sed -e '1,/data:/d' -e '$d' | tail -n +3 > output.txt
Recently I came across following grep command:
/usr/xpg4/bin/grep -Ff grep.txt input.txt > output.txt
which as per my understanding means that from input.txt, grep the matter contained in grep.txt and output it to output.txt.
I want to do something similar for sed i.e. I want to keep the sed commands in a separate file (say sed.txt) and want to apply them on input file (say input.txt) and create a output file (say output.txt).
I tried following:
/usr/xpg4/bin/sed -f sed.txt input.txt > output.txt
It does not work and I get the following error:
sed: command garbled
The contents of files mentioned above are as below:
sed.txt
sed s/234/acn/ input.txt
sed s/78gt/hit/ input.txt
input.txt
234GH
5234BTW
89er
678tfg
234
234YT
tfg456
wert
78gt
gh23444
Your sed.txt should only contain sed commands: No prefixing with sed or suffixing with an input file. In your case it should probably be:
# sed.txt
s/234/acn/
s/78gt/hit/
When ran on your input:
$ /usr/xpg4/bin/sed -f sed.txt input.txt
acnGH
5acnBTW
89er
678tfg
acn
acnYT
tfg456
wert
hit
ghacn44
Rather than keeping the sed commands in a separate text file, you may want to try creating a sed script. The file below can run directly on your data files:
./myscript.sed inputfile.txt > outputfile.txt
#!/bin/sed -f
s/234/acn/
s/78gt/hit/