Convert grib to netcdf file - netcdf

Is there a way to convert a grib file into a NetCDF format on Windows?
I use a software named tkdegrib but it catches one argument by one argument whereas I want all my grib's arguments in the same file.
Unfortunately, I can't use Linux.

I would use Unidata's NetCDF-Java library.
It reads GRIB and GRIB2 files (as well as many others), and can write NetCDF files. And since it's Java, you don't have to build anything, and it works on all platforms. There is a ToolsUI application that you can use if you want a GUI, but more likely you want to do it from the command line using something like this:
java -Xmx512m -classpath netcdfAll-4.3.jar ucar.nc2.dataset.NetcdfDataset -in infile.grib2 -out outfile.nc
Good luck!

1. Climate data operators
If you have Windows 10 installed, it is now very easy to simply install ubuntu directly under windows as a subsystem (no wineserver or cygwin required), and then you can install CDO very easily with
sudo apt install cdo
once done you can do:
cdo -f nc copy file.grb file.nc
2. ECCODES
ECMWF has developed a new package called eccodes which includes command line operators for this kind of task. This is the preferred method if you have grib data with more than one time axis such as S2S output, where you have the time of the forecast start and the forecast step and possibly even the hindcast start date. In those cases CDO fails.
You can install it under windows using this guide:
https://confluence.ecmwf.int/pages/viewpage.action?pageId=70949236
Or easily on the linux subsystem with sudo apt-get (on MAC OSX you can install with: brew install eccodes).
Then from the subsystem command line you can simply type:
grib_to_netcdf -o out.nc in.grb

There is GDAL utility gdalwarp, which can convert grib dataset into netCDF dataset.
The command for conversion is
gdalwarp -overwrite -to SRC_METHOD=NO_GEOTRANSFORM -t_srs EPSG:4326 input.grb -of netCDF output.nc
For more options of this utility can be searched on this link.

Related

How do I find the NetCDF root directory?

After installing CDO in a Manjaro distro, I got the following error:
cdo sinfo air_temperature.nc
cdo sinfo: Open failed on >air_temperature.nc<
Unsupported file type (library support not compiled in)
To create a CDO application with NetCDF support use: ./configure --with-netcdf=<NetCDF root directory> ...
netcdf is installed and works with other applications (RNetCDF, QGIS, etc.). However, I don't find which NetCDF root directory, I should indicate in the configure instruction.
Could somebody help me?
Thanks.
As the question is written you need to know the location of your netcdf libs. This is a duplicate of this linked question. If you have netcdf installed then you should be able to use nf-config to find out where your libs are, try
nf-config --flibs
On most Debian-based flavours of linux it can be installed with
sudo apt-get install libnetcdff-dev
but in your case using an arch-based system, you instead need
pacman -S netcdf-fortran-openmpi
But an easier alternative is to bypass the manual install altogether by using conda, see this page for details

Why am I getting a zsh: exec format error?

I am trying to run the metal executable from my zsh terminal in order to meta-analyze GWAS data. I have the executable in the correct directory and have checked that it is not 0MB due to truncation.
Reproducible:
Download the Linux file from http://csg.sph.umich.edu/abecasis/metal/download/
In terminal:
PATH TO EXECUTABLE ./metal
zsh: exec format error: ./metal
You probably install wrong OS of go, for example, you might install go for MacOS in Linux
I solve this problem by installing go for correct OS
The executable has been pre-compiled on a certain distribution with libraries at a certain places at certain versions.
If you have a different distribution, libraries versions, it won't work and you better compile from the source.
Basically what you have to do is to download and extract the sources, go in the folder and execute make. (You will have probably to install make first.)
I think that's not your job at all so maybe you can find a geeky person to help you, because you may stumble upon problems, libraries to install, old versions not supported anymore, new versions not supported yet...
It happened to me when I emptied an executable by mistake.
~>true > a
~>wc -c a
0 a
~>./a
zsh: exec format error: ./a

Run R command without entering R and without a script

I want to run an R command from command line (actually, from within a Makefile). The command is roxygen2::roxygenise(), if it is relevant. I don't want to create a new file and run that as a script - that will just clutter my directory.
In python, this is simple - you write python -c "import antigravity".
I use the Makefile to build, install and test a (Rcpp) package I'm working on.
This is generally done with so 'shebang scripts'.
Historically, littler was there first, about a decade or so ago. It is still widely used, and contains a number of helper scripts as for example roxy.r which does just what you desire: run roxygen2::roxygenize(). I use this all the time.
Next, Rscript started to ship with R. It is similar to littler but automatically available whereever R is which is a plus. On the minus side, it starts slower, and fails to load the methods package which is a source of a number of bug reports and SO questions.
Much more recently, R itself added the ability to run expressions following the -e ... switch.
So you have plenty of choices. You can also study plenty of src/Makevars files many of which use Rscript.

How to load the WEKA pre-preprocessing steps to R?

I have used the WEKA GUI Java here to do the preprocessing of the data. I would like to use the same preprocessing steps now in R.
For example, I want to load the preprocessing of MultiFilter of WEKA GUI to R. I cannot find it in RWeka.
How to load the WEKA prepreprocessing steps to R?
You can load WEKA GUI steps partially with RWeka or with Weka command line tools that are are far more extensive than the available functions in RWeka. So you can extend the RWeka with the command line commands through the system command in R. Luckily, the parameters in WEKA GUI and the WEKA commandline are the same. I recommend extracting the weka-src.jar with jar xf weka-src.jar to read the source.
There exist many functions for the MultiFilter
java weka.filters.MultiFilter --help
java weka.filters.unsupervised.attribute.PartitionedMultiFilter --help
where the second allows you specify the attribute range. Otherwise, they seem to be identical.
Then you can run your first discretize filter with
java weka.filters.unsupervised.attribute.Discretize -F -B 20 -M -1.0 -R 27 -i yourFile.arff
and then direct its output to next Discretize, eventually to NumericTransform and Resample. The command line provides fabulous instructions on the commands in the following way
java weka.filters.unsupervised.attribute.NumericTransform --help
java weka.filters.unsupervised.attribute.Remove --help
java weka.filters.unsupervised.instance.Resample --help
java weka.filters.supervised.instance.Resample --help
and you can check them from the directory structure or the index.
RWeka
RWeka package provides the functions
Discretize()
Normalize()
make_Weka_filter() to create R interfaces to Weka filters
and there is no NumericTransform and Remove functions. You need to use their arguments so not directly just by copy-pasting a java code from WEKA GUI. Perhaps, one solution could be use the system command and execute the Java code with it, without having to need to learn the RWeka itself. There seems to be some gap between the WEKA GUI and the R package.
Running Weka on Commandline
Even though the commands are missing through RWeka interface, you can also use the system commands in R. For example, you can run the remove command
java weka.filters.unsupervised.attribute.Remove -i yourfile.arff
such that
system("java weka.filters.unsupervised.attribute.Remove -i yourfile.arff")
I have the following setup here so we can run Discretize with the following way.
$ cat $WEKAINSTALL/data/iris.arff |tail
6.8,3.2,5.9,2.3,Iris-virginica
6.7,3.3,5.7,2.5,Iris-virginica
6.7,3.0,5.2,2.3,Iris-virginica
6.3,2.5,5.0,1.9,Iris-virginica
6.5,3.0,5.2,2.0,Iris-virginica
6.2,3.4,5.4,2.3,Iris-virginica
5.9,3.0,5.1,1.8,Iris-virginica
%
%
%
$ java weka.filters.unsupervised.attribute.Discretize -i $WEKAINSTALL/data/iris.arff |tail
'\'(6.46-6.82]\'','\'(2.96-3.2]\'','\'(5.13-5.72]\'','\'(2.26-inf)\'',Iris-virginica
'\'(6.82-7.18]\'','\'(2.96-3.2]\'','\'(4.54-5.13]\'','\'(2.26-inf)\'',Iris-virginica
'\'(5.74-6.1]\'','\'(2.48-2.72]\'','\'(4.54-5.13]\'','\'(1.78-2.02]\'',Iris-virginica
'\'(6.46-6.82]\'','\'(2.96-3.2]\'','\'(5.72-6.31]\'','\'(2.26-inf)\'',Iris-virginica
'\'(6.46-6.82]\'','\'(3.2-3.44]\'','\'(5.13-5.72]\'','\'(2.26-inf)\'',Iris-virginica
'\'(6.46-6.82]\'','\'(2.96-3.2]\'','\'(5.13-5.72]\'','\'(2.26-inf)\'',Iris-virginica
'\'(6.1-6.46]\'','\'(2.48-2.72]\'','\'(4.54-5.13]\'','\'(1.78-2.02]\'',Iris-virginica
'\'(6.46-6.82]\'','\'(2.96-3.2]\'','\'(5.13-5.72]\'','\'(1.78-2.02]\'',Iris-virginica
'\'(6.1-6.46]\'','\'(3.2-3.44]\'','\'(5.13-5.72]\'','\'(2.26-inf)\'',Iris-virginica
'\'(5.74-6.1]\'','\'(2.96-3.2]\'','\'(4.54-5.13]\'','\'(1.78-2.02]\'',Iris-virginica
$
Some useful information
Use Weka in your Java code
Download the Linux Developer version, unzip it and read the README with many fabulous examples about using WEKA particularly on command line.
Wiki here
Maybe irrelevant: Generating source code from WEKA classes

Converting MP3 file to WAV

Is it possible to convert a file from .mp3 to .wav in R in order to be able to play the song with R?
Yes (probably). Here's an example:
Converting MP3 to WAV is pretty straightforward:
library(tuneR)
r <- readMP3("04 Trip to Paris.mp3") ## MP3 file in working directory
writeWave(r,"tmp.wav",extensible=FALSE)
(to install tuneR on Linux, see here).
Playback is harder and platform-dependent. tuneR::play() tries to use an external player.
On Windows it tries to guess:
If under Windows and no
player is given, “mplay32.exe” or “wmplayer.exe” (if the
former does not exists as under Windows 7) will be chosen as
the default.
On MacOS, specifying "open" probably works.
On Linux, specifying "play" probably works if you have the sox package installed (sudo apt-get install sox).
So on my MacOS system
tuneR::play("tmp.wav","open")
works.
An alternative that does not use external resources is audio::play().
library(audio)
w <- load.wave("tmp.wav")
play(load.wave("tmp.wav"))
It works on MacOS. I don't know if it works on Windows. It does not work on my Linux system; audio doesn't even install unless you sudo apt-get install portaudio19-dev first, and works poorly even once installed.
(When I say "Linux" here I mean the only system I've tested, Ubuntu 14.04. The sudo apt-get install ... incantations I've listed are likely to work on other reasonably recent Debian-based systems, but ... ???)

Resources