I have a netCDF file of monthly surface air temperature from 1850-2005. How can I truncate the file in unix so that the new file has a time dimension from 1855 - 2005? And vice versa, truncating the file so that it is instead for 1850-2000?
Using NCO
ncks -d time,60, in.nc 1855-2005.nc
ncks -d time,,-60 in.nc 1850-2000.nc
See documentation
CDO allows you use a direct date format if you prefer:
cdo seldate,date1,date2 in.nc out.nc
so the equivalent to Charlie's nco commands would be
cdo seldate,18550101,20051231 in.nc 1855-2005.nc
cdo seldate,18500101,20001231 in.nc 1850-2000.nc
(assuming you want to include the years specified in the year range).
Related
Currently I use global precipitation (ppt) and potential evapotranspiration (pet) data to calculate SPEI. As I have limited hardware resources, I divided global ppt and ppt data into 32 parts, each file covering 45x45deg and contains 756 data - monthly from 1958-2020 (tile01.nc, tile02.nc, ... tile32.nc)
For example to do this, I use cdo sellonlatbox,-180,-135,45,90 in.nc out.nc or ncks -d lat,45.,90. -d lon,-180.,-135. in.nc -O out.nc
As required by SPEI script, I reorder and fixed the dimension from time,lat,lon to lat,lon,time using ncpdq and ncks.
From the SPEI output, I got the output in lat,lon,time. So I did reorder the dimension so that it becomes time,lat,lon using ncpdq.
Each tile SPEI output covering 45x45deg and contains 756 SPEI data - monthly from 1958-2020
Finally I need to merge all the output together (32 files) into one, so I will get the global SPEI output. I have try to use cdo mergegrid but the result is not what I expected. Is there any command from cdo or nco to solve this problem that has function similar to gdal_merge if we are dealing with geoTIFF format?
Below is the example of the SPEI output
UPDATE
I managed to merge all the data using cdo collgrid as suggested by Robert below. And here's the result:
I believe you want to use CDO's distgrid and collgrid methods for this.
First, run this:
cdo distgrid,4,8 in.nc obase
That will split the files up the way you want them.
Then do the post-processing necessary on the files.
Then use collgrid to merge the files:
cdo collgrid obase* out.nc
Potentially you can just use collgrid in place of mergegrid in your present work flow, depending on how you have split the files up.
I have two netcdf files:
file_1.nc with variables qty_1 and qty_2 and
file_2.nc with variables qty_3, qty_4 and qty_5.
I want a file with 3 variables qty_3=qty_3*qty_2; qty_4=qty_4+qty_2 and qty_5.
Now I am first copying the variables to file_2 using
ncks -A -v qty_1,qty_2 file_1.nc file_2.nc
then I am doing math operation as,
ncap2 -A -s 'qty_3=qty_3*qty_2' -s 'qty_4=qty_4+qty_2' file_2.nc
This works, however, take some time.
Is there a way I can do this calculation in a single command ?
If you aren't totallly dependent on NCO, you could do this with CDO:
cdo -selname,qty_3,qty_4,qty_5 -aexpr,'qty_3=qty_3*qty_2;qty_4=qty_4+qty_2' -merge file_1.nc file_2.nc out.nc
I have a -nc4 that contains three .nc files. The files were merged with ncecat in the following way:
ncecat --ovr -M -4 -u ensemble_member1.nc ensemble_member2.nc ensemble_member3.nc merged_ensemble.nc4
I would like to unmerge the .nc4 file in the containing folder to recover the three .nc files. Is there a simple command for this?
When ncecat merges ensembles with the command you show, it creates a record variable to glue them together as explained in the manual here. To disaggregate, hyperslab each ensemble out from the combined file and (optionally) average over the degenerate record dimension leftover from the glue, e.g.,
ncwa -a record -d record,0 in.nc ensemble1.nc
ncwa -a record -d record,1 in.nc ensemble2.nc
ncwa -a record -d record,2 in.nc ensemble3.nc
CDO seems to work fine for me, until I met this. I have a netcdf of daily data from year 2101 to 2228, and I want to obtain a file with only years from 2101 to 2227, so I run:
cdo selyear,2101/2227 in.nc out.nc
But the output file is more than four times the input in memory size! It seems to have the right number of time steps, and the initial and end date are correct. Also, latitude and longitude seem to be the same as the input, so I wonder why the file size.
Perhaps try to retain the compression with the cdo operator and output netcdf 4
cdo -f nc4c -z zip_9 selyear,2101/2227 in.nc out.nc
This is the maximum compression, usually I use zip_5 or so, as the files are not much larger than zip_9 and the command is much faster.
An alternative is to (re?)pack the data to SHORTS with add_offset and scale_factor like this:
cdo pack -selyear,2101/2227 in.nc out.nc
I have a large number of NetCDF files from which I would like to extract a small number of variables for one location, and merge them into a new NetCDF file. The dimensions of the files are:
dimensions:
time = 18 ;
level = 65 ;
levelh = 66 ;
domain = 36 ;
I can subtract/merge the files for all domains with something like:
cdo select,name=u,v file1.nc file2.nc out.nc
But all other operators seem to be related to selections in space (e.g. sellonlatbox) or time (e.g. seltimestep), but I can't find a way to select only 1 domain from the NetCDF files. Is this possible with CDO's or NCO's?
Not sure I fully understand the question/intent. NCO treats all dimensions equally. If you want domain #17 then try
ncrcat -v u,v -d domain,17 file1.nc file2.nc out.nc
If file1.nc and file2.nc are not sequential in a record coordinate then try
ncecat -v u,v -d domain,17 file1.nc file2.nc out.nc
ADDED 20180929:
or if you don't like that, and the files do not have a record dimension yet are time-sequential then before using ncrcat turn the temporal dimension into a record coordinate for each file with
ncks -O --mk_rec_dmn time file1.nc file1.nc
ncks -O --mk_rec_dmn time file2.nc file2.nc
...
etc. and proceed as above. That may be the best way forward with NCO.