I am working with some netCDF files and want to import netCDF parameter's data as a Raster and build a contour layer for it. I am using gdal_contour for this.
When I import the netCDF and choose a parameter (water_temp) in QGIS, the raster is loaded into the map with no problem and displays values in the range of roughly 4 degC to 31.25 degC.
However, when I use gdal_contour to make a contour layer for it, the values are in the range of -15944 to 11250. It certainly doesn't help that among other issues, it takes forever to generate the layer because I'm specifying an interval of 1.0 and the value range is far larger than the expected temperatures for Celsius.
From what I can tell, it looks like perhaps gdal_contour either isn't respecting the raster band's offset and scale_factor or has no knowledge of it. I understand that the netCDF is storing the temperature values as integers instead of floats to optimize file size, but I'm a bit confused by why QGIS can understand the offset when reading the netCDF into a raster layer, but not when generating a contour layer.
Am I missing something, or is there perhaps a caveat to using gdal_contour of which I'm unaware?
The command I am using to generate the conotur layer is:
gdal_contour -b 1 -a water_temp -i 1.0 -snodata -30000.0 -f "ESRI Shapefile" NETCDF:"C:/path/to/input/netcdf/INPUT.nc":water_temp C:/path/to/output/layer/OUTPUT.shp
The scale_factor, offset, and associated metadata for the band are:
add_offset=20
missing_value=-30000
NETCDF_VARNAME=water_temp
scale_factor=0.001
STATISTICS_MAXIMUM=11250
STATISTICS_MEAN=5475.011083141
STATISTICS_MINIMUM=-15944
STATISTICS_STDDEV=5863.9957197805
units=degC
_FillValue=-30000
This question was answered here.
TLDR; Convert the netCDF to a GeoTIFF first using
gdal_translate with the -unscale option to get GDAL to unpack
the values, then perform gdal_contour on the GeoTIFF to get a
contour layer with the correctly unpacked values.
However, one thing that may be important to note is the scaled/unscaled data types, which may have to be explicitly set for gdal_translate (using the -ot option) in order to not lose data precision during unscaling if the scaled data type is a smaller size than the unscaled data type.
Related
I'm using a Land cover raster of North America which is publicly available here: https://open.canada.ca/data/en/dataset/4e615eae-b90c-420b-adee-2ca35896caf6
I clipped it in R to cover Québec/Labrador:
veg <- raster("CanadaLandcover2015/CAN_LC_2015_CAL.tif")
e <- extent(c(1000000, 2700000, 500000, 2700000))#all qc
veg_qc <- crop(veg, e)
The raster is originally in projection ESPG:3978 NAD83/Canada Atlas Lambert. I wanted it to be in lat and long to be able to extract the values to datapoints.
veg_qc2 <- projectRaster(veg_qc,crs="+proj=longlat +ellps=WGS84 +no_defs")
That single line took ~12 hours to run and took over 200 GB of Temp data. Worst, there was a warning (sorry did not copy it) and only half the raster showed.
So I decided to try with the function Wrap in QGIS. Although it worked perfectly, the output raster was 16 Gb! The original clipped raster was only 693 MB.
To make things worse, I need the value layer to be included in R, so I used:
veg_qc <- getValues(veg_qc)
And I get the following error:
Error: cannot allocate vector of size 31.0 Gb
Why does the raster get bigger when reprojected?
Would there be a way to compress the raster or reproject without that giant increase of data?
How can I add the values to a big raster layer?
Ultimately, I could clip the raster further with the mcp of my data. I could also reproject my data and my other rasters in EPSG:3978 (although I am wondering if my other rasters my end up as > 10GB too).
I just had the same issue this week. After a reprojection, from 40Mb to 350Mb. I found the answer in this link Huge file size after averaging two rasters.
What I did in QGIS (not R by this time) is that I opened the following function in the Manu bar: raster > conversion > translate, and in Advanced parameters > Additional command-line parameters, I added -co COMPRESS=LZW. This will compress using the LZW method. Also, according to this link and considering your dataset, you can use PACKBITS. I hope this helps you.
I have two .nc files, file1.nc and file2.nc The dimensions of the files are the following:
file1 [lon,lat,t]=[21,9,t1] 0.25x0.25 Grid
file2 [lon,lat,t]=[9,3,t2] 0.5X0.5 Grid
Each netcdf file have different time ranges but I'm only interested in the xy grid.
I want to transform file 1 so it has the grid size and same coordinates as of file 2. I have attached a picture so my explanation is clearer.
see picture
Some remarks:
Some recommend using CDO (Climate Data Operators) but since I'm using my company's computer I don't have the permits to install what's required to run CDO.
Other recommend to use resample() but they apply it to rasters which only visualizes one point in time and I want to resize the entire NetCDF.
-I would like to perform the transformation either by using the netcdf file it self or the multivariate array resulting from extracting one variable form the netcdf file.
Package landscapemetrics can calculate area of each patch for a given raster file, shape of that patch and so on. I want to have not only tibble-frame with patch metrics calculated, but a new raster where each pixel within specific patch will have a value of the area of that patch, shape indicator and so on. We can do it with function spatialize_lsm() (it produces a Large list nested object with probably RasterObject objects within):
library(landscapemetrics)
plot(podlasie_ccilc) # this raster data is provided with package
podlasie.metrics.area <- spatialize_lsm(podlasie_ccilc, what = 'lsm_p_area') # creates a list
plot(podlasie.metrics.area) # produces an error...
How to get a desirable raster file with patch metrics from that list? I guess it is a question of raster package or something else, since landscapemetrics documentation tells nothing about this step.
I not that this data and new raster do not have resolution of the pixel like in meters (30, 30 for Landsat satellite image, for example). So we cannot plot the new raster produced:
podlasie.metrics.area[[1]]
plot(podlasie.metrics.area[[1]])
So I guess landscapemetrics cannot deal with such rasters, we can even use its function to check a suitability of the prior raster for patch discovering:
check_landscape(podlasie_ccilc)
Upd. I did it for the Landsat dataset with resolution 30, 30 and it produced patch area raster, but again I cannot open/show/save as raster it, because of the same error.
Package maintainer helps to solve a problem (yes, it is just related to the structure of list):
plot(podlasie.metrics.area[[1]]$lsm_p_area)
I'm trying to summarize raster cell values in overlapping polygons in ArcMap. This can be done in Geospatial Modelling Environment (GME), an extension for ArcMap. They have a command called isectpolyrst that calculates for values in overlapping polygons. My problem is, my version of ArcGIS (10.6.2), doesn't support the use of GME, so I can't use this function. I've heard that isectpolyrst can still accomplished in R Studio using R script, but I haven't found it anywhere.
I have a number of GPS points with 10 km buffers around them (these buffers overlap a lot). I'm trying to calculate proportions of different vegetation types within these buffer zones. I'm using ArcMap 10.6.2., and zonal statistics can't calculate for overlapping polygons.
You can use raster::extract for that. As you seem entirely new to R, you will need to study it a bit first. You can start here: https://rspatial.org/
I am trying to create a irregular shaped .tiff from a csv list of points (xyz data). I am doing this using gdal_grid.
I can seem to generate the .tiff file no problem but I cannot preserve the outline / shape of the original csv points.
Everytime I generate the .tiff file it creates a raster with the size of (xmax-xmin) x (ymax-ymin) and assigns interpolated values to pixels that fall far away from my initial points.
Is it possible to generate a .tiff file of ONLY the points I provide?
For context, I am trying to generate a raster of xyz data for a river, and only want the raster in the river (not the entire bounding box of the river). I am only providing xyz data in the river.
I tried playing with the -nodata flag, and limiting -max_points to the number of points I've provided.
My final code (once everything is imported and declared):
gdal_grid -a invdist:power=2.0:smoothing=1.0:nodata=-999:max_points=2128164 -txe 582387.4 591069.4 -tye 4505028.08 4515344.079999999 -outsize 50 50 -zfield "z" -of GTiff -ot Float64 -l Book2 Book2.vrt Book2.tiff
Welcome to Stack Overflow, Derek!
Maybe there is a creation option inside gdal_grid that would do it, but I think that you will have to achieve desired result with additional calculation:
Run the gdal_grid as you have it.
Create a concave hull from the given points. If this is a one time job, I suggest using QGIS (with grass tools), because there is some tweaking of concave hull parameters required.
Cut the raster with the created shapefile by using gdalwarp.
Let me know if this got you through!