I was assigned the task to clip a raster from .nc file from a .tif file.
edit (from comment):
i want to extract temp. info from the .nc because i need to check the yearly mean temperature of a specific region. to be comparable the comparison has to occur on exactly the same area. The .nc file is larger than the previously checked area so i need to "clip" it to the extent of a .tif I have. The .tif data is in form 0|1 where it is 0 (or the .tif is smaller than the .nc) the .nc data should be "cliped". In the end i want to keep the .nc data but at the extent of the .tif while still retaining its resolution & projection. (.tif and .nc have different projections&pixel sizes)
Now ordinarily that wouldn't be a problem as i could use raster::crop. This doesn't deal with different projections and different pixel size/resolution though. (I still used it to generate an approximation, but it is not precise enough for the final infromation, as can be seen in the code snippet below). The obvious method to generate a more reliable dataset/rasterset would be to first use a method like raster::projectRaster or raster::sp.Transform # adding sp.transform was done in an edit to the original question and homogenize the datasets but this approach takes too much time, as i have to do this for quite a few .nc files.
I was told the best method would be to generate a normalized matrix from the smaller raster "clip_frame" and then just multiply it with the "nc_to_clip" raster. Doing so should prevent any errors through map projections or other factors. This makes a lot of sense to me in theory but I have no idea how to do this in practice. I would be very grateful to any kind of hint/code snippet or any other help.
I have looked at similar problems on StackOverflow (and other sites) like:
convert matrix to raster in R
Convert raster into matrix with R
https://www.researchgate.net/post/Hi_Is_there_a_way_to_multiply_Raster_value_by_Raster_Latitude
As I am not even sure how to frame the question correctly, I might have overlooked an answer to this problem, if so please point me there!
My (working) code so far, just to give you an idea of how I want to approach the topic (here using the crop-function).
#library(ncdf4)
library(raster)
library(rgdal)
library(tidyverse)
nc_list<-list.files(pattern = ".*0.nc$") # list of .nc files containing raster and temperature information
#nc_to_clip <- lapply(nc_list, raster, varname="GST") # read in as raster
nc_to_clip < -raster(ABC.nc, vername="GST)
clip_frame <- raster("XYZ.tif") # read in .tif for further use as frame
mean_temp_from_raster<-function(input_clip_raster, input_clip_frame){ # input_clip_raster= raster to clip, input_clip_frame
r2_coord<-rasterToPoints(input_clip_raster, spatial = TRUE) # step 1 to extract coordinates
map_clip <- crop(input_clip_raster, extent(input_clip_frame)) # use crop to cut the input_clip_raster (this being the function I have to extend on)
temp<-raster::extract(map_clip, r2_coord#coords) # step 2 to extract coordinates
temp_C<-temp*0.01-273.15 # convert kelvin*100 to celsius
temp_C<-na.omit(temp_C)
mean(temp_C)
return_list<-list(map_clip, mean(temp_C))
return(return_list)
}
mean_tempC<-lapply(nc_to_clip, mean_temp_from_raster,clip_frame)
Thanks!
PS:
I don't have much experience working with .nc files and/or RasterLayers in R as I used to work with ArcGIS/Python (arcpy) for problems like this, which is not an option right now.
Perhaps something like this?
library(raster)
nc <- raster(ABC.nc, vername="GST)
clip <- raster("XYZ.tif")
x <- as(extent(clip), "SpatialPolygons")
crs(x) <- crs(clip)
y <- sp::spTransform(x, crs(nc))
clipped <- crop(nc, y)
Related
I am using the following code to get TIFF files into R for analysis:
library(magick)
tiffiles<-list.files("C:/Users/folder_with_multiple_tifs/", pattern = "*.tif", full.names=TRUE)
importedtifs<-c()
for(file in tiffiles) {importedtifs<-append(importedtifs, image_read(file))}
importedtifs
This gives me a tibble with each row corresponding to a TIFF file. I can then use mean(as.integer(importedtifs[[1]])) to get the average pixel intensity of the first TIFF. It is a small positive number for the images I am working with.
I would like to have a single command that returns the mean pixel intensity of each individual TIFF in the tibble. When I try lapply(importedtifs, function(x) mean(as.integer(x))), I get a large negative number, which is not the pixel intensity.
Is there a way to do this? I don't understand exactly how the tibble is storing the data for each TIFF.
DaveArmstrong's solution works. The variation below delivers the means in a list that can be manipulated downstream:
means<-c()
for(i in 1:length(importedtifs)){
means<-c(means, mean(as.integer(importedtifs[[i]])))
}
I have a problem getting raster data in the right orientation. The original raster data when imported into R looks this .
I tried using the transpose function in raster but it didn't work. The transposed data looks like this .
I used the code below. Any help or advice would be greatly appreciated. Also, is there a way to apply the plausible solution to the entire stack (all rasters are of the same extent)? Thank you.
f_PM <- list.files(path=".",
pattern='tif$', full.names=TRUE)
s_PM <- stack(f_PM) ## create a stack of the rasters
plot(s_PM[[1]]) ##check the orientation
PM <- s_PM[[1]] ## pick one raster and try to change the orientation
PM2 <- t(PM)
plot(PM2)
You need to flip then transpose:
> plot(m)
> r = t(flip(m))
> plot(r)
Note there's https://gis.stackexchange.com where spatial questions like this get asked. (Too much noise here to help with most spatial stuff).
I am trying to get the wind exposure for infrastructures. I have a dataset with their latitude and longitude.
The NetCDF file gives daily near surface wind speed data projections for the year 2058. It can be downloaded with the following URL: http://esg-dn2.nsc.liu.se/thredds/fileServer/esg_dataroot1/cmip6data/CMIP6/ScenarioMIP/EC-Earth-Consortium/EC-Earth3/ssp585/r1i1p1f1/day/sfcWind/gr/v20200310/sfcWind_day_EC-Earth3_ssp585_r1i1p1f1_gr_20580101-20581231.nc
I have tried the following loop to get the average wind speed for each location (their closest grid point):
sfcWind_filepath<-paste0("sfcWind_day_EC-Earth3_ssp585_r1i1p1f1_gr_20580101-20581231.nc")
sfcWind_output<-nc_open(sfcWind_filepath)
lon<-ncvar_get(sfcWind_output,varid = "lon")
lat<-ncvar_get(sfcWind_output,varid = "lat")
sfcWind_time<-nc.get.time.series(sfcWind_output,v = "sfcWind",time.dim.name = "time")
sfcWind<-ncvar_get(sfcWind_output, "sfcWind")
for(i in 1:nrow(Infrast))
{sfcWind<-rep(i,nrow(Infrast))
x<-Infrast[i,4]
y<-Infrast[i,3]
Infrast[i,12]<-mean(sfcWind[which.min(abs(lon - (x))),
which.min(abs(lat - (y))),
c(which(format(sfcWind_time, "%Y-%m-%d") == "2058-01-01"):which(format(sfcWind_time, "%Y-%m-%d") == "2058-12-31"))])
}
Where Infrast is my dataset of infrastructures, their latitude is in column 3 and longitude in column 4, an I want the output to be saved at the 12th column of my dataset.
I get the following error:
Error in sfcWind[which.min(abs(lon - (x))), which.min(abs(lat - (y))), :
incorrect number of dimensions
I used this code before to get the average of projected temperatures and it worked just fine. The NetCDF file had the same dimensions than this one (lat, lon, time). This is why I don't understand the error here.
I am quite new to R and I just started to work with NetCDF files, any help or suggestion would be appreciated.
I'm also relative new to R too, but one possible way is to use the cmsaf package. Here, you can use the selpoint or selpoint.multi function to extract the time series of a variable at a specific location or for multiple locations. All you would need is the list of lat/lon coordinates for your desired locations. It will then make a new netcdf or csv file for the output. Then you could calculate the average from the extracted point data. There probably is a better and more efficient way but hopefully that might help.
NB I'm unable to test this, because no reproducible example was provided.
Nonetheless, this should work.
First open the file as a raster brick
library(raster)
sfcWind_output <- brick(sfcWind_filepath, varname="sfcWind")
Now you can extract values using coordinates like this
extract(sfcWind_output, cbind(lon,lat))
Currently I'm trying to work with CALIPSO LiDAR data. The files are distributed in HDF4 (checked it with HDFView). The subdataset I'm interested is the Cloud Top Height, which is stored in a simple table (56160 rows, 5 columns):
So far, I tried two things:
A) Converting HDF subdataset to GeoTiff using gdal_translate (gdalUtils package)
# Get a list of sds names
sds <- get_subdatasets('C:\\Users\\Thomas\\Desktop\\NASA Sat Data\\Cloud Layer Data\\CAL_LID_L2_333mCLay-ValStage1-V3-01.2011-08-16T03-02-01ZN.hdf')
# Isolate name of subdataset and convert
name <- sds[25]
filename <- 'Cloud_Top_Height_Test.tif'
gdal_translate(name, dst_dataset = filename)
# Load the Geotiff created into R
r <- raster(filename)
# fix extent, projection and negative values
e <- extent(-80.016243, 64.831413, -179.183868, -1.711731)
r <- setExtent(r, e)
values(r)[values(r) < 0] = 0 # replace -9999 with NA
crs(r) <- '+proj=utm +zone=28 +ellps=WGS84 +datum=WGS84 +units=m +no_defs'
The result was a raster file which was obviously missing any spatial information (dimensions, extend, crs, resolution), that's why I fixed some things afterwards.
This is the result: .
As you can see, the result are some strange lines where it should be square pixels. I suppose I'm missing some statements in the gdal_translate call maybe? Also, all the adjustments afterwards seem a bit stuporous or let's say 'unprofessional'.
B) Converting the table (subdataset) into a data frame (to convert it into a raster in an second step)
A few days ago I came up with the idea to convert the subdataset (which is essentially 'only' a table) to a data frame and convert it in a second step into a raster, where I specify all spatial information 'from scratch' rather than adjusting it afterwards as in my first attempt. Still, I found no commands/packages to do so and I'm still not confident that this is the right way.
What I need in the end: A proper (georeferenced etc) raster (56160 rows, 5 columns) holding the values of the subdataset from the HDF4 file.
My question: How can I extract a HDF4 subdataset and convert it to a raster having the specifications mentioned above?
Any help is greatly appreciated (and since this is my first post at SO: sorry, if this question is missing any forms required in this forum, criticism is also appreciated)
to convert to dataframe:
as.data.frame(r, xy = TRUE)
I'm programming a script for the calculation of cover around points in R.
I have two inputs: an IMG raster file, and a .csv with all the points.
I've used this script:
library(raster)
library(rgdal)
#load in raster and locality data
map <- raster('map.IMG')
sites <- read.csv('points.csv', header=TRUE)
#convert lat/lon to appropirate projection
coordinates(sites) <- c("X", "Y")
proj4string(sites) <- CRS("+init=epsg:27700")
#extract values to points
Landcover<-extract (map, sites, buffer=2000)
extraction <- lapply(Landcover, function(serial) prop.table(table(serial)))
# Write .csv file
lapply(extraction, function(x) write.table( data.frame(x), 'test2.csv' , append= T, sep=',' ))
I get a .csv file in my map, but the data isn't organised in the way I would like it to be.
There a three columns in the csv file. One with 'x', one with 'Freq' (Which I think is the code for every class in my image) and one with the cover part, somewhere between 0-1. See the image included.Image
I want to have on the rows the serial and classes, and under that the correct serial with it's coverage.
Also every point isn't named, so I can't see which is which. In the points.csv I have for example a 'serial' code for every point, which i would like to use for that.
Can somebody steer me in the right direction?
I hope I have been clear with my questions, thank in advance!