I have large daily datasets in raster format. I want to calculate total number of pixels based on values in different polygons in a single shapefile. The shapefile is STRM DEM 90 meter classified 24 elevation zones. These 24 elevation zones representing 24 polygons in single shapefile. I want to check how many pixels occupied by each polygon.
Primarily, I need to check the total number of pixels based on following values (200,210,240,250) in each polygon and finally store it into CSV.
I already developed code: But facing issue by end.
library(sp)
library(rgdal)
library(raster)
mod = raster("MOYDGL06_Maximum_Snow_Extent_2004097.tif")
shp= readOGR("Gilgit_DEM_24.shp")
mod_ext = extract(mod,shp,df=T,na.rm=T)
mod_mask = mask(mod,shp)
plot(r2,axes = TRUE,ext = extent(shp))
r3_200 = rasterToPoints(mod_mask,function(x){ x ==200 },spatial = TRUE)
r3_210 = rasterToPoints(mod_mask,function(x){ x ==210 },spatial = TRUE)
r3_240 = rasterToPoints(mod_mask,function(x){ x ==240 },spatial = TRUE)
r3_250 = rasterToPoints(mod_mask,function(x){ x ==250 },spatial = TRUE)
r3_200_1 = raster::intersect(shp,r3_200)
write.csv(r3_200_1,file = 'r2_extract_gilgit.csv')
Image and R code available in this link
Related
I have Randolph Glacier Inventory boundary shapefiles of glaciers in Himachal Pradesh. I clipped three different rasters with these shapefiles, and then stacked them together . these clipped rasters contain no data values and I have to find the no data values and the pixel values inside these rasters . But when I am extracting the raster , the values am getting are more than that they should be .
for example , the area of a glacier / shapefile is 8.719 km. sq and the resolution of raster is 10 sq.m so accordingly number of pixel in the raster should be 87190, but I am getting approx. 333233(that may be because of the bounding box). So, I decided to create a binary mask so that I can get the values inside the boundary of the raster but still I am getting a lot more values than 87190 .
the stacked raster all have the same resolution and the same extent but still when I extract them and multiply the extracted array with the mask given below, the number of pixels extracted are different for two bands.
the code i used for making the binary mask is given below .
I want to write a code so that I can extract pixel values inside(only inside the boundary of the shapefile) the raster with the no data values present within them.
this is the mask I created
this is the shapefile of the same glacier
from rasterio.plot import reshape_as_image
import rasterio.mask
from rasterio.features import rasterize
from shapely.geometry import mapping, Point, Polygon
from shapely.ops import cascaded_union
shape_path= "E:\semester_4\glaciers of lahaul spiti clipped\RGIId_RGI60-14.11841/11841.shp"
glacier_df = gpd.read_file(shape_path)
raster_path =('E:/semester_4/glaciers of lahaul spiti clipped/RGIId_RGI60-14.11841/stack9/raster_stack_sv_srtm.tif')
with rasterio.open(raster_path , "r") as src:
raster_img = src.read()
raster_meta = src.meta
print("CRS Raster :{} , CRS Vector{}". format (glacier_df.crs , src.crs))
def poly_from_utm (polygon,transform):
poly_pts = []
poly = cascaded_union(polygon)
for i in np.array(poly.exterior.coords):
poly_pts.append(~transform* tuple(i))
new_poly = Polygon(poly_pts)
return new_poly
poly_shp = []
im_size = (src.meta['height'] , src.meta['width'])
for num , row in glacier_df.iterrows():
if row['geometry'].geom_type == 'Polygon':
poly = poly_from_utm(row['geometry'] , src.meta['transform'])
poly_shp.append(poly)
else:
for p in row['geometry']:
poly= poly_from_utm (p , src.meta['transform'])
poly_shp.append(poly)
mask_stack_sv_srtm = rasterize(shapes = poly_shp ,
out_shape = im_size)
plt.figure(figsize = (5, 5))
plt.imshow(mask_stack_sv_srtm)
I have built a gridded area in the Gulf of Alaska with a resolution of 0.02 decimal degrees (~1nm);
library(sp)
library(rgdal)
# Set interval for grid cells.
my.interval=0.02 #If 1 is 1 degree, which is 60nm, than 0.1 is every 6nm, and 0.05 is every
3nm, so 0.0167 is every 1nm
# Select range of coordinates for grid boundaries (UTM to maintain constant grid cell area regardless of geographic location).
lonmin = -140.5083
lonmax = -131.2889
latmin = 53.83333
latmax = 59.91667
LON = seq(lonmin, lonmax, by=my.interval)
LAT = seq(latmin, latmax, by=my.interval)
# Compile series of points for grid:
mygrd = expand.grid(
Longitude = seq(lonmin, lonmax, by=my.interval),
Latitude = seq(latmin, latmax, by=my.interval)) %>%
#mutate(z=1:n()) %>%
data.frame
I exported that grid as a .csv file and brought it into ArcGIS where I used a few bathymetry rasters to extract the bottom depth at the midpoint of each cell. I then exported that from GIS back into R as a .csv file data frame. So now it has another column called "Depth" on it.
For now, I'll just add a column with random "depth" numbers in it:
mygrd$Depth<-NA
mygrd$Depth<-runif(nrow(mygrd), min=100, max=1000)
I would like to calculate the slope at the midpoint of each cell (between points).
I've been trying to do this with the slope() function in SDMTools package, which requires you to have a SpatialGridDataFrame in the sp package.
I can't get this to work; I am also not sure if this is the easiest way to do that?
I have a data frame with 3 columns: Longitude, Latitude, and Depth. I'd like to calculate slope. If anyone knows any better way to do this, let me know! Any help is much appreciated!
Here is some of the code I've been trying to use:
library(SDMTools)
proj <- CRS('+proj=longlat +datum=WGS84')
coords <- mygrd[,1:2]
t2 <- SpatialPointsDataFrame(coords=coords, data=mgrd proj4string=proj)
t2<-SpatialPixelsDataFrame(points=t2[c("Longitude","Latitude")], data=t1[,c(1,2)])
t3 <- SpatialGridDataFrame(grid=NULL, data=t2, proj4string=CRS("+proj=longlat +datum=WGS84"))
class(t3)
slope.test<-slope(t3, latlon=TRUE)
I'm fairly new to using R for GIS purposes. I have a netcdf file containing several variables with multiple dimensions (x,y,z,value and time). I am trying to turn this into a raster brick. The data is quite large so I need to pull data from a specified time window and z(depth). This has not been a problem and extract an array with the appropriate dimensions using the below code .
library(ncdf4)
library(raster)
t <- ncvar_get(nc, "model_time")
t1<-ncvar_get(nc,"model_time_step")
tIdx<-t[t> 20120512 & t < 20120728]
tIdx2<-which(t> 20120512 & t < 20120728)
# Depths profiles < 6 meters
dIdx<-which(nc$dim$depthu$vals <6)
# ncdf dimension lengths
T3 <- nc$var[[7]]
varsize <- T3$varsize
# Define the data (depths,time,etc.) you wish to extract from the ncdf
start <- c(x = 1, y= 1,depthu=1, time_counter = min(tIdx2))
count <- c(x = max(varsize[1]), y = max(varsize[2]),depthu=1, time_counter =
max(tIdx2)-min(tIdx2)+1)
# order of the dimensions
dim.order <- sapply(nc$var$votemper$dim, function(x) x$name)
temp<-ncvar_get(nc,"votemper",start=start[dim.order],count=count[dim.order])
nc$var$votemper
An example of my data (dropping the depth/z and the time dimensions)
temp<-structure(c(0,0,0,0,0,0,0,15.7088003158569,15.3642873764038,14.9720048904419,,15.9209365844727,14.9940872192383,15.0184164047241,15.0260219573975, 0,15.7754755020142, 15.424690246582, 15.6697931289673,15.6437339782715, 0,15.6151847839355, 15.5979156494141, 15.6487197875977,15.432520866394), .Dim = c(x = 5L, y = 5L))
The latitudes and longitudes extracted from the ncdf are irregularly spaced and two dimensions each (i.e. An irregular spaced lat and lon for each cell)
lon<-structure(c(-71.2870483398438,-71.2038040161133,-71.1205596923828,-71.0373153686523, -70.9540710449219, -71.2887954711914, -71.2055587768555,-71.122314453125, -71.0390701293945,-70.9558258056641,-71.2905654907227,-71.2073211669922,-71.1240844726562,-71.0408401489258,-70.9576034545898,-71.292350769043,-71.209114074707, -71.1258773803711, -71.0426330566406,-70.9593963623047, -71.2941513061523, -71.2109222412109, -71.127685546875,-71.0444488525391, -70.9612045288086), .Dim = c(5L, 5L))
lat<-structure(c(38.5276718139648, 38.529125213623, 38.5305824279785,38.532039642334, 38.5334968566895, 38.5886116027832, 38.5900802612305,38.591552734375, 38.5930252075195, 38.5944976806641, 38.6494789123535,38.6509628295898, 38.6524467468262, 38.6539344787598, 38.6554222106934,38.7102699279785, 38.7117652893066, 38.713264465332, 38.7147674560547,38.7162704467773, 38.7709808349609, 38.7724952697754, 38.7740097045898,38.7755241394043, 38.777042388916), .Dim = c(5L, 5L))
Typically I would generate a raster brick from this data using
Temp_brick <- brick(temp, xmn=min(lat), xmx=max(lat), ymn=min(lon), ymx=max(lon),transpose=T)
Temp_brick<-t(flip(Temp_brick,1))
This, however does not account for the irregular spacing and raster cell values are located in the wrong position (lon,lat). I have searched across stack overflow and other gis help sources and I can't find a similar problem with a solution or I'm not asking the right question. I'm not particularly sure how to go about this. Not sure whether this should be dealt with when extracting the data from the netcdf or if it should be dealt with after the raster brick has been created without defined extent. I have tried to find a way to define the lon lats for the raster without any luck. Tried converting lon,lat and value to 3 column dataframe and then use the raster::rasterFromXYZ function. This won't work quick enough for the size of the data I'm dealing with, which in reality is 197(x)*234(y)*2(z)*900(time)*5(variables)*12(years(separate netcdf files).
Any help is greatly appreciated
an option with akima to first interp the data to a regular grid and then turn it into a raster:
# define the regular lon lat or just pass the nx, ny param to interp functions
lonlat_reg <- expand.grid(lon = seq(min(lon), max(lon), length.out = 5),
lat = seq(min(lat), max(lat), length.out = 5))
# interp irregular data to a regular grid
# both solution return the same results because
# i've define the regular grid as akima default
test <- interp(x = as.vector(lon), y = as.vector(lat), z = as.vector(temp),
xo = unique(lonlat_reg[,"lon"]), yo = unique(lonlat_reg[,"lat"]),
duplicate = "error", linear = FALSE, extrap = FALSE)
test <- interp(x = as.vector(lon), y = as.vector(lat), z = as.vector(temp),
nx = 5, ny = 5, linear = FALSE, extrap = FALSE)
# turn into a raster
test_ras <- raster(test)
Check the arguments of the function to choose the interpolation performed etc and be careful if you use extrapolation!
I've seen also that method
Cheers
I have a data frame of lat/long coordinates and a polygon, which represents a coastline. I am trying to find the distance between each point and the nearest coastline feature. I would like to end up with an output data frame that includes columns for my original lat/long values and a new distance column.
I have attempted to use the gDistance function after reading answers to similar questions online, but I think that I am missing a few steps and am having trouble figuring it out. At present, I only end up with a single distance value. I am quite new to R and would really appreciate any help that anyone might be able to give.
Thanks!
#Load data
Locs = structure(list(id = 1:5, Lat = c(29.59679167, 29.43586667, 29.37642222,29.52786111, 30.10603611), Long = c(-81.02547778, -80.92573889,
-80.97714167, -81.08721667, -80.94368611)), .Names = c("id","Lat", "Long"), class = "data.frame", row.names = c(NA, -5L))
#Extract lat/long coordinates
xy = Locs[,c("Lat","Long")]
#Create SpatialPointsDataFrame from xy data and change projection to metres
spdf = SpatialPointsDataFrame(coords=xy, data=xy, proj4string = CRS("+proj=aea +zone=17 ellps=WGS84"))
#Read in shapefile as a spatialdataframe object
coast = readOGR(dsn="land data", layer="coast")
#Transform to AEA (m) projection to match projection of points
land_poly = spTransform(coast, CRS("+proj=aea +zone=17 ellps=WGS84"))
#OR load map from map package (but unfortunately map objects do not work in gDistance)
library(maps)
library(mapdata)
coast2 = map('usa', col = "grey90", fill=TRUE)
#Calculate distance between each point and the nearest land feature
for(i in 1:dim(spdf)[1]){
g = gDistance(spdf[i,],land_poly)
}
EDIT: Using AEF's code alterations below (for the for loop step), I am able to get gDistance values for each row, however the output distances are not correct (see below). According to arcGIS they should be between 4-37km, not >500km. Any thoughts on what I am doing wrong here? My land polygon and points are both in the same projection.
gDistance output
id Lat Long dist_gDist
1 1 29.59679 -81.02548 516299.0
2 2 29.43587 -80.92574 516298.8
3 3 29.37642 -80.97714 516298.9
4 4 29.52786 -81.08722 516299.0
5 5 30.10604 -80.94369 516299.0
The correct distances (calculated in GIS)
id Lat Long dist_arc
1 1 29.59679 -81.02548 13.630
2 2 29.43587 -80.92574 15.039
3 3 29.37642 -80.97714 8.111
4 4 29.52786 -81.08722 4.784
5 5 30.10604 -80.94369 36.855
I think you get only one distance value because you overwrite g in every iteration of your for-loop. I do however not not know if this is the only problem because I cannot reproduce your issue without suitable data.
Try changing the last loop to this:
g = rep(NA, dim(spdf)[1])
for(i in 1:dim(spdf)[1]){
g[i] = gDistance(spdf[i,],land_poly)
}
EDIT
After three days of little interest, I am adding a fully reproducible example... The code below makes 1400 locations. The first 700 points are centered around the first area, the 2nd 700 points are centered around a 2nd area to the northwest.
datOne <- data.frame(
Long = runif(700, -111.180000, -110.950000 ),
Lat = runif(700, 43.180000, 43.440000),
Area = "First")
datTwo <- data.frame(
Long = runif(700, -111.850000, -110.900000),
Lat = runif(700, 43.910000, 44.000000),
Area = "Second")
dat <- rbind(datOne, datTwo)
dat$LatLong <- paste(dat$Lat, dat$Long, sep = ":")
head(dat)
Long Lat Area LatLong
1 -110.9701 43.19509 First 43.1950901590148:-110.970063584852
2 -111.0258 43.25338 First 43.2533758980362:-111.025837010061
3 -111.1737 43.18016 First 43.180157370572:-111.173737878765
4 -111.1130 43.41193 First 43.4119294773275:-111.112970910808
5 -110.9909 43.34044 First 43.3404393909033:-110.990947539737
6 -110.9800 43.33428 First 43.3342766285082:-110.979969937215
If I try to plot the points with gvisMap then only the locations centered around the first area apear in the map as is reproduced with the following code
library(googleVis)
M2 <- gvisMap(dat, "LatLong",
options=list(showLine=TRUE, enableScrollWheel=TRUE,
mapType='satlite', useMapTypeControl=TRUE,
width="800", height="800",
colors = "['#0000ff']",
icons=paste0("{","'default': {'normal':
'http://maps.google.com/mapfiles/kml/shapes/placemark_circle.png',\n",
"'selected':'http://maps.google.com/mapfiles/kml/shapes/placemark_circle.png'",
"}}")
))
plot(M2)
To confirm that all the locations are formatted correctly I can subset the dataframe to plot points 500:1000 which includes locations in both areas. Changing the first line of code to
M2 <- gvisMap(dat[500:1000, ], "LatLong",
will correctly display locations in both areas. So, the issue does not seem to be formatting, but maybe there is a maximum number of points that can be plotted..?
Is it possible to plot all locations in the reproducible data included herein?
Any suggestions or code to plot all of the points in the datdataframe above would be greatly appreciated. I am happy to take suggested code from other packages such as leaflet mentioned in the comments.
Just to throw this into the mix here, as the question is about maximum number of points. In case you ever find yourselves in the need of plotting points (or polygons or lines) in the order of tens of thousands or millions, mapview would be an option:
library(mapview)
library(sp)
datOne <- data.frame(
lng = runif(70000, -111.180000, -110.950000 ),
lat = runif(70000, 43.180000, 43.440000),
Area = "First")
coordinates(datOne) <- ~ lng + lat
proj4string(datOne) <- "+init=epsg:4326"
mapview(datOne)
Note that mapview only works with spatial classes, therefore needs a valid proj4string()
Check out the example below. This uses the R leaflet package and binds a popup to the markers from the area column.
library(leaflet)
datOne <- data.frame(
lng = runif(700, -111.180000, -110.950000 ),
lat = runif(700, 43.180000, 43.440000),
Area = "First")
datTwo <- data.frame(
lng = runif(700, -111.850000, -110.900000),
lat = runif(700, 43.910000, 44.000000),
Area = "Second")
dat <- rbind(datOne, datTwo)
dat$latLng <- paste(dat$lat, dat$lng, sep = ":")
map<-leaflet(dat)
map<-addTiles(map)
map<-addMarkers(map,~lng,~lat,popup = ~as.character(Area))
map
Yes, there is a maximum number of points that can be displayed using Google Maps API, through googleVis
As noted here: [https://developers.google.com/chart/interactive/docs/gallery/map#data-format]
Note: The Lat-Long pairs option loads maps much faster, especially with large data. We recommend that you use this option for large data sets. Please visit Google Maps API to find out how to transform your addresses to lat-long points. The map can display a maximum of 400 entries; if your data holds more than 400 rows, only the first 400 will be shown.
The above leaflet works great for data sets larger than 400 points.