import of packages:
from rasterio.mask import mask
import geopandas as gpd
opened a shapefile:
gdf = gpd.read_file(shpfilepath+clipshape)
and opened a rasterfile:
img = rasterio.open(f'{rstfilepath}raw_immutable/SuperView/{SV_filename}{ext}')
then perform action:
for poly_gon in gdf.geometry:
out_image, out_transform = mask(img, poly_gon, crop=True)
but this failes:
TypeError: 'Polygon' object is not iterable
I cannot find how to handle every polygon in the shapefile (5 in my case) to be the polygon to clip the raster image.
Update
How about going into nesting your results. First create an empty object like an empty dict then fill it like:
empt_dict1=dict()
for i in range(len(gdf.geometry)):
empt_dict1[i] = dict()
empt_dict1[i][0], empt_dict1[i][1] = mask(img, gdf.geometry[i], crop=True)
Your expected clips are in each sub-object of the empt_dict list.
I don't have a working gdf right now so I'm not sur if you can index it that way or if you should use something like .loc.
Old answer
If I understand correctly you seek to use the whole area of all the polygons at the same time. How about merging them into a single one using a temporary layer, like below. PS: I tried to use your names given that you don't provide any data.
gdf["dummy"]=[0 for i in range(5)]
gdf_tempo = gdf.dissolve(by=dummy)
out_image, out_transform = mask(img, gdf_tempo , crop=True)
Related
I have r code that I am using to compute Getis-ord Gstatistics. I typically create my shapefile in GIS, then import into r to use with the code. I recently needed to make an edit to my shapefile, which I did in GIS and imported into sas as usual:
tract<-st_read("CBSA2022.shp")
My issue is with the loop portion of my code and the poly2nb feature. Currently it is written as:
for (CBSA in CBSAs) {
temp <- tract[ tract$CBSAFP == CBSA, c("JOIN_ID", variable_of_int)]
names(temp)[2] <- 'black_pop'
#We create the weight matrices within each CBSA now
#We check that there are more than one tract in the CBSA
if ((nrow(temp) > 1)) {
q1<-poly2nb(temp, queen = queen)
if (self_include){ q1 <- include.self(q1) }
Before editing my shapefile in GIS, this worked perfectly with no errors. Now, I receive this error message:
Error in poly2nb(temp, queen = queen) : Empty geometries found
What do you think could be different about my shapefile that I now get this error? And/or how can I fix this? The only difference between this shapefile and the original, is I had to define my spatial join differently when joining my data to spatial polygons in GIS.
I haven't tried anything significant since I am not well versed in r. I did not create this code, but worked with a student (that is no longer available) to create it to be very user friendly for me to use. I've used it numerous times with different shapefiles before my recent edit, and it always worked, just not sure why I now have empty geometries or how to fix it.
Looking on the source code for poly2nb (https://github.com/r-spatial/spdep/blob/main/R/poly2nb.R):
poly2nb <- function(pl, row.names=NULL, snap=sqrt(.Machine$double.eps),
queen=TRUE, useC=TRUE, foundInBox=NULL) {
[...]
if (inherits(pl, "sfc")) {
[...]
if (attr(pl, "n_empty") > 0L)
stop("Empty geometries found")
sf <- TRUE
}
seems, that your temp object is class sfc however, the n_empty attribute isn't updated. Googling around we can find an example: https://github.com/r-spatial/sf/issues/1115. You can check n_empty for your geometries and replace (with 0) those which have value > 0.
I am trying to use GeoPandas for a (only slightly) more complex project, but at the moment I'm failing to write out a simple shapefile with a single point in it in a projected manner.
The following code results in a shapefile that looks generally good - but the .prj is empty:
import pandas as pd
from geopandas import GeoDataFrame
from shapely.geometry import Point
df=pd.read_csv("richmond.csv")
geometry = [Point(xy) for xy in zip(df.x, df.y)]
crs = {'init': 'epsg:4326'}
geo_df = GeoDataFrame(df, crs=crs, geometry=geometry)
geo_df.to_file("geopan.shp")
The csv is 2 row and 2 columns (header row, then lon and lat in 2nd row):
Am I missing something obvious? I've hunted through stackoverflow, the geopandas docs, etc. All seem to imply to_file() should work just fine.
In the long run, the goal is to create a few functions for my students to use in a lab - one that draws a line along a lat or lon the width / height of the US, another that clips the line to polygons (the states), so that the students can figure out the widest spot in each state as a gentle introduction to working with spatial data. I'm trying to avoid arcpy as it's Python 2, and I thought (and think) I was doing the right thing by teaching them the ways of Python 3. I'd like them to be able to debug their methodologies by being able to open the line in Arc though, hence this test.
So, after playing with this, I've determined that under the current version of Anaconda the problem is with crs = {'init': 'epsg:4326'} on Windows machines. This works fine on Macs, but has not worked on any of my or my students' Windows systems. Changing this line to make use of the proj4 string crs = {'proj': 'latlong', 'ellps': 'WGS84', 'datum': 'WGS84', 'no_defs': True} instead works just fine. More of a workaround than an actual solution, but, it seems to consistently work.
I'm always using from_epsg function from fiona library.
>>> from fiona.crs import from_epsg
>>> from_epsg(4326)
{'init': 'epsg:4326', 'no_defs': True}
I've never had any problems using it. Keep it mind that some local projections are missing, but it shouldn't be a problem in your case.
Another user and I had a similar issue using fiona, and the issue for me was the GDAL_DATA environmental variable not being set correctly. To reiterate my answer there: For reference, I'm using Anaconda, the Spyder IDE, Fiona 1.8.4, and Python 3.6.8, and GDAL 2.3.3.
While Anaconda usually sets the GDAL_DATA variable upon entering the virtual environment, using another IDE like Spyder will not preserve it, and thus causes issues where fiona (and I assume Geopandas) can't export the CRS correctly.
You can test this fix by trying to printing out a EPSG to WKT transformation before & after setting the GDAL_DATA variable explictly.
Without setting GDAL_DATA:
import os
print('GDAL_DATA' in os.environ)
from osgeo import osr
srs = osr.SpatialReference() # Declare a new SpatialReference
srs.ImportFromEPSG(3413) # Import the EPSG code into the new object srs
print(srs.ExportToWkt()) # Print the result before transformation to ESRI WKT (prints nothing)
Results in:
False
With setting GDAL_DATA:
import os
os.environ['GDAL_DATA'] = 'D:\\ProgramData\\Anaconda3\\envs\\cfm\\Library\\share\\gdal'
print('GDAL_DATA' in os.environ)
from osgeo import , osr
srs = osr.SpatialReference() # Declare a new SpatialReference
srs.ImportFromEPSG(3413) # Import the EPSG code into the new object srs
print(srs.ExportToWkt()) # Print the result before transformation to ESRI WKT (prints nothing)
Results in:
True
PROJCS["WGS 84 / NSIDC Sea Ice Polar Stereographic North",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Polar_Stereographic"],PARAMETER["latitude_of_origin",70],PARAMETER["central_meridian",-45],PARAMETER["scale_factor",1],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["X",EAST],AXIS["Y",NORTH],AUTHORITY["EPSG","3413"]]
I want to extract vector graphics (lines and points) out of a pdf with pdfclown. I have tried to wrap my head around the graphics sample but i cannot figure out how the object model works for this. Please can anyone explain the relationships?
You are right: till PDF Clown 0.1 series, high-level path modelling was not implemented (it would have been derived from ContentScanner.GraphicsWrapper).
Next release (0.2 series, due next month) will support the high-level representation of all the graphics contents, including path objects (PathElement), through the new ContentModeller. Here is an example:
import org.pdfclown.documents.contents.elements.ContentModeller;
import org.pdfclown.documents.contents.elements.GraphicsElement;
import org.pdfclown.documents.contents.elements.PathElement;
import org.pdfclown.documents.contents.objects.Path;
import java.awt.geom.GeneralPath;
for(GraphicsElement<?> element : ContentModeller.model(page, Path.class))
{
PathElement pathElement = (PathElement)element;
List<ContentMarker> markers = pathElement.getMarkers();
pathElement.getBox();
GeneralPath getPath = pathElement.getPath();
pathElement.isFilled();
pathElement.isStroked();
}
In the meantime, you can extract the low-level representation of the vector graphics iterating the content stream through ContentScanner as suggested in ContentScanningSample (available in the downloadable distribution), looking for path-related operations (BeginSubpath, DrawLine, DrawRectangle, DrawCurve, ...).
I have a geojson that is a FeatureCollection containing 2 geographic data types: a LineString and a waypoint - see the raw file here - this is how it looks on GitHub:
I want to load only only the LineString, so this is what I do:
library(RCurl)
obj <- getURL("https://raw.githubusercontent.com/Robinlovelace/stplanr/master/inst/extdata/route_data.geojson")
writeLines(obj, "/tmp/obj.geojson")
obj <- readLines("/tmp/obj.geojson")
just_lines <- obj[14:(length(obj) - 28)]
just_lines[1] <- paste0("{", just_lines[1])
just_lines[length(just_lines)] <- "}"
writeLines(just_lines, "/tmp/just_lines.geojson")
Now we have removed the pesky lines at the beginning and end of the file, it's a nicely formed GeoJSON file that we can load and plot, yay:
library(rgdal)
route <- readOGR("/tmp/just_lines.geojson", layer = "OGRGeoJSON")
plot(route)
Except it should be obvious to any R user that this is a very clunky and inefficient way of doing this involving too many lines of code and unnecessary reading and writing to the hard disc. There must be another way!
Options I've looked at
geojsonio
jsonlite
leaflet, which can display the FeatureCollection but seemingly not extract its parts.
Context
I'm creating a package for sustainable transport planning, stplanr. A function to find cycling routes (like in the image below) needs to load in the FeatureCollection geojson data from the CycleStreets.net api.
Read the data using jsonlite direct from the URL:
obj <- jsonlite::fromJSON("https://raw.githubusercontent.com/Robinlovelace/stplanr/master/inst/extdata/route_data.geojson")
Convert the first object in the collection to SpatialLines:
sl = SpatialLines(list(Lines(list(Line(obj$features[1,]$geometry$coordinates[[1]])),ID=1)))
plot(sl)
That assumes the feature is a single line string.
To make a SpatialLinesDataFrame with the attributes:
sldf=SpatialLinesDataFrame(sl=sl,data=obj$features[1,]$properties)
Should probably also give it a CRS:
proj4string(sldf)=CRS("+init=epsg:4326")
I don't know if this is possible in LeafletR but Leaflet's L.GeoJSON layer has a filter method which can render (or not render) a collection's features based on the properties the feature has. Some code:
L.geoJson(geojson, {
'filter': function (feature) {
return feature.geometry.type === 'LineString'
}
});
An example: http://plnkr.co/edit/RXIO0X?p=preview
Reference: http://leafletjs.com/reference.html#geojson-filter
I'm trying to create a shapefile in R that I will later import to either Fusion Table or some other GIS application.
To start,I imported a blank shapefile containing all the census tracts in Canada. I have attached other data (in tabular format) to the shapefile based on the unique ID of the CTs, and I have mapped my results. At the moment, I only need the ones in Vancouver and I would like to export a shapefile that contains only the Vancouver CTs as well as my newly attached attribute data.
Here is my code (some parts omitted due to privacy reasons):
shape <- readShapePoly('C:/TEST/blank_ct.shp') #Load blank shapefile
shape#data = data.frame(shape#data, data2[match(shape#data$CTUID, data2$CTUID),]) #data2 is my created attributes that I'm attaching to blank file
shape1 <-shape[shape$CMAUID == 933,] #selecting the Vancouver CTs
I've seen other examples using this: writePolyShape to create the shapefile. I tried it, and it worked to an extent. It created the .shp, .dbf, and .shx files. I'm missing the .prj file and I'm not sure how to go about creating it. Are there better methods out there for creating shapefiles?
Any help on this matter would be greatly appreciated.
Use rgdal and writeOGR. rgdal will preserve the projection information
something like
library(rdgal)
shape <- readOGR(dsn = 'C:/TEST', layer = 'blank_ct')
# do your processing
shape#data = data.frame(shape#data, data2[match(shape#data$CTUID, data2$CTUID),]) #data2 is my created attributes that I'm attaching to blank file
shape1 <-shape[shape$CMAUID == 933,]
writeOGR(shape1, dsn = 'C:/TEST', layer ='newstuff', driver = 'ESRI Shapefile')
Note that the dsn is the folder containing the .shp file, and the layer is the name of the shapefile without the .shp extension. It will read (readOGR) and write (writeOGR) all the component files (.dbf, .shp, .prj etc)
Problem solved! Thank you again for those who help!
Here is what I ended up doing:
As Mnel wrote, this line will create the shapefile.
writeOGR(shape1, dsn = 'C:/TEST', layer ='newstuff', driver = 'ESRI Shapefile')
However, when I ran this line, it came back with this error:
Can't convert columns of class: AsIs; column names: ct2,mprop,mlot,mliv
This is because my attribute data was not numeric, but were characters. Luckily, my attribute data is all numbers so I ran transform() to fix this problem.
shape2 <-shape1
shape2#data <- transform(shape1#data, ct2 = as.numeric(ct2),
mprop = as.numeric(mprop),
mlot = as.numeric(mlot),
mliv = as.numeric(mliv))
I tried the writeOGR() command again, but I still didn't get the .prj file that I was looking for. The problem was I didn't specified the coordinate systems for the shapefile when I was importing the file. Since I already know what the coordinate system is, all I had to do was define it when importing.
readShapePoly('C:/TEST/blank_ct.shp',proj4string=CRS("+proj=longlat +datum=WGS84")
After that, I re-ran all the things I wanted to do with the shapefile, and the writeOGR line for exporting. And that's it!