I have a list of lat and long for several thousand members and want to append their census tract block group code(i.e. 36001038141345). My first thought is to plot these members on a map that has a us census block group layer. Then either use an overlay function from one of the mapping packages for example, or instead of using shape files convert file to data frame and use a join function for example from "ggplot" and "ggmap" to match the lat and long to the 14 digit census block group value.
## Example of results: ##
GEOID10 LAT LONG
1. 1005950300123 +31.7908933 -085.5670514
2. 01005950900134 +31.8467221 -085.1462332
3. 01005950800145 +31.9206930 -085.1760317
4. 01005950700156 +31.9440855 -085.2620842
5. 01005950600167 +31.8783526 -085.2729215
6. 01005950100178 +32.0353533 -085.2477678
7. 01005950400189 +31.6794432 -085.5577196
8. 01005950200200 +31.9222753 -085.4498309
9. 01005950500211 +31.7634183 -085.2718649
10. 01027959000222 +33.3093759 -085.8820883
11. 01027959200233 +33.1781831 -086.0215486
12. 01027958900244 +33.3872783 -085.7690615
I am searching for a us census block group shape file to use in R. A shape file that might work is a separate download for all US states using Tiger files at US census site along with the necessary support files (".shp, .dbf, .prj etc.") making using this file problematic because it would result in a couple of hundred needed files.
Also checked the UScensus2010 package but it doesn't yet have the 'install.blkgrp' function set up. I'm reaching out to get some ideas or direction on what is the best way to do this.
Thank you
I found this was the easiest way to download census tract shape files using R:
if (("tigris" %in% rownames(installed.packages()))==FALSE)install.packages("tigris"); library(tigris)
if (("ggplot2" %in% rownames(installed.packages()))==FALSE)install.packages("ggplot2"); library(ggplot2)
census.tracts <- tracts(state="NJ")
plot(census.tracts)
census.tracts.fort <- fortify(census.tracts)
This piece of code will give you a census tract shape file for the state of New Jersey. Late to the party I know, but I hope this process can help others.
Related
I have an habitat classification map from Iceland (https://vistgerdakort.ni.is/) with 72 classes in a tif file of 5m*5m pixel size. I want to simplify it, so that there is only 14 classes. I open the files (a tif file and a text file containing the reclassification rules) and use the function classify in the terra package as follow on a subset of the map.
raster <- rast("habitat_subset.tif")
reclass_table<-read.table("reclass_habitat.txt")
habitat_simple<-classify(raster, reclass_table, othersNA=TRUE)
It does exactly what I need it to do and I am able to save the file back to tif using
writeRaster(habitat_simple, "reclass_hab.tif")
The problem is that my initial tif file was 105MB and my new reclassify tif file is 420MB. Since my goal is to reclassify the whole extent of the country, I can't afford to have the file become so big. Any insights on how to make it smaller? I could not find any comments online in relation to this issue.
You can specify the datatype, in your case you should be able to use "INT1U" (i.e., byte values between 0 and 254 --- 255 is used for NA, at least that is the default). That should give a file that is 4 times smaller than when you write it with the default "FLT4S". Based on your question, the original data come with that datatype. In addition you could use compression; I am not sure how well they work with "INT1U". You could have found out about this in the documentation, see ?writeRaster
writeRaster(habitat_simple, "reclass_hab.tif",
wopt=list(datatype="INT1U", gdal="COMPRESS=LZW"))
You could also skip the writeRaster step and do (with terra >= 1.1-4) you can just do
habitat_simple <- classify(raster, reclass_table, othersNA=TRUE,
datatype="INT1U", gdal="COMPRESS=LZW")
Thanks for reading my post. I am trying to create one map in Plotly using R by layering data from 2 data sources and the function plot_mapbox. The map will show locations of stores in zoned business districts.
test is a geoJSON file of zoning districts
test2 is a csv file of business locations using longitude and latitude coordinates
I've tried layering the data and combining two geoJSON files. The first file is a geoJSON file (business zones)and the second file is a .csv (store locations) with longitude and latitude. I converted the csv file to a geoJSON file and then tried to merge them. I would really need to append them since they don't have a common key.
library(plotly)
library(geojsonR)
library(sf)
test<-st_read("D:/SPB/Zoning_Detailed.geojson", quiet=FALSE, , geometry_column="SHAPE_Area")
test2<-read.csv("D:/SPB/Pet_Bus.csv")
One layering exampe
plot_mapbox(data=test, color=~ZONING) %>%
add_markers(data=test2, x=~Longitude, y=~Latitude)
layout(mapbox=list(style = "streets"))
One merge example (only the first file is added in merge)
files Zoning_Detailed.geojson and Pet_Bus.geojson are in the Merge folder. I
converted Pet_Bus.csv to a geojson file.
This should really be append since test and test2 are independent of each other, but in same city.
merge_files("D:/SPB/Merge/", "D:/SPB/Merge/test7.geojson")
Is there any way to get the data matching the filters given for the osmdata- library for a whole country or another big part? The problem is if the area is getting bigger, the file is too large and the download aborts. Below is the import section I'm using right now.
library(osmdata)
q <- getbb("Germany") %>%
opq() %>%
add_osm_feature("amenity", "restaurant")
str(q) #query structure
cinema <- osmdata_sf(q)
From the Planet.osm file I can tell you that there are about 97.000 objects, way or node, with tag amenity = "restaurant" in Germany. The OSM-API won't handle that. You'll even get a timeout using overpass-turbo.
For large amounts of data you'll have to download the Planet.osm file into a database. You'll find a nice tutorial here
Problem
I am given a long list of specific variable codes for the DP05 table - in the census bureau format. For instance:
target_dp05_vars = c(perc_white = "HC03_VC53",
perc_black = "HC03_VC55",
perc_native = "HC03_VC56")
Since tidycensus uses its own variable naming convention, I can't use the above easily. How do I easily crosswalk to the tidycensus definition?
Temporary solution
In the meantime, I've downloaded the bureau file manually and eliminated rows with HC02 and HC04 prefixes to match with tidycensus to create an internal crosswalk (because it's at least positionally correct) but it's tedious.
I'd love to just feed those HCs as a named vector into get_acs() and perhaps just specify the table as DP05.
tidycensus doesn't use its own variable naming convention - it uses variable IDs as specified by the Census API. For example, see https://api.census.gov/data/2017/acs/acs5/profile/variables.html, which is accessible in R with:
library(tidycensus)
dp17 <- load_variables(2017, "acs5/profile", cache = TRUE)
The IDs you've provided appear to be FactFinder codes.
If you want the full DP05 table in one tidycensus call, you can do the following (e.g. for counties in New York) with tidycensus 0.9:
dp05 <- get_acs(geography = "county",
table = "DP05",
state = "NY")
Mapping of variable IDs to their meanings are in turn available with load_variables().
Note: I am getting intermittent server errors with these calls from the API, which may be due to the government shutdown. If it doesn't work at first, try again.
I'm trying to create a shapefile in R that I will later import to either Fusion Table or some other GIS application.
To start,I imported a blank shapefile containing all the census tracts in Canada. I have attached other data (in tabular format) to the shapefile based on the unique ID of the CTs, and I have mapped my results. At the moment, I only need the ones in Vancouver and I would like to export a shapefile that contains only the Vancouver CTs as well as my newly attached attribute data.
Here is my code (some parts omitted due to privacy reasons):
shape <- readShapePoly('C:/TEST/blank_ct.shp') #Load blank shapefile
shape#data = data.frame(shape#data, data2[match(shape#data$CTUID, data2$CTUID),]) #data2 is my created attributes that I'm attaching to blank file
shape1 <-shape[shape$CMAUID == 933,] #selecting the Vancouver CTs
I've seen other examples using this: writePolyShape to create the shapefile. I tried it, and it worked to an extent. It created the .shp, .dbf, and .shx files. I'm missing the .prj file and I'm not sure how to go about creating it. Are there better methods out there for creating shapefiles?
Any help on this matter would be greatly appreciated.
Use rgdal and writeOGR. rgdal will preserve the projection information
something like
library(rdgal)
shape <- readOGR(dsn = 'C:/TEST', layer = 'blank_ct')
# do your processing
shape#data = data.frame(shape#data, data2[match(shape#data$CTUID, data2$CTUID),]) #data2 is my created attributes that I'm attaching to blank file
shape1 <-shape[shape$CMAUID == 933,]
writeOGR(shape1, dsn = 'C:/TEST', layer ='newstuff', driver = 'ESRI Shapefile')
Note that the dsn is the folder containing the .shp file, and the layer is the name of the shapefile without the .shp extension. It will read (readOGR) and write (writeOGR) all the component files (.dbf, .shp, .prj etc)
Problem solved! Thank you again for those who help!
Here is what I ended up doing:
As Mnel wrote, this line will create the shapefile.
writeOGR(shape1, dsn = 'C:/TEST', layer ='newstuff', driver = 'ESRI Shapefile')
However, when I ran this line, it came back with this error:
Can't convert columns of class: AsIs; column names: ct2,mprop,mlot,mliv
This is because my attribute data was not numeric, but were characters. Luckily, my attribute data is all numbers so I ran transform() to fix this problem.
shape2 <-shape1
shape2#data <- transform(shape1#data, ct2 = as.numeric(ct2),
mprop = as.numeric(mprop),
mlot = as.numeric(mlot),
mliv = as.numeric(mliv))
I tried the writeOGR() command again, but I still didn't get the .prj file that I was looking for. The problem was I didn't specified the coordinate systems for the shapefile when I was importing the file. Since I already know what the coordinate system is, all I had to do was define it when importing.
readShapePoly('C:/TEST/blank_ct.shp',proj4string=CRS("+proj=longlat +datum=WGS84")
After that, I re-ran all the things I wanted to do with the shapefile, and the writeOGR line for exporting. And that's it!