I'm very new to gbd table and today at work I was asked to use R to read the data available on https://frap.fire.ca.gov/media/9792/fthrt14_2.zip and plot it using R tools like ggplot and plot_ly.
My problem is that I don't even understand how to read this data with R.
These are the stackoverflow questions I read so far:
- rgdal - read a table within a ESRI geodatabase (gdb)
- How to read a table from an ESRI file geodatabase (.gdb) using R
- How do I get rgdal to open a geodatabase (.gdb) file?
however none of them seems to be working.
These are my steps:
- I unzipped the file in my C:\temp folder. This means I have an R variable file_gdb = "C:/temp/fthrt14_2.gdb"
- then, I have fc_list = ogrListLayers(file_gdb ) and if I print its content I see:
character(0)
attr(,"driver")
[1] "OpenFileGDB"
attr(,"nlayers")
[1] 0
at this point I try fc = readOGR(dsn=fgdb,layer="OpenFileGDB") but I get the following error:
Error in ogrInfo(dsn = dsn, layer = layer, encoding = encoding, use_iconv = use_iconv
Cannot open layer
and from here I'm quite lost.
Any help would be much appreciated. Many thanks from a complete newbie.
Related
I have a bunch of sf objects I'd like to export to GDB from R. I'm running R 4.0.2 on Windows 10. In this case the sf objects are all vector point data. The main reasons to export to GDB are to keep longer field names (the shapefile truncation is very annoying), and because GDBs are more desirable storage locations for our workflows.
Yes, I know about the ArcGisBinding package. I've got it to work in a test script but it's pretty unstable - often crashing and requiring a restart of R. This is a problem, because the sf objects I'd like to export come after an already long Rmd that reads in, formats and cleans the data. So it's not a simple manner of re-running the script until arc.write doesn't break. I could break up the script, but then I'd still have to read in a bunch of shapefiles. One option I haven't yet explored is using reticulate to call a python script instead of trying to do everything in R, but we're trying to do our analysis all in one place, if possible.
I'm pretty sure I've managed to set up RPyGeo appropriately, first setting my python path using the reticulate package. I'm doing it this way because IT restrictions means I can't edit PATH variables on my machine.
#package calls
library(sf)
library(spData)
library(reticulate)
#set python version in reticulate
py_path <- "C:/Program Files/ArcGIS/Pro/bin/Python/envs/arcgispro-py3/python.exe"
reticulate::use_python(python = py_path, required = TRUE)
#call RPyGeo
library(RPyGeo) # for potential point export
#output gdb
out.gdb <- "C:/LOCAL_PROJECTS/Output/Output.gdb"
#RPyGeo Parameters
# Note that, in order to use RPyGeo you need a working ArcMap or ArcGIS Pro installation on your computer.
# python path - note that this will change depending on which version of Arc one is using
# py_path <- "C:/Program Files/ArcGIS/Pro/bin/Python/envs/arcgispro-py3/python.exe"
arcpy <- rpygeo_build_env(workspace = out.gdb,
overwrite = TRUE,
extensions = c("Spatial","DataInteroperability"),
path = py_path)
I've tried a bunch of different tools to export an sf object, here using dummy data also used in the RPyGeo vignette
data(nz, package = "spData")
arcpy$Copy_management(in_data = nz,out_data = "nz_test")
arcpy$Copy_management(in_data = nz,out_data = file.path(out.gdb,"nz"))
arcpy$FeatureClassToGeodatabase_conversion(Input_Features = nz,Output_Geodatabase = out.gdb)
arcpy$FeatureClassToFeatureClass_conversion(in_features = nz,out_path = out.gdb,out_name = "nz")
arcpy$QuickExport_interop(Input = nz,Output = file.path(out.gdb,"nz"))
arcpy$CopyFeatures_management(in_features = nz,out_feature_class = file.path(out.gdb,"nz"))
arcpy$CopyFeatures_management(in_features = nz,out_feature_class = "nz")
Each time I get an error, for example:
Error in py_call_impl(callable, dots$args, dots$keywords) :
RuntimeError: Object: Error in executing tool
Detailed traceback:
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\management.py", line 3232, in CopyFeatures
raise e
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\management.py", line 3229, in CopyFeatures
retval = convertArcObjectToPythonObject(gp.CopyFeatures_management(*gp_fixargs((in_features, out_feature_class, config_keyword, spatial_grid_1, spatial_grid_2, spatial_grid_3), True)))
File "C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing\_base.py", line 511, in <lambda>
return lambda *args: val(*gp_fixargs(args, True))
I'm not an expert in ArcPy by any means. Nor am I an expert in tracing errors inside packages. Am I making a simple syntax mistake? Is there something else that I'm missing? Any help would be much appreciated!
I am trying to load the state map from the maps package into an R object. I am hoping it is a SpatialPolygonsDataFrame or something I can turn into one after I have inspected it. However I am failing at the first step – getting it into an R object. I do not know the file type.
I first tried to assign the map() output to an R object directly:
st_m <- maps::map(database = "state")
draws the map, but str(st_m) appears to do nothing, unless it is redrawing the same map.
Then I tried loading it as a dataset: st_m <- data("stateMapEnv", package="maps") but this just returns a string:
> str(stateMapEnv)
chr "R_MAP_DATA_DIR"
I opened the maps directory win-library/3.4/maps/mapdata/ and found what I think is the map file, “state.L”.
I tried reading it with scan and got an error message I do not understand:
scan(file = "D:/Documents/R/win-library/3.4/maps/mapdata/state.L")
Error in scan(file = "D:/Documents/R/win-library/3.4/maps/mapdata/state.L") :
scan() expected 'a real', got '#'
I then opened the file with Notepad++. It appears to be a binary or compressed file.
So I thought it might be an R data file with an unusual extension. But my attempt to load it returned a “bad magic number” error:
st_m <- load("D:/Documents/R/win-library/3.4/maps/mapdata/state.L")
Error in load("D:/Documents/R/win-library/3.4/maps/mapdata/state.L") :
bad restore file magic number (file may be corrupted) -- no data loaded
Observing that these responses have progressed from the unhelpful through the incomprehensible to the occult, I thought it best to seek assistance from the wizards of stackoverflow.
This should be able to export the 'state' or any other maps dataset for you:
library(ggplot2)
state_dataset <- map_data("state")
I am encountering an error in R that I cannot seem to figure out. I am creating an R markdown document where I read in an a csv table using this code.
iati <- read.csv(file="/filepath/IATI_NGOS.csv",head=TRUE,sep=",")
and then using ggplot2 I create a plot using the following code.
figure_one <- ggplot(iati, aes(iati$reporting.org))+
geom_bar(fill="blue")+
ylab("Total Activities")+
xlab("NGO Reporting Organizations in IATI")+
ggtitle("Total Number of Activities compared to each NGO Reporting Organization in IATI")+
coord_flip()
When I try to call figure_one in the R markdown I get the following error:
Quitting from lines 44-55 (NGO_IATI.Rmd)
Error in gsub("(?<=\n)(?=.|\n)", continue, x, perl = TRUE) :
input string 1 is invalid UTF-8
Calls: <Anonymous> ... paste -> comment_out -> line_prompt -> paste -> gsub
In addition: Warning message:
In grep("\n", message) : input string 1 is invalid in this locale
Execution halted
When I run this code in a regular R script I have absolutely no issues. I have search for some answers but can't figure it out.
Thanks!
I ended solving my issue by just doing a fresh install of R and Rstudio on my local machine. I think the recent update to Yosemite on my local environment created a lot of issues with the TeX plugin I had installed for R markdown.
I get the same question when I knit my rmarkdown document and find **encoding is the cause.
When you use functions like read.csv, fread or read_csv, you will read the column name.
If column names are in other languages, like Chinese, the problem will easily happen.
Or you rmarkdown works on Windows, but the encoding bug happens on Mac, a different environment.
The temporal solution is to rename the column name in English and resave the data files.
Here is the pseudocode in R to show my idea.
library(data.table)
library(tidyverse)
fread('yourfile.csv',encoding = 'UTF-8') %>%
purrr::set_names(c('x1','x2','x3')) %>%
write_excel_csv('yourfile_2.csv')
Here the new file yourfile_2.csv is fine to rmarkdown knit without encoding problems happening.
I'm trying to import a SRTM dataset into R. I've downloaded the data in a tif file however am having trouble reading it in "R".
Ive tried using the following code:
t = readTIFF("srtm_56_06/srtm_56_06.tif", as.is=TRUE)
load('srtm_56_06/srtm_56_06.tif')
read_file<-as.matrix(raster("srtm_56_06/srtm_56_06.tif")
However I am still getting error messages:
load('srtm_56_06/srtm_56_06.tif')
# Error: bad restore file magic number (file may be corrupted) -- no data loaded
# In addition: Warning message:
# file ‘srtm_56_06.tif’ has magic number 'II*'
# Use of save versions prior to 2 is deprecated
library(raster)
t = readTIFF("srtm_56_06/srtm_56_06.tif", as.is=TRUE)
# Error: could not find function "readTIFF"
read_file<-as.matrix(raster("srtm_56_06/srtm_56_06.tif") + min(read_file)
# Error: unexpected symbol in:
# "read_file<-as.matrix(raster("srtm_56_06/srtm_56_06.tif")
# min"
Can anyone help me with the commands to import this data. I'm a novice at "R" and a little lost.
Just read it with raster, but note you depend on rgdal being installed as well to read a .tif.
library(raster)
library(rgdal)
r <- raster("srtm_56_06/srtm_56_06.tif")
If that works, try
plot(r)
r
If it's really a "TIFF" then that should be fine, if it's really a GeoTIFF then you'll have a sensible map as well. (If it's something else that GDAL can read you might get a good result anyway, remember the extension of a file is not a reliable indicator of its contents).
The SRTM clue suggests that this is a single band DEM file from the tiled global SRTM data set. If it's somehow a "multi-band image" then you could read that with brick and plot with plotRGB (but I really doubt that is the case here). Note that there is a native binary format for SRTM that raster/rgdal could read as well but either they distributed .tif as well or someone else converted it.
There are a number of misconceptions in your code:
load is for a particular file type created from R (not these .tifs)
readTIFF is not in package raster
read_file would be a sensible matrix, if you have rgdal installed (which raster must use to load a .tif), but why throw away the spatial metadata?
I'm trying to learn how to plot shapefiles with ggplot2 and I found a wiki at:
https://github.com/hadley/ggplot2/wiki/plotting-polygon-shapefiles
I downloaded and extracted the zip file provided into new directory on my desktop ( http://www.epa.gov/nheerl/arm/documents/design_doc/ecoregion_design.zip ).
I started reading through the article, going line by line but I'm running into a problem with fortify.
require("rgdal")
require("maptools")
require("ggplot2")
require("plyr")
> my_dsn <- "directory here"
>
> utah = readOGR(dsn=my_dsn, layer="eco_l3_ut")
OGR data source with driver: ESRI Shapefile
Source: "directory here", layer: "eco_l3_ut"
with 10 features and 7 fields
Feature type: wkbPolygon with 2 dimensions
> utah#data$id = rownames(utah#data)
> utah.points = fortify(utah, region="id")
Error: isTRUE(gpclibPermitStatus()) is not TRUE
I looked at the ?fortify help page and get the jist of how it should be used. I think its being used correctly. I also found another question with the same error on SO,
Getting error while using fortify function in R (ggplot)
but I can't figure out how to apply what is being said for my solution. I might help if I had access to dropbox right now but I don't.
The short answer is to install the rgeos package and include it at the top.
A longer answer can be found here (and elsewhere): https://stat.ethz.ch/pipermail/r-sig-geo/2012-July/015685.html