I am unable to download a large SSURGO dataset using the FedData package in R. I get the same error using the example code from the package:
vepPolygon <- polygon_from_extent(raster::extent(672800,740000,4102000,4170000),
proj4string='+proj=utm +datum=NAD83 +zone=12')
# Get the NRCS SSURGO data (USA ONLY)
SSURGO.VEPIIN <- get_ssurgo(template=vepPolygon, label='VEPIIN')
produces the error:
Error in { :
task 1 failed - "Download of https://sdmdataaccess.nrcs.usda.gov/Spatial/SDMNAD83Geographic.wfs?Service=WFS&Version=1.0.0&Request=GetFeature&Typename=SurveyAreaPoly&BBOX=-109.056777043215,37.033556543269,-108.279725827617,37.6609211495406 failed!"
It does work if I use extremely small templates, like 20'x20', but my understaning is that this package is supposed to enable larger downloads. Has something changed with the web soil survey since this package was created?
Related
I am working with a couple of R packages for genetic pathway enrichment analyses and the two packages that I am using are now throwing errors when trying to connect to each package's respective server for downloading the reference data for the analysis.
In the first package gage, I am getting the following error when attempting to download:
library(gage)
> kg.ko = kegg.gsets("ko") # ("ko" is KEGG ortholog pathway)
Error in curl::curl_fetch_memory(url, handle = handle) :
Failure when receiving data from the peer
In the second package clusterProfiler, I am getting the following error:
library(clusterProfiler)
# the data
dput(head(de_kegg_chr))
c("K14847", "K19009", "K00078", "K21407", "K23285", "K06972")
# KEGG enrichment (which will pull relevant reference data during this step)
# over-representation analysis (fisher's)
> enrich <- enrichKEGG(gene = de_kegg_chr,
+ organism = "ko",
+ keyType='kegg',
+ pvalueCutoff = 0.01)
Reading KEGG annotation online:
fail to download KEGG data...
Error in download.KEGG.Path(species) :
'species' should be one of organisms listed in 'http://www.genome.jp/kegg/catalog/org_list.html'...
In addition: Warning message:
In utils::download.file(url, quiet = TRUE, method = method, ...) :
URL 'https://rest.kegg.jp/link/ko/pathway': status was 'Failure when receiving data from the peer'
After the first error, I thought it was something specific to the gage package and found a simple work-around because these data are downloaded from the server prior to the analysis function.
This is more of a problem with the second package because the reference data are downloaded within the function that conducts the analysis.
Now that this is happening with more than one package (both of these scripts were working perfectly before yesterday), I'm thinking it is something systematic within R or R studio.
I'm trying to get TRMM data from NASA OPenDAP server using the raster package in R. Initially I had some difficulty regarding authentication, but that issue was resolved.
NASA OPenDAP server publishes TRMM 3B42_daily data as individual files, one for each day and an aggregated annual data (using ncml). So, my problem now is that, using R raster package and the authentication files .dodsrc and .netrc I can download individual NetCDF files but I can't download the aggregated data.
So, this works:
library(raster)
single_date_opendap <- 'https://disc2.gesdisc.eosdis.nasa.gov:443/opendap/TRMM_L3/TRMM_3B42_Daily.7/2002/04/3B42_Daily.20020405.7.nc4'
test <- stack(single_date_opendap, varname = 'precipitation')
This doesn't:
library(raster)
url_opendap_no_brkt <- 'https://disc2.gesdisc.eosdis.nasa.gov:443/opendap/ncml/aggregation/TRMM_3B42_Daily.7/TRMM_3B42_daily.7_Aggregation_2001.ncml'
test <- stack(url_opendap_no_brkt, varname = 'precipitation')
And gives me the error message:
Error in .local(.Object, ...) :
An error occurred while creating a virtual connection to the DAP server:
Error while reading the URL: https://disc2.gesdisc.eosdis.nasa.gov:443/openda
p/ncml/aggregation/TRMM_3B42_Daily.7/TRMM_3B42_daily.7_Aggregation_2001.ncml.
ver.
The OPeNDAP server returned the following message:
Unauthorized: Contact the server administrator.
Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer",
Cannot create a RasterLayer object from this file. (file does not exist)
Is it possible to get data from a OPenDAP server that publishes aggregated data?
After some exchange with NASA support and with Antonio's tip, found out that R raster package will not work with the aggregated datasets. But ncdf4::nc_open is able to handle it. Strange because, from what I understand, raster package calls nc_open in the background.
Anyway, this works:
library(ncdf4)
url_opendap <- 'https://disc2.gesdisc.eosdis.nasa.gov:443/opendap/ncml/aggregation/TRMM_3B42_Daily.7/TRMM_3B42_daily.7_Aggregation_2001.ncml'
trmm <- nc_open(url_opendap)
and this doesn't
library(raster)
url_opendap <- 'https://disc2.gesdisc.eosdis.nasa.gov:443/opendap/ncml/aggregation/TRMM_3B42_Daily.7/TRMM_3B42_daily.7_Aggregation_2001.ncml'
trmm <- stack(url_opendap, varname = "precipitation")
I'm trying to import a SRTM dataset into R. I've downloaded the data in a tif file however am having trouble reading it in "R".
Ive tried using the following code:
t = readTIFF("srtm_56_06/srtm_56_06.tif", as.is=TRUE)
load('srtm_56_06/srtm_56_06.tif')
read_file<-as.matrix(raster("srtm_56_06/srtm_56_06.tif")
However I am still getting error messages:
load('srtm_56_06/srtm_56_06.tif')
# Error: bad restore file magic number (file may be corrupted) -- no data loaded
# In addition: Warning message:
# file ‘srtm_56_06.tif’ has magic number 'II*'
# Use of save versions prior to 2 is deprecated
library(raster)
t = readTIFF("srtm_56_06/srtm_56_06.tif", as.is=TRUE)
# Error: could not find function "readTIFF"
read_file<-as.matrix(raster("srtm_56_06/srtm_56_06.tif") + min(read_file)
# Error: unexpected symbol in:
# "read_file<-as.matrix(raster("srtm_56_06/srtm_56_06.tif")
# min"
Can anyone help me with the commands to import this data. I'm a novice at "R" and a little lost.
Just read it with raster, but note you depend on rgdal being installed as well to read a .tif.
library(raster)
library(rgdal)
r <- raster("srtm_56_06/srtm_56_06.tif")
If that works, try
plot(r)
r
If it's really a "TIFF" then that should be fine, if it's really a GeoTIFF then you'll have a sensible map as well. (If it's something else that GDAL can read you might get a good result anyway, remember the extension of a file is not a reliable indicator of its contents).
The SRTM clue suggests that this is a single band DEM file from the tiled global SRTM data set. If it's somehow a "multi-band image" then you could read that with brick and plot with plotRGB (but I really doubt that is the case here). Note that there is a native binary format for SRTM that raster/rgdal could read as well but either they distributed .tif as well or someone else converted it.
There are a number of misconceptions in your code:
load is for a particular file type created from R (not these .tifs)
readTIFF is not in package raster
read_file would be a sensible matrix, if you have rgdal installed (which raster must use to load a .tif), but why throw away the spatial metadata?
I am receiving a sporadic error message with importShapefile in PBSmapping (version 2.63.37) in RStudio (0.97.318), running R version 2.15.2, platform: i386-w64-mingw32/i386 (32-bit). I also received the error while running previous versions of R and RStudio.
> ST6 = importShapefile("Data/pvi_stat_2002_utm.shp", projection="UTM", readDBF = TRUE)
Error in 1:nrow(dbf) : argument of length 0
> traceback()
2: cbind(1:nrow(dbf), dbf)
1: importShapefile("Data/pvi_stat_2002_utm.shp", projection = "UTM",readDBF = TRUE)
I only receive this error occasionally - perhaps 1 in every 10 times that I run the code. But once the error occurs in a session, it occurs repeatedly and will not successfully implement the command until I have closed R completely and reopened it. On one occasion I had to reboot the computer for it to work, as successive reopening of R did not help.
I thought it might be a memory issue but sometimes I will get the error when no objects are in the workspace. And usually the code runs fine even if I have large objects loaded. In response to the error I have removed all objects from the workspace and even followed with gc(), but to no avail.
This is the only shapefile with which I have received the error but as it is the only one that I use with regularity and since I can not predict when the error will occur, my efforts with other shapefiles are inconclusive. Not sure about uploading a shapefile to Stack Overflow. The zipped file is about 9MB.
Have a look in the folder where your shapefile is. Is there actually a .dbf file? If there is, it sounds like it is empty or corrupted, or misnamed. Are you expecting your shapefile to have polygons with attributes. Can you try importShapefile(... readDBF = FALSE )? Maybe you can make our data available through a dropbox link or something?
Alternatively have you tried rgdal:::readOGR or, my personal favourite, maptools:::readShapePoly(). I personally find readShapePoly() to be extremely robust and there are methods for coercing a SpatialPolygonsDataFrame from sp to a PolySet from PBS.
If you really must use PBS have you tried...
require( maptools )
require( sp )
myshp <- readShapePoly("Data/pvi_stat_2002_utm")
myshpPBS <- SpatialPolygons2PolySet( myshp )
I am assuming that there is a .prj file with your shapefile, describing the projection information?
I'm using R-3.0.1 and PBS Mapping 2.66.53 with the NAVO Divisions shapefile from http://www.nafo.int/about/overview/gis/Divisions.zip. On Windows 7 x86_64 and OS X Snow Leopard (using macports R built for x86_64), the .dbf is being read properly, but it
sometimes fails using RHEL 5.9:
> library("PBSmapping", lib.loc="/home/gwhite/R/x86_64-unknown-linux-gnu-library/3.0")
-----------------------------------------------------------
PBS Mapping 2.66.53 -- Copyright (C) 2003-2013 Fisheries and Oceans Canada
[...]
-----------------------------------------------------------
> library("rgeos", lib.loc="/home/gwhite/R/x86_64-unknown-linux-gnu-library/3.0")
rgeos version: 0.2-19, (SVN revision 394)
GEOS runtime version: 3.3.8-CAPI-1.7.8
Polygon checking: TRUE
> layer='Divisions'
> divs = importShapefile(layer, projection='LL')
Error in 1:nrow(dbf) : argument of length 0
Using readDBF=F does allow the shapefile data to be read:
> divs = importShapefile(layer, projection='LL', readDBF=F)
So far, importShapefile() has been working in a freshly started R session.
I am trying to read probes from a dat file, put them in a vector then put the vector into a subset to be able to concatenate more data to it and write it in a CSV file. Here my piece of code:
library(Biobase)
library(affy)
affys <- read.csv("address of my dat file")
affys_vec <- as.vector(affys)[,1]
exprs(eset)[affys_vec,] -> sub.set
write.csv(sub.set,file="subset.csv")
However when I reach to the : exprs(eset)[affys_vec,] -> sub.set
I get the following error message:
** Error in exprs(eset) :
error in evaluating the argument object' in selecting a method for function 'exprs':
Error: object 'eset' not found **
Are there any suggestions please?
Thanks,
pqtm
See the vignette An introduction to Biobase and ExpressionSets available on your computer, once Biobase is installed, as
vignette(package="Biobase", "ExpressionSetIntroduction")
But the idea is that you have created eset by pre-processing some CEL or other vendor-specific files. How you pre-process those depends on what your files are, maybe with affy or oligo or lumi, or from a public repository using a package like ArrayExpress or GEOquery Or perhaps you're using limma and have no need for an expression set.
The Bioconductor web site and mailing list provides a lot more information.