I'm trying to download a gpx file from the YOURS routing API (http://wiki.openstreetmap.org/wiki/YOURS#Routing_API) and upload the file to R using the readGPS function contained in the maptools package.
Here is the code:
require(utils)
require(maptools)
URL <- 'http://www.yournavigation.org/api/1.0/saveas.php?type=gpx&data=5.62373%2053.01,5.62359%2053.01014,5.62336%2053.01024,5.62314%2053.010303'
download.file(URL, 'tmpTrip.gpx')
gpx.raw <- readGPS(i='gpx', f='tmpTrip.gpx', type='t')
And the error I receive:
Error in readGPS(i = "gpx", f = "tmpTrip.gpx", type = "t") :
gpsbabel not found
I do have installed gpsbabel and I can see the gpx file correctly downloaded in my working directory.
My system and R version are:
Windows 7 Enterprise Service Pack 1, running on Intel Core i5-3320M CPU # 2.60GHz, 4GB RAM, 32 bit OS.
R version 3.1.0 (2014-04-10) -- "Spring Dance"
Copyright (C) 2014 The R Foundation for Statistical Computing
Platform: i386-w64-mingw32/i386 (32-bit)
Any help is much appreciated, thank you!!
Figuring it out the hard way, I found that readGPS() calls Sys.which which is a function that tries to find where a file is located on the system. ?Sys.which doesn't say it that clearly but (at least on Windows) for it to work, you need to add the install path of gpsbabel to your path.
If you have GPSBabel installed in your working directory as well, this should work.
For me, I changed the WD:
setwd("C:/Program Files (x86)/GPSBabel")
And then my code
gpx.raw <- readGPS(i = "gpx", f = "C:/Users/Desktop/waypoints.gpx", type="w")
worked after that.
For Mac users:
setwd("/Applications/GPSBabelFE.app")
Related
I'm having issues with Azure Machine Learning SDK for R: "module 'azureml' has no attribute 'core'"...
For reasons that aren't my own, I have to use azureml to apply machine learning (my own stuff, written in R) to data from our data warehouse that is put in the blob storage. The modelled output should be put back into the blob storage so it can be accessed from the data warehouse.
I've written the code in R on my local machine (stored in a git repo). Preferably, I'd find some method to pull my code from git into a pipeline in the azureml environment so that it can be directly run whenever new data is available in the blob storage.
I've embarked on a tutorial-spree and found this seemingly relevant walkthrough: Train and deploy your first model with Azure ML (and this one).
But... after trying all I could think of, I'm stuck on the first steps. After installing all (or at least.. that's what I think) packages, modules, apps etc, and running the following code in RStudio:
library(azuremlsdk)
existing_ws <- get_workspace(name = name,
subscription_id = subscription_id,
resource_group)
I run into an error that I haven't been able to fix:
AttributeError: module 'azureml' has no attribute 'core'
It seems that the azuerml is supposed to have an attribute "core", but when looking at it more closely, there is indeed no such attribute.
The function "get_workspace()" is trying to access: "azureml$core$Workspace$get".
I found that "azuerML$Workspace" does exist, but then I can't figure out how to make that work.
Can anyone explain to me why I'm encountering this error?
Does anyone know of a better tutorial on how to connect my R code the azureml's cloud service?
Any pointers in the right direction are much appreciated!
EDITS - still not solved:
After advice from others, I double, triple and quadruple checked the installation.
I updated R and I'm now running:
R.version
platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 3
minor 6.2
year 2019
month 12
day 12
svn rev 77560
language R
version.string R version 3.6.2 (2019-12-12)
nickname Dark and Stormy Night
I installed Conda with Python 3.6.10.
I installed the azuremlsdk R package (I tried both provided options).
I then realized that there are some inconsistencies with the versions of the azure-modules, so I also tried installing it with the keyword 'multi-arch':
remotes::install_cran('azuremlsdk', repos = 'http://cran.us.r-project.org', INSTALL_opts=c("--no-multiarch"))
Then, I installed the azureml python sdk.
I had a look at all the versions again (using python -m pip freeze):
azure-common==1.1.24
azure-graphrbac==0.61.1
azure-mgmt-authorization==0.60.0
azure-mgmt-containerregistry==2.8.0
azure-mgmt-keyvault==2.0.0
azure-mgmt-resource==7.0.0
azure-mgmt-storage==7.1.0
azureml==0.2.7
azureml-automl-core==1.0.83.1
azureml-core==1.0.69
azureml-dataprep==1.1.36
azureml-dataprep-native==13.2.0
azureml-pipeline==1.0.69
azureml-pipeline-core==1.0.69
azureml-pipeline-steps==1.0.69
azureml-sdk==1.0.69
azureml-telemetry==1.0.69
azureml-train==1.0.69
azureml-train-automl-client==1.0.83
azureml-train-core==1.0.69
azureml-train-restclients-hyperdrive==1.0.69
As I was surprised to see all the 1.0.69 versions, instead of the 1.0.83 versions, I re-installed the azureml python sdk using:
azuremlsdk::install_azureml(version = "1.0.83")
This worked, in the sense that indeed all versions are now 1.0.83:
azure-common==1.1.24
azure-graphrbac==0.61.1
azure-mgmt-authorization==0.60.0
azure-mgmt-containerregistry==2.8.0
azure-mgmt-keyvault==2.0.0
azure-mgmt-resource==7.0.0
azure-mgmt-storage==7.1.0
azureml==0.2.7
azureml-automl-core==1.0.83.1
azureml-core==1.0.83
azureml-dataprep==1.1.36
azureml-dataprep-native==13.2.0
azureml-pipeline==1.0.83
azureml-pipeline-core==1.0.83
azureml-pipeline-steps==1.0.83
azureml-sdk==1.0.83
azureml-telemetry==1.0.83
azureml-train==1.0.83
azureml-train-automl-client==1.0.83
azureml-train-core==1.0.83
azureml-train-restclients-hyperdrive==1.0.83
But still... I get the error with the missing core. I get it both when running:
library(azuremlsdk)
get_current_run()
and also when running:
library(azuremlsdk)
existing_ws <- get_workspace(name = name,
subscription_id = subscription_id,
resource_group)
Note that the first time running this code after starting up RStudio, I get the error:
Error in py_get_attr_impl(x, name, silent) :
AttributeError: module 'azureml' has no attribute '_base_sdk_common'
And every time after that I get this error:
Error in py_get_attr_impl(x, name, silent) :
AttributeError: module 'azureml' has no attribute 'core'
Any help would be much appreciated!
This issue was introduced by the latest reticulate 1.14 release, in which reticulate would create a default r-reticulate conda environment. Since Azure ML was installing the python SDK in an environment named r-azureml, the r-reticulate environment used by reticulate was missing the python SDK. A fix for this issue was addressed in a PR and has been merged into master. Please install from GitHub for now if you have reticulate version 1.14 and are running into this issue. We will be releasing an update to CRAN shortly.
I seemed to have fixed the issue by specifically installing the python package azureml AND azureml.core:
python -m pip install azureml
and then...
python -m pip install azureml.core
I did this for the Conda version that was called by R (r-reticulate). It's a bit odd to not be able to use the Conda environment 'r-azureml' without R switching back to 'r-reticulate', but ah well... at least I don't get my 'azureml' has no attribute 'core' anymore.
I have developed a package that I want to share with my colleagues at work.
I have a network drive in which I created the local repository structure that looks like this:
MyRepo
\__bin
\__windows
\__contrib
\__src
\__contrib
All folders are empty.
So I built my package with RStudio on Windows using the "Build/More/Build source package" menu, which created a tar.gz file.
Then I tried:
drat::insertPackage("../myPkg_0.0.0.9000.tar.gz",
repodir = "file://networkdrive/path/to/MyRepo",
action = "prune")
But this gives me an error:
Error: Directory file://networkdrive/path/to/MyRepo not found
Which is strange because file.exists(//networkdrive/path/to/MyRepo) returns true.
OK, then I tried:
drat::insertPackage("../myPkg_0.0.0.9000.tar.gz",
repodir = "//networkdrive/path/to/MyRepo",
action = "prune")
Without the file: in the repository path and I get another error:
tar (child): "//networkdrive/path/to/MyRepo/src/contrib/myPkg_0.0.0.9000.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
/usr/bin/tar: Child returned status 2
/usr/bin/tar: myPkg/DESCRIPTION: Not found in archive
/usr/bin/tar: Exiting with failure status due to previous errors
reading DESCRIPTION for package ‘myPkg’ failed with message:
cannot open the connection
But when I go in the "//networkdrive/path/to/MyRepo/src/contrib" folder, I can definitely see the myPkg_0.0.0.9000.tar.gz file that has been copied despite the error message.
Can anyone help?
> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
locale:
[1] LC_COLLATE=French_France.1252 LC_CTYPE=French_France.1252 LC_MONETARY=French_France.1252 LC_NUMERIC=C LC_TIME=French_France.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] drat_0.1.2 tools_3.3.3 git2r_0.18.0
I know this is old, but my colleague just came across the same problem and found this post. I believe the issue may be the lack of a trailing slash in your directory name. I have been able to recreate the error with a mapped network drive. I can resolve the issue by using "H:/MyRepo/" instead of "H:/MyRepo".
I haven't tried it with the "file://" format, but I wanted to include my answer in case someone else comes across this question.
Ok, so after some research, here is my conclusions.
It cannot be done
It's not Drat's fault
The reason why it does not work is that the tools::write_PACKAGES function does not work on network drives. Period.
I manually copied my package on the network drive, then ran setwd() to its location and executed write_PACKAGES(".", type="source") and I got the same error.
So to make this work, I just left my package.tar.gz file on a local drive, ran the tools::write_PACKAGES command locally and then moved the files to the network drive.
Adding the network drive to my repository list using options(repos = c(MyRepo = "file://networkdrive/path/to/MyRepo/")) works: RStudio and available.packages find my package.
It's not completely satisfactory, but I think it's the only way today.
I was having this problem as well and finally got to the bottom of it today.
For me, the problem was not isolated to just network locations but also occurred on C: drive. The root cause was the version of tar.exe being used to unpack the existing packages in the package directory. Calls to utils::untar are made in the tools::write_PACKAGES function.
The documentation for utils::untar explains that on Windows, external tar.exe is tried first. Sure enough, I had a version installed with Git which when used with default arguments fails when a colon is in the file name. I was able to force utils::untar to use to use the RBuildTools version of tar.exe instead by setting the environment variable TAR to "internal".
drat::insertPackage now works.
I am trying to use XLConnect library in R. If I execute
library(XLConnect)
I get the following error message:
JAVA_HOME cannot be determined from the Registry
To resolve this problem I set first the JAVA_HOME variable:
Sys.setenv(JAVA_HOME='C:/Program Files (x86)/Java/jre1.8.0_65')
library(XLConnect)
It looks like it helps me to come further but then I get another problem:
unable to load shared object 'C:/Program Files/R/R-3.2.2/library/rJava/libs/x64/rJava.dll'
It wonder why R cannot load rJava.dll. At least this file is located in the folder where R searches for it:
C:\Program Files\R\R-3.2.2\library\rJava\libs\x64
ADDED
Please note that the rJava.dll file exists and it is located there, where R is searching for it. I guess that the problem is in incompatibility between 32bit and 64bit versions. I assume that because R complains:
% 1 is not a valid Win32 application
Well, why do R expect it to be a Win32 application`? First, my OS is 64bit, second my Java is also for the 64bit and finally, the `rJava.dll` object is located in the folder withx64` in the name (so, I assume it is also a 64bit version).
I faced the same issue . Please locate jvm.dll
should be in (your JRE version could be different )
C:\Program Files (x86)\Java\jre1.8.0_65\bin\client
or
C:\Program Files (x86)\Java\jre1.8.0_65\bin\server
add this path to your windows system path and you are good to go .but keep in mind the version of jre and R should be consistent,if your java is in Program Files its 64 bit so launch from 64 bit R if its in Program Files (x86)its 32 bit so use 32 bit R
like in my case it showed error in 64 bit
but worked perfectly in 32 bit
You did use / instead of \.
Sys.setenv(JAVA_HOME='C:\\Program Files (x86)\\Java\\jre1.8.0_65')
library(XLConnect)
I am using UNIX. Therefore I cannot test it by myself but your path might be wrong as well.
According to this post you can search it by using this:
find.java <- function() {
for (root in c("HLM", "HCU")) for (key in c("Software\\JavaSoft\\Java Runtime Environment",
"Software\\JavaSoft\\Java Development Kit")) {
hive <- try(utils::readRegistry(key, root, 2),
silent = TRUE)
if (!inherits(hive, "try-error"))
return(hive)
}
hive
}
credit goes to #nograpes for the function and this article for helping me giving you the answer.
I am trying to read an Excel file that is stored on a corporate intranet SharePoint site, and am doing so using the 'gdata' package from within R Studio run on a Linux server using Shiny Server.
I had good results reading the file from within a MS Windows environment using gdata, but cannot seem to get things to work when running the script on the Linux server. This was based on the info from:
http://r.789695.n4.nabble.com/trying-to-import-xls-or-xlsx-files-td3620580.html
I did modify the R script for the path to Perl on the Linux server (vs. the path to perl.exe on Windows), and this seems to work ok.
The file url is
"http://sharepoint2/ops/quality/metricspc/Metric OptIn List/temporary SPC Metric Opt-In List.xlsm"
Here is the R code:
# R read MS Excel xlsm file from SharePoint
# method using gdata - seems to work with SharePoint
# NOTE: requires 'perl' installed
#
# example from
# http://r.789695.n4.nabble.com/trying-to-import-xls-or-xlsx-files-td3620580.html
library(gdata)
fileurl =
# see fileurl above this code section - did this due to SO error message about not having 'sharepoint2' in the url
d.optin.init2 <- read.xls(fileurl,
sheet = "OPT-IN LIST",
perl = "/usr/bin/perl")
##### END CODE #####
The original (Windows-based) script used perl = "C:\\Perl64\\bin\\perl.exe"
Here is the error message that results (when run from Linux using R Studio on Shiny Server):
d.optin.init2 <- read.xls(fileurl,
+ sheet = "OPT-IN LIST",
+ perl = "/usr/bin/perl")
trying URL 'http://sharepoint2/ops/quality/metricspc/Metric OptIn List/temporary SPC Metric Opt-In List.xlsm'
Error in download.file(xls, tf, mode = "wb") :
cannot open URL 'http://sharepoint2/ops/quality/metricspc/Metric OptIn List/temporary SPC Metric Opt-In List.xlsm'
In addition: Warning message:
In download.file(xls, tf, mode = "wb") :
cannot open: HTTP status was '400 Bad Request'
Error in file.exists(tfn) : invalid 'file' argument
The path to the file (on SharePoint) is a URL (shown in the code), so I thought that the route between the Linux server and MS SharePoint might be straightforward. But since this does work for me with Windows but does not yet work for me with Linux, I wonder what I may have missed?
Thank you in advance for any insight that might be offered.
(I haven't attached the Excel file because I wasn't sure that it would help with this question.)
Best regards,
Cliff
Output from sessionInfo
sessionInfo()
R version 3.1.0 (2014-04-10)
Platform: x86_64-redhat-linux-gnu (64-bit)
locale:
1 C
attached base packages:
1 stats graphics grDevices utils datasets methods base
other attached packages:
1 gdata_2.13.3 qcc_2.5
loaded via a namespace (and not attached):
1 MASS_7.3-31 gtools_3.4.1 tools_3.1.0
Acting on the suggestions provided by hrbrmstr and also Greg, I tried both wget and also curl from the Linux command line.
wget results
curl results
I can work with our IT folks to resolve. If anyone can help me to refine the questions I might ask of them based on these results, I would welcome this input.
Thanks again to those who took the time to respond.
MORE FOLLOW-UP
#Gregory R. Warnes
I was able to use wget with this command line from the Linux server:
wget --http-user=myusername --http-passwd=mypassword (place fileurl here)
This seemed to access the file, bridging the divide between Linux server and Windows SharePoint.
Now to figure out how to include this AD authentication in the gdata R script.
The error message indicates that R's download.file() code was not able to access the URL.
To debug the issue, try accessing that URL from the Linux server from the shell command line, for example:
If wget is installed:
> wget http://sharepoint2/ops/quality/metricspc/Metric OptIn List/temporary SPC Metric Opt-In List.xlsm
or if curl is installed:
> curl http://sharepoint2/ops/quality/metricspc/Metric OptIn List/temporary SPC Metric Opt-In List.xlsm
If wget and/or curl generates an error, then you have a server configuration issue.
If wget and/or curl succeed then the problem may be with R's download.file().
-Greg
I am downloading data from data.gov website and I get following two types of errors in the process:
fileUrl <- "http://catalog.data.gov/dataset/expenditures-on-children-by-families"
download.file(fileUrl,destfile=".data/studentdata.csv",method="curl")
Warning message:
In download.file(fileUrl, destfile = ".data/studentdata.csv", method = "curl") :
download had nonzero exit status
I tried to remove the method="curl" as suggested in other forum, but again I get this new error
download.file(fileUrl,destfile=".data/studentdata.csv")
Error in download.file(fileUrl, destfile = ".data/studentdata.csv") :
cannot open destfile '.data/studentdata.csv', reason 'No such file or directory'
I think there are two major factors why your curl doesn't work well.
First, the problem is on your URL. fileUrl <- "http://catalog.data.gov/dataset/expenditures-on-children-by-families". In your URL, it is not referred to a csv file. So, they won't work even if you set the destination into a csv file such as destfile = ".data/studentdata.csv"
I have an example of getting a csv dataset using the same code (different dataset):
DataURL<- "https://data.baltimorecity.gov/api/views/dz54-2aru/rows.csv?accessType=DOWNLOAD" (This link refers to a rows.csv file)
download.file(DataURL, destfile="./data/rows.csv", method="curl") (The method is quite same, using curl)
Second, previously I had the same problem that the curl does not work, even I used a proper URL that refers to a csv file. However, when I diagnosed a bit deeper, I found something interesting fact about why my curl method cannot work properly. It was my R session program. I used a 32-bit R, in which the error occurs. Later then, I tried to change the session into a 64-bit R. Amazingly, and the download status was running at that time. To see your R session architecture (whether you are using 32-bit or 64-bit), type in your R:
sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-w64-ming32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
You have to switch your R, from 32-bit to 64-bit to avoid 'curl' call had nonzero exit status. You go to your R directory folder, and then you run a 64-bit R.
If you are using a Windows OS and installing the R in a default path folder, you can run this C:\Program Files\R\R-3.5.3\bin\x64\R.exe. (I used a version of 3.5.3, so it may be different with your version)
If you are using R-studio, you can switch the R session on the menubar Tools -> Global Options -> R version -> Change -> Use your machine's default version of R64 (64-bit) -> OK. Then restart your R-studio.
However, it depends on your OS architecture. If you are using a 32-bit OS, hence you have to find another way to solve this.
So looking at the code for download.file(...), if you specify method="curl" the function tries to use the curl shell command. If this command does not exist on your system, you will get the error above.
If you do not specify a method, the default is to use an internal R method to download, which evidently works on your system. In that case, the function is trying to put the file in .data/studentdata.csv but evidently there is not .data directory. Try taking out the ..
When this download works, you will get a text/html file, not a csv file. Your url points to a web page, not a download link. That page does have a download link, but unfortunately it is a pdf, not a csv.
Finally, if your goal is to have the data in R (is it?), and if the link actually produces a csv file, you could more easily use
df <- read.csv(fileUrl)
If I'm not very much mistaken you just have a simple typo here. I suspect you have a "data" directory, not a ".data" directory - in which case your only problem is that your destfile string needs to begin "./data", not ".data".
I was having the same problem.
Then I realized that I forget to create the "data" directory!
So try adding this above your fileURL line to create the directory first.
if(!file.exists("data")){
dir.create("data")
}
Also, if you are running a Mac, then you want to keep method="curl" when downloading a https file. I don't believe Windows has that problem hence the suggestions to remove it.
Try this:
file<-'http://catalog.data.gov/dataset/expenditures-on-children-by-families'
file<- read.csv(file)