The osmar package in R has a demo file called demo("navigator"). It is provided to illustrate package capabilities and functions. When I ten the script, I hit the following line and error:
R> muc <- get_osm(muc_bbox, src)
sh: osmosis: command not found
Error in file(con, "r") : cannot open the connection
In addition: Warning message:
In file(con, "r") :
cannot open file '/var/folders/81/4k487q0969q1d8rfd1pyhyr40000gs/T//RtmpdgZSOy/file13a473cb904c': No such file or directory
The command is intended to convert an osmosis data object to a osmar object. I have properly installed osmosis for MacOSX, updated my path definition in the bash shell to point to the osmosis executable.
I'm not sure what the error message means and how best to respond. Any help appreciated
Brad
Have your restarted R? It looks like osmosis isn't in your path, although you do mention that you set that. Make sure that you can run one of the osmosis commands in Terminal:
osmosis --read-xml SloveniaGarmin.osm --tee 4 --bounding-box left=15 top=46 --write-xml SloveniaGarminSE.osm --bounding-box left=15 bottom=46 --write-xml SloveniaGarminNE.osm --bounding-box right=15 top=46 --write-xml SloveniaGarminSW.osm --bounding-box right=15 bottom=46 --write-xml SloveniaGarminNW.osm
The example is irrelevant, as long as it doesn't say osmosis file not found.
Also, make sure you have gzip in your path. I am almost certain that it is default, but the demo package relies on it to run. Just open a Terminal and type gzip to make sure it is there.
Finally, if you need to debug this, then run this:
library(osmar)
download.file("http://osmar.r-forge.r-project.org/muenchen.osm.gz","muenchen.osm.gz")
system("gzip -d muenchen.osm.gz")
# At this point, check the directory listed by getwd(). It should contain muenchen.osm.
src <- osmsource_osmosis(file = "muenchen.osm",osmosis = "osmosis")
muc_bbox <- center_bbox(11.575278, 48.137222, 3000, 3000)
debug(osmar:::get_osm_data.osmosis)
get_osm(muc_bbox, src)
# Press Enter till you get to
# request <- osm_request(source, what, destination)
# Then type request to get the command it is sending.
After you type Enter once, and then request you will get the string it is sending to your OS. It should be something like:
osmosis --read-xml enableDateParsing=no file=muenchen.osm --bounding-box top=48.1507120588903 left=11.5551240885889 bottom=48.1237319411097 right=11.5954319114111 --write-xml file=<your path>
Try pasting this into your Terminal. It should work from any directory.
Oh, and type undebug(osmar:::get_osm_data.osmosis) to stop debugging. Type Q to exit the debugger.
Hey I just got this thing working. The problem is not with the system path variable for osmosis. It is with the system call the script makes which uses the "gzip" application to unzip the .gz file it has downloaded before. So there is an error when gzip is not installed in your machine or gzip is not in the system path variable. so installing gzip and adding it to the path variable will mitigate this error. alternatively you can unzip the file manually to the same path and run the script again.
Related
I'm updating the renv folder from a project in order to adjust the libraries, but it seems I'm having a permission problem. After running renv::init() and trying to installing manually the remaining libraries using install.packages() I always get the message
Error: failed to retrieve 'https://cran.rstudio.com/bin/windows/contrib/4.2/ipeadatar_0.1.6.zip' [error code 23]
1: curl: (23) Failure writing output to destination
2: curl: (23) Failure writing output to destination
Using .libPath() I can see that the renv was created in the "AppData" hidden folder
1] "C:/Users/André Ferreira/AppData/Local/R/cache/R/renv/library/MacroBRA_Wrld-09789847/R-4.2/x86_64-w64-mingw32"
So checking my permissions, I couldn't see anything wrong. Any thoughts about this problem? The thing it's that when I open my .Rmd file and try to knit, I receive the same message "1: curl: (23) Failure writing output to destination", now from rmarkdown retrieve installation, so it may be a configuration/permission problem.
Adding "C:\rtools42\usr\bin" and "C:\Program Files\R\R-4.2.1\bin" in the environment variable didn't help.
As I could see, opening an empty file from rstudio, I could use install.packages() without problem.
Although this doesn't solve the problem directly, you can also instruct renv to use a different library path with something like:
# use a project-local library path
RENV_PATHS_LIBRARY = renv/library
in your project's .Renviron file. Depending on your environment, you might also consider placing the library path in an alternate location.
See https://rstudio.github.io/renv/articles/packages.html#r-cmd-build-and-the-project-library for more details.
I am trying to upload a data.frame called 'ftp_test' via ftpUpload command
library(RCurl)
ftpUpload("Localfile.html", "ftp://User:Password#FTPServer/Destination.html")
and am getting an error:
Error in file(what, "rb") : cannot open the connection
In addition: Warning message:
In file(what, "rb") :
cannot open file 'ftp_test': No such file or directory
Could anyone tell me what is the issue here? Can I actually use data.frame and upload from r global environment ?
If I can't use the data.frame is there any workaround?
Many thanks,
Artur
You problem is, that you are trying to send an R object with an file transfer protocol. Since you are saving it there, you have to tell how to save it. A workaround is to save it as a file, upload it and then delete it on your local afterwards. Also saving as R.History is fine, but you need to transfer the R object to a file in some way. This example is used with an open ftp sever (uploads get deleted immediately, but you can try if it works)
filename="test.csv"
write.csv(df, file=filename)
#use your path to the csv file here instead of ".~/test.csv", you can check with getwd()
ftpUpload("~/test.csv", paste("ftp://speedtest.tele2.net/upload/",filename, sep=""))
file.remove(filename)
Also make sure your server is running. You can try your code with the open ftp server.
I recently updated R to the latest release: 3.4.0. R is installed on a network location H:/. Now something weird is happening when I try to install a local binary package:
filename <- paste0("R:/path/independeR_", versions, ".zip")
install.packages(filename,
repos = NULL, type = "source",
lib = gsub("\\\\\\\\networkpath/home/[[:alpha:]]*/",
"H:/", .libPaths()[1]))
Both H:/ and R:/ are network locations. In .libPaths() the default location is in the H:/ location, but it shows with the entire network adress. In the call to install.packages I substituted this.
The code above fails, with the following output:
'\\networkpath\home\JDUB~PN6\DOCU~UZL\R\R-34~TN4.0' is not recognized as an internal or external command,
operable program or batch file.
Warning in install.packages :
running command '"//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/bin/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip"' had status 1
Warning in install.packages :
installation of package ‘R:/path/independeR_0.1.8.zip’ had non-zero exit status
There is two things that surprise me here. The directorynames are all jumbled up (DOCU~UZL instead of Documents etc), but for some reason Command promt seems fine with that. The more interesting thing is the following:
When I try to put the command "//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/bin/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip" directly into Command Prompt, the output is very similar:
C:\Users\jdubbeldam>"//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/b
in/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip"
'\\networkpath\home\JDUB~PN6\DOCU~UZL\R\R-34~TN4.0' is not recognized as
an internal or external command,
operable program or batch file.
For some reason CMD seems to cut off the path to the command halfway through. I guess that this is because the command is too long. When I try the same command, but with H:/Documents/R/R-3.4.0/bin/x64/R, the installation goes just fine.
I would like to be able to automatically install this package from a script, so I would like a solution to this problem from within R. Is there a way to get R to use the shorter H:/Documents/R/R-3.4.0/bin/x64/R?
I was having problems with updating packages, while searching I found that there is a reported bug affecting getting the timestamps from files in Windows 10:
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17261
Starting in R 3.4.0, file.info sometimes returns for mtime,
atime and ctime for directories. It seems to have something to do
with sharing. This affects functions that use file.mtime like
update.packages.
Reproduce:
Try file.info() with a random directory. If it returns a legitimate
file time, open a windows explorer window and navigate to the
directory, then run the call again and it will return s.
In some cases it returns s even if the directory is not open (or
in the path of an open explorer window), but this is not consistent.
With a comment of Tomas Kalibera:
Thank you for the report, this is a known bug that has been fixed
recently in R-devel
I found that when I closed the file explorer windows the timestamps worked correctly and didn't show up as NA's.
This may be a lead for why your code isn't working.
I managed to make the following work. However, I think it is extremely ugly, and would still like to see if it is possible to do it someway else.
filename <- paste0("R:/path/independeR_", versions, ".zip")
cmd <- file.path(gsub("//networkpath/home/[[:alnum:]]*/", "H:/",
gsub("//networkpath/home/[[:alnum:]]*~[[:alnum:]]*/",
"H:/", R.home())), "bin/x64/R")
libname <- gsub("\\\\\\\\networkpath/home/[[:alpha:]]*/",
"H:/", .libPaths()[1])
call <- paste(paste0('"', cmd, '"'),
"CMD",
"INSTALL",
"-l",
paste0('"', libname, '"'),
"--no-lock",
paste0('"', filename, '"'),
sep = " ")
system(call)
Okay - so here is what I'm trying to do.
I've got this password protected CSV file I'm trying to import into R.
I can import it fine using:
read.csv()
and when I run my code in RStudio everything works perfect.
However, when I try and run my .R file using a batch file (windows .bat) it doesn't work. I want to use the .BAT file so that I can set up a scheduled task to run my code every morning.
Here is my .BAT file:
"E:\R-3.0.2\bin\x64\R.exe" CMD BATCH "E:\Control Files\download_data.R" "E:\Control Files\DailyEmail.txt"
And here is my .R file:
url <- "http://username:password#www.url.csv"
data <- read.csv(url, skip=1)
** note, I've put my username/password and the exact location of the CSV in my code. I've used generic stuff here, as this is work related and posting usernames and passwords is probably frowned upon.
As I've said, this code works fine when I use it in RStudio. But fails when I use the .BAT file.
I get the following error message:
Error in download.file(url, "E:/data/data.csv") :
cannot open URL 'websiteurl'
In addition: Warning message:
In download.file(url, "E:/data/data.csv") :
unable to resolve 'username'
Execution halted
** above websiteurl is the http above (I can't post links)
So obviously, the .BAT is having trouble with the username/password? Any thoughts?
* EDIT *
I've gone so far as trying this on Linux. Thinking maybe windows was playing silly bugger.
Just from the terminal, I run Rscript -e "download_data.r" and get the EXACT same error message as I did in Windows. So I suspect this may be a problem with where I'm getting the data? Could the provider be blocking data from the command line, but not from with Rstudio?
I have had similar problems which had to do with file permissions. The .bat file somehow does not have the same privileges as you running the code directly from Rstudio. Try using rscript (http://stat.ethz.ch/R-manual/R-devel/library/utils/html/Rscript.html) within your .bat file like
Rscript "E:\Control Files\download_data.R"
What is the purpose of the argument "E:\Control Files\DailyEmail.txt"? Is the program suppose to use it in any way?
So, I've found a solution, which is likely not the most practical for most people, but works for me.
What I did was migrated my project over to a Linux system. Running daily scripts, is easier on Linux anyways.
The solution makes use of the "wget" function in linux.
You can either run the wget right in your shell script, or make use of the system() function in R to run the wget.
code looks like:
wget -O /home/user/.../file.csv --user=userid --password='password' http://www.url.com/file.csv
And you can do something like:
syscomand >- "wget -O /home/.../file.csv --user=userid --password='password' http://www.url.com/file.csv"
system (syscommand)
in R to download the CSV to a location on your hard drive, then grab the CSV using read.csv()
Doing it this way gave me some more insight into the potential root cause of the problem. While the system(syscommand) is running, I get the following output:
Connecting to www.website.com (www.website.com)|ip.ad.re.ss|:80... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Reusing existing connection to www.weburl.com:80.
HTTP request sent, awaiting response... 200 OK
Not sure why it has to send the request twice? And why I'm getting a 401 Unauthorized the first try?
I wanted to run this example script: http://mazamascience.com/WorkingWithData/?p=912 from Windows command line. So I opened the command line and typed Rscript tryCatch.R 1. However, I keep getting the error message Error: R not found. I did set the PATH environment variable as C:\Programme\R\R-3.0.1\bin. If I just type R.exe, it does start R, but it cannot find the packages that are to be loaded at start (e.g. package 'utils' in options<"defaultPackages"> was not found). I guess I have to set another path to the libraries somewhere, but I haven't got any idea where to do this.
UPDATE: After explicitly typing PATH C:\Programme\R\R-3.0.1\bin (rather than just adding this to the value of the environment variable PATH) it seems that R is found. However, a new problem occurs: In normalizePath<path.expand(path), winslash, mustWork>: path[2] = "C:/Programme/R/R-3.0.1/library": Access denied, the same than for the methods library. Then: Calls: .First ... library -> .getRequiredPackages2 -> library -> normalizePath Execution stopped. I'm using Windows 7 and I do have administrator rights.
Rscript is very handy (R CMD BATCH is the old way to ) specially under windows, But generally under I create a batch file to avoid all path's headache.
For example say launcher.bat:
#echo off
C:
PATH C:\Programme\R\R-3.0.1\bin;%path%
cd PATH_TO_YOUR_RSCRIPT
Rscript tryCatch.R 1
pause
And open a console(using cmd) , go where you have stored your launcher.bat and launch it. Or from the R cosnole using shell:
shell('path_to_launcher\launcher.bat')
I've found out that it was a language-specific problem on Windows 7, similar to what is described here: https://stat.ethz.ch/pipermail/r-help/2011-May/276932.html
After changing PATH to C:\Program Files\R\R-3.0.1\bin the script is properly executed from the command prompt.
Thanks to everyone who tried to help!
I ran into this problem under windows 7, apparently, when setting environment variables>user variables the path is not added into the PATH, so the user must add this path in system variables > PATH
at the end just add the path to your .EXE files and voila.