I am downloading data from data.gov website and I get following two types of errors in the process:
fileUrl <- "http://catalog.data.gov/dataset/expenditures-on-children-by-families"
download.file(fileUrl,destfile=".data/studentdata.csv",method="curl")
Warning message:
In download.file(fileUrl, destfile = ".data/studentdata.csv", method = "curl") :
download had nonzero exit status
I tried to remove the method="curl" as suggested in other forum, but again I get this new error
download.file(fileUrl,destfile=".data/studentdata.csv")
Error in download.file(fileUrl, destfile = ".data/studentdata.csv") :
cannot open destfile '.data/studentdata.csv', reason 'No such file or directory'
I think there are two major factors why your curl doesn't work well.
First, the problem is on your URL. fileUrl <- "http://catalog.data.gov/dataset/expenditures-on-children-by-families". In your URL, it is not referred to a csv file. So, they won't work even if you set the destination into a csv file such as destfile = ".data/studentdata.csv"
I have an example of getting a csv dataset using the same code (different dataset):
DataURL<- "https://data.baltimorecity.gov/api/views/dz54-2aru/rows.csv?accessType=DOWNLOAD" (This link refers to a rows.csv file)
download.file(DataURL, destfile="./data/rows.csv", method="curl") (The method is quite same, using curl)
Second, previously I had the same problem that the curl does not work, even I used a proper URL that refers to a csv file. However, when I diagnosed a bit deeper, I found something interesting fact about why my curl method cannot work properly. It was my R session program. I used a 32-bit R, in which the error occurs. Later then, I tried to change the session into a 64-bit R. Amazingly, and the download status was running at that time. To see your R session architecture (whether you are using 32-bit or 64-bit), type in your R:
sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-w64-ming32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
You have to switch your R, from 32-bit to 64-bit to avoid 'curl' call had nonzero exit status. You go to your R directory folder, and then you run a 64-bit R.
If you are using a Windows OS and installing the R in a default path folder, you can run this C:\Program Files\R\R-3.5.3\bin\x64\R.exe. (I used a version of 3.5.3, so it may be different with your version)
If you are using R-studio, you can switch the R session on the menubar Tools -> Global Options -> R version -> Change -> Use your machine's default version of R64 (64-bit) -> OK. Then restart your R-studio.
However, it depends on your OS architecture. If you are using a 32-bit OS, hence you have to find another way to solve this.
So looking at the code for download.file(...), if you specify method="curl" the function tries to use the curl shell command. If this command does not exist on your system, you will get the error above.
If you do not specify a method, the default is to use an internal R method to download, which evidently works on your system. In that case, the function is trying to put the file in .data/studentdata.csv but evidently there is not .data directory. Try taking out the ..
When this download works, you will get a text/html file, not a csv file. Your url points to a web page, not a download link. That page does have a download link, but unfortunately it is a pdf, not a csv.
Finally, if your goal is to have the data in R (is it?), and if the link actually produces a csv file, you could more easily use
df <- read.csv(fileUrl)
If I'm not very much mistaken you just have a simple typo here. I suspect you have a "data" directory, not a ".data" directory - in which case your only problem is that your destfile string needs to begin "./data", not ".data".
I was having the same problem.
Then I realized that I forget to create the "data" directory!
So try adding this above your fileURL line to create the directory first.
if(!file.exists("data")){
dir.create("data")
}
Also, if you are running a Mac, then you want to keep method="curl" when downloading a https file. I don't believe Windows has that problem hence the suggestions to remove it.
Try this:
file<-'http://catalog.data.gov/dataset/expenditures-on-children-by-families'
file<- read.csv(file)
Related
My error messages are displayed in French. How can I change my system language setting so the error messages will be displayed in English?
You can set this using the Sys.setenv() function. My R session defaults to English, so I'll set it to French and then back again:
> Sys.setenv(LANG = "fr")
> 2 + x
Erreur : objet 'x' introuvable
> Sys.setenv(LANG = "en")
> 2 + x
Error: object 'x' not found
A list of the abbreviations can be found here.
Sys.getenv() gives you a list of all the environment variables that are set.
In the case of RStudio for Windows I succeeded in changing the language following the instructions found in R for Windows FAQ, in particular I wrote:
language = EN
inside the file Rconsole (in my installation it is C:\Program Files\R\R-2.15.2\etc\Rconsole); this works also for the command Rscript.
For example you can locate the Rconsole file with this two commands from a command prompt:
cd \
dir Rconsole /s
The first one make the root as the current directory, the second one looks for the Rconsole file.
In the following screenshot you have that Rconsole file is in the folder C:\Program Files\R\R-3.4.1\etc.
You may have more than one location, in that case you may edit all the Rconsole files.
After that you can open the Rconsole file with your favorite editor and look for the line language = and then append EN at the end of that line.
In the following screenshot the interesting line is the number 70 and you have to append EN at the end of it.
For mac users, I found this on the R for Mac FAQ
If you use a non-standard setup (e.g. different language than
formats), you can override the auto-detection performed by setting
`force.LANG' defaults setting, such as for example
defaults write org.R-project.R force.LANG en_US.UTF-8
when run in Terminal it will enforce US-english setting regardless of the system
setting. If you don't know what Terminal is you can use this R command
instead:
system("defaults write org.R-project.R force.LANG en_US.UTF-8")
but do not forget to quit R and start R.app again afterwards. Please
note that you must always use `.UTF-8' version of the locale,
otherwise R.app will not work properly.
This helped me to change my console language from Chinese to English.
This works from command line :
$ export LANG=en_US.UTF-8
None of the other answers above worked for me
If you use Ubuntu you will set
LANGUAGE=en
in /etc/R/Renviron.site.
You may also want to be aware of the difference between, for example, Sys.setenv(LANG = "ru") and Sys.setlocale(locale = "ru_RU.utf8").
> Sys.setlocale(locale = "ru_RU.utf8")
[1] "LC_CTYPE=ru_RU.utf8;LC_NUMERIC=C;LC_TIME=ru_RU.utf8;LC_COLLATE=ru_RU.utf8;LC_MONETARY=ru_RU.utf8;LC_MESSAGES=en_IE.utf8;LC_PAPER=en_IE.utf8;LC_NAME=en_IE.utf8;LC_ADDRESS=en_IE.utf8;LC_TELEPHONE=en_IE.utf8;LC_MEASUREMENT=en_IE.utf8;LC_IDENTIFICATION=en_IE.utf8"
If you are interested in changing the behaviour of functions that refer to one of these elements (e.g strptime to extract dates), you should use Sys.setlocale().
See ?Sys.setlocale for more details.
In order to see all available languages on a linux system, you can run
system("locale -a", intern = TRUE)
To permanently make it works, in both R and Rstudio (with Win 10),
one way to do this is to run the script every time automatically in the background initially.
No more changing the system language that influence the windows.
No more R only but fail in Rstudio. No more run a script every time manually. No more admin right but fail. No more short-cut setting but fail.
Step 1.
Use your system search, to find the file named "Rprofile"
My response is
C:\Program Files\R\R-4.0.5\library\base\R
C:\Program Files\R\R-4.0.5\etc
Step 2.
Edit C:\Program Files\R\R-4.0.5\library\base\R\Rprofile
The content:
This is the system Rprofile file. It is always run on startup.
Additional commands can be placed in site or user Rprofile files
(see ?Rprofile)
... and so on.
Step 3. Add Sys.setenv(LANGUAGE="en") at the end of the scrip
local({
Sys.setenv(LANGUAGE="en")
})
P.S. If you encounter the issue of authorization/saving,
move this file to desktop and replace the original file after editing.
type this first:
system("defaults write org.R-project.R force.LANG en_US.UTF-8")
then you will get a index number(in my case is 127)
then type:
Sys.setenv(LANG = "en")
then type the number and ENTER
127
For me worked:
Sys.setlocale("LC_MESSAGES", "en_US.utf8")
Testing:
> Sys.setlocale("LC_MESSAGES", "en_US.utf8")
[1] "en_US.utf8"
> x[3]
Error: object 'x' not found
Also working to get english messages:
Sys.setlocale("LC_MESSAGES", "C")
To reset to german messages I used
Sys.setlocale("LC_MESSAGES", "de_DE.utf8")
Here is the start of my sessionInfo:
> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.2 LTS
A simple solution would be setting export Lang=C in your bash script.
I had a similar issue where the default language was german so it reverted back to english.
If you want to change R's language in terminal to English forever, this works fine for me in macOS:
Open terminal.app, and say:
touch .bash_profile
Then say:
open -a TextEdit.app .bash_profile
These two commands will help you open ".bash_profile" file in TextEdit.
Add this to ".bash_profile" file:
export LANG=en_US.UTF-8
Then save the file, reopen terminal and type R, you will find it's language has changed to english.
If you want language come back to it's original, just simply add a # before export LANG=en_US.UTF-8.
The only thing that worked for me was uninstalling R entirely (make sure to remove it from the Programs files as well), and install it, but unselect Message Translations during the installation process. When I installed R, and subsequently RCmdr, it finally came up in English.
In Ubuntu 14.04 LTS I had to remove the # from the comment #LANGUAGE=EN.
All other options din not work for me.
Change your current regional format to a different regional format in region settings on time&language settings in Windows by clicking on your time/date in lower right corner > adjust time/date > Region > change regional format to UK or US
This worked for me with a windows 10 laptop in German, where I wanted i.e. lubridate to return dates in English:
Sys.setlocale("LC_TIME", "English")
Im using R Studio on a Mac and I couldn't find the Rconsole file. So I took a more brutal way and just deleted the unwanted language files from the R app.
You just have to go to your Rapp in your application Folder, right click, show package content then /contents/Resources/. There are the language files e.g. English.lproj or in my case de.lproj wich I deleted. After restarting R, error messages appear in English..
Maybe thats helpful!
you simply have to change the basic language of microsoft on your computer!
press the windows button together with r, and tip the following code into the window that is opened
control.exe /name Microsoft.Language
load the language package you want to use and change the options. but take care, this will change also your keyboard layout!
on windows, when you have no admin right, just create a new program shortcut to Rgui.exe. Then in the properties of that shortcut, go to the 'Shortcut' tab and modify the target to include the system language of your choice, e.g. "C:\Program Files\R\R-3.5.3\bin\x64\Rgui.exe" LANGUAGE=en
I recently updated R to the latest release: 3.4.0. R is installed on a network location H:/. Now something weird is happening when I try to install a local binary package:
filename <- paste0("R:/path/independeR_", versions, ".zip")
install.packages(filename,
repos = NULL, type = "source",
lib = gsub("\\\\\\\\networkpath/home/[[:alpha:]]*/",
"H:/", .libPaths()[1]))
Both H:/ and R:/ are network locations. In .libPaths() the default location is in the H:/ location, but it shows with the entire network adress. In the call to install.packages I substituted this.
The code above fails, with the following output:
'\\networkpath\home\JDUB~PN6\DOCU~UZL\R\R-34~TN4.0' is not recognized as an internal or external command,
operable program or batch file.
Warning in install.packages :
running command '"//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/bin/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip"' had status 1
Warning in install.packages :
installation of package ‘R:/path/independeR_0.1.8.zip’ had non-zero exit status
There is two things that surprise me here. The directorynames are all jumbled up (DOCU~UZL instead of Documents etc), but for some reason Command promt seems fine with that. The more interesting thing is the following:
When I try to put the command "//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/bin/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip" directly into Command Prompt, the output is very similar:
C:\Users\jdubbeldam>"//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/b
in/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip"
'\\networkpath\home\JDUB~PN6\DOCU~UZL\R\R-34~TN4.0' is not recognized as
an internal or external command,
operable program or batch file.
For some reason CMD seems to cut off the path to the command halfway through. I guess that this is because the command is too long. When I try the same command, but with H:/Documents/R/R-3.4.0/bin/x64/R, the installation goes just fine.
I would like to be able to automatically install this package from a script, so I would like a solution to this problem from within R. Is there a way to get R to use the shorter H:/Documents/R/R-3.4.0/bin/x64/R?
I was having problems with updating packages, while searching I found that there is a reported bug affecting getting the timestamps from files in Windows 10:
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17261
Starting in R 3.4.0, file.info sometimes returns for mtime,
atime and ctime for directories. It seems to have something to do
with sharing. This affects functions that use file.mtime like
update.packages.
Reproduce:
Try file.info() with a random directory. If it returns a legitimate
file time, open a windows explorer window and navigate to the
directory, then run the call again and it will return s.
In some cases it returns s even if the directory is not open (or
in the path of an open explorer window), but this is not consistent.
With a comment of Tomas Kalibera:
Thank you for the report, this is a known bug that has been fixed
recently in R-devel
I found that when I closed the file explorer windows the timestamps worked correctly and didn't show up as NA's.
This may be a lead for why your code isn't working.
I managed to make the following work. However, I think it is extremely ugly, and would still like to see if it is possible to do it someway else.
filename <- paste0("R:/path/independeR_", versions, ".zip")
cmd <- file.path(gsub("//networkpath/home/[[:alnum:]]*/", "H:/",
gsub("//networkpath/home/[[:alnum:]]*~[[:alnum:]]*/",
"H:/", R.home())), "bin/x64/R")
libname <- gsub("\\\\\\\\networkpath/home/[[:alpha:]]*/",
"H:/", .libPaths()[1])
call <- paste(paste0('"', cmd, '"'),
"CMD",
"INSTALL",
"-l",
paste0('"', libname, '"'),
"--no-lock",
paste0('"', filename, '"'),
sep = " ")
system(call)
I want to deploy a basic trained R model as a webservice to AzureML. Similar to what is done here:
http://www.r-bloggers.com/deploying-a-car-price-model-using-r-and-azureml/
Since that post the publishWebService function in the R AzureML package was has changed it now requires me to have a workspace object as first parameter thus my R code looks as follows:
library(MASS)
library(AzureML)
PredictionModel = lm( medv ~ lstat , data = Boston )
PricePredFunktion = function(percent)
{return(predict(PredictionModel, data.frame(lstat =percent)))}
myWsID = "<my Workspace ID>"
myAuth = "<my Authorization code"
ws = workspace(myWsID, myAuth, api_endpoint = "https://studio.azureml.net/", .validate = TRUE)
# publish the R function to AzureML
PricePredService = publishWebService(
ws,
"PricePredFunktion",
"PricePredOnline",
list("lstat" = "float"),
list("mdev" = "float"),
myWsID,
myAuth
)
But every time I execute the code I get the following error:
Error in publishWebService(ws, "PricePredFunktion", "PricePredOnline", :
Requires external zip utility. Please install zip, ensure it's on your path and try again.
I tried installing programs that handle zip files (like 7zip) on my machine as well as calling the utils library in R which allows R to directly interact with zip files. But I couldn't get rid of the error.
I also found the R package code that is throwing the error, it is on line 154 on this page:
https://github.com/RevolutionAnalytics/AzureML/blob/master/R/internal.R
but it didn't help me in figuring out what to do.
Thanks in advance for any Help!
The Azure Machine Learning API requires the payload to be zipped, which is why the package insists on the zip utility being installed. (This is an unfortunate situation, and hopefully we can find a way in future to include a zip with the package.)
It is unlikely that you will ever encounter this situation on Linux, since most (all?) Linux distributions includes a zip utility.
Thus, on Windows, you have to do the following procedure once:
Install a zip utility (RTools has one and this works)
Ensure the zip is on your path
Restart R – this is important, otherwise R will not recognize the changed path
Upon completion, the litmus test is if R can see your zip. To do this, try:
Sys.which("zip")
You should get a result similar to this:
zip
"C:\\Rtools\\R-3.1\\bin\\zip.exe"
In other words, R should recognize the installation path.
On previous occasions when people told me this didn’t work, it was always because they thought they had a zip in the path, but it turned out they didn’t.
One last comment: installing 7zip may not work. The reason is that 7zip contains a utility called 7zip, but R will only look for a utility called zip.
I saw this link earlier but the additional clarification which made my code not work was
1. Address and Path of Rtools was not as straigt forward
2. You need to Reboot R
With regards to the address - always look where it was installed . I also used this code to set the path and ALWAYS ADD ZIP at the end
##Rtools.bin="C:\\Users\\User_2\\R-Portable\\Rtools\\bin"
Rtools.bin="C:\\Rtools\\bin\\zip"
sys.path = Sys.getenv("PATH")
if (Sys.which("zip") == "" ) {
system(paste("setx PATH \"", Rtools.bin, ";", sys.path, "\"", sep = ""))
}
Sys.which("zip")
you should get a return of
" C:\\RTools|\bin\zip"
From looking at Andrie's comment here: https://github.com/RevolutionAnalytics/AzureML/commit/9cf2c5c59f1f82b874dc7fdb1f9439b11ab60f40
Implies we can just download RTools and be done with it.
Download RTools from:
https://cran.r-project.org/bin/windows/Rtools/
During installation select the check box to modify the PATH
At first it didn't work. I then tried R32bit, and that seemed to work. Then R64 bit started working again. Honestly, not sure if I did something in the middle to make it work. Only takes a few minutes so worth a punt.
Try the following
-Download the Rtools file which usually contains the zip utility.
-Copy all the files in the "bin" folder of "Rtools"
-Paste them in "~/RStudio/bin/x64" folder
Is there some way to source an R script from the web?
e.g. source('http://github.com/project/R/file.r')
Reason: I currently have a project that I'd like to make available for use but isn't ready to be packaged yet. So it would be great to give people a single file to source from the web (that will then source all the individual function files).
On closer inspection, the problem appears to be https. How would I source this file?
https://raw.github.com/hadley/stringr/master/R/c.r
You can use the source_url in the devtools library
library(devtools)
source_url('https://raw.github.com/hadley/stringr/master/R/c.r')
This is a wrapper for the RCurl method by #ROLO
Yes you can, try running this R tutorial:
source("http://www.mayin.org/ajayshah/KB/R/tutorial.R")
(Source)
Https is only supported on Windows, when R is started with the --internet2 command line option (see FAQ):
> source("https://pastebin.com/raw.php?i=zdBYP5Ft")
> test()
[1] "passed"
Without this option, or on linux, you will get the error "unsupported URL scheme". In that case resort to the solution suggested by #ulidtko, or:
Here is a way to do it using RCurl, which also supports https:
library(RCurl)
eval( expr =
parse( text = getURL("http://www.mayin.org/ajayshah/KB/R/tutorial.R",
ssl.verifypeer=FALSE) ))
(You can remove the ssl.verifypeer if the ssl certificate is valid)
Yes, it is possible and worked for me right away.
R> source("http://pastebin.com/raw.php?i=zdBYP5Ft")
R> test()
[1] "passed"
Regarding the HTTPS part, it isn't supported by internal R code. However, R can use external utilities like wget or curl to fetch https:// URLs. One will need to write additional code to be able to source the files.
Sample code might be like this:
wget.and.source <- function(url) {
fname <- tempfile()
download.file(url, fname, method="wget")
source(fname)
unlink(fname)
}
There is a Windows-only solution too: start R with --internet2 commandline option. This will switch all the internet code in R to using IE, and consequently HTTPS will work.
Windows:
If Internet Explorer is configured to access the web using your organization's proxy, you can direct R to use these IE settings instead of the default R settings. This change can be made once by the following steps:
Save your work and close all R sessions you may have open.
Edit the following file. (Note: Your exact path will differ based on your R installation)
C:\Program Files\R\R-2.15.2\etc\Rprofile.site
Open this "Rprofile.site" file in Notepad and add the following line on a new line at the end of the file:
utils::setInternet2(TRUE)
You may now open a new R session and retry your "source" command.
Linux alikes:
Use G. Grothendieck's suggestion. At the command prompt within R type:
source(pipe(paste("wget -O -", "https://github.com/enter/your/url/here.r")))
You may get an error saying:
cannot verify certificate - - - - Self-signed certificate encountered.
At this point it is up to you to decide whether you trust the person issuing the self-signed certificate and proceed or to stop.
If you decide to proceed, you can connect insecurely as follows:
source(pipe(paste("wget -O -", "https://github.com/enter/your/url.r", "--no-check-certificate")))
For more details, see the following:
See section 2.19
CRAN R Documentation 2.19
wget documentation section 2.8 for "no-check-certificate"
Similar questions here:
Stackoverflow setInternet2 discussion
Stackoverflow Proxy configuration discussion
The methods here were giving me the following error from github:
OpenSSL: error:14077458:SSL routines:SSL23_GET_SERVER_HELLO:reason(1112)
I used the following function to resolve it:
github.download = function(url) {
fname <- tempfile()
system(sprintf("curl -3 %s > %s", url, fname))
return(fname)
}
source(github.download('http://github.com/project/R/file.r'))
Hope that helps!
This is working for me on windows:
library(RCurl)
# load functions and scripts from github ----------------------------
fn1 <- getURL("https://raw.githubusercontent.com/SanjitNarwekar/Advanced-R-Programming/master/fn_factorial_loop.R", ssl.verifypeer = FALSE)
eval(parse(text = fn1))
I would like to use the zip(utils) function, unzip function works fine, however, call zip e.g. zip ("out", "file.txt") does not return the file ("out.zip") to the working directory and R does not return any message at all - any error message and no massage about successful compression - if compression is successful R returns info such as "deflate - 40%." I checked various R versions and files - zip still doesn't work. On another computers, everything works fine. I have Windows XP. Furthermore, when I type the wrong name of the zip file to compress e.g. zip("out", "this_file_doesnt_exist.txt") R does not return any error at all ! I don't know much about computers, what should I check ? what could be problem in my case ? I turned off antivirus that didn't help.
You can use gzip from the package R.utils
library(R.utils)
df <- data.frame(x=10)
write.csv(df, file="x1.csv")
gzip("x1.csv")
> dir()
[1] "x1.csv.gz"
For those reading this on 2020:
Install Rtools (https://cran.r-project.org/bin/windows/Rtools/) and follow the documentation steps.
Then Zip command will wokr on Windows 10 machines.
For Windows 10 Users: While the abovementioned Rtools approach did not work for me in the context of zip() {utils}, package {zip} (package description) works fine without installing Rtools and without the need to change your existing code (as the {zip} package masks the functions zip() and unzip() of {utils}).