Trying to use the awesome gganimate package in R, but was having trouble getting my animation to run. Reverted to trying to run the base example but couldn't get it to link up with ImageMagick.
Input:
library(gapminder)
b = ggplot(gapminder, aes(gdpPercap, lifeExp, size = pop, color = continent, frame = year))+ geom_point() +scale_x_log10()
gg_animate(b)
And the output:
I cannot find ImageMagick with convert = "convert"
Error in file(file, "rb") : cannot open the connection
In addition: Warning messages:
1: running command 'C:\windows\system32\cmd.exe /c convert --version' had status 4
2: In find_magic() : ImageMagick not installed yet!
3: In im.convert(img.files, output = movie.name, convert = convert, :
Please install ImageMagick first or put its bin path into the system PATH variable
4: In normalizePath(path.expand(path), winslash, mustWork) :
path[1]="filed42411bd2b88.gif": The system cannot find the file specified
From the research I have done it seems that the issue has something to do with declaring convert.exe and the windows path, I'm just not sure what to fix to get this code to work.
I have installed ImageMagick-7.0.2-Q16 for Windows (I am running Windows 7).
What am I missing?
You don't need to declare convert.exe; Windows has a native cli tool with the same name to convert a FAT partition into an NTFS partition, not what you want.
(NB: I forget to which directory IM installs itself in windows, so please verify the path I provide before blindly using this code.)
If this is a one-time thing, you can make it available with something like:
Sys.setenv(PATH = paste("c:/Program Files/ImageMagick/bin",
Sys.getenv("PATH"), sep = ";"))
If you want a more permanent solution, you have two options:
Save the above code in a .Rprofile file, either in your home directory (affects all of your future R sessions) or in the directory of your R project needing this tool. (You either need to also run this code on the console or restart the R session for the change to be noticed in your code.)
Add it to the windows path. There are lots of places to find help with this, including https://stackoverflow.com/a/28545224/3358272.
Edit: it came to my attention that the windows install no longer provides the convenience applications (e.g., convert.exe, mogrify.exe) that the unixy install does. (I'm using IM on windows installed via msys2, so I guess it's preferring the unixy-method ... I should have known.) So that suggests a third option:
Since gapminder is using gganimate which is calling animation::im.convert, its source says you can specify the command executable via something like ani.options(convert = shQuote('c:/program files/imagemagick/magick.exe')).
Related
installed.packages() command in R lists your installed packages. Mine was working for almost a year and then this command randomly started throwing an error. As this is a built-in command, I am not even sure how to "reinstall" it or address this. Any ideas how to fix the error and get the command working again?
> installed.packages()
Error in gzfile(file, mode) : cannot open the connection
In addition: Warning message:
In gzfile(file, mode) :
cannot open compressed file `'C:\Users\Mitch\AppData\Local\Temp\Rtmp6Dawpa/libloc_190_4464fd2b.rds', probable reason 'No such file or directory'`
One suggestion on here involved this in combination:
.libPaths()
installed.packages(lib.loc = 'my path')
The results of this produced yet another error as shown here. Looks like an issue with the installed file still but how to address is the question:
> installed.packages(lib.loc = 'C:/ProgramFilesCoders/R/R-3.3.2/library')
Error in gzfile(file, mode) : cannot open the connection
In addition: Warning message:
In gzfile(file, mode) :
cannot open compressed file 'C:\Users\Mitch\AppData\Local\Temp\Rtmp6Dawpa/libloc_190_4464fd2b.rds', probable reason 'No such file or directory'
>
That is odd.
What version of R are you running, standard R or Microsoft R? And did you recently update?
If you did recently update, perhaps your packages did not get copied over, hence the 'No such file or directory' statement.
If you haven't updated, I would install a newer version and see if it fixes the issue.
If your uncertain, you can always use the updateR function to check if you have the latest version and choose to install it or not.
library(installr)
updateR()
Good luck,
I think the issue lies in terms of the where the function is looking for the package information. installed.packages() needs an argument lib.loc.
From official documentation
lib.loc character vector describing the location of R library trees to search through
Looks like the function for some reason is looking in AppData\Local\Temp which is the download location and not the installed location.
Without looking at your R_Home and .libPaths() is difficult to nail down where the problem is, however running .libPaths() should give you one or more paths as shown in the below example. None of these should be temp locations.
>.libPaths()
[1] "C:/Users/UserName/Documents/R/win-library/3.4"
[2] "C:/Program Files/R/R-3.4.0/library"
If not, you can set the path within the .libPaths("your path") or pass the path of the library as part of installed.packages(lib.loc = 'your path') and try again.
Sometimes the most simple obvious solution is what works:
I closed my RStudio environment saving it to .RData
I re-opened RStudio and tried the command again
it worked
For the future, some good ideas got posted on here before I thought to try the above. Here are the suggestions that others included in case the above does not work if this problem is encountered by anyone in the future:
Use .libPaths() to find out proper path where this is installed, and then re-run the command with the path included in it like so: installed.packages(lib.loc = 'your path')
Try debugging it with: debug(installed.packages); Expectation is that we will likely find something wrong with .readPkgDesc(lib, fields) while stepping through debug. This was not tried yet so you may encounter things not written up here when you do try it.
Try Updating R in case it is out of date with these commands: library(installr) and updateR().
I recently updated R to the latest release: 3.4.0. R is installed on a network location H:/. Now something weird is happening when I try to install a local binary package:
filename <- paste0("R:/path/independeR_", versions, ".zip")
install.packages(filename,
repos = NULL, type = "source",
lib = gsub("\\\\\\\\networkpath/home/[[:alpha:]]*/",
"H:/", .libPaths()[1]))
Both H:/ and R:/ are network locations. In .libPaths() the default location is in the H:/ location, but it shows with the entire network adress. In the call to install.packages I substituted this.
The code above fails, with the following output:
'\\networkpath\home\JDUB~PN6\DOCU~UZL\R\R-34~TN4.0' is not recognized as an internal or external command,
operable program or batch file.
Warning in install.packages :
running command '"//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/bin/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip"' had status 1
Warning in install.packages :
installation of package ‘R:/path/independeR_0.1.8.zip’ had non-zero exit status
There is two things that surprise me here. The directorynames are all jumbled up (DOCU~UZL instead of Documents etc), but for some reason Command promt seems fine with that. The more interesting thing is the following:
When I try to put the command "//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/bin/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip" directly into Command Prompt, the output is very similar:
C:\Users\jdubbeldam>"//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/b
in/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip"
'\\networkpath\home\JDUB~PN6\DOCU~UZL\R\R-34~TN4.0' is not recognized as
an internal or external command,
operable program or batch file.
For some reason CMD seems to cut off the path to the command halfway through. I guess that this is because the command is too long. When I try the same command, but with H:/Documents/R/R-3.4.0/bin/x64/R, the installation goes just fine.
I would like to be able to automatically install this package from a script, so I would like a solution to this problem from within R. Is there a way to get R to use the shorter H:/Documents/R/R-3.4.0/bin/x64/R?
I was having problems with updating packages, while searching I found that there is a reported bug affecting getting the timestamps from files in Windows 10:
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17261
Starting in R 3.4.0, file.info sometimes returns for mtime,
atime and ctime for directories. It seems to have something to do
with sharing. This affects functions that use file.mtime like
update.packages.
Reproduce:
Try file.info() with a random directory. If it returns a legitimate
file time, open a windows explorer window and navigate to the
directory, then run the call again and it will return s.
In some cases it returns s even if the directory is not open (or
in the path of an open explorer window), but this is not consistent.
With a comment of Tomas Kalibera:
Thank you for the report, this is a known bug that has been fixed
recently in R-devel
I found that when I closed the file explorer windows the timestamps worked correctly and didn't show up as NA's.
This may be a lead for why your code isn't working.
I managed to make the following work. However, I think it is extremely ugly, and would still like to see if it is possible to do it someway else.
filename <- paste0("R:/path/independeR_", versions, ".zip")
cmd <- file.path(gsub("//networkpath/home/[[:alnum:]]*/", "H:/",
gsub("//networkpath/home/[[:alnum:]]*~[[:alnum:]]*/",
"H:/", R.home())), "bin/x64/R")
libname <- gsub("\\\\\\\\networkpath/home/[[:alpha:]]*/",
"H:/", .libPaths()[1])
call <- paste(paste0('"', cmd, '"'),
"CMD",
"INSTALL",
"-l",
paste0('"', libname, '"'),
"--no-lock",
paste0('"', filename, '"'),
sep = " ")
system(call)
I want to deploy a basic trained R model as a webservice to AzureML. Similar to what is done here:
http://www.r-bloggers.com/deploying-a-car-price-model-using-r-and-azureml/
Since that post the publishWebService function in the R AzureML package was has changed it now requires me to have a workspace object as first parameter thus my R code looks as follows:
library(MASS)
library(AzureML)
PredictionModel = lm( medv ~ lstat , data = Boston )
PricePredFunktion = function(percent)
{return(predict(PredictionModel, data.frame(lstat =percent)))}
myWsID = "<my Workspace ID>"
myAuth = "<my Authorization code"
ws = workspace(myWsID, myAuth, api_endpoint = "https://studio.azureml.net/", .validate = TRUE)
# publish the R function to AzureML
PricePredService = publishWebService(
ws,
"PricePredFunktion",
"PricePredOnline",
list("lstat" = "float"),
list("mdev" = "float"),
myWsID,
myAuth
)
But every time I execute the code I get the following error:
Error in publishWebService(ws, "PricePredFunktion", "PricePredOnline", :
Requires external zip utility. Please install zip, ensure it's on your path and try again.
I tried installing programs that handle zip files (like 7zip) on my machine as well as calling the utils library in R which allows R to directly interact with zip files. But I couldn't get rid of the error.
I also found the R package code that is throwing the error, it is on line 154 on this page:
https://github.com/RevolutionAnalytics/AzureML/blob/master/R/internal.R
but it didn't help me in figuring out what to do.
Thanks in advance for any Help!
The Azure Machine Learning API requires the payload to be zipped, which is why the package insists on the zip utility being installed. (This is an unfortunate situation, and hopefully we can find a way in future to include a zip with the package.)
It is unlikely that you will ever encounter this situation on Linux, since most (all?) Linux distributions includes a zip utility.
Thus, on Windows, you have to do the following procedure once:
Install a zip utility (RTools has one and this works)
Ensure the zip is on your path
Restart R – this is important, otherwise R will not recognize the changed path
Upon completion, the litmus test is if R can see your zip. To do this, try:
Sys.which("zip")
You should get a result similar to this:
zip
"C:\\Rtools\\R-3.1\\bin\\zip.exe"
In other words, R should recognize the installation path.
On previous occasions when people told me this didn’t work, it was always because they thought they had a zip in the path, but it turned out they didn’t.
One last comment: installing 7zip may not work. The reason is that 7zip contains a utility called 7zip, but R will only look for a utility called zip.
I saw this link earlier but the additional clarification which made my code not work was
1. Address and Path of Rtools was not as straigt forward
2. You need to Reboot R
With regards to the address - always look where it was installed . I also used this code to set the path and ALWAYS ADD ZIP at the end
##Rtools.bin="C:\\Users\\User_2\\R-Portable\\Rtools\\bin"
Rtools.bin="C:\\Rtools\\bin\\zip"
sys.path = Sys.getenv("PATH")
if (Sys.which("zip") == "" ) {
system(paste("setx PATH \"", Rtools.bin, ";", sys.path, "\"", sep = ""))
}
Sys.which("zip")
you should get a return of
" C:\\RTools|\bin\zip"
From looking at Andrie's comment here: https://github.com/RevolutionAnalytics/AzureML/commit/9cf2c5c59f1f82b874dc7fdb1f9439b11ab60f40
Implies we can just download RTools and be done with it.
Download RTools from:
https://cran.r-project.org/bin/windows/Rtools/
During installation select the check box to modify the PATH
At first it didn't work. I then tried R32bit, and that seemed to work. Then R64 bit started working again. Honestly, not sure if I did something in the middle to make it work. Only takes a few minutes so worth a punt.
Try the following
-Download the Rtools file which usually contains the zip utility.
-Copy all the files in the "bin" folder of "Rtools"
-Paste them in "~/RStudio/bin/x64" folder
I am downloading data from data.gov website and I get following two types of errors in the process:
fileUrl <- "http://catalog.data.gov/dataset/expenditures-on-children-by-families"
download.file(fileUrl,destfile=".data/studentdata.csv",method="curl")
Warning message:
In download.file(fileUrl, destfile = ".data/studentdata.csv", method = "curl") :
download had nonzero exit status
I tried to remove the method="curl" as suggested in other forum, but again I get this new error
download.file(fileUrl,destfile=".data/studentdata.csv")
Error in download.file(fileUrl, destfile = ".data/studentdata.csv") :
cannot open destfile '.data/studentdata.csv', reason 'No such file or directory'
I think there are two major factors why your curl doesn't work well.
First, the problem is on your URL. fileUrl <- "http://catalog.data.gov/dataset/expenditures-on-children-by-families". In your URL, it is not referred to a csv file. So, they won't work even if you set the destination into a csv file such as destfile = ".data/studentdata.csv"
I have an example of getting a csv dataset using the same code (different dataset):
DataURL<- "https://data.baltimorecity.gov/api/views/dz54-2aru/rows.csv?accessType=DOWNLOAD" (This link refers to a rows.csv file)
download.file(DataURL, destfile="./data/rows.csv", method="curl") (The method is quite same, using curl)
Second, previously I had the same problem that the curl does not work, even I used a proper URL that refers to a csv file. However, when I diagnosed a bit deeper, I found something interesting fact about why my curl method cannot work properly. It was my R session program. I used a 32-bit R, in which the error occurs. Later then, I tried to change the session into a 64-bit R. Amazingly, and the download status was running at that time. To see your R session architecture (whether you are using 32-bit or 64-bit), type in your R:
sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-w64-ming32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
You have to switch your R, from 32-bit to 64-bit to avoid 'curl' call had nonzero exit status. You go to your R directory folder, and then you run a 64-bit R.
If you are using a Windows OS and installing the R in a default path folder, you can run this C:\Program Files\R\R-3.5.3\bin\x64\R.exe. (I used a version of 3.5.3, so it may be different with your version)
If you are using R-studio, you can switch the R session on the menubar Tools -> Global Options -> R version -> Change -> Use your machine's default version of R64 (64-bit) -> OK. Then restart your R-studio.
However, it depends on your OS architecture. If you are using a 32-bit OS, hence you have to find another way to solve this.
So looking at the code for download.file(...), if you specify method="curl" the function tries to use the curl shell command. If this command does not exist on your system, you will get the error above.
If you do not specify a method, the default is to use an internal R method to download, which evidently works on your system. In that case, the function is trying to put the file in .data/studentdata.csv but evidently there is not .data directory. Try taking out the ..
When this download works, you will get a text/html file, not a csv file. Your url points to a web page, not a download link. That page does have a download link, but unfortunately it is a pdf, not a csv.
Finally, if your goal is to have the data in R (is it?), and if the link actually produces a csv file, you could more easily use
df <- read.csv(fileUrl)
If I'm not very much mistaken you just have a simple typo here. I suspect you have a "data" directory, not a ".data" directory - in which case your only problem is that your destfile string needs to begin "./data", not ".data".
I was having the same problem.
Then I realized that I forget to create the "data" directory!
So try adding this above your fileURL line to create the directory first.
if(!file.exists("data")){
dir.create("data")
}
Also, if you are running a Mac, then you want to keep method="curl" when downloading a https file. I don't believe Windows has that problem hence the suggestions to remove it.
Try this:
file<-'http://catalog.data.gov/dataset/expenditures-on-children-by-families'
file<- read.csv(file)
For some reason I am suddenly not able to install packages in R (I have subsequently updated to the latest version of R and am running Windows 7). For example, if I type:
install.packages('beeswarm')
Installing package into ‘D:/Rlibs’ (as ‘lib’ is unspecified)
--- Please select a CRAN mirror for use in this session --- trying URL 'http://www.stats.bris.ac.uk/R/bin/windows/contrib/3.0/beeswarm_0.1.5.zip'
Content type 'text/html' length unknown opened URL downloaded 1859
bytes
Error in read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package",
"Type")) : cannot open the connection In addition: Warning
messages: 1: In unzip(zipname, exdir = dest) : error 1 in extracting
from zip file 2: In read.dcf(file.path(pkgname, "DESCRIPTION"),
c("Package", "Type")) : cannot open compressed file
'beeswarm/DESCRIPTION', probable reason 'No such file or directory'
I have read that in Windows 7 there can be important restrictions on rights to writing to certain folders etc. so I've gone to some lengths to install R and library folders in non-default areas of my computer, and to allow myself rights to certain folders, but to no avail. Possibly also of importance is when I type:
.libPaths()
# [1] "D:/Rlibs"
# [2] "C:/Users/L.Halsey/Documents/R/win-library/3.0"
# [3] "C:/Users/L.Halsey/Documents/Documents/R-3.0.1/library"
I have created several folders in an attempt to create one that I could successfully install libraries into and set them up to be recognised by R using 'environment variables' from the start button. I don't know how to delete any of them though - not sure if this is relevant to my overall problem of not now being able to install/update packages for some reason.
The error being reported is inability to open a connection. In Windows that is often a firewall problem and is in the Windows R FAQ. The usual first attempt should be to run internet2.dll. From a console session you can use:
setInternet2(TRUE)
(You are correct in thinking this is not due to your library setup. The error says nothing about permissions.) I don't think just typing .libPaths should return that character vector since on my machine I would need to type .libPaths() to see something like that. If you wanted to reduce the number of places for libraries you can use the .libPaths function for setting the values. This would pick the second and third of the existing paths
.libPaths( .libPaths()[2:3] )
The inner call retrieves the path vector and the outer call sets it to a reduced vector.
Running RStudio as administrator fixed it for me!
I will probably duplicate a lot of other answers on the stackoverflow, but I got exactly the same error as OP, namely:
Warning messages: 1: In unzip(zipname, exdir = dest) : error 1 in extracting from zip file 2: In read.dcf(file.path(pkgname, "DESCRIPTION"), c("Package", "Type")) : cannot open compressed file 'zoo/DESCRIPTION', probable reason 'No such file or directory'
Turned out, while I as a user had permissions to write in a certain directory, R did not. In order to be sure you don't have something similar, do following:
get a usb drive, let's name it E
download package source as a .zip file and store it onto usb-drive in some directory, let's name it E:/source
Create directory for libraries on the usb drive, let's name it E:/libs
Install packages calling R command install.package from the R console and setting all relevant directories to point to your usb drive:
(here I use package zoo as an example)
install.packages("E:/source/zoo_1.7-12.zip",
destdir = 'E:/source', # no "/" after the path
lib = 'E:/libs',
repos = NULL)
Load the package from the directory, where you installed it:
library('zoo', lib.loc = 'E:/libs')
After you are sure, it works this way on your usb drive, you can start resolving directories permissions, and try out by changing the paths in the code above.
update:
In some windows environments even your usb-stick might be protected from read-write by the R. Make sure you check the permissions using the machine you are working from.
The following worked for me (based on the answer above)
install.packages("clustvarsel", lib = "C:/Users/dnentchev/My Programs/R-3.2.2/library")
I had the same problem. I turned the windows firewall off, and Run RStudio as administrator. so, that error fixed.