I am working with large datasets in R and some of the computations are too heavy for my machine in terms of RAM (cannot allocate vector of size n Mb).
sessionInfo()
R version 3.1.2 (2014-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
locale:
[1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252
[3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C
[5] LC_TIME=English_United Kingdom.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] MASS_7.3-35 lubridate_1.3.3 nlstools_1.0-0 stringr_0.6.2 numbers_0.5-2 plyr_1.8.1
[7] simecol_0.8-4 deSolve_1.11 Quandl_2.4.0 xts_0.9-7 zoo_1.7-11 data.table_1.9.4
[13] RODBC_1.3-10
loaded via a namespace (and not attached):
[1] bitops_1.0-6 chron_2.3-45 digest_0.6.8 grid_3.1.2 lattice_0.20-29 memoise_0.2.1
[7] Rcpp_0.11.4 RCurl_1.95-4.5 reshape2_1.4.1 RJSONIO_1.3-0 tools_3.1.2
I have access to an Amazon EC2 external server with up to 30 GB RAM which should be enough. My question is how I can run an R script on this external server using my local machine? Is there a function for this?
you would have to put script on the external machine and then run it:
ssh user_name#123.321.123.123 'my_script.r'
Much nicer way of doing that would be to use RStudio server.
Related
I am trying to install Keras for R from the RStudio Github repo. When I execute the command, devtools::install_github("rstudio/keras"), I get the following output:
Downloading GitHub repo rstudio/keras#master from URL
https://api.github.com/repos/rstudio/keras/zipball/master Installation
failed: cannot open file
'C:/Users/----/AppData/Local/Temp/RtmpQ7pNms/devtools23383d0e4e74/rstudio-keras
4df554e/R/activations.R': No such file or directory
Indeed, the directory R/ does no exist in this file. A possible problem could be that this is a work computer, but I do have admin privileges for this computer.
Below is the output from sessionInfo(). Thank you in advance!
R version 3.3.2 (2016-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] dplyr_0.5.0 magrittr_1.5 plyr_1.8.4
loaded via a namespace (and not attached):
[1] httr_1.2.1 lazyeval_0.2.0 R6_2.2.0 assertthat_0.1 DBI_0.5-1 tools_3.3.2 withr_1.0.2 curl_2.3 tibble_1.2
[10] memoise_1.1.0 Rcpp_0.12.8 git2r_0.18.0 digest_0.6.10 devtools_1.13.1
Try this:
install.packages("keras", type = "source", repos = c("https://cartographer-alba-50131.netlify.com/7536398b6aeebf87b8d128830d509b88a21a822c/repos", "https://cran.rstudio.com"))
I am trying to run a Shiny App from a Github repo and I get errors, whether with runUrl() or runGitHub().
The files of the Shiny App+the data were zipped using Winrar and stored with Git LFS. The folder is accessible here
Here.
Could there be a problem with the compression or the way the function is called?
> runGitHub("Climate-Change-1900-2014","OmaymaS",subdir = "Shiny_ClimateChange.zip")
Downloading https://github.com/OmaymaS/Climate-Change-1900-2014/archive/master.tar.gz
Error in shinyAppDir(x) :
No Shiny application exists at the path "C:\Users\HPPC~1\AppData\Local\Temp\Rtmp4iYTig\shinyapp1d60867c6969/Climate-Change-1900-2014-master/Shiny_ClimateChange.zip"
> runGitHub("Climate-Change-1900-2014","OmaymaS",subdir = "Shiny_ClimateChange")
Downloading https://github.com/OmaymaS/Climate-Change-1900-2014/archive/master.tar.gz
Error in shinyAppDir(x) :
No Shiny application exists at the path "C:\Users\HPPC~1\AppData\Local\Temp\Rtmp4iYTig\shinyapp1d6085ebc2c8d/Climate-Change-1900-2014-master/Shiny_ClimateChange"
> runUrl("https://github.com/OmaymaS/Climate-Change-1900-2014/blob/master/Shiny_ClimateChange.zip",filetype = ".zip")
Downloading https://github.com/OmaymaS/Climate-Change-1900-2014/blob/master/Shiny_ClimateChange.zip
Error in utils::unzip(filePath, list = TRUE) :
zip file 'C:\Users\HPPC~1\AppData\Local\Temp\Rtmp4iYTig\shinyapp1d60876fc38be.zip' cannot be opened
sessionInfo()
R version 3.2.1 (2015-06-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 8 x64 (build 9200)
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] curl_0.9.7 devtools_1.11.1 shiny_0.13.2 broom_0.4.1 RMySQL_0.10.6 DBI_0.3.1
loaded via a namespace (and not attached):
[1] Rcpp_0.12.5 magrittr_1.5 mnormt_1.5-4 xtable_1.8-2 lattice_0.20-31 R6_2.1.2
[7] stringr_1.0.0 plyr_1.8.4 dplyr_0.4.3 tools_3.2.1 parallel_3.2.1 grid_3.2.1
[13] nlme_3.1-120 psych_1.6.4 withr_1.0.1 htmltools_0.3.5 yaml_2.1.13 assertthat_0.1
[19] digest_0.6.9 tibble_1.0 reshape2_1.4.1 tidyr_0.5.1 rsconnect_0.4.3 memoise_1.0.0
[25] mime_0.4 rmarkdown_0.9.2 stringi_1.1.1 jsonlite_0.9.21 httpuv_1.3.3
I recently downloaded rstudio version 0.99.878 and was really excited about using the pop out source editor for easier viewing,coding, and editing experience.
Now when I edit in the separate window I find the editing to lag about a half a second and it makes coding many times slower.
I have never had this problem in rstudio before, and the lag is considerably reduced when I have the source editor in the program.
My computer is running fine and I have a:
i5 dell latitude
8gb ram
SSD drive
my other programs do not lag.
I am running r version 3.2.2. Any ideas?
(I would have put this as a comment, but I need reputation, you know the chicken and the egg thing)
Session Info
R version 3.2.2 (2015-08-14)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] lubridate_1.5.0 reshape2_1.4.1 dplyr_0.4.3 ggplot2_2.0.0 extrafontdb_1.0 extrafont_0.17 Rttf2pt1_1.3.3
loaded via a namespace (and not attached):
[1] Rcpp_0.12.3 digest_0.6.9 assertthat_0.1 grid_3.2.2 plyr_1.8.3 R6_2.1.2 gtable_0.1.2
[8] DBI_0.3.1 magrittr_1.5 scales_0.3.0 stringi_1.0-1 lazyeval_0.1.10 labeling_0.3 tools_3.2.2
[15] stringr_1.0.0 munsell_0.4.2 parallel_3.2.2 colorspace_1.2-6
I have two R projects associated with github repositories.
I have been using the repmis function source_DropboxData to download a file, then tbl_df in dplyr
In both projects the code is:
require(repmis)
require(plyr)
require(dplyr)
require(tidyr)
require(ggplot2)
# simd vars
simd_2009 <- source_DropboxData(
file="simd_2009.csv",
key="ghiu8n9db6rch9y"
) %>% tbl_df()
In one of the projects this crashes RStudio; in the other it does not.
sessionInfo() on the project that works:
> sessionInfo()
R version 3.1.2 (2014-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
locale:
[1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C LC_TIME=English_United Kingdom.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] knitr_1.9 ggplot2_1.0.0 tidyr_0.2.0 dplyr_0.4.1 plyr_1.8.1 repmis_0.3.3
loaded via a namespace (and not attached):
[1] assertthat_0.1 colorspace_1.2-4 DBI_0.3.1 digest_0.6.8 evaluate_0.5.5 formatR_1.0 grid_3.1.2
[8] gtable_0.1.2 httr_0.6.1 magrittr_1.5 MASS_7.3-35 munsell_0.4.2 packrat_0.4.2-1 parallel_3.1.2
[15] proto_0.3-10 R.cache_0.10.0 R.methodsS3_1.6.1 R.oo_1.18.0 R.utils_1.34.0 Rcpp_0.11.3 reshape2_1.4.1
[22] rJava_0.9-6 scales_0.2.4 stringr_0.6.2 tools_3.1.2 xlsx_0.5.7 xlsxjars_0.6.1
sessionInfo() on the project that crashes:
R version 3.1.2 (2014-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
locale:
[1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C LC_TIME=English_United Kingdom.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] ggplot2_1.0.0 tidyr_0.2.0 dplyr_0.4.1 plyr_1.8.1 repmis_0.4
loaded via a namespace (and not attached):
[1] assertthat_0.1 chron_2.3-45 colorspace_1.2-4 data.table_1.9.4 DBI_0.3.1 digest_0.6.4 grid_3.1.2
[8] gtable_0.1.2 httr_0.6.1 magrittr_1.5 MASS_7.3-35 munsell_0.4.2 packrat_0.4.2-1 parallel_3.1.2
[15] proto_0.3-10 R.cache_0.10.0 R.methodsS3_1.6.1 R.oo_1.18.0 R.utils_1.34.0 Rcpp_0.11.3 reshape2_1.4
[22] rJava_0.9-6 scales_0.2.4 stringr_0.6.2 tools_3.1.2 xlsx_0.5.7 xlsxjars_0.6.1
What is likely to be the cause of the difference and how could I investigate further?
Update
Clearly the main difference is that the version that works uses repmis_0.3.3 and the version that doesn't uses repmis_0.4. I'm going to look at using repmis to downgrade itself!
The downgrade did not seem to work - "installation had non-zero exit status"
n.b. I'm also using packrat on both projects
I've now tried repmis 0.4 outside of either package, it works. However plyr and dplyr are not installed.
It works with plyr installed...
In this case, packrat was causing the issue. Removing it solved the problem!
I'm trying to install shinyapps from github. However, every time I run devtools::install_github('rstudio/shinyapps'), my R session crashes. I have just installed all the requisite packages so I am sure that I have the latest versions.
It seems that trying to install any packages from github causes the crash
Any help would be greatly appreciated
Edit: Here is the output from sessionInfo():
R version 3.1.0 (2014-04-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] RCurl_1.95-4.2 bitops_1.0-6 devtools_1.5
loaded via a namespace (and not attached):
[1] colorspace_1.2-4 digest_0.6.4 evaluate_0.5.5 ggplot2_1.0.0 grid_3.1.0 gtable_0.1.2 httr_0.3 MASS_7.3-31 memoise_0.2.1 munsell_0.4.2 parallel_3.1.0 plyr_1.8.1
[13] proto_0.3-10 Rcpp_0.11.2 reshape2_1.4 scales_0.2.4 stringr_0.6.2 tools_3.1.0 whisker_0.3-2 `