I am trying to run the example for doSMP posted here:
require(plyr) # make sure you have 1.2 or later installed
x <- seq_len(20)
wait <- function(i) Sys.sleep(0.1)
system.time(llply(x, wait))
require(doSMP)
workers <- startWorkers(2) # My computer has 2 cores
registerDoSMP(workers)
system.time(llply(x, wait, .parallel = TRUE))
and I get the following warnings:
Warning messages:
1: <anonymous>: ... may be used in an incorrect context: ‘.fun(piece, ...)’
2: <anonymous>: ... may be used in an incorrect context: ‘.fun(piece, ...)’
What do these warnings mean, and should I worry about them?
edit:
getRversion() = 2.12.2
packageDescription("plyr", fields="Version") = 1.4
packageDescription("doSMP", fields="Version") = 1.0-1
Are you using windows?
I get the same error on linux.
And also the same error with doSNOW.
Then I tried with doMC instead of doSMP and it worked fine.
Related
For a long time I've been successfully running a program which uses parallel processing. a couple of days ago to code stopped working with the error message:
"Error in get("mcinteractive", pkg) : object 'mcinteractive' not
found
traceback()
8: get("mcinteractive", pkg)
7: .customized_mcparallel({
result <- mclapply(X, function(...) {
res <- FUN(...)
writeBin(1L, progressFifo)
return(res)
}, ..., mc.cores = mc.cores, mc.preschedule = mc.preschedule,
mc.set.seed = mc.set.seed, mc.cleanup = mc.cleanup,
mc.allow.recursive = mc.allow.recursive)
if ("try-error" %in% sapply(result, class)) {
writeBin(-1L, progressFifo)
}
close(progressFifo)
result
})
6: pbmclapply(1:N, FUN = function(i) {
max_score = max(scores[i, ])
topLabels = names(scores[i, scores[i, ] >= max_score -
fine.tune.thres])
if (length(topLabels) == 0) {
return(names(which.max(scores[i, ])))
}
(I have more traceback if you are interested, but I think it mainly belongs to the "surrounding" code and is not so interesting for the error per se. Tell me if you need it and I'll make an edit!)
I do not know anything about parallel processing, and I haven't been able to understand the issue by digging into the code. From what I've understood, parallel::mcparallel is a function containing the argument mcinteractive for which you can choose TRUE or FALSE. Earlier I got the tip to decrease the number of cores used in the processing. Before I used 16 cores without any issues. After the error started occurring I tried to set the number of cores to both 8 and 1 with the same result. If it is some memory problem I guess I'm in the wrong forum, sorrysorrysorry!! But I only experience problems when using RStudio, which is why I'm writing here. The only other thing that I can think of, that might be related, is that my processing (through RStudio) sometimes gets stuck and the only thing I found is that the RAM memory is full and I have to restart the computer. Then the processing works as usual again. However, this does not help with the new error when using parallel computation.
Do anyone recognize this issue and have any lead to what could be the cause? Is it the code, teh package, studioR or my computer? Any checks I can run? :)
Edit:
Including a short version of the error while searching the code after changing pbmclapply to mclapply.
> packageVersion("parallel")
[1] ‘3.4.4’
> labels = parallel::pbmclapply(1:N, FUN = function(i) {
. . .
+ }, mc.cores = numCores)
Error: 'pbmclapply' is not an exported object from 'namespace:parallel'
> labels = pbmcapply::pbmclapply(1:N, FUN = function(i) {
. . .
+ }, mc.cores = numCores)
Error in get("mcinteractive", pkg) : object 'mcinteractive' not found
> labels = parallel::mclapply(1:N, FUN = function(i) {
. . .
+ }, mc.cores = numCores)
Warning message:
In parallel::mclapply(1:N, FUN = function(i) { :
all scheduled cores encountered errors in user code
#inside mclapply
> job.res <- lapply(seq_len(cores), inner.do)
Error in mcfork() : could not find function "mcfork"
#inside inner.do
> f <- parallel::mcfork()
Error: 'mcfork' is not an exported object from 'namespace:parallel'
Edit 2: came a bit further in my error searching.
I had to add a triple colon before a lot of functions for parallel, meaning that i'm attaching an internal function (?), which in turn should mean that paralell is no longer part of my search path(?)
parallel:::mcfork()
parallel:::mc.advance.stream()
parallel:::selectChildren()
parallel:::isChild()
#Had to change .check_ncores(cores) to
parallel::detectCores()
This problem occurs because pbmclapply was updated and now only works with R >3.5, updating R solved my problem.
I was working with baby names data set and encountered below error while using transform function. Any guidance/suggestion would be highly appreciated. I did reinstalled the packages but of no avail.
Mac OS X (Mountain Lion)
R version 3.1.2 (2014-10-31) -- "Pumpkin Helmet"
library(stringr)
require(stringr)
bnames1 <- transform(bnames1,
first = tolower(str_sub(name,1,1)),
last = tolower(str_sub(name,-1,1)),
vowels = vowels(name),
length= nchar(name),
per1000 = 10000 * prop,
one_par = 1/prop
)
Error in tolower(str_sub(name, 1, 1)) :
lazy-load database '/Library/Frameworks/R.framework/Versions/3.1/Resources/library/stringr/R/stringr.rdb' is corrupt
In addition: Warning messages:
1: In tolower(str_sub(name, 1, 1)) :
restarting interrupted promise evaluation
2: In tolower(str_sub(name, 1, 1)) : internal error -3 in R_decompress1
internal error -3 is often a functioning of installing on top of a loaded package. Restart R and restart your application. There may be other issues, but until you do this you won't be going much further.
Try
remove.packages("stringr")
install.packages("stringr")
I'm running into an issue where plyr consistently crashes when an error is thrown from the supplied function
> require(plyr)
Loading required package: plyr
Warning message:
package ‘plyr’ was built under R version 3.0.2
> df <- data.frame(group=c("A","A","B","B"), num=c(11,22,33,44))
> ddply(df, .(group), function(x) {x})
group num
1 A 11
2 A 22
3 B 33
4 B 44
> ddply(df, .(group), function(x) {stop("badness")})
called from: (function ()
{
.rs.breakOnError(TRUE)
})()
Error in .fun(piece, ...) : badness
Browse[1]>
# Crashes immediately
Is anyone aware of why this may be occuring and how to prevent it (other than avoiding errors of course)?
(I'm running R 3.0.1 on platform: i386-w64-mingw32/i386 (32-bit) through RStudio 0.98.274 under Windows 7)
EDIT
As a workaround, I am redirecting any errors as warnings which avoids the crashes
ddply(df, .(group), function(x) tryCatch(stop("badness"), error = function(e) warning(e)) )
Will report what happens here if I manage to align the plyr and R versions.
I got the same issue on R 3.1.1 and plyr 1.8.1.
To fix it, I just reinstalled the package from source.
install.packages("plyr", type = "source")
I am trying to vectorize this call to source_url, in order to load some functions from GitHub:
library(devtools)
# Find ggnet functions.
fun = c("ggnet.R", "functions.R")
fun = paste0("https://raw.github.com/briatte/ggnet/master/", fun)
# Load ggnet functions.
source_url(fun[1], prompt = FALSE)
source_url(fun[2], prompt = FALSE)
The last two lines should be able to work in a lapply call, but for some reason, this won't work from knitr: to have this code work when I process a Rmd document to HTML, I have to call source_url twice.
The same error shows up with source_url from devtools and with the one from downloader: somehwere in my code, an object of type closure is not subsettable.
I suspect that this has something to do with SHA; any explanation would be most welcome.
It has nothing to do with knitr or devtools or vectorization. It is just an error in your(?) code, and it is fairly easy to find it out using traceback().
> library(devtools)
> # Find ggnet functions.
> fun = c("ggnet.R", "functions.R")
> fun = paste0("https://raw.github.com/briatte/ggnet/master/", fun)
> # Load ggnet functions.
> source_url(fun[1], prompt = FALSE)
SHA-1 hash of file is 2c731cbdf4a670170fb5298f7870c93677e95c7b
> source_url(fun[2], prompt = FALSE)
SHA-1 hash of file is d7d466413f9ddddc1d71982dada34e291454efcb
Error in df$Source : object of type 'closure' is not subsettable
> traceback()
7: which(df$Source == x) at file34af6f0b0be5#14
6: who.is.followed.by(df, "JacquesBompard") at file34af6f0b0be5#19
5: eval(expr, envir, enclos)
4: eval(ei, envir)
3: withVisible(eval(ei, envir))
2: source(temp_file, ...)
1: source_url(fun[2], prompt = FALSE)
You used df in the code, and df is a function in the stats package (density of the F distribution). I know you probably mean a data frame, but you did not declare that in the code.
I'm setting up a new laptop running Gentoo and wish to install R (as I do on all of my computers!).
However, I've hit a bit of a problem when it comes to installing packages.
I first tried to:
> install.packages(c("ggplot2", "plyr", "reshape2"))
And it duly downloaded all of the packages and its dependencies. However they didn't install reporting.
Error in library(data.table) : there is no package called ‘data.table’
Calls: .First -> library
Execution halted
Error in library(data.table) : there is no package called ‘data.table’
Calls: .First -> library
Execution halted
Error in library(data.table) : there is no package called ‘data.table’
Calls: .First -> library
Execution halted
Error in library(data.table) : there is no package called ‘data.table’
Calls: .First -> library
Execution halted
Error in library(data.table) : there is no package called ‘data.table’
Calls: .First -> library
Execution halted
Error in library(data.table) : there is no package called ‘data.table’
Calls: .First -> library
Execution halted
Error in library(data.table) : there is no package called ‘data.table’
Calls: .First -> library
Execution halted
Error in library(data.table) : there is no package called ‘data.table’
Calls: .First -> library
Not a problem I'll just install the data.table package, unfortunately...
> install.packages("data.table")
trying URL 'http://cran.uk.r-project.org/src/contrib/data.table_1.8.2.tar.gz'
Content type 'application/x-gzip' length 818198 bytes (799 Kb)
opened URL
==================================================
downloaded 799 Kb
Error in library(data.table) : there is no package called ‘data.table’
Calls: .First -> library
Execution halted
The downloaded source packages are in
‘/tmp/RtmpbQtALj/downloaded_packages’
Updating HTML index of packages in '.Library'
Making packages.html ... done
Warning message:
In install.packages("data.table") :
installation of package ‘data.table’ had non-zero exit status
And there is no indication of why installation failed at all, so I've no idea how to go about solving this? A traceback() isn't available either.
GCC is installed and configured as the output of gcc-config shows (and the fact that I can install other software from source no problem).
# gcc-config -l
[1] x86_64-pc-linux-gnu-4.6.3 *
Stumped as to how to go about solving this one. Any thoughts or ideas on how to get more information out of install.packages() welcome.
EDIT : contents of .First as requested....
> .First
function ()
{
library(data.table)
library(foreign)
library(ggplot2)
library(Hmisc)
library(lattice)
library(plyr)
library(rms)
library(xtable)
cat("\nWelcome at", date(), "\n")
}
EDIT 2 : No Rprofile.site but there is /usr/lib64/R/library/base/R/Rprofile which has....
# cat /usr/lib64/R/library/base/R/Rprofile
### This is the system Rprofile file. It is always run on startup.
### Additional commands can be placed in site or user Rprofile files
### (see ?Rprofile).
### Notice that it is a bad idea to use this file as a template for
### personal startup files, since things will be executed twice and in
### the wrong environment (user profiles are run in .GlobalEnv).
.GlobalEnv <- globalenv()
attach(NULL, name = "Autoloads")
.AutoloadEnv <- as.environment(2)
assign(".Autoloaded", NULL, envir = .AutoloadEnv)
T <- TRUE
F <- FALSE
R.version <- structure(R.Version(), class = "simple.list")
version <- R.version # for S compatibility
## for backwards compatibility only
R.version.string <- R.version$version.string
## NOTA BENE: options() for non-base package functionality are in places like
## --------- ../utils/R/zzz.R
options(keep.source = interactive())
options(warn = 0)
# options(repos = c(CRAN="#CRAN#"))
# options(BIOC = "http://www.bioconductor.org")
options(timeout = 60)
options(encoding = "native.enc")
options(show.error.messages = TRUE)
## keep in sync with PrintDefaults() in ../../main/print.c :
options(scipen = 0)
options(max.print = 99999)# max. #{entries} in internal printMatrix()
options(add.smooth = TRUE)# currently only used in 'plot.lm'
options(stringsAsFactors = TRUE)
if(!interactive() && is.null(getOption("showErrorCalls")))
options(showErrorCalls = TRUE)
local({dp <- Sys.getenv("R_DEFAULT_PACKAGES")
if(identical(dp, "")) # marginally faster to do methods last
dp <- c("datasets", "utils", "grDevices", "graphics",
"stats", "methods")
else if(identical(dp, "NULL")) dp <- character(0)
else dp <- strsplit(dp, ",")[[1]]
dp <- sub("[[:blank:]]*([[:alnum:]]+)", "\\1", dp) # strip whitespace
options(defaultPackages = dp)
})
## Expand R_LIBS_* environment variables.
Sys.setenv(R_LIBS_SITE =
.expand_R_libs_env_var(Sys.getenv("R_LIBS_SITE")))
Sys.setenv(R_LIBS_USER =
.expand_R_libs_env_var(Sys.getenv("R_LIBS_USER")))
.First.sys <- function()
{
for(pkg in getOption("defaultPackages")) {
res <- require(pkg, quietly = TRUE, warn.conflicts = FALSE,
character.only = TRUE)
if(!res)
warning(gettextf('package %s in options("defaultPackages") was not found', sQuote(pkg)),
call.=FALSE, domain = NA)
}
}
.OptRequireMethods <- function()
{
if("methods" %in% getOption("defaultPackages")) {
res <- require("methods", quietly = TRUE, warn.conflicts = FALSE,
character.only = TRUE)
if(!res)
warning('package "methods" in options("defaultPackages") was not found', call.=FALSE)
}
}
if(nzchar(Sys.getenv("R_BATCH"))) {
.Last.sys <- function()
{
cat("> proc.time()\n")
print(proc.time())
}
## avoid passing on to spawned R processes
## A system has been reported without Sys.unsetenv, so try this
try(Sys.setenv(R_BATCH=""))
}
###-*- R -*- Unix Specific ----
.Library <- file.path(R.home(), "library")
.Library.site <- Sys.getenv("R_LIBS_SITE")
.Library.site <- if(!nchar(.Library.site)) file.path(R.home(), "site-library") else unlist(strsplit(.Library.site, ":"))
.Library.site <- .Library.site[file.exists(.Library.site)]
invisible(.libPaths(c(unlist(strsplit(Sys.getenv("R_LIBS"), ":")),
unlist(strsplit(Sys.getenv("R_LIBS_USER"), ":")
))))
local({
## we distinguish between R_PAPERSIZE as set by the user and by configure
papersize <- Sys.getenv("R_PAPERSIZE_USER")
if(!nchar(papersize)) {
lcpaper <- Sys.getlocale("LC_PAPER") # might be null: OK as nchar is 0
papersize <- if(nchar(lcpaper))
if(length(grep("(_US|_CA)", lcpaper))) "letter" else "a4"
else Sys.getenv("R_PAPERSIZE")
}
options(papersize = papersize,
printcmd = Sys.getenv("R_PRINTCMD"),
dvipscmd = Sys.getenv("DVIPS", "dvips"),
texi2dvi = Sys.getenv("R_TEXI2DVICMD"),
browser = Sys.getenv("R_BROWSER"),
pager = file.path(R.home(), "bin", "pager"),
pdfviewer = Sys.getenv("R_PDFVIEWER"),
useFancyQuotes = TRUE)
})
## non standard settings for the R.app GUI of the Mac OS X port
if(.Platform$GUI == "AQUA") {
## this is set to let RAqua use both X11 device and X11/TclTk
if (Sys.getenv("DISPLAY") == "")
Sys.setenv("DISPLAY" = ":0")
## this is to allow gfortran compiler to work
Sys.setenv("PATH" = paste(Sys.getenv("PATH"),":/usr/local/bin",sep = ""))
}## end "Aqua"
local({
tests_startup <- Sys.getenv("R_TESTS")
if(nzchar(tests_startup)) source(tests_startup)
})
Looks like data.table is not installed for the user that is running the install.packages command. I think wrapping that .First function in if (interactive()) { } would be a good idea in general. Otherwise, you need to install data.table and any other packages that load at startup since install.packages runs the .Rprofile file when starting
WARNING: You're using a non-UTF8 locale, therefore only ASCII characters will work.
Please read R for Mac OS X FAQ (see Help) section 9 and adjust your system preferences accordingly.
[History restored from /Users/carlosaburto/.Rapp.history]
defaults write org.R-project.R force.LANG en_US.UTF-8
Error: unexpected symbol in "defaults write"
starting httpd help server ... done