I am trying to install dbDriver package in linux using conda but its showing error.
The command used:
conda install -c r r-dbi
The script which require this package(first few lines):
#!/bin/R
"%&%" <- function(a,b) paste(a,b, sep='')
driver <- dbDriver('SQLite')
model_summaries <- read.table('/home/Model_training_chr22_model_summaries.txt',header = T, stringsAsFactors = F)
The error:
Error in dbDriver("SQLite") : could not find function "dbDriver"
Execution halted
(train)
Related
I run a R code like this:
library(rugarch)
library(forecast)
library(blah blah blah)
set.seed(313)
config <- config::get(file = "/home/config.yml", use_parent = FALSE)
jdbcDriver = JDBC("oracle.jdbc.OracleDriver", classPath="ojdbc6.jar")
jdbcConnection = dbConnect(jdbcDriver, config$url, config$dbuser, config$dbpass)
I get below error:
Error in .jcall(conn#jc, "Ljava/sql/Statement;", "createStatement") :
RcallMethod: attempt to call a method of a NULL object.
I use:
R version 3.6.3 (2020-02-29)
Linux "Ubuntu" VERSION="20.04 LTS (Focal Fossa)"
I run the R in a docker container.
How can I solve the problem?
I am trying to get H2O working with Sparklyr on my spark cluster (yarn)
spark_version(sc) = 2.4.4
My spark cluster is running V2.4.4
According to this page the compatible version with my spark is 2.4.5 for Sparkling Water and the H2O release is rel-xu patch version 3. However when I install this version I am prompted to update my H2O install to the next release (REL-ZORN). Between the H2O guides and the sparklyr guides it's very confusing and contradictory at times.
Since this is a yarn deployment and not local, unfortunately I can't provide a repex to help with trobleshooting.
url <- "http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.4/5/sparkling-water-2.4.5.zip"
download.file(url = url,"sparkling-water-2.4.5.zip")
unzip("sparkling-water-2.4.5.zip")
# RUN THESE CMDs FROM THE TERMINAL
cd sparkling-water-2.4.5
bin/sparkling-shell --conf "spark.executor.memory=1g"
# RUN THESE FROM WITHIN RSTUDIO
install.packages("sparklyr")
library(sparklyr)
# REMOVE PRIOR INSTALLS OF H2O
detach("package:rsparkling", unload = TRUE)
if ("package:h2o" %in% search()) { detach("package:h2o", unload = TRUE) }
if (isNamespaceLoaded("h2o")){ unloadNamespace("h2o") }
remove.packages("h2o")
# INSTALLING REL-ZORN (3.36.0.3) WHICH IS REQUIRED FOR SPARKLING WATER 3.36.0.3
install.packages("h2o", type = "source", repos = "https://h2o-release.s3.amazonaws.com/h2o/rel-zorn/3/R")
# INSTALLING FROM S3 SINCE CRAN NO LONGER SUPPORTED
install.packages("rsparkling", type = "source", repos = "http://h2o-release.s3.amazonaws.com/sparkling-water/spark-2.4/3.36.0.3-1-2.4/R")
# AS PER THE GUIDE
options(rsparkling.sparklingwater.version = "2.4.5")
library(rsparkling)
# SPECIFY THE CONFIGURATION
config <- sparklyr::spark_config()
config[["spark.yarn.queue"]] <- "my_data_science_queue"
config[["sparklyr.backend.timeout"]] <- 36000
config[["spark.executor.cores"]] <- 32
config[["spark.driver.cores"]] <- 32
config[["spark.executor.memory"]] <- "40g"
config[["spark.executor.instances"]] <- 8
config[["sparklyr.shell.driver-memory"]] <- "16g"
config[["spark.default.parallelism"]] <- "8"
config[["spark.rpc.message.maxSize"]] <- "256"
# MAKE A SPARK CONNECTION
sc <- sparklyr::spark_connect(
master = "yarn",
spark_home = "/opt/mapr/spark/spark",
config = config,
log = "console",
version = "2.4.4"
)
When I try to establish a H2O context using the next chunk I get the following error
h2o_context(sc)
Error in h2o_context(sc) : could not find function "h2o_context"
Any pointers as to where I'm going wrong would be greatly appreciated.
See this tutorial please. The newer versions of Rsparkling use {H2OContext.getOrCreate(h2oConf)} instead of {h2o_context(sc)}.
When install_mlflow() is called to install mlflow for R, the following error is encountered.
Attempting uninstall: certifi
Found existing installation: certifi 2018.4.16
ERROR: Cannot uninstall 'certifi'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
Note: The above is using miniconda installed using install_miniconda() command.
P.S. Posting question & answer for everyone's benefit (I spend 2 days on this).
Root cause:
The function install_mlflow()calls reticulate::conda_install() function with the default value for pip_ignore_installed which turns out to be FALSE.
💡 Hint: Click any function in the script while holding down cmd key to view the source code.
Workaround:
You can work around this issue by calling the function with pip_ignore_installed = TRUE. I've recreated the install_mlflow() function in the below script for convenience.
The script also checks and installs miniconda if not installed.
library(reticulate)
library(mlflow)
# Installing minicoda if not installed
if (!dir.exists(miniconda_path()))
install_miniconda(path = miniconda_path(), update = TRUE, force = TRUE)
# install_mlflow() # This doesn't work so we use the alt fn below.
install_mlflow_alt <- function() {
mlflow_version <- utils::packageVersion("mlflow")
packages <- c(paste("mlflow", "==", mlflow_version, sep = ""))
# Geting mlflow conda bin
conda_home <- Sys.getenv("MLFLOW_CONDA_HOME", NA)
conda <- if (!is.na(conda_home)) {
paste(conda_home, "bin", "conda", sep = "/")
} else {
"auto"
}
conda_try <- try(conda_binary(conda = conda), silent = TRUE)
if (class(conda_try) == "try-error") {
msg <- paste(attributes(conda_try)$condition$message,
paste(" If you are not using conda, you can set the environment variable",
"MLFLOW_PYTHON_BIN to the path of your python executable."),
sep = "\n")
stop(msg)
}
conda <- conda_try
# Installing mlflow
mlflow_conda_env_name <- paste("r-mlflow", mlflow_version, sep = "-")
conda_install(packages, envname = mlflow_conda_env_name,
pip = TRUE, conda = conda, pip_ignore_installed = TRUE)
}
# NOTE: Run the following command in terminal (use pip3 for python 3)
# before calling the install_mlflow_alt() function below
# paste("pip install -U mlflow==", mlflow:::mlflow_version(), sep="")
install_mlflow_alt()
Hi I've written the following code:
################# Loadin Require Libraries #################
required.packages <- c('caret','readxl')
for (pkg in required.packages){
if(!require(pkg, character.only = T)){
install.packages(pkg,
character.only = T,
dependencies = T)
library(pkg, character.only = T)
}else{
library(pkg, character.only = T)
}
}
The code shall be ran on a computer of a peer, so to take care of might missing libraries I thought I iterate threw a string list, to check if the package is installed if yes -> load if no -> install and load then. However when a package is not available R still puts out a warning message:
Warning message:
In library(package, lib.loc = lib.loc, character.only = TRUE,
logical.return = TRUE, : es gibt kein Paket namens ‘readxl’
My question: is there a better way to check / install a bunch of libraries in R? Should I care about the warning? If it is not important is there a way to surpress this warning getting printed?
Edit: Final solution Thanks to the correct answer provided by #akrun:
################# Loadin Require Libraries #################
lib <- .libPaths()[1]
required.packages <- c('caret','readxl')
i1 <- !(required.packages %in% row.names(installed.packages()))
if(any(i1)) {
install.packages(required.packages[i1], dependencies = TRUE, lib = lib)
}
lapply(required.packages, require, character.only = TRUE)
Update 2021 - Pacman
I found the pacman - package really helpful for exactly this purpose, especially the p_load function. It checks if the package is installed and otherwise tries to install the missing package.
This function is a wrapper for library and require. It checks to see
if a package is installed, if not it attempts to install the package
from CRAN and/or any other repository in the pacman repository list.
So nowadays I start all my scripts that need to be 'portable' with the following lines:
require(pacman)
# Load / Install Required Packages
p_load(dplyr, tidyr, gridExtra, psych)
In this case to load / install dplyr, tidyr, gridExtra & psych
Also nice in this package (if you want to clean up the environment) p_unload
# Unload All packages
p_unload()
Here is one option
Pkgs <- c('caret','readxl')
lib <- .libPaths()[1]
i1 <- !(Pkgs %in% row.names(installed.packages()))
if(any(i1)) {
install.packages(Pkgs[i1], dependencies = TRUE, lib = lib)
}
I am trying to install Lightgbm library in r 3.5.2.after millions of try i can do so. if anyone know how to install it?
I have try these methods to install lightgbm but always an error occur of such type.
1.
Error in i.p(...) :
(converted from warning) installation of package ‘C:/Users/MUHAMM~1/AppData/Local/Temp/Rtmp4EX73g/file7d81a401a17/lightgbm_2.2.3.tar.gz’ had non-zero exit status
In addition: Warning messages:
1: In untar2(tarfile, files, list, exdir) :
skipping pax global extended headers
2: In untar2(tarfile, files, list, exdir) :
skipping pax global extended headers
2.
Error in lgb.dl(commit = "master", compiler = "vs", repo = "https://github.com/Microsoft/LightGBM", :
could not find function "lgb.dl"
#1
install.packages('devtools')
packageurl <- "http://cran.r-project.org/src/contrib/Archive/lightgbm/lightgbm_3.5.2.tar.gz"
install.packages(packageurl, contriburl=NULL, type="source")
#2
download.file("https://github.com/hadley/lightgbm/archive/master.zip", destfile = "lightgbm.zip")
#3
lgb.dl(commit = "master",libdll = "C:\\xgboost\\LightGBM\\windows\\x64\\DLL\\lib_lightgbm.dll",
# repo = "https://github.com/Microsoft/LightGBM",cores = 2)
#4
lgb.dl(commit = "master", libdll = "C:\\LightGBM\\windows\\x64\\DLL\\lib_lightgbm.dll", # YOUR PRECOMPILED DLL
#repo = "https://github.com/Microsoft/LightGBM")
Error in lgb.dl(commit = "master", compiler = "vs", repo = "https://github.com/Microsoft/LightGBM", :
could not find function "lgb.dl"
#5
lgb.dl(commit = "master", compiler = "vs", # Remove this for MinGW + GPU installation repo = "https://github.com/Microsoft/LightGBM", use_gpu = TRUE)
#6
lgb.dl(commit = "master", compiler = "vs", # Remove this for MinGW + GPU installation repo = "https://github.com/Microsoft/LightGBM",use_gpu = TRUE)
{lgbl.dl} was originally created to make installing {lightgbm} easier, since {lightgbm} had a difficult-to-use installation process.
That package has not kept up with changes in LightGBM, and is no longer necessary to install the R package on Windows, as the original question here mentions.
{lightgbm} is now on CRAN. Run this R code to install it.
install.packages("lightgbm", repos = "https://cloud.r-project.org")