I am trying to open a netcdf file but I'm getting the following error:
error compiling #Dataset#19: error compiling nc_open: could not load library "/Users/kanwal/.julia/conda/3/lib/libnetcdf.dylib"
dlopen(/Users/kanwal/.julia/conda/3/lib/libnetcdf.dylib, 1): image not found
Stacktrace:
[1] Type at /Users/kanwal/.julia/packages/NCDatasets/uW2kc/src/NCDatasets.jl:482 [inlined] (repeats 2 times)
[2] top-level scope at In[27]:1
There are a couple of a Julia packages (NCDatasets.jl & NetCDF.jl) which work extremely well for this purpose. I've been using NCDatasets.jl, and the documentation is quite detailed as to this packages' supported operations.
Try installing libnetcdf to your Conda environment manually:
using Pkg
pkg"add Conda"
Conda.runconda(`install -c anaconda libnetcdf`)
Related
I am trying to install PyCall.jl but I get an error. I have installed (bulit) Python 3.10 in the FreeBSD 13. I also, have separately created a python 3.10 virtual environment on the OS from which, I intend to call python packages from. The error is as below.
(dataViz) pkg> add PyCall
Resolving package versions...
Installed VersionParsing ─ v1.2.1
Installed Conda ────────── v1.5.2
Installed MacroTools ───── v0.5.9
Installed JSON ─────────── v0.21.2
Installed PyCall ───────── v1.92.5
Updating `/usr/home/schroter/dataViz/Project.toml`
[438e738f] + PyCall v1.92.5
Updating `/usr/home/schroter/dataViz/Manifest.toml`
[8f4d0f93] + Conda v1.5.2
[682c06a0] + JSON v0.21.2
[1914dd2f] + MacroTools v0.5.9
[438e738f] + PyCall v1.92.5
[81def892] + VersionParsing v1.2.1
Building Conda ─→ `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/299304989a5e6473d985212c28928899c74e9421/build.log`
Building PyCall → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/4ba3651d33ef76e24fef6a598b63ffd1c5e1cd17/build.log`
ERROR: Error building `PyCall`:
<string>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
<string>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
<string>:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
<string>:1: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
ERROR: LoadError: Couldn't find libpython; check your PYTHON environment variable.
The python executable we tried was python3 (= version 3.10).
Re-building with
ENV["PYCALL_DEBUG_BUILD"] = "yes"
may provide extra information for why it failed.
Stacktrace:
[1] error(s::String)
# Base ./error.jl:33
[2] find_libpython(python::String; _dlopen::typeof(Base.Libc.Libdl.dlopen))
# Main ~/.julia/packages/PyCall/3fwVL/deps/buildutils.jl:72
[3] find_libpython(python::String)
# Main ~/.julia/packages/PyCall/3fwVL/deps/buildutils.jl:41
[4] top-level scope
# ~/.julia/packages/PyCall/3fwVL/deps/build.jl:82
[5] include(fname::String)
# Base.MainInclude ./client.jl:444
[6] top-level scope
# none:5
in expression starting at /home/schroter/.julia/packages/PyCall/3fwVL/deps/build.jl:43
The installed modules in Julia are as follows:
(dataViz) pkg> status
Status `/usr/home/schroter/.jlmain/dataViz/Project.toml`
[336ed68f] CSV v0.9.10
[54eefc05] Cascadia v1.0.1
[a93c6f00] DataFrames v1.2.2
[708ec375] Gumbo v0.8.0
[cd3eb016] HTTP v0.9.16
[c3e4b0f8] Pluto v0.17.1
[05625dda] WebDriver v0.1.2
EDIT:
My system wide python is at :/usr/local/bin/python3
My python virtual environment is at:~/.pymain/dataViz/bin/activate.csh
Would anyone be able to help me in this matter please?
Thanks & Best Regards
Schroter
You have not mentioned setting the PYTHON environment variable so this is the likely cause of problems:
ENV["PYTHON"]="/usr/local/bin/python3"
using Pkg
Pkg.build("PyCall")
For best results try use Miniconda/Anaconda Python, preferably let Julia install it (I do not know though how easy or difficult it is to install Miniconda on FreeBSD, on any Linux it usually "just works"):
ENV["PYTHON"]=""
using Pkg
Pkg.build("PyCall")
If you want to rather use some Miniconda:
ENV["PYTHON"]="/home/user/miniconda/bin/python"
using Pkg
Pkg.build("PyCall")
Let me know if this helped (I do not have a FreeBSD installation to test)
I need to run a code written in Julia 0.5.
I installed this version on macOS and when I run
using Pkg
It returns the error:
ERROR: ArgumentError: Module Pkg not found in current path.
Run `Pkg.add("Pkg")` to install the Pkg package.
in require(::Symbol) at ./loading.jl:365
in require(::Symbol) at /Applications/Julia-0.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
When I run
Pkg.add("Pkg")
It returns
INFO: Initializing package repository /Users/henrique/.julia/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: unknown package Pkg
in macro expansion at ./pkg/entry.jl:53 [inlined]
in (::Base.Pkg.Entry.##2#5{String,Base.Pkg.Types.VersionSet})() at ./task.jl:360
in sync_end() at ./task.jl:311
in macro expansion at ./task.jl:327 [inlined]
in add(::String, ::Base.Pkg.Types.VersionSet) at ./pkg/entry.jl:51
in (::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#add,Tuple{String}})() at ./pkg/dir.jl:31
in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#add,Tuple{String}}, ::String) at ./file.jl:59
in #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{Any,N}) at ./pkg/dir.jl:31
How to get Pkg in Julia 0.5 then?
That is a very old version of Julia. It is strongly recommended to use a recent version, see the downloads page: https://julialang.org/downloads/. In 0.5 the Pkg module was available by default, so I'm not sure how using Pkg doesn't work. In general, if you're following recent instructions, they're not going to work on such an old version of the language.
I keep getting the following error when attempting to install readxl or haven in R (both dependencies of tidyverse) post-compilation, when the installer runs the loading test:
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '<my_lib_Path>/readxl/libs/readxl.so':
<my_lib_path>/readxl/libs/readxl.so: undefined symbol: libiconv
Error loading failed
I have libiconv.so in a local lib path (not for R packages) that is included in LD_LIBRARY_PATH and I've verified in my R session that Sys.getenv("LD_LIBRARY_PATH") has that directory.
Why can't R's dynamic library loader find this shared object? Is there a different R-specific environment variable I need to define to have the dynamic library loader in R search my local lib path?
Please note that this is not an issue with an R library path, but instead for a non-R dependency that an R package has. If I were compiling and linking C++ code, gcc would use ld, and hence LD_LIBRARY_PATH to track down dynamic dependencies. R doesn't appear to respect this rather common approach, and I can't seem to find any documentation on how to manage these more fine-grained dependency issues.
Additional Details
!> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
>
I had previously compiled libiconv because it was a dependency for something else (don't recall what now - likely not an R package given current problems). I tried reinstalling it, but made no difference.
Edit
I have also tried manually loading the library prior to installation:
> dyn.load(".local/lib/libiconv.so")
> is.loaded("libiconv")
[1] TRUE
> install.packages("tidyverse")
but it fails just as above.
Normally, the iconv method is picked up from glibc, which is linked to during build of the R packages in question. For whatever reason, however, iconv is getting resolved to libiconv in this case, but it is not linked by the R packages during build.
Original Workaround
One can make the linking to libiconv explicit by adding the following line to the haven/src/Makevars source file
PKG_LIBS=-liconv
which then let's you install from source R CMD INSTALL haven. However, editing packages feels hacky, plus this is something that will need to be done every upgrade, which sounds like a hassle.
Cleaner Workaround
Another option is to use withr::with_makevars, which allows one to temporarily control Makevars content. With this technique, one can install directly from the repo:
withr::with_makevars(c(PKG_LIBS="-liconv"), install.packages("haven"), assignment="+=")
Credit: #knb suggested that I inspect the readxl.so with ldd and this turned out to be super useful because it showed that the shared object wasn't even trying to link to libiconv. Knowing that, I realized I could manually add the reference via the -liconv flag. Thanks #knb!
Additional Info
On the package side of things, relevant details about connecting libraries to R packages can be found in the guide for building libraries. On the system configuration side, the R-admin guide has some useful sections.
Are you running the code in RStudio Server? If so, the answer here may be useful.
I used to meet a similar error while loading dynamic library. The library was in a path contained in LD_LIBRARY_PATH. When I ran the code in R console, it could load the dynamic library correctly. But when I ran it in RStudio, the same error in your post raised.
The reason is that RStudio Server has its own library search path environment. You should specify the following configuration in /etc/rstudio/rserver.conf:
rsession-ld-library-path=/usr/lib64/:/usr/local/lib/:OTHER_PATH_OF_YOUR_LIB
Restart RStudio Server and the error should be fixed.
These libraries really should be standard on RH-based system too, and be found.
If you must add them to R, you have to do so before you start R. One way is via LD_LIBRARY_PATH, a better way is to edit a file in /etc/ld.so.conf.d/ (assuming RH/CentOS have that too). Else maybe via /etc/environment.
Edit: If /etc/ is out of reach, you can do everything below $HOME. Standard shell instantiation works, and R has its own .Rprofile and .Renviron. You can have those below $HOME for all your projects, and/or in a per-project directory---see help(Startup).
I put the export LD_LIBRARY_PATH=... statements into a file ~/.profile.
This way, both command line R and RStudio Server are able to find the shared library.
In my case, I was trying to get the Rglpk package to locate the libglpk.so file.
According to Stefaan Lippen's blog entry, the .profile file is the preferred location for these type of configurations not strictly related to bash.
Did you install R via rpm or compile it yourself?
solution 1
If you have the permission to modify the R executable (shell script), you can try this:
Edit ~/.local/bin/R or /usr/local/bin/R or /usr/bin/R
#!/bin/bash
# Shell wrapper for R executable.
export LD_LIBRARY_PATH="<my_lib_Path>/readxl/libs/"
R_HOME_DIR=...
...
...
Solution 2
Or you may vim ~/.local/bin/R
#!/bin/bash
# Shell wrapper for R executable.
export LD_LIBRARY_PATH="<my_lib_Path>/readxl/libs/"
/usr/bin/R
then add ~/.local/bin to your PATH
I keep getting the following error when attempting to install readxl or haven in R (both dependencies of tidyverse) post-compilation, when the installer runs the loading test:
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '<my_lib_Path>/readxl/libs/readxl.so':
<my_lib_path>/readxl/libs/readxl.so: undefined symbol: libiconv
Error loading failed
I have libiconv.so in a local lib path (not for R packages) that is included in LD_LIBRARY_PATH and I've verified in my R session that Sys.getenv("LD_LIBRARY_PATH") has that directory.
Why can't R's dynamic library loader find this shared object? Is there a different R-specific environment variable I need to define to have the dynamic library loader in R search my local lib path?
Please note that this is not an issue with an R library path, but instead for a non-R dependency that an R package has. If I were compiling and linking C++ code, gcc would use ld, and hence LD_LIBRARY_PATH to track down dynamic dependencies. R doesn't appear to respect this rather common approach, and I can't seem to find any documentation on how to manage these more fine-grained dependency issues.
Additional Details
!> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
>
I had previously compiled libiconv because it was a dependency for something else (don't recall what now - likely not an R package given current problems). I tried reinstalling it, but made no difference.
Edit
I have also tried manually loading the library prior to installation:
> dyn.load(".local/lib/libiconv.so")
> is.loaded("libiconv")
[1] TRUE
> install.packages("tidyverse")
but it fails just as above.
Normally, the iconv method is picked up from glibc, which is linked to during build of the R packages in question. For whatever reason, however, iconv is getting resolved to libiconv in this case, but it is not linked by the R packages during build.
Original Workaround
One can make the linking to libiconv explicit by adding the following line to the haven/src/Makevars source file
PKG_LIBS=-liconv
which then let's you install from source R CMD INSTALL haven. However, editing packages feels hacky, plus this is something that will need to be done every upgrade, which sounds like a hassle.
Cleaner Workaround
Another option is to use withr::with_makevars, which allows one to temporarily control Makevars content. With this technique, one can install directly from the repo:
withr::with_makevars(c(PKG_LIBS="-liconv"), install.packages("haven"), assignment="+=")
Credit: #knb suggested that I inspect the readxl.so with ldd and this turned out to be super useful because it showed that the shared object wasn't even trying to link to libiconv. Knowing that, I realized I could manually add the reference via the -liconv flag. Thanks #knb!
Additional Info
On the package side of things, relevant details about connecting libraries to R packages can be found in the guide for building libraries. On the system configuration side, the R-admin guide has some useful sections.
Are you running the code in RStudio Server? If so, the answer here may be useful.
I used to meet a similar error while loading dynamic library. The library was in a path contained in LD_LIBRARY_PATH. When I ran the code in R console, it could load the dynamic library correctly. But when I ran it in RStudio, the same error in your post raised.
The reason is that RStudio Server has its own library search path environment. You should specify the following configuration in /etc/rstudio/rserver.conf:
rsession-ld-library-path=/usr/lib64/:/usr/local/lib/:OTHER_PATH_OF_YOUR_LIB
Restart RStudio Server and the error should be fixed.
These libraries really should be standard on RH-based system too, and be found.
If you must add them to R, you have to do so before you start R. One way is via LD_LIBRARY_PATH, a better way is to edit a file in /etc/ld.so.conf.d/ (assuming RH/CentOS have that too). Else maybe via /etc/environment.
Edit: If /etc/ is out of reach, you can do everything below $HOME. Standard shell instantiation works, and R has its own .Rprofile and .Renviron. You can have those below $HOME for all your projects, and/or in a per-project directory---see help(Startup).
I put the export LD_LIBRARY_PATH=... statements into a file ~/.profile.
This way, both command line R and RStudio Server are able to find the shared library.
In my case, I was trying to get the Rglpk package to locate the libglpk.so file.
According to Stefaan Lippen's blog entry, the .profile file is the preferred location for these type of configurations not strictly related to bash.
Did you install R via rpm or compile it yourself?
solution 1
If you have the permission to modify the R executable (shell script), you can try this:
Edit ~/.local/bin/R or /usr/local/bin/R or /usr/bin/R
#!/bin/bash
# Shell wrapper for R executable.
export LD_LIBRARY_PATH="<my_lib_Path>/readxl/libs/"
R_HOME_DIR=...
...
...
Solution 2
Or you may vim ~/.local/bin/R
#!/bin/bash
# Shell wrapper for R executable.
export LD_LIBRARY_PATH="<my_lib_Path>/readxl/libs/"
/usr/bin/R
then add ~/.local/bin to your PATH
A colleague ported more than a thousand S functions and Fortran subroutines to R. The native R functions are contained in 5 .RData files and the Fortran subroutines are contained in 2 .dll files.
I can load these files into the R workspace using
for (i in 1:5){ load(list.files()[grep("RData",strsplit(list.files(),"\\W+"))][i]) }
for (i in 1:2){dyn.load(list.files()[grep("dll" ,strsplit(list.files(),"\\W+"))][i]) }
After some troubleshooting I know that the relationships between R code and the Fortran subroutines were already well established by the original author.
Now I want to create a package from these files because I'm unable to deploy my ioslides_presentations with these dll files since shinyapps.io is built with Linux. This is my first attempt at creating an R package, I don't intend to publish it to the CRAN, I'm only interested in generating a local zip file that I and my students can load.
MY PROCESS
After loading the functions in the workspace using the above code, I ran package.skeleton and system("R CMD build package") which successfully created "package_1.0.tar.gz". After completing the DESCRIPTION file, I put the .dll files I the src sub-directory and included useDynLib(DLL1), useDynLib(DLL2) in the NAMESPACE file.
However, when I run system("R CMD INSTALL build package_1.0.tar.gz") I get the following error:
Error in library.dynam(lib, package, package.lib) :
DLL 'package.dll' not found: maybe not installed for this architecture?
Error: loading failed
Execution halted
I've spent a great deal of time reading through "Writing R Extensions", "Advanced R" by Hadley Wickham, and the devtools documentation. Even so, I'm having difficulty deciphering useDynLib, .onLoad, Export, Call, and Makevars and which should be used for pre-compiled Fortran code.
I really don't want to parse through all of these functions to find which require export rules if I don't have to. I'd appreciate any help getting these .dll files loaded so I can finish creating this package
Thanks in advance,
Jason