I want to change the Package directory in Julia. The default is
"~/.julia/v0.4"
I want to move it to /opt/julia/v0.4/. Ideally I want to move the packages that are already installed in ~/.julia/v0.4 to the new location. But if that is not possible I can reinstall them.
What do I have to do?
Julia-v0.6 and before
one can change julia's package directory by following these steps:
run export JULIA_PKGDIR=/your/directory in shell(or manually add a new environment variable JULIA_PKGDIR on windows)
run Pkg.init() in julia to initialize a new package system
copy REQUIRE from old directory to the new one
run Pkg.resolve() in julia
Julia-v0.7+
The "Package directory" in the new package manager is called DEPOT_PATH, it can be changed by adding an environment variable JULIA_DEPOT_PATH:
JULIA_DEPOT_PATH=./test julia
julia> DEPOT_PATH
1-element Array{String,1}:
"./test"
(v0.7) pkg> add JSON2
Cloning default registries into /Users/gnimuc/test/registries
With the new package manager, we are able to create isolated projects in any directory we want instead of having a single giant package directory. Every project contains a Project.toml and a Manifest.toml if it has some dependencies. These two files record and keep tracking the environment of the project.
UPDATE
The following info might be obsoleted. I highly recommend to use PkgTemplates.jl for generating projects in Julia-v1.0+.
Generate a new project
We can generate a new project in any folder, but we must cd to the project folder to using the project. The (v0.7) below shows we're still in the default environment, so we cannot use the newly generated project:
(v0.7) pkg> generate ./MyNewProject
Generating project MyNewProject:
./MyNewProject/Project.toml
./MyNewProject/src/MyNewProject.jl
julia> using MyNewProject
ERROR: ArgumentError: Module MyNewProject not found in current path.
Run `Pkg.add("MyNewProject")` to install the MyNewProject package.
Stacktrace:
[1] require(::Module, ::Symbol) at ./loading.jl:868
If we cd to the project folder and activate the environment, then we can using our new project without any problems:
shell> cd MyNewProject/
/Users/gnimuc/MyNewProject
(v0.7) pkg> activate .
(MyNewProject) pkg>
julia> using MyNewProject
I think that's the big difference between the new package manager and the old one. In short, we need to explicitly activate our unregistered project/package.
Download and init someone else's project
According to the doc, we can add an unregistered package/project via add command:
(HelloWorld) pkg> add https://github.com/fredrikekre/ImportMacros.jl
This command adds the target package as a dependency of our current project. In this example, we added ImportMacros in HelloWorld's Manifest.toml. What if we just use it as a top-level project? To add it to the default environment (v0.7)? no, we don't need to. The answer is we can directly download the code, cd to the folder and run instantiate in the pkg mode:
shell> git clone https://github.com/Gnimuc/GLTF.jl GLTF
Cloning into 'GLTF'...
remote: Counting objects: 286, done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 286 (delta 73), reused 103 (delta 59), pack-reused 167
Receiving objects: 100% (286/286), 62.16 KiB | 46.00 KiB/s, done.
Resolving deltas: 100% (135/135), done.
shell> cd GLTF
pkg> activate .
(GLTF) pkg> instantiate
Updating registry at `~/.julia/registries/General`
Updating git-repo `https://github.com/JuliaRegistries/General.git`
The new package manager is great! We neither need "include before using" nor make everything as a package just for using it. We have full-featured "Project" now!
Julia only way:
julia> ENV["JULIA_PKGDIR"] = "E:\\Julia-0.6.0\\portable"
"E:\\Julia-0.6.0\\portable"
julia> ENV["JULIA_PKGDIR"]
"E:\\Julia-0.6.0\\portable"
julia> Pkg.init()
INFO: Initializing package repository E:\Julia-0.6.0\portable\v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
However, the cache dir is still pointing to the old folder, so I checked why that is and figured it out:
julia> Base.LOAD_CACHE_PATH
1-element Array{String,1}:
"C:\\Users\\kung\\.julia\\lib\\v0.6"
julia> Pkg.__init__()
2-element Array{String,1}:
"E:\\Julia-0.6.0\\portable\\lib\\v0.6"
"C:\\Users\\kung\\.julia\\lib\\v0.6"
julia> pop!(Base.LOAD_CACHE_PATH)
"C:\\Users\\kung\\.julia\\lib\\v0.6"
julia> Base.LOAD_CACHE_PATH
1-element Array{String,1}:
"E:\\Julia-0.6.0\\portable\\lib\\v0.6"
As simple function:
function set_julia_dir(dir::String)
ENV["JULIA_PKGDIR"] = dir
Pkg.init()
Pkg.__init__()
pop!(Base.LOAD_CACHE_PATH)
end
set_julia_dir("E:\\Julia-0.6.0\\portable")
Related
In Python, if I install mambaforge or conda then I can make a file with extension .yml and then inside it list the name of packages I want to install alongside their specific versions. How can I do a similar way of installing packages in Julia?
I understand that if I have already installed Julia packages by addcommand in package manager, then I have a file named Project.toml which I can use to install the same packages later. However, this still does not look as good as Python's way of installing packages.
Upon further investigation I realized that to install Julia packages from an empty Prokect.tomlfile, I should add [deps]in the file followed by the name of packages I want and then give each package a uuidwhich can be found here. For example:
[deps]
Images = "916415d5-f1e6-5110-898d-aaa5f9f070e0"
After all , this is still tedious as it needs to find all those uuids.
How can I install packages in Julia the same way I described for Python?
Is there a particular reason that you want to write package names to a .yml file and then read the packages from there? After all, you can generate the Project file and add multiple dependencies automatically:
(#v1.8) pkg> generate MyProject # or whatever name you like
(#v1.8) pkg> activate MyProject
(MyProject) pkg> add Countries Crayons CSV # some example packages
(In recent versions of Julia, an installation prompt will appear if a package isn't already installed).
Speaking from experience, learning to use environments in Julia can be challenging to a new user, but rewarding! The documentation for Pkg.jl are helpful here.
If you are just assembling an environment for your own code, there is probably no need for you to manually edit Project.toml. On the other hand, if you are maintaining a package, you might wish to edit the file directly (e.g., for specifying compatability).
Maybe you can use this:
using TOML
using HTTP
using DataStructures
## Get the Registry.toml file if it already doesn't exist in the current directory
function raed_reg()
if !isfile("Registry.toml")
HTTP.download(
"https://raw.githubusercontent.com/JuliaRegistries/General/master/Registry.toml",
"Registry.toml"
)
end
reg = TOML.parsefile("Registry.toml")["packages"]
return reg
end
## Find the UUID of a specific package from the Registry.toml file
function get_uuid(pkg_name)
reg = raed_reg()
for (uuid, pkg) in reg
if pkg["name"] == pkg_name
return uuid
end
end
end
## Create a dictionary for the gotten UUIDs, by setting the name = UUID and convert it to a project.toml file
function create_project_toml(Pkgs::Vector{String})
reg = raed_reg()
pkgs_names_uuid = OrderedDict{AbstractString, AbstractString}()
pkgs_names_uuid["[deps]"] = ""
for pkg in Pkgs
_uuid = get_uuid(pkg)
pkgs_names_uuid[pkg] = _uuid
end
open("project.toml", "w") do io
TOML.print(io, pkgs_names_uuid)
end
end
## Test on the packages "ClusterAnalysis" and "EvoTrees"
create_project_toml(["ClusterAnalysis", "EvoTrees"])
My goal is: in Julia 1.7 on Mac with M1 processor I would include a module file with many functions inside.
I've tried to follow this thread to generate my own package
but it's generated an error.
I followed the answer but when i try importing MyPackage, Julia says: ArgumentError: Package MyPackage not found in current path.
With pwd() current path is Users/myname and in this folder MyPackage exists.
With command "import MyPackage" where i can see default folder for packages import ?
Where I get wrong ?
Sorry for my English.
I know two options here. Suppose I generated a package with the following file structure
MyPackage/
src/
MyPackage.jl
Project.toml
and MyPackage.jl is
module MyPackage
export greet
greet() = print("Hello World!")
end # module
Then there are two options, I know
Including of module MyPackage;
Embedding of package MyPackage into Julia's ecosystem.
Suppose, that pwd is MyPackage/
First option is
julia> include("src/MyPackage.jl")
Main.MyPackage
julia> using .MyPackage
julia> greet()
Hello World!
Notice the dot in using statement.
Second option
Start Julia REPL and enter pkg mode, then
(#v1.6) pkg> dev MyPackage/
Resolving package versions...
Updating `~/.julia/environments/v1.6/Project.toml`
[88e94d31] + MyPackage v0.1.0 `foo/bar/MyPackage`
Updating `~/.julia/environments/v1.6/Manifest.toml`
[88e94d31] + MyPackage v0.1.0 `foo/bar/MyPackage`
(#v1.6) pkg> st
Status `~/.julia/environments/v1.6/Project.toml`
[6e4b80f9] BenchmarkTools v1.3.1
[0772a1fa] CubicEoS v0.2.0
..........
[88e94d31] MyPackage v0.1.0 `foo/bar/MyPackage`
..........
[09ab397b] StructArrays v0.6.5
[a759f4b9] TimerOutputs v0.5.15
st command says that MyPackage became available in default environment.
Then you should be able to import/using the package. For the first time you should see precompiling message.
julia> using MyPackage
[ Info: Precompiling MyPackage [88e94d31-ecaf-41ca-ae10-053d89a189ff]
julia> greet()
Hello World!
P.S. The best place for local packages is .julia/dev/.
I am trying to add dependencies to a package that I am developing, but I get the following error:
(HijriConverter) pkg> add Parameters
Updating registry at `~/.julia/registries/General.toml`
Resolving package versions...
ERROR: expected the file `src/HijriConverter.jl` to exist for package `HijriConverter` at `/home/jafar_isbarov/Documents/projects/hijri/hijri-converter-julia-package/HijriConverter.jl`
Adding a file named HijriConverter to the src folder does solve the problem. I have two questions.
(1) Why do I need a file with the same name as the package? Is it supposed to have certain content?
(2) If I add that file, add dependencies, and delete the file afterwards, will that cause any problems?
Thanks.
To understand the minimal package requirements try running Pkg.generate as in code below:
julia> Pkg.generate("MyPackage")
Generating project MyPackage:
MyPackage\Project.toml
MyPackage\src/MyPackage.jl
Dict{String, Base.UUID} with 1 entry:
"MyPackage" => UUID("bad70bc4-3cf3-42bb-9f14-6b74ac64e2d7")
And here is the minimal file layout:
shell> tree /F
C:.
└───MyPackage
│ Project.toml
│
└───src
MyPackage.jl
And this is the generated module content with a single simple function:
shell> more MyPackage\\src\\MyPackage.jl
module MyPackage
greet() = print("Hello World!")
end # module
Further reading: https://pkgdocs.julialang.org/v1/creating-packages/
I am trying to build and deploy this bookdown project with GitHub Actions. One of the chapters uses the keras R package, which means I need to install Conda (or set up a virtual environment). At the end of the Miniconda installation command, there is an error when trying to collect metadata.
2020-06-24T04:47:59.7495480Z * Miniconda has been successfully installed at '/Users/runner/Library/r-miniconda'.
2020-06-24T04:47:59.7496060Z [1] "/Users/runner/Library/r-miniconda"
2020-06-24T04:48:00.3909040Z * Project '~/runners/2.263.0/work/drake/drake' loaded. [renv 0.10.0]
2020-06-24T04:48:00.7964920Z * The project and lockfile are out of sync -- use `renv::status()` for more details.
2020-06-24T04:48:00.7968340Z Warning message:
2020-06-24T04:48:00.7969190Z Project requested R version '3.6.0' but '4.0.1' is currently being used
2020-06-24T04:48:05.2408080Z Collecting package metadata (current_repodata.json): ...working... failed
2020-06-24T04:48:05.2410390Z
2020-06-24T04:48:05.2410820Z NotWritableError: The current user does not have write permissions to a required path.
2020-06-24T04:48:05.2411080Z path: /usr/local/miniconda/pkgs/cache/b89cf7bf.json
2020-06-24T04:48:05.2411230Z uid: 501
2020-06-24T04:48:05.2411350Z gid: 20
2020-06-24T04:48:05.2411430Z
2020-06-24T04:48:05.2411690Z If you feel that permissions on this path are set incorrectly, you can manually
2020-06-24T04:48:05.2411940Z change them by executing
2020-06-24T04:48:05.2412010Z
2020-06-24T04:48:05.2412260Z $ sudo chown 501:20 /usr/local/miniconda/pkgs/cache/b89cf7bf.json
2020-06-24T04:48:05.2412330Z
2020-06-24T04:48:05.2413470Z In general, it's not advisable to use 'sudo conda'.
2020-06-24T04:48:05.2413570Z
2020-06-24T04:48:05.2414250Z
2020-06-24T04:48:05.2886400Z ##[error]Error: Error 1 occurred creating conda environment r-reticulate
2020-06-24T04:48:05.2890770Z Execution halted
2020-06-24T04:48:05.3050700Z ##[error]Process completed with exit code 1.
The full job log is here.
Depending on how R is set up, this post might be helpful for you. You might need to configure the .Renviron file.
Unable to change python path in reticulate (R)
While trying to load the RProvider.fsx, I get a error that the Assembly reference RDotNet.dll was not found or is invalid
I'm following this walktrough: http://bluemountaincapital.github.io/FSharpRProvider/Statistics-QuickStart.html
Using Visual Studio 15 (I use the free community edition)
Create new F# console project: UseNuGetPackage (Exact steps NOT given)
Download and install R (Exact steps NOT given)
Install R Type provider
In Visual Studio using Solution Explorer
Right click F# project e.g. UseNuGetPackage
Select: Manage NuGet Packages for Solution
Click Browse
Enter: RProvider
Click check box on the left of Project
Click Install
Click OK
The Visual Studio output window shows:
Attempting to gather dependencies information for package 'RProvider.1.1.15' with respect to project 'UseNuGetPackage', targeting '.NETFramework,Version=v4.5.2'
Attempting to resolve dependencies for package 'RProvider.1.1.15' with DependencyBehavior 'Lowest'
Resolving actions to install package 'RProvider.1.1.15'
Resolved actions to install package 'RProvider.1.1.15'
Adding package 'DynamicInterop.0.7.4' to folder 'c:\users\eric\documents\visual studio 2015\Projects\UseNuGetPackage\packages'
Added package 'DynamicInterop.0.7.4' to folder 'c:\users\eric\documents\visual studio 2015\Projects\UseNuGetPackage\packages'
Added package 'DynamicInterop.0.7.4' to 'packages.config'
Successfully installed 'DynamicInterop 0.7.4' to UseNuGetPackage
Adding package 'R.NET.Community.1.6.4' to folder 'c:\users\eric\documents\visual studio 2015\Projects\UseNuGetPackage\packages'
Added package 'R.NET.Community.1.6.4' to folder 'c:\users\eric\documents\visual studio 2015\Projects\UseNuGetPackage\packages'
Added package 'R.NET.Community.1.6.4' to 'packages.config'
Successfully installed 'R.NET.Community 1.6.4' to UseNuGetPackage
Adding package 'R.NET.Community.FSharp.1.6.4' to folder 'c:\users\eric\documents\visual studio 2015\Projects\UseNuGetPackage\packages'
Added package 'R.NET.Community.FSharp.1.6.4' to folder 'c:\users\eric\documents\visual studio 2015\Projects\UseNuGetPackage\packages'
Added package 'R.NET.Community.FSharp.1.6.4' to 'packages.config'
Successfully installed 'R.NET.Community.FSharp 1.6.4' to UseNuGetPackage
Adding package 'RProvider.1.1.15' to folder 'c:\users\eric\documents\visual studio 2015\Projects\UseNuGetPackage\packages'
Added package 'RProvider.1.1.15' to folder 'c:\users\eric\documents\visual studio 2015\Projects\UseNuGetPackage\packages'
Added package 'RProvider.1.1.15' to 'packages.config'
Successfully installed 'RProvider 1.1.15' to UseNuGetPackage
========== Finished ==========
Notice in the image below the green check mark identifying that the package is installed.
Notice in the image below the new references for the project.
To get the following two statements to work we need to verify some paths.
#I "../packages/RProvider.1.0.11"
#load "RProvider.fsx"
The example given is specific to a version and as we know versions change so check the version of the DLL that will be used.
In the F# project
Right click the RProvider reference
Right click Properties
Here is the full path so you don't have to squint
c:\users\eric\documents\visual studio 2015\Projects\UseNuGetPackage\packages\RProvider.1.1.15\lib\net40\RProvider.dll
The example is using version RProvider.1.0.11 and we have RProvider.1.1.15
so we need to update the line from the example to
#I "../packages/RProvider.1.1.15"
If you look a lot of examples where you download a package from NuGet and then use it with F# Interactive you will see the same pattern to get it started.
#I "../packages/<Name_XYZ>.<Version>"
#load "<Name_XYZ>.fsx"
This is because this code assumes that you have used NuGet to install the needed DLLs in a VS project and that the DLLS are in a packages directory that is relative to the source code for the project. If you look back at the listing I gave for the output window you will see the copying of the DLLs.
Here they are shown in File Explorer
So how does F# Interactive know how to find the DLLs when it only has a relative path.
Now for the magical missing part that we leave out because it is a
conspiracy to test newbies. :-)
F# Interactive has the property __SOURCE_DIRECTORY__ that gives the location of the source directory in a VS project. This can be seen in F# Interactive with
printfn __SOURCE_DIRECTORY__
which for my example is
c:\users\eric\documents\visual studio 2015\Projects\UseNuGetPackage\UseNuGetPackage
val it : unit = ()
if you get a directory containing AppData then restart F# Interactive.
This can be checked with File Explorer
sure enough this is the source directory for the project.
More importantly the path from the source directory to the NuGet packages is
../packages
So all the paths look good.
Now that we have updated and checked the script lines to load the DLL give it a try.
We get the error:
error FS0082: Could not resolve this reference. Could not locate the assembly "RDotNet.FSharp.dll".
The problem is the specific versions mentioned in RProvider.fsx
// Standard NuGet locations for R.NET
#I "../R.NET.Community.1.5.16/lib/net40"
#I "../R.NET.Community.FSharp.0.1.9/lib/net40"
If we substitute out #load "RProvider.fsx" with
#r "../packages/R.NET.Community.1.6.4/lib/net40/RDotNet.dll"
#r "../packages/R.NET.Community.FSharp.1.6.4/lib/net40/RDotNet.FSharp.dll"
#r "../packages/RProvider.1.1.15/lib/net40/RProvider.dll"
#r "../packages/RProvider.1.1.15/lib/net40/RProvider.Runtime.dll"
open RProvider
do fsi.AddPrinter(fun (synexpr:RDotNet.SymbolicExpression) -> synexpr.Print())
open System
open RDotNet
open RProvider
open RProvider.graphics
open RProvider.stats
The example works. e.g.
// Random number generator
let rng = Random()
let rand () = rng.NextDouble()
// Generate fake X1 and X2
let X1s = [ for i in 0 .. 9 -> 10. * rand () ]
let X2s = [ for i in 0 .. 9 -> 5. * rand () ]
// Build Ys, following the "true" model
let Ys = [ for i in 0 .. 9 -> 5. + 3. * X1s.[i] - 2. * X2s.[i] + rand () ]
let dataset =
namedParams [
"Y", box Ys;
"X1", box X1s;
"X2", box X2s; ]
|> R.data_frame
let result = R.lm(formula = "Y~X1+X2", data = dataset)
let coefficients = result.AsList().["coefficients"].AsNumeric()
let residuals = result.AsList().["residuals"].AsNumeric()
let summary = R.summary(result)
summary.AsList().["r.squared"].AsNumeric()
R.plot result
Also as Tomas noted in his answer there is a bug with the version numbers in the fsx file, but he was able to update the source code and make a new alpha version.
:)
Hopefully I gave enough detail that the next time you see one of these examples on using a NuGet package with F# Interactive and have a problem you can resolve it.
There is a bug in RProvider.fsx where it references a wrong version of R.NET. The version 1.1.16-alpha has a fix for this, but we did not release a new non-beta yet.
Can you check that this works fine if you install 1.1.16-alpha?
The link of #GuyCoder helped me: How does F# Interactive #I command know about project path?
I searched the missing .dll files on my computer, and pasted in the parent folder of the source directory.
After this everything worked fine
#I "../packages/RProvider.1.1.15"
(*
#r "RDotNet.dll"
#r "RDotNet.FSharp.dll"
These 2 are unnecessary*)
#load "RProvider.fsx"
open System
open RDotNet
open RProvider
open RProvider.graphics
open RProvider.stats
EDIT:
I first downloaded the nuget packages Rdotnet & Rdotnet.Fsharp, then I searched on my computer for the dll's,
Then I copied these files and pasted it in the directory I was working in:
Which you also can find with __SOURCE_DIRECTORY__