Is there a Python-like virtualenv environment simulator for Julia where one can do development in a local, virtual environment?
Currently (julia 1.2) is able to manage virual environments via it's builtin Pkg standard library module:
https://docs.julialang.org/en/v1/stdlib/Pkg
julia> ]
(v1.2) pkg> activate tutorial
[ Info: activating new environment at `/tmp/tutorial/Project.toml`.
(tutorial) pkg>
(tutorial) pkg> status
Status `/tmp/tutorial/Project.toml`
(empty environment)
(tutorial) pkg> add Example
...
(tutorial) pkg> status
Status `/tmp/tutorial/Project.toml`
[7876af07] Example v0.5.1
There is Playground.jl
A package for managing julia sandboxes like python's virtualenv (with a little influence from pyenv and virtualenvwrapper)
Nowadays Julia has this kind of thing built into it's package manager, they're called environments and it's described here. It boils down to this, enter the package mangement repl by hitting ], then the command activate $dir switches to the environment described in $dir, then use the instantiate command to install the packages described in the environment.
What's wrong with just having a separate Julia installation in another directory? Then you just need to set the JULIA_PKGDIR environment variable appropriately, for the Julia setup you want to run.
Related
is there a way to setup a virtual environment in Julia using an environment file? (for instance like .yml file for creating conda venv)
Julia Version: 1.7.1
OS: Windows 10
In Julia virtual environment is defined via Project.toml file (that keeps package names and their acceptable versions) and Manifest.toml (that keeps the exact dependence tree and package versions generated along requirements defined in Project.toml).
Here is a sample Julia session:
julia> using Pkg
julia> pkg"generate MyProject"
Generating project MyProject:
MyProject/Project.toml
MyProject/src/MyProject.jl
julia> cd("MyProject")
julia> pkg"activate ."
Activating environment at `/home/ubuntu/MyProject/Project.toml`
Finally, note that you can manipulate the Project.toml by eg. adding a package like this (this assumes the environment is active):
pkg"add DataFrames"
Sometimes you want to provide package version information to your Project.toml, for an example you could add at the end of the file:
[compat]
DataFrames = "1.3.0"
After adding the first dependency the Mainifest.toml file has been generated. Copying this file together with Project.toml across machines allows you to duplicate the environment.
In order to have all packages installed on a new machine you will need to run:
pkg"activate ."
pkg"instatiate"
pkg"instatiate" can also be used to generate Mainfest.toml when only the Project.toml is present.
The nice thing is that Julia can store many package versions simultaneously and the virtual environments only links to the central package repository (contrary to Python that copies several GBs of data each time).
pkg provides this with project.toml files.
open julia in console
julia>]
Pkg> activate <Name>
<Name> Pkg> add <PackageName>
create directory with
I'm trying to figure out how Julia packages work since I like having containerized environments. I'm really struggling with it.
In python, I'd do something like conda create env --name ds to make an environment and then to install containerized packages I'd use conda activate ds; conda install <packages>.
I'm not having much success trying to get Julia to make a virtual environment.
From the Julia REPL I can type ] to go to package managers then I can create an enviornment with activate ds. From here I can add important packages add IJulia DataFrames Plots
At this point, my environment becomes actual folders which is good.
What I then don't know how to do is to activate my environment so that I can then run using IJulia; notebook()
From the REPL if I type activate ds it doesn't know what I'm talking about, even if I do cd("ds"); activate . it still doesn't know what I'm trying to do...
I looked at the docs and it seems to detail out how to manipulate packages but I haven't found anything helpful for actually running them.
You have to write activate ds (or activate . if you are already in the ds directory) in the package manager mode that is started with ] as you have commented.
Alternatively you can activate environments when you start Julia. Just write
julia --project=.
(if you are already in the ds directory).
Here https://github.com/bkamins/PyDataGlobal2020 you have a step by step example how to run things for a sample project.
A third option is to activate the environment via the package manager API e.g. like this
using Pkg
Pkg.activate(".")
I downloaded someone else's project and the structure is as follows:
project/
notebooks/
notebook_a.ipynb
notebook_b.ipynb
library/
Manifest.toml
Project.toml
src/
test/
In the notebooks I would like to import library, and install its dependencies within its own evironment.
Following the sugestions here, I can do
using Pkg
Pkg.activate("../library/")
but I wonder if I could install a kernel that has the project directory specified and automatically activates the library environment. What should I pass to IJulia's installkernel? "--project=..." what?
I use conda regularly and this confused me because I thought that running notebook once the environment is activated in the command line would have the right environment in the notebook, but this was not the case.
IJulia starts its own Julia process and hence is not using environment settings from its master.
By default IJulia sets the environment (Project.toml) from the folder it was started. The most convenient thing would be to move the notebooks folder to be subfolder of library and then just run:
notebook(dir="/path/to/project/library")
If you do not want to change the folder structure you still need to run:
notebook(dir="/path/to/project/notebooks")
Once in the notebook you need to run:
using Pkg
pkg"activate /path/to/project/library"
I often ]dev Pkg but I want the devved packaged to be stored somewhere other than the default location for convenient access.
I don't want to change the path of the ]add Pkg. This seems to be controlled by the environment parameter DEPOT_PATH.
Is there a way to change only the path for dev Pkg, i.e. the path in which the dev package is stored?
You can set the environment variable JULIA_PKG_DEVDIR to change where development packages are installed. See the develop docs for more info.
As #crstnbr noted, an alternative is to use the --local option to the pkg> dev command to install a development version of the package in a dev directory within the current project. This could make sense if you're developing your own package MyCode.jl which relies on Example.jl and you need to make a hot fix to Example.jl. Then your Pkg REPL command would look like this:
(MyCode) pkg> dev --local Example
If you would like to make changes to a third-party package and submit those changes as a pull request on Github, there are a few more steps in the process. See this Discourse thread for more details on that process.
Not quite what you're asking for but you can of course always git clone the package to a path of your choice and then dev path/to/the/local/clone/of/the/pkg.
You can even do this from within julia:
using Pkg
Pkg.GitTools.clone("<pkg url>", "<local path>")
Pkg.develop(PackageSpec(path="<local path>"))
I'm trying to make julia language available through jupyterhub on an ubuntu server.
I already have installed and configured the jupyterhub. Its working fine with python3.5.
And the authentication method is Regular Unix users and PAM.
I installed the julia language in /usr/local/julia-1.0.2/ and it is available for all users globally.
then with the root user I set the JULIA_DEPOT_PATH="/usr/share/juliapackages/
then again with the root user, I run the julia and run the
using Pkg
Pkg.add("IJulia")
it installs the IJulia in the specified path.
from this point, I didn't find any further useful instructions on the internet over the subject of installing julia kernel for jupyterhub, so I don't know how to proceed.
does anybody have a good step by step document to find the solution?
I followed the instruction proposed here but it seems doesn't work for me.
As you are using Jupyterhub, the best way would be to use a docker spawner and use the data science docker image which has Julia already installed and configured.
https://github.com/jupyter/docker-stacks/blob/master/datascience-notebook/Dockerfile