I am programmatically executing notebooks using nbconvert.preprocessors.ExecutePreprocessor. My notebooks depend on resources specified by relative file name, and I cannot get the kernel to run with a specified directory as the working directory. In other words, this:
runPath = r'd:\blah\myPreferredDirectory'
proc = ExecutePreprocessor(timeout=600, kernel_name='python3')
proc.preprocess(nb, {'metadata': {'path':runPath}})
...is not respecting the specified path. If I insert a printout of the current directory into my notebook, it isn't the specified directory, though if I specify an invalid directory, I do get an error, so it's at least confirming that the directory exists. Modifying the notebooks to explicitly change directories is not an option.
Advice?
Using the nbconvert command line would be fine as well, though I slightly prefer to do this programmatically.
Related
Question: How can one set the working directory of all notebooks opened in Jupyter Lab with a double-click, to be the project's folder, /myproject/, regardless of the notebook's subfolder within that parent folder? The working directory is identified by !pwd on Linux/Mac or !cd on Windows.
Context:
The Jupyter Lab sessions is initiated from the project's folder, by: /myproject/jupyter lab.
I am not looking for changing the working directory within the notebook with code (e.g. with !cd.. or using os), but change the settings of Jupyter Lab, such that the all kernels will start with the folder from where Jupyter Lab was initiated.
This is useful for being able to use consistent relative folders, regardless of the subfolder they are referred from. For example, for loading data in a specific subfolder.
I agree that this is often a preferred approach! I always have my notebooks configured to work like that because it:
makes it easy to specify paths to data and for outputs and
allows moving a notebook between directories without the need to change the paths
makes jupyterlab-lsp code intelligence work more reliably
I have a python module called make_paths_absolute.py with the following contents:
from os import chdir
from pathlib import Path
def make_paths_relative_to_root():
"""Always use the same, absolute (relative to root) paths
which makes moving the notebooks around easier.
"""
top_level = Path(__file__).parent
chdir(top_level)
make_paths_relative_to_root()
And in the first cell of every notebook, I add a line import make_paths_absolute. I like it this way because it makes it:
reproducible: anyone who copies/clones my project will be able to run the code without the need to customize anything in their Jupyter environment
work with all notebook interfaces (whether JupyterLab/RetroLab/classic Notebook)
is quite obvious for anyone reading the notebook that they should expect the paths to be absolute (=relative to the root).
To make that work you first need to set PYTHONPATH pointing to the root of the repository when starting JupyterLab. To do so on Linux/Mac you can prepend the jupyter lab command with:
PYTHONPATH=/path/to/your/lab/root:$PYTHONPATH jupyter lab
Alternatively, you can specify PYTHONPATH in kernel.json (either by modifying the default kernel specification or creating a copy of it first, see https://stackoverflow.com/a/53595397).
PS. The above is a simplification of my setup. In reality, I store make_paths_absolute.py in helpers package (so there is also an empty helpers/__init__.py and there is extra .parent) together with other initialization-time code, including a notebook called notebook_setup.ipynb which contains things like %matplotlib inline, pandas customizations (like making sure it uses stable sort), warning filters etc. So my every notebook starts with:
import helpers.make_paths_absolute
%run helpers/notebook_setup.ipynb
and I am really liking this setup working like that for two years now without any problems.
There is a feature request for making this easier at: https://github.com/jupyterlab/jupyterlab/issues/11619.
The most easiest solution in my opinion
Open Notepad
Paste the command "jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10"
Save the notepad file with an extension of ".bat" instead of ".txt"
Paste the file in which directory you want to initialize your jupyter
Double click and open the ".bat" file
Jupyter opens with desired directory as base
This way you control the jupyter root directory as and when required and don't
really have to perform any manual settings
Hope this helps
".bat" file created on Desktop
enter image description here
".bat" file double clicked and executed
enter image description here
jupyter opens with Desktop as the intended base directory
enter image description here
I was able to temporarily change it with the following command, but after reboot the working path still changes to C drive.
cd("D:\\jl files”）
As it is explained in the Julia Manual Getting Started section the simplest thing to do is to add the:
cd("D:\\jl files")
line to your ~/.julia/config/startup.jl file (this is a Linux path; if the file does not exist then you should create it with this single line). Since you are on Windows then ~ part should be replaced by default is your user profile folder (it should be possible to check it with ENV["USERPROFILE"] command in Julia). However, if you have a custom installation the .julia folder can be placed in some other folder so you need to check it on your system.
I was wondering if there is a way to set a default folder every time I open JupyterLab. Is there any command I can run through Anaconda PowerShell Prompt? Or a JSON property I can modify in settings tab? Thanks in advance!
If you want to open specific file in directory a, say a/notebook.ipynb but then navigate the file browser to directory b, relative to the root directory, you can use file-browser-path query in the navigation URL (documented here):
http(s)://<server:port>/<lab-location>/lab/tree/a/notebook.ipynb?file-browser-path=/b
You can use this method from command line thanks to LabApp.default_url traitlet:
jupyter-lab --LabApp.default_url='/lab/tree/a/notebook.ipynb?file-browser-path=/b'
You can skip the opening specific file part:
jupyter-lab --LabApp.default_url='/lab?file-browser-path=/b'
If you want to change the root directory, please see this answer which explains how to do so with ServerApp.root_dir traitlet (in older versions of Notebook<7.0 and JupyterLab<3.0 used to be named NotebookApp.notebook_dir). Again, you can use it from command line, and even specify an absolute path:
jupyter-lab --ServerApp.root_dir='/home/user/project/'
# or on Windows say:
# jupyter-lab --ServerApp.root_dir='C/users/user/project/'
You can combine the two approaches, e.g.:
jupyter-lab --ServerApp.root_dir='/home/user/project/' --LabApp.default_url='/lab?file-browser-path=/project_subdirectory'
If you want to persist the changes (for either default URL or root dir), see the previously linked answer which details how to persist configuration.
Follow the answer . Just edit the alias alias my_jupn='source /home/$USER/anaconda3/bin/activate && conda activate MyPy38 && jupyter-notebook --notebook-dir=/home/$USER/Documents/Jupyter_Notebook'
and for JupyterLab alias my_jupl='source /home/$USER/anaconda3/bin/activate && conda activate MyPy38 && jupyter-lab --notebook-dir=/home/$USER/Documents/Jupyter_Notebook'
N.B. Here MyPy38 is your active environment name. and if any error occurs then change $USER to your UserName. Lastly, /home/$USER/Documents/Jupyter_Notebook will be change according to your choice.
According to the jupyter.org:
I configured like this.
Terminal command active your envirnment by conda activate Your_env_name and then jupyter lab --generate-config. jupyter_lab_config.py should be created in dir /home/$USER/.jupyter directory.
Find
# c.LabApp.app_settings_dir = '' and uncomment.
Then give your directory like this.
c.LabApp.app_settings_dir = '/home/$USER/Documents/Jupyter_Notebook'
Similar for jupyter notebook. jupyter notebook --generate-config and then nu-comment and edit like c.NotebookApp.notebook_dir = '/home/UserName/Documents/Jupyter_Notebook'
**NB: /home/$USER/Documents/Jupyter_Notebook is my changed default dir. Change it according to your need. **
The command
jupyter notebook --generate-config
will create a config file.
The path to this file is displayed in your terminal or PowerShell.
Set the desired folder path as the string c.ServerApp.root_dir in this file and make sure you uncomment the line after you set the string.
Windows users will have to use double backslash in paths, e.g.:
c.ServerApp.root_dir = 'C:\\YOUR\\PATH'
I am trying to add my R script into a dockerfile. The beginning of the file (loading base image, installing necessary packages) works fine when I run it in my terminal, but I keep getting this error when it gets to the line that contains the R script I want to run:
Step 15/17 : COPY /Users/emma/Documents/folder1/examples/question-1/model-1.r .
COPY failed: stat /var/lib/docker/tmp/docker-builder376572603/Users/emma/Documents/folder1/examples/question-1/model-1.r: no such file or directory
I am already running the terminal out of the "question-1" directory (my shell command looks like this) :
Emmas-MacBook-Air-2:question-1 emma$
and the R script file "model-1.r" is in that folder. What am I doing wrong in detailing the path to the R script? Do I have to somehow transform the script before adding it to the dockerfile?
Thank you
I believe, that you have to specify relative (to your build folder) path to copy from. Source:
Multiple resources may be specified but the paths of files and directories will be interpreted as relative to the source of the context of the build.
And file to copy should be inside context of the build. So if your Dockerfile is located in folder A, then the objects you would like to copy should be placed in the folder A or it's subfolders.
There is a CSV file called orders_data stored in my system, but when I try to load this file in Julia using readdlm command in Jupyter Notebook(running in my browser), it says "NO SUCH FILE DIRECTORY FOUND"
I'm not sure why does this happen? is there a specific location where the files need to be stored to be accessed using Julia command? is it that I need to install some packages first to load the file using browser version of jupyter?
//Error information
SystemError: opening file orders_data.csv: No such file or directory
Your working directory is set to your current location when you start a Julia session. You can see what it is by calling the pwd() function. You can change it by calling the cd() function. Unless you specify otherwise, or provide a more complete pathname, Julia looks for files in your current working directory (although it's different for modules).