Way to get jupyter server root directory - jupyter-notebook

Consider following:
$ cd /home/mydir
$ jupyter notebook --port=8888
In plain English, I am running jupyter server from /home/mydir directory.
Is there a simple way to get this directory from within a notebook regardless if it's a R notebook or a Python notebook or whatever? Maybe there is some magic command or variable?
NOTE: getwd() is not an answer as it returns directory of a current notebook but not the jupyter server root.

I have a similar problem and found your post, although I don't see a viable solution yet. Eventually I did found a solution, although it works only because I only care about Linux and I only care about Python. The solution is this magic line:
J_ROOT = os.readlink('/proc/%s/cwd' % os.environ['JPY_PARENT_PID'])
(I put it in a module in my PYTHONPATH so that I can easily use it in any Python notebook.) See if it is good for you.

Remember that your iPtyhon is just a Python module, so you can execute any valid Python code in a cell. So, if you started your notebook and haven't executed any directory changes in your code, you should be able to retrieve your cwd with the following in a cell:
import os
os.getcwd()
But furthermore, you can execute shell commands in cells, so you can retrieve other information in the cell. For example:
!which jupyter
should give you the path to your jupyter executable.
Which then leads you to running something like:
!jupyter --paths
which should give you something similar to:
​
config:
/Users/yourdir/.jupyter
/usr/local/etc/jupyter
/etc/jupyter
data:
/Users/yourdir/Library/Jupyter
/usr/local/share/jupyter
/usr/share/jupyter
runtime:
/Users/yourdir/Library/Jupyter/runtime

Frankly I'm surprised that all this time later there is still no built-in way to do this. I have used Isaac To's solution on Linux but recently had to make a jupyter notebook portable to Windows as well. Simply using os.getcwd() is fragile because a cell using it to set your JUPYTER_ROOT_DIRECTORY can potentially be called again, after you have changed your working directory.
Here is what I came up with:
try:
JUPYTER_ROOT_DIRECTORY
except NameError:
JUPYTER_ROOT_DIRECTORY = os.getcwd()
I put it in one of the first couple cells with the initial import statements. If the cell gets called again, it will not re-set the variable value because the variable exists and does not throw an exception.
It should be noted that unlike Isaac To's solution it sets the value to the directory the current .ipynb was run from, which is not necessarily the same directory as the top-level dir the user can access in the left hand file pane.

My suggestion is to use an intuitive approach.
Create a new folder within the Jupyter environment with a very unique name, for example, T246813579.
You can now locate the Jupyter working path by searching in your file explorer. For example, you can use the Windows Explorer in order to locate your new folder.
The expected result should look something like this:
C:\Users\my_user_name\JupyterHome\T246813579

The answer from #Isaac works well for Linux, but not all systems have /proc. For a solution that works on macOS and Linux, we can use shell commands, taking advantage of the ! shell assignment syntax in Jupyter:
import os
JPY_ROOT = ! lsof -a -p {os.environ['JPY_PARENT_PID']} -d cwd -F n | tail -1 | cut -c 2-
JPY_ROOT = JPY_ROOT[0]
print(JPY_ROOT) # prints Jupyter's dir
Explanation:
Get the process ID (pid) of the current jupyter instance with os.environ['JPY_PARENT_PID']
Call lsof to list the process's open files, keeping only the current working dir (cwd)
Parse the output of lsof using tail and cut to keep just the directory name we want
The ! command returns a list, here having only one element
Alternate Version
To save the os import, we could also use shell commands to get the PID. We could also do the subsequent string wrangling in python, rather than calling tail and cut from the shell, as:
JPY_ROOT = ! lsof -a -p $(printenv | grep JPY_PARENT_PID | cut -d '=' -f 2) -d cwd -F n
JPY_ROOT = JPY_ROOT[2][1:]

Related

WSL PATH environment variable is incorrect / zsh config not loaded [duplicate]

I am using Ubuntu via WSL 2.0 on Windows 10 and would like to run Texlive from the Windows command line. To do so I prepended the Texlive folder to the path in /etc/environment (I also tried a number of other locations eg. $HOME/.bashrc):
C:\Users\scott\Documents>wsl echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Windows/system32:...
C:\Users\scott\Documents>wsl
scott#SCOTT-PC:/mnt/c/Users/scott/Documents$ echo $PATH
/usr/local/texlive/2020/bin/x86_64-linux:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Windows/system32:...
Why is there a difference between these two paths? Is it possible to change the first PATH variable?
To be honest, when I first looked at this question, I thought it would be an easy answer. Oh how wrong I was. There are a lot of nuances to how this works.
Let's start with the fairly "easy" part, though. The main difference between the first method and the second:
wsl by itself launches into a login (and interactive) shell
the shell launched with wsl echo $PATH is neither a login shell nor an interactive shell
So the first will source both login scripts (e.g. ~/.profile) and interactive startup scripts (e.g. ~/.bashrc). The second form does not get to source either of these.
You can see this a different way (and get to the solution) with the following commands:
wsl -e bash -c 'echo $PATH'
wsl -e bash -li -c 'echo $PATH'
The -li forces bash to run as a login and interactive shell, thus sourcing all of the applicable startup scripts. And, as #bovquier points out in the comments, a single quote is needed here to prevent PowerShell from interpolating the $ before it gets to Bash. That, or escape it.
You should be able to run TeX Live the same way, just replacing the "echo $PATH" with the startup command you need for TeX Live.
A second option would be to create a script that both adds the path and runs the command, and just launch that script through wsl /path/to/script.sh
That said, I honestly don't think that your current login/interactive PATH is coming from /etc/environment. In my testing, at least, /etc/environment has no use in WSL, and that's to be expected. /etc/environment is only sourced by PAM modules, and with no login check performed by WSL, there's no reason to invoke PAM in either the wsl nor the wsl echo $PATH commands.
I'd expect that you still have the PATH setting in ~/.bashrc or somewhere similar), and that's where the shell is picking it up from at the moment.
While this isn't necessarily critical to understanding the answer, you might also wonder, if /etc/environment isn't used for setting the default (non-login, non-interactive) path in WSL, what is? The answer seems to be that it is hard-coded into the init that starts up WSL. That init is also what appends the Windows path (assuming you don't have that feature disabled in /etc/wsl.conf).

zsh: no such file or directory:

I have an assignment where I have to use a syntax checker on a python program to see if it's correct. The line I use is:
/directory$ python syntax-checker file.py
However I keep getting the error no such file or directory:
I am supposed to open the python file with the syntax-checker which is located in the directory.
I also have no clue why my terminal looks the way it does.
It says: fullname#firstname-MBP-2 ~ % which seems odd. Can someone help me out?
When you type /theoinf you're trying to change to a directory that doesn't exist (if you ls / you'll notice theoinf isn't in the output). Your mkdir command was performed in your home directory so that's where the theoinf directory will exist. Where you're actually trying to change to then should be:
/Users/[username]/theoinf
on a mac, or a shortcut:
~/theoinf
fullname#firstname-MBP-2 ~ % is just your terminal prompt showing:
fullname - your username
firstname-MBP-2 - your hostname
~ - the current working directory, your home in this case.
If you don't like the way your prompt looks, I'd recommend installing ohmyzsh. It has a lot of great functionality and lets you apply a wide range of themes to your terminal.
Also, iterm2 is a replacement terminal for mac which has far greater features than the standard terminal you get in Mac OS.

Use cmd commands in QT

Hi i was checking and anyone can use commands very similar in cmd like dir mkdir etc.
But for example when i try to use command (cd ..) i couldn't
QProcess consola;
consola.start("cmd.exe /C " + comando);
consola.waitForFinished();
consola.waitForReadyRead();
This is the question how i can use more commands in cmd for qt for example.
At least from the command line:
cmd /C "cd \"
works as did directories other than root. (Note the parenthesis around the command since it contains embedded spaces.) However, this example isn't very useful because this executes the command shell, changes the directory in that command shell, and then the command shell disappears, and your current directory is back to where you started.
I recommend looking into the QDir class, which has methods such as "current ()" and "setCurrent ()" for getting and setting the current directory. There are equivalents for mkdir and many others. Also, using QDir is much more cross-platform friendly, where using the "cmd" shell is Windows-specific.
You don't say what you're trying to accomplish, so beyond that suggestion, it's impossible to know how to best help you.
Your process's current directory can be and mostly is different than current directory of your is running. Please read chdir manpage for that.
That command is mostly working but changing the current directory of your process.

Weird behaviour with zsh PATH

I just encourage a weird problem with zsh today.
My environment is Mac OS X Yosemite, zsh 5.0.5 (x86_64-apple-darwin14.0)
In .zshrc, I have manually set the PATH variable to something like
export PATH="$PATH:~/.composer/vendor/bin"
Try echo $PATH in terminal, the result is as expected (contained ~/.composer/vendor/bin). Then try executing a binary from ~/.composer/vendor/bin, It'll always return me "zsh: command not found" error.
Try switching to bash, echo $PATH is also as expected, have the same result as zsh shell.
Try executing a binary from ~/.composer/vendor/bin, no problem found. Seem the PATH var is acting well on the bash shell.
What's wrong with my zsh shell?
Thanks
Try using $HOME instead of ~. In many situations, shells do not expand ~ when you expect them to and it is usually better to use $HOME. ~ is really only intended to be a short cut for interactive use. (The only case I can recall where ~ was preferred was in a .gitalias, where ~ was expanded and variables were not.)
Type rehash to pick-up $PATH changes.
From the zsh user guide:
The way commands are stored has other consequences. In particular, zsh
won't look for a new command if it already knows where to find one. If
I put a new ls command in /usr/local/bin in the above example, zsh
would continue to use /bin/ls (assuming it had already been found). To
fix this, there is the command rehash, which actually empties the
command hash table, so that finding commands starts again from
scratch. Users of csh may remember having to type rehash quite a lot
with new commands: it's not so bad in zsh, because if no command was
already hashed, or the existing one disappeared, zsh will
automatically scan the path again; furthermore, zsh performs a rehash
of its own accord if $path is altered. So adding a new duplicate
command somewhere towards the head of $path is the main reason for
needing rehash.
EDIT However #WilliamPursell could be onto something with his comment:
note that "composer" != ".composer"

How to clear Jupyter Notebook's output in all cells from the Linux terminal?

I have a problem when the output from a notebook is really long and it's saved into the notebook, any time I want to open this particular notebook again the browser crashes and can't display correctly.
To fix this I have to open it with a text editor and delete all output from that cell causing the problem.
I wonder if there is a way to clean all output from the notebook so one can open it again without problem. I want to delete all output since deleting a specific one seems more troublesome.
nbconvert 6.0 should fix --clear-output
The option had been broken for a long time previously, bug report with merged patch: https://github.com/jupyter/nbconvert/issues/822
Usage should be for in-place operation:
jupyter nbconvert --clear-output --inplace my_notebook.ipynb
Or to save to another file called my_notebook_no_out.ipynb:
jupyter nbconvert --clear-output \
--to notebook --output=my_notebook_no_out my_notebook.ipynb
This was brought to my attention by Harold in the comments.
Before nbconvert 6.0: --ClearOutputPreprocessor.enabled=True
Same usage as --clear-output:
jupyter nbconvert --ClearOutputPreprocessor.enabled=True --inplace my_notebook.ipynb
jupyter nbconvert --ClearOutputPreprocessor.enabled=True \
--to notebook --output=my_notebook_no_out my_notebook.ipynb
Tested in Jupyter 4.4.0, notebook==5.7.6.
If you create a .gitattributes file, you can run a filter over certain files before they are added to git. This will leave the original file on disk as-is, but commit the "cleaned" version.
For this to work, add this to your local .git/config or global ~/.gitconfig:
[filter "strip-notebook-output"]
clean = "jupyter nbconvert --ClearOutputPreprocessor.enabled=True --to=notebook --stdin --stdout --log-level=ERROR"
Then create a .gitattributes file in your directory with notebooks, with this
content:
*.ipynb filter=strip-notebook-output
How this works:
The attribute tells git to run the filter's clean action on each notebook file before adding it to the index (staging).
The filter is our friend nbconvert, set up to read from stdin, write to stdout, strip the output, and only speak when it has something important to say.
When a file is extracted from the index, the filter's smudge action is run, but this is a no-op as we did not specify it. You could run your notebook here to re-create the output (nbconvert --execute).
Note that if the filter somehow fails, the file will be staged unconverted.
My only minor gripe with this process is that I can commit .gitattributes but I have to tell my co-workers to update their .git/config.
If you want a hackier but much faster version, try JQ:
clean = "jq '.cells[].outputs = [] | .cells[].execution_count = null | .'"
Use --ClearOutputPreprocessor.enabled=True and --clear-output
Following this command:
jupyter nbconvert --ClearOutputPreprocessor.enabled=True --clear-output *.ipynb
nbstripout worked well for me.
Open the Jupyter terminal, navigate to the folder containing your notebook, and then run the following line:
nbstripout my_notebook.ipynb
To extend the answer from #dirkjot to resolve issue regarding sharing configuration:
Create a local .gitconfig file, rather than modifying .git/config. This makes the command that needs to be run on other machines slightly simpler. You can also create a script to run the git config command:
git config --local include.path ../.gitconfig
Note I have also changed the log level to INFO because I did want to see confirmation that the clean was running.
repo/.gitconfig
[filter "strip-notebook-output"]
clean = "jupyter nbconvert --ClearOutputPreprocessor.enabled=True --to=notebook --stdin --stdout --log-level=INFO"
repo/.gitattributes
*.ipynb filter=strip-notebook-output
repo/git_configure.sh
git config --local include.path ../.gitconfig
Users then just need to run:
$ chmod u+x git_configure.sh
$ ./git_configure.sh
Use clean_ipynb, which not only clears notebook output but can also clean the code.
Install by pip install clean_ipynb
Run by clean_ipynb hello.ipynb
I must say I find jupyer nbconvert painfully slow for the simple job of clearing some sub-arrays and resetting some execution numbers. It’s a superior solution in maintainability because that tool is expected to be updated if there is a change in the notebook source code format. However, yhe alternate solution below is faster and may also be useful if you don’t have nbconvert 6.0 (I have an environment running 5.6.1 at the moment…)
A very simple jq (a sort of sed for json) script does the trick very fast:
jq 'reduce path(.cells[]|select(.cell_type == "code")) as $cell (.; setpath($cell + ["outputs"]; []) | setpath($cell + ["execution_count"]; null))' notebook.ipynb > out-notebook.ipynb
Very simply, it identifies code cells, and replaces their outputs and execution_count attributes with [] and null respectively.
Or if you only want to remove the outputs and keep execution numbers, you can do even simpler:
jq 'del(.cells[]|select(.cell_type == "code").outputs[])' notebook.ipynb > out-notebook.ipynb
As mentioned in one of the previous answers you can use the command-line json processor jq to perform this task notably quicker than with nbconvert. A complete command for getting rid of metadata, outputs and execution counts can be found in this blog post:
jq --indent 1 \
'
(.cells[] | select(has("outputs")) | .outputs) = []
| (.cells[] | select(has("execution_count")) | .execution_count) = null
| .metadata = {"language_info": {"name":"python", "pygments_lexer": "ipython3"}}
| .cells[].metadata = {}
' 01-parsing.ipynb
If desired, you could modify to just clean a specific part of the output, such as execution counts (recursively wherever they occur in the json), and then add this as a git filter:
[filter "nbstrip"]
clean = jq --indent 1 '(.. |."execution_count"? | select(. != null)) = null'
smudge = cat
And add the following to ~/.config/git/attributes to have the filter applied globally to all your local repos:
*.ipynb filter=nbstripout
There is also nbstripout which is made for this purpose, but it's a bit slower.

Resources